Hello, I'm trying to tap data from hubspot. After browsing for a while, I find this singer-tap for hubspot: https://www.singer.io/tap/hubspot/ To add it into my meltano project, I invoke this:

meltano add --custom extractor tap-hubspot

I set

pip_url

into

tap-hubspot

and I left everything else as is. After the extractor was added, I add some configuration manually in my

meltano.yaml

. So my yaml is now looks like this:

version: 1
send_anonymous_usage_stats: true
project_id: a08f7b37-b962-410b-ba91-e9aabd25008a
plugins:
  extractors:
  - name: tap-hubspot
    namespace: tap_hubspot
    pip_url: tap-hubspot
    executable: tap-hubspot
    capabilities:
    - state
    #- catalog
    #- discover
    config:      
      redirect_uri: "<https://redirect-url-from-my-hubspot-app.com/>"
      client_id: "my-client-id"
      client_secret: "my-client-secret"
      refresh_token: "i-am-not-sure-about-this"
      start_date: "2017-01-01T00:00:00Z"
      disable_collection: false
  loaders:
  - name: target-sqlite
    variant: meltano
    pip_url: git+<https://gitlab.com/meltano/target-sqlite.git>
schedules:
- name: hubspot-to-sqlite
  extractor: tap-hubspot
  loader: target-sqlite
  transform: skip
  interval: '@once'
  start_date: 2021-05-31 07:14:31.743914

To run the elt, I run

meltano elt tap-hubspot target-sqlite

Now the problem is, I'm not sure about two things: • The configuration, especially the "refresh token" • The capabilities I should put for

tap-hubspot

. Using the configuration above, I got this output:

(meltano) gofrendi@sanctuary [12:07:53] [~/kata/meltano/hubspot]
-> % meltano elt tap-hubspot target-sqlite
meltano       | Running extract & load...
meltano       | No state was found, complete import.
tap-hubspot   | INFO No properties were selected
meltano       | Extract & load complete!
meltano       | Transformation skipped.
meltano       | /home/gofrendi/.local/share/virtualenvs/meltano-OykesmMm/lib/python3.8/site-packages/meltano/core/job/finder.py:63: SAWarning: TypeDecorator IntFlag() will not produce a cache key because the ``cache_ok`` flag is not set to True.  Set this flag to True if this type object's state is safe to use in a cache key, or False to disable this warning.
meltano       |   self.with_payload(session, **kwargs)
meltano       | /home/gofrendi/.local/share/virtualenvs/meltano-OykesmMm/lib/python3.8/site-packages/meltano/core/plugin/singer/tap.py:198: SAWarning: TypeDecorator IntFlag() will not produce a cache key because the ``cache_ok`` flag is not set to True.  Set this flag to True if this type object's state is safe to use in a cache key, or False to disable this warning.
meltano       |   for state_job in incomplete_state_jobs:

But I don't see any sqlite db file created. I also try to add

[state, catalog, discover]

capability, but I got this errors:

Exception: Giving up on request after 1 tries with url ('<https://api.hubapi.com/properties/v1/contacts/properties',>) and params {}

Has anyone ever try to do this? Also, I hope this is the correct channel to share my problems. Otherwise please let me know.

hubspot taps uses a deprecated option

properties

Do try the

properties

capabilities flag and see if that helps it. You can calso rune with `

meltano --log-level=debug

to get some more info

Hi again, thx for helping. Apparently I didn't setup my OAuth correctly. I tried with

hapikey

instead of OAuth. It didn't really work as I expect (the data wasn't saved into

warehouse.db

), but at least I can see the list of schema. As for the capabilities, I include everyting

[state, discover, catalog, properties]

I’ve got an MR open to add some additional detail around properties (among other things)https://gitlab.com/meltano/meltano/-/merge_requests/2149