Hello, I'm trying to tap data from hubspot. After...
# plugins-general
g
Hello, I'm trying to tap data from hubspot. After browsing for a while, I find this singer-tap for hubspot: https://www.singer.io/tap/hubspot/ To add it into my meltano project, I invoke this:
Copy code
meltano add --custom extractor tap-hubspot
I set
pip_url
into
tap-hubspot
and I left everything else as is. After the extractor was added, I add some configuration manually in my
meltano.yaml
. So my yaml is now looks like this:
Copy code
version: 1
send_anonymous_usage_stats: true
project_id: a08f7b37-b962-410b-ba91-e9aabd25008a
plugins:
  extractors:
  - name: tap-hubspot
    namespace: tap_hubspot
    pip_url: tap-hubspot
    executable: tap-hubspot
    capabilities:
    - state
    #- catalog
    #- discover
    config:      
      redirect_uri: "<https://redirect-url-from-my-hubspot-app.com/>"
      client_id: "my-client-id"
      client_secret: "my-client-secret"
      refresh_token: "i-am-not-sure-about-this"
      start_date: "2017-01-01T00:00:00Z"
      disable_collection: false
  loaders:
  - name: target-sqlite
    variant: meltano
    pip_url: git+<https://gitlab.com/meltano/target-sqlite.git>
schedules:
- name: hubspot-to-sqlite
  extractor: tap-hubspot
  loader: target-sqlite
  transform: skip
  interval: '@once'
  start_date: 2021-05-31 07:14:31.743914
To run the elt, I run
Copy code
meltano elt tap-hubspot target-sqlite
Now the problem is, I'm not sure about two things: • The configuration, especially the "refresh token" • The capabilities I should put for
tap-hubspot
. Using the configuration above, I got this output:
Copy code
(meltano) gofrendi@sanctuary [12:07:53] [~/kata/meltano/hubspot]
-> % meltano elt tap-hubspot target-sqlite
meltano       | Running extract & load...
meltano       | No state was found, complete import.
tap-hubspot   | INFO No properties were selected
meltano       | Extract & load complete!
meltano       | Transformation skipped.
meltano       | /home/gofrendi/.local/share/virtualenvs/meltano-OykesmMm/lib/python3.8/site-packages/meltano/core/job/finder.py:63: SAWarning: TypeDecorator IntFlag() will not produce a cache key because the ``cache_ok`` flag is not set to True.  Set this flag to True if this type object's state is safe to use in a cache key, or False to disable this warning.
meltano       |   self.with_payload(session, **kwargs)
meltano       | /home/gofrendi/.local/share/virtualenvs/meltano-OykesmMm/lib/python3.8/site-packages/meltano/core/plugin/singer/tap.py:198: SAWarning: TypeDecorator IntFlag() will not produce a cache key because the ``cache_ok`` flag is not set to True.  Set this flag to True if this type object's state is safe to use in a cache key, or False to disable this warning.
meltano       |   for state_job in incomplete_state_jobs:
But I don't see any sqlite db file created. I also try to add
[state, catalog, discover]
capability, but I got this errors:
Copy code
Exception: Giving up on request after 1 tries with url ('<https://api.hubapi.com/properties/v1/contacts/properties',>) and params {}
Has anyone ever try to do this? Also, I hope this is the correct channel to share my problems. Otherwise please let me know.