Does anyone have any experience with the tap-hubspot from potloc (https://github.com/potloc/tap-hubspot)? I've been working on it today and cannot seem to figure out how the incremental loading works. See my configuration below. I'm trying to write it to a MSQL database. The initial

meltano run tap-hubspot target-mssql

writes all records, even despite what the

start_date

is configured to. All subsequent runs also keep extracting and loading all records. Feel like I'm just missing something trivial as the in the git repository it explicitly mentions that the tap: "Incrementally pulls data based on the input state" Any pointers and suggestions are welcome!

- name: tap-hubspot
    variant: potloc
    pip_url: git+<https://github.com/potloc/tap-hubspot.git>
    config:
      access_token: ${HUBSPOT_TOKEN}
      start_date: "2023-08-20T00:00:00Z"
      flattening_enabled: true
      flattening_max_depth: 1
    select:
    - contacts.properties.*
    - "!contacts.associations.*"
    - "!contacts.associations"

@Stéphane Burwash may have an idea as they maintain the tap, as far as I am aware.

@bernard_wezeman sadly currently start_date is not functional. I've heard some comments also regarding the incremental loading not being functional -> I'll have to look into it

@Stéphane Burwash have you had the chance to look into this at all yet? Today I did look into it a bit myself and found out that filtering isn't supported within the Hubspot API on the default GET endpoints (e.g.

/crm/v3/objects/contacts

) whereas this is possible when utilizing the search (POST) endpoint (e.g.

/crm/v3/objects/contacts/search

). Forked the repo and got incremental to work (only for

ContactsStream

atm), which also automatically leads the start_date config to be functional. Can check it out here: https://github.com/BernardWez/tap-hubspot. Do you think this concept is something you guys would want to consolidate into the Potloc variant or would you prefer going about it a different way? If incremental loading isn't a priority rn, lmk then I'll stop bugging ya and just stick with the fork for what I'm doing rn 😉

Hey @bernard_wezeman! Did not take the time to investigate (been pretty busy these days, sorry 😅 ) but this is absolutely something I've always wanted to try and implement. This isin't a priority for us, but feel free to make a PR to add this to the tap or I'll try to look at it when I get the chance 😄

No worries, I understand things get busy! I'll have a look into cleaning up the code a bit and make a PR sometime later this week. Those objects are probably also the only ones I'm really interested in tbh and would be a good starting point indeed.