I received a (tool agnostic) request from a cowork...
# best-practices
k
I received a (tool agnostic) request from a coworker to pull customer data from Stripe and update contacts on Hubspot accordingly. In Meltano, I can extract customer data from Stripe smoothly. But my hunch is that the workflow of loading updated data to hubspot does not lend itself well to the ELT process. I'm considering using Zapier, as there is no target-hubspot ...plus I like analytics to be analytics. Any thoughts or clarifications?
t
@kathryn_cowie you could definitely use Meltano and Singer extractors to move data from any point to point source. It "just" requires a target for any destination you'd want to send data to. We've got a few examples on https://hub.meltano.com/loaders/ which can be considered "SaaS API" targets. Part of the challenge with SaaS targets is formatting the data for persnickety APIs. A lot of folks do this formatting in the warehouse but in theory you could do it in the target itself. This is what led to the creation of a lot of the "Reverse ETL" tools. All that said, you can definitely do reverse etl with Metlano and we have a lot of folks in the community doing that, but for your specific case of sending data to Hubspot there may not be an existing target yet (as you've stated). Does that help clarify? 😄
c
A lot of folks do this formatting in the warehouse
Just quickly chiming in here with a "This is how we do it" comment: • Meltano ELT runs from SaaS sources (many different) into Postgres. And DBT models manage the different "shapes" of analytics exposures according to requirements (i.e. some models do the analytics insights transformations into the traditional datawarehouse "shapes". And then other dedicated dbt models have the task of shaping the necessary JSON objects that have the requisite format to be loaded back into external data stores. • Postgres happens to have fantastic support for working with JSON (both parsing and generating it) and makes this task very easy. • At the end, meltano does the reverse EL step of reading the Postgres relations with the correctly shaped data in it and dump that into a JSONL file for actual loading into the external system.
k
thanks, that does help clarify!
c
There was another conversation happening in parallel in this thread with a more detailed chat about "Reverse ETL" https://meltano.slack.com/archives/C01TCRBBJD7/p1658192407828719
k
thanks, I will look out to see if anyone whips up a
target-hubspot
and in the mean time work on getting closely acquainted with DBT