Can a few folks confirm, before I take a stab at c...
# random
c
Can a few folks confirm, before I take a stab at creating it, there is no Salesforce Singer target?
d
Correct, just a tap
a
Actually....
I don't know the status on this, but I do remember some threads on folks working on it.
cc @alexander_butler, @hassan_syyid, @kelly_mondor who may have additional context on prior iterations and how far each have gotten in their efforts.
k
I did not get very far and just ended up using hightouch to get the data in quickly 🙂
h
@aaronsteers / @connor_lough We built this: https://gitlab.com/hotglue/target-salesforce-v2
a
Nice! 🔥
c
@aaronsteers thanks for surfacing this to @hassan_syyid! And thank you @hassan_syyid for building it! Do you have plans to add more objects in the near future? Maybe I'll try to clone it, make a branch and add? For context, I'm evaluating different ways to get .json stored in S3 (about a dozen or so different objects) into my Salesforce account.
a
I reverse ETL data to salesforce with meltano but I ended up going the Meltano Utility approach for a number of reasons.
h
Yeah we are planning to add more objects in the future, PRs are more than welcome 🙂
c
@alexander_butler this weekend was my first time looking at the Meltano Utilities Can you speak about the general process you took? I'm imagining a source-to-s3 pipeline via Meltano, and then some utility package to transform and send to Salesforce?
a
@connor_lough 1. I have a subdirectory in my meltano project which is its own poetry project. 2. I have a core controller which every integration inherits from a. specific integrations inherit and override the
run
function and have some niceties available like a connector to DWH + GCS storage + Feature Flag client 3. Each integration is its own
.py
file with a
main
function exposed as a poetry script (executable) a.
main
just instantiates the relevant class and executes the
run
method of the class,
__enter__
and
__exit__
are conveniences for clean up 4. Meltano use
pip_url: -e modules/reverse_etl
to install the sub dir 5. Currently I use
inherits_from
to expose each function (which poetry creates as an executable) as its own utility. It could be valid to just use
commands
and the "executable" config option exposed in meltano to keep them as a single utility 6. In production, just run it as needed. so
meltano run tap-salesforce target-bigquery py_001_opportunity_enrichment
as an example pipeline reasons are a bit long to get into but mostly a business-specific solution for reverse etl made more sense for us than a generic solution and had lower risk for critical operational systems