Anyone using Meltano to do data integration for things like HR system to Directory Systems. ie BambooHR <-> Google Workspace, or even something like BambooHR <-> Greenhouse? I think with Taps/Targets it'd look something like BambooHR Tap, Spreadsheet Target (or pick a DB). Tap for the Target system ie Google Workspace to something like that target. (Insert some glue here for correlating your desired state in the system you loaded all the data to). Spreasheet Tap, Google Workspace target. Looks like the answer is pretty clearly that no one's doing this exact setup, but maybe there's something close? Been diving through the docs and doing some quick searches, haven't found the answer. Sorry if this is obvious!

@visch Writing a BambooHR tap shouldn't be too difficult, assuming they have a reasonable API, which would then work with any target that accepts arbitrary streams and properties (tables and fields). A target for Google Workspace would probably not take arbitrary streams and properties, since it's not an arbitrary data store. It would either accept incoming data matching a specific schema (e.g.

users

stream,

email

property), which would limit the taps it would work with to ones specifically built for it, kind of defeating the point of Singer, or it would allow you to configure a mapping from source streams/properties to things the target/destination understands (e.g.

users_stream: 'people'

,

email_property: 'email_verified'

), so that it can work with any tap as long as the mapping is configured correctly.

i'd use a

tap-google-workspace

if you created it @visch 😄

Hope there isn't too much babble in there 😮

Not at all! I've just left a comment with some thoughts

https://github.com/airbytehq/airbyte/issues/2110 , stumbled on this. Airbyte might be making a google tap. I'll keep an eye on this

Hey @douwe_maan - Just finding this thread. I'm a data engineer at Bonobos. We've been using the singer framework for a while but havent used meltano yet since we built a custom tap/target manager. Anyways - we've built plenty of custom targets (kafka/apis/salesforce/sftp/etc.) and the pattern has been working pretty well for us. We love the direction the singer-sdk is going, hoping to get support in there for targets. Me/my teammates would be happy to contribute to help build that out and potentially open source some of our targets if possible. I'm going to follow up on the open issue on gitlab

@pnadolny there is a WIP MR in right now on the singer-sdk for targets. It's very unpolished but not a bad place to take a peek!

@pnadolny Welcome! We're very eager to collaborate on the SDK for targets, and especially targets that load into SaaS destinations as you've alluded to in https://gitlab.com/groups/meltano/-/epics/91#note_552915677. We also just created #C01RKUVUG4S specifically for this topic

Awesome! Thanks for the links, I'll start taking a look through.