Hi, folks! I need to extract data from Postgresql and load them on Elasticsearch. Could you point me how I can perform that ETL?

Hi, @adriano_vieira! Welcome! The

transferwise

org has a tap for Postgres which you can use to extract the data. You can add this to meltano with the

meltano add --custom extractor …

command here. I’m not personally aware of a singer target for Elasticsearch specifically, but depending on your existing stack and the size of the data, you can target jsonl output (or another file-based output target) and use the Filebeat or another Elasticsearch connector to ingest from the file storage. There are probably lots of other ways to do this as well but does this help?

Yes! Can you point me what is the steps to build a target for Elasticsearch, please?

@adriano_vieira @aaronsteers and the community are still working on the basics of an SDK for targets. there’s generally bit more complexity than in a tap so coming up with good abstractions takes a bit more time. Do checkout https://gitlab.com/meltano/singer-sdk/-/issues/96 though!

Hi! Thanks Looking forward to having an SDK for targets so we probably could work on elasticsearch one.

@adriano_vieira - The target SDK is still a work in progress but we announced the preview and are actively looking for early feedback. By chance are you able to join tomorrow for our #C01QS0RV78D? I’ll be demoing an SDK-based target for AWS Athena. Although the Target SDK is still in preview, that should give you a pretty good idea of the interface. We’ll also record the office hours session, and I think we’ll do a (less-technical) demo in this Friday’s #C01S996HGFK.