Hi, I recently started researching Meltano. First of all, it looks amazing! What I don't seem to find is a way to have multiple extractors/taps (let's say PostgreSQL extractors) pointing to different databases. I've been searching the Gitlab issues and I can find threads with the same type of requests (also some threads in this slack mention this). However to be clear, because there's a lot of information out there and I surely could've missed a bunch of things, there is at this point no easy way (or at least UI-way) to have extractors of the same type for different instances. What is the best approach ... add different (custom) plugins for the same extractor?

Hi, I recently started researching Meltano. First of all, it looks amazing!

@gregory__tengu_ Thank you! 😊

there is at this point no easy way (or at least UI-way) to have extractors of the same type for different instances.

That's correct, but this is changing very soon, since I started working on https://gitlab.com/meltano/meltano/-/issues/2390 yesterday!

What is the best approach ... add different (custom) plugins for the same extractor?

You can either configure the same plugin differently each time you run it using environment variables: https://meltano.com/docs/integration.html#pipeline-specific-configuration, or you can add the same plugin multiple times as a custom plugin, with a different name each time. This approach will be easy to migrate to the new inheritance approach once that lands 🙂

Great! Thanks for the reply.

@douwe_maan I'm really looking forward to this landing. I started playing around with Meltano a few days ago and I came across your MR last night in looking for a solution to this. I actually had to fork Meltano anyway in order to bump up the OUTPUT_BUFFER_SIZE. Do you think it would be safe to merge your MR into my fork right now?

@tyler_martin It's not quite ready still, once the pipeline goes green again it should be in a better state, but I don't expect it to be ready for prime time until next week

Thanks! I'll check back next week on that. Ha yes I did see that issue. I'm working on getting this set up for a client but after I get their ETL pipeline working I might circle back and take a crack at that. I had been thinking about working on a fix for that issue

Sounds good!

Oh sweet ok. I'll try to get around to that later tonight. I've seen that there is a particular way that MRs are supposed to be formatted for Meltano

It just needs a changelog item, nothing too scary 😄

Awesome ok I will do that tonight