Hi, is it possible to pass custom catalog file to a tap when calling the elt command?

@or_barda

meltano elt

will look for a catalog file at

.meltano/run/<tap>/tap.properties.json

. If the tap has the

discover

capability, Meltano will automatically run it, it apply the selection, metadata and schema rules, and put the file there, overriding anything you may have put there yourself. However, if you remove the

discover

capability, Meltano will not try to generate its own catalog and will use the one you provided at that location instead. That means that you could put the catalog file in your project outside of

.meltano

, and

cp

it to the desired location ahead of calling

meltano elt

. Would that work for you?

Got it, and that's actually what I'm doing but I have the same schema for multiple taps of the same type and I didn't want to copy this for all of them.

Ah, all right, makes sense. Would you mind creating an issue so we can consider adding a

--catalog

flag or something like that? 🙂

So what happen is that I want to optimize one of the taps, so what I'm doing is splitting its objects and write them to the meltano.yml as different taps and by this I'm able to run them in parallel

Ah, clever 🙂 When you say "splitting its objects", do you mean each tap selects a different set of Singer streams? Or they all use the same catalog, but select different objects based on some setting?