Hi team, what’s the easiest way to “unselect” a st...
# best-practices
e
Hi team, what’s the easiest way to “unselect” a stream? I know how to do it by changing the metadata in the catalog. But this is super cumbersome and requires passing in the whole catalog after just editing a few “selected” values to false. My use case is that in the
tap-github
a few streams require extra authentication, so I would like to run all the streams but one or two. Any thoughts?
t
e
Thanks! What needs to happen in the tap to make it available? I get
Error: No such option: --exclude
Or I need to call the tap using meltano directly?
How about doing something like this in the tap? https://github.com/MeltanoLabs/tap-github/pull/109
t
Oh so you’re calling the tap from outside Meltano? I think I’m confused on the workflow here
e
Yes im calling the tap from outside meltano (in that specific case just for testing target actually)
t
Sorry - coming back to this now. I suppose we could add something like this to the SDK to act as a filter on the catalog file… that doesn’t feel like the right place for it though but perhaps @edgar_ramirez_mondragon has opinions 😄 or @aaronsteers (he’s OOO this week though)
e
I think this could a valuable addition to the SDK, but we should probably also bring it to the singer working group, since this feature would compete with the catalog and might be seen as a divergence from the standard. Wdyt @taylor?
t
Yeah, would be worth bring to the working group. The simplest version of it could just be a pre-filter on the passed-in catalog to filter things out. If under-the-hood the tap still uses just the catalog then it’s probably fine