Sorry for all the noise in here : ) Struggling to get this project finished Any reason why the output of

meltano select --list {my_tap}

shows that all streams / columns are selected but they get skipped anyway when running the tap?

Hi @philip_johnson! This is probably a custom Meltano plugin of a tap built with the SDK, so you’re probably missing the

catalog

capability in

meltano.yml

:

plugins:
  extractors:
    - name: <my-tap>
      capabilities:
      - state
      - catalog
      - discover

https://docs.meltano.com/tutorials/custom-extractor#add-the-plugin-to-your-meltano-project

Yeah it is, I'm bringing this over from a legacy singer tap I wrote a few years ago. I have the catalog capability in the config and also provided a path to the catalog manually (which is maybe the problem since I'm not sure the select commands reads the catalog file or not) Trying running it without a manual catalog now and see if it works - is that advisable?

and also provided a path to the catalog manually

ah yeah, that may be it.

select

indeed does not work together with a manual catalog file

ah ok, good to know - we'll see how this run goes and I'll report back : )

@edgar_ramirez_mondragon - Should we throw an error if

select

is used while

catalog

capability is not declared, and a warning if manual catalog is combined with

select

?

I wonder if the first case should also just warn, and instead have

meltano run

/`meltano invoke` log a warning when catalog rules could not be applied because the capability is missing

Edgar, thanks for logging: https://github.com/meltano/meltano/issues/6717