Can someone sanity check me before I file another bogus bug? I think

meltano run

is failing to pick up new tables from the source. I have a three-part pipeline - extractor, mapper, loader. The select list in the extractor is a wildcard, e.g.

select:
    - MySchema-NewTableTest*.*

The first time I run the pipeline all tables matching that spec are pushed to the loader - no problem there. On subsequent runs new and updated rows in those tables are pushed to the loader - also no problem there. But if I add a new table it never gets picked up. So for example if on the first run there's a table named NewTableTest1 it gets included and all updates to it get replicated on an on-going basis. But if I create a table called NewTableTest2... it never shows up in the destination. It seems to me like the catalog being sent to the extractor isn't being updated after the first run, but that's just a hunch - I don't know how to prove that. I tried this with

meltano elt

and it works as expected. I swear it used to work with

meltano run

too but I don't think I have any proof of that. This is not the expected behavior though, right?

the cached catalog is used if the plugin's

config

,

schema

and

metadata

don't change so the catalog isn't refreshed. A few options until #2915 or #2907 are addressed: •

meltano install extractor <tap name>

. This will try to upgrade the package and keep the existing venv. More relevant here: it will remove all the run files (!2267) • manually delete

.meltano/run/<tap name>/tap.properties.cache_key

Thanks @edgar_ramirez_mondragon. That explains everything. Until very recently I was using a local copy of the MySQL tap to deal with MySQL 8 compatibility issues; not long ago PW released a version that works with MySQL 8 so I switched to using a released version. So the exception for editable plugins was producing the desired behavior and I just didn't realize it. 🤦🏻‍♂️