I'm encountering an issue with tap-postgres when I specify a catalog where config fails to test successfully; if I run:

meltano --environment=stg config tap-postgres--secondary-svc-db test
2024-06-02T00:49:46.723050Z [info     ] Environment 'stg' is active
Plugin configuration is valid

that's fine, but if the catalog has expired it takes around 4 or 5 minutes to regenerate the catalog (subsequent runs for a very short period of time, 1-2 minutes, use the cached catalog, otherwise it gets regenerated which takes another 4-5 minutes). So I figured I'd run

meltano --environment=stg invoke tap-postgres--secondary-svc-db --discover > ./extract/tap-postgres--secondary-svc-db-catalog.json

then if I specify the catalog path in my configuration in

meltano.yml

like:

...
catalog: extract/tap-postgres--secondary-svc-db-catalog.json
...

and re-run the config/test command I get this:

meltano --environment=stg config tap-postgres--secondary-svc-db test
2024-06-02T01:00:13.541060Z [info     ] Environment 'stg' is active
2024-06-02T01:00:15.068077Z [info     ] Found catalog in /Users/cfeduke/Projects/startengine/data-warehouse-regulatory/extract/tap-postgres--secondary-svc-db-catalog.json
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
No RECORD message received

Additionally, running commands like

meltano --environment=stg select tap-postgres--secondary-svc-db --list --all

shows all of the fields as excluded when the catalog is specified, and shows the correctly filtered selected fields if no catalog is specified.

Calling

--discover

will cause Meltano to not pass selection rules to the tap. You probably want instead

meltano --environment=stg invoke --dump=catalog tap-postgres--secondary-svc-db > ./extract/tap-postgres--secondary-svc-db-catalog.json