josh_lloyd
07/23/2021, 3:36 PMmeltano invoke
? I’m trying to run a very specific command for a custom tap but the command seems to be ignoring the arguments I’m trying to pass to the tap. Specifically I’m trying to run meltano invoke tap-rest-api --infer_schema
but the first line in the error message is
Catalog discovery failed: command ['/Users/.../meltano/.meltano/extractors/tap-rest-api/venv/bin/tap-rest-api', '--config', '/Users/.../meltano/.meltano/run/tap-rest-api/tap.config.json', '--discover'] returned 1: INFO Loading Schemas
which makes me think that it’s ignoring the argument --infer_schema
entirely because I don’t see it in the command it’s spitting back at me. It seems to just be running --discover
instead.douwe_maan
07/23/2021, 4:06 PMmeltano invoke
with a tap will always call the tap in discovery mode first if the discovery
capability is present, except when meltano invoke <tap> --discovery
is run directly, because discovery/catalog is needed for sync mode (without any --
args). I think the better behavior would be to only run discovery mode ahead of the regular requested invocation if it’s actually running in sync mode, i.e. not getting any user-provided arguments. That would require changing https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin/singer/tap.py#L213douwe_maan
07/23/2021, 4:06 PM--discover
instead of --infer_schema
, it’s running it before running --infer_schema
, and if --discover
fails, it won’t get to --infer_schema
douwe_maan
07/23/2021, 4:07 PM--discover
doesn’t fail (which may be impossible if that depends on having inferred the schema first?), removing the discovery
capability (temporarily), or fixing that check in https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin/singer/tap.py#L213. I agree this is undesireable behavior, so I suggest filing an issue either wayjosh_lloyd
07/23/2021, 4:08 PMpnadolny
08/05/2021, 7:34 PMpnadolny
08/05/2021, 7:35 PMjosh_lloyd
08/05/2021, 7:51 PM