what is the expected behavior of `meltano invoke`?...
# troubleshooting
j
what is the expected behavior of
meltano invoke
? I’m trying to run a very specific command for a custom tap but the command seems to be ignoring the arguments I’m trying to pass to the tap. Specifically I’m trying to run
meltano invoke tap-rest-api --infer_schema
but the first line in the error message is
Copy code
Catalog discovery failed: command ['/Users/.../meltano/.meltano/extractors/tap-rest-api/venv/bin/tap-rest-api', '--config', '/Users/.../meltano/.meltano/run/tap-rest-api/tap.config.json', '--discover'] returned 1: INFO Loading Schemas
which makes me think that it’s ignoring the argument
--infer_schema
entirely because I don’t see it in the command it’s spitting back at me. It seems to just be running
--discover
instead.
d
@josh_lloyd
meltano invoke
with a tap will always call the tap in discovery mode first if the
discovery
capability is present, except when
meltano invoke <tap> --discovery
is run directly, because discovery/catalog is needed for sync mode (without any
--
args). I think the better behavior would be to only run discovery mode ahead of the regular requested invocation if it’s actually running in sync mode, i.e. not getting any user-provided arguments. That would require changing https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin/singer/tap.py#L213
So to be clear, it’s not running
--discover
instead of
--infer_schema
, it’s running it before running
--infer_schema
, and if
--discover
fails, it won’t get to
--infer_schema
The easiest way to work around this would be to either make sure
--discover
doesn’t fail (which may be impossible if that depends on having inferred the schema first?), removing the
discovery
capability (temporarily), or fixing that check in https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/plugin/singer/tap.py#L213. I agree this is undesireable behavior, so I suggest filing an issue either way
j
That makes sense. I would agree that a better default behavior would be the most desirable. I’ll file an issue
p
@josh_lloyd did you ever file an issue for this? Would you mind posting the link here?
I had trouble with something that might be along the same lines - at least in terms of how it could be resolved https://meltano.slack.com/archives/C01TCRBBJD7/p1628190421111500
j