Hi, is there a way to use --dump for catalog file ...
# announcements
o
Hi, is there a way to use --dump for catalog file without running the elt?
j
IIRC you can use
meltano select
to see what streams will get selected, and running either that command or
meltano invoke
(could be any invoke call to the tap, even
--help
) will generate a catalog file in
.meltano/run/[tap-name]
that you can inspect. Is that what you're asking?
actually it has to be
meltano select --list
oh I see what you are saying. Sorry,
--dump
is new functionality I had missed. Try adding
--dry
to the command so it doesn't actually run the pipeline @or_barda
d
@or_barda @julian_knight
meltano elt --dump=<file>
already doesn't actually run the pipeline! I tried to make this clear in the docs (https://meltano.com/docs/command-line-interface.html#elt, emphasis mine):
A
--dump
option can be passed (along with any of the other options) to dump the content of a pipeline-specific generated file to STDOUT instead of actually running the pipeline. This can aid in debugging extractor catalog generation, pipeline state lookup, and pipeline-specific configuration.
The reason
--dump
is an option on
meltano elt
and not a command of its own is that the entire ELT context (extractor, loader, optional transformer, job ID) is needed to correctly generate the files, which are then dumped to STDOUT instead of passed on to the plugin.
o
Got it, and it worked for me thanks