I'm also kind of confused from the CLI and all its subcommands, (e.g

invoke

,

el

,

elt

,

run

). Some of them have overlaps and it can get overwhelming 😅

https://docs.meltano.com/reference/command-line-interface/#how-to-use-5 The docs do a pretty good job of defining each subcommand: •

el

run a extract-load task (a tap, and a target) •

elt

run extract-load-transform task (a tap, and a target, an optional utility/transformer) •

invoke

run a single plugin, such as a tap in discovery mode to get the schema, or run a target to get the configuration •

run

run a series of plugins, I believe you can chain together an arbitrary amount of plugins with this command

el

is currently an alias for

elt

with the

--transform=skip

option as default. We don’t recommend using the transformation capability anymore of

elt

and instead recommend

run

for doing that. Expect

elt

to be deprecated likely in a v4 release.

Makes sense, I am using ELT without a transform step in a few cases so I'll swap those over to

el

or

run

el

gives you more control over config, catalog, etc while

run

is more for multiple plugins. so if you’re needing more customizability then

el

may still be for you 🙂

Thanks! What CLI subcommand am I supposed to use if I want to use the dbt-clickhouse plugin (from Clickhouse)? (or in general, how can I use dbt with clickhouse 😅) https://meltano.slack.com/archives/CMN8HELB0/p1692122264689919

Using DBT with Clickhouse is probably out of scope for here, but if you can run DBT core with Clickhouse independently of Meltano initially, then it shouldn't be too hard to integrate as a utility step and getting it added to the catalog. I maintain a WIP Clickhouse Target for loading data into Clickhouse but I haven't had a chance to use DBT with Clickhouse yet

my confusion is mainly from the existence of the meltano dbt-* plugins, e.g dbt-postrges or dbt-bigquery – what are they used for? speaking of the clickhouse plugin – why are there 2 different plugins for clickhouse

One is a target for loading data into Clickhouse, the other is a utility for running a local environment of Clickhouse for testing or running queries over data files