Siddu Hussain
05/15/2024, 7:55 AMmeltano el <extractor> <loader> [--state-id TEXT]
I see that when we run just
meltano el <extractor> <loader>
I see that a state is created but not used in the next run. Is it expected that I have to pass the state ID every time?
2---
Does using a stream map like the one below only disable data load, or do extractions also stop? I have a use case to exclude the stream without doing anything.
{
"stream_maps": {
"addresses": null
}
}
3--
Is there a way to see all metrics of an el run in a tabular format?Edgar Ramírez (Arch.dev)
05/15/2024, 3:36 PMIs it expected that I have to pass the state ID every time?Yes,
meltano el
expects an explicit state ID for incremental pipelines
Does using a stream map like the one below only disable data load, or do extractions also stop? I have a use case to exclude the stream without doing anything.Using a stream map like that will cause the stream to be dropped after it's been extracted. To avoid extracting the stream in the first place, you want to use https://docs.meltano.com/concepts/plugins/#select-extra
Is there a way to see all metrics of an el run in a tabular format?Not at the moment but do log an issue and include which metrics would be most helpful 🙏
Siddu Hussain
05/15/2024, 3:38 PM