Working on getting Meltano working for our use case here for the first time. A few odd messages, I notice and would like some input on: 1. We have a huge backlog to sync, so it isn't realistic to wait for it to sync the whole table.... I've let it run overnight and see messages like

Incremental state has been updated at 2021-04-23 19:00:11.605346.

. However, when I rerun the same

elt

command, it says:

No state was found, complete import.

Does this mean it isn't resuming where it had left off? 2.

target-snowflake  | INFO Failed to import optional packages, pyarrow

Although, Pyarrow is installed in the meltano venv, I'm not sure why it has this issue.

What's the complete command you ran the first time?

meltano elt tap-postgres-tipping_receivetiptransaction target-snowflake

both times

You may need the

--job_id

option: https://www.meltano.com/docs/command-line-interface.html#elt

Oh, yeah, I see it says

The --job_id option identifies related EL(T) runs when storing and looking up incremental replication state.

... so maybe it doesn't save the state without it? Good find... I have it running via the UI right now, but I'll try this option when I get a chance

I have not. Are you running meltano in docker? I think it starts the UI in readonly mode when you run in docker

I answered the Explore question here: https://meltano.slack.com/archives/C01TCRBBJD7/p1619212038438200?thread_ts=1619205903.437200&cid=C01TCRBBJD7

Thanks, it seems like it would be better if there was a default job_id so that it 'just works'... but either way the docs should clarify this