Hi, I’m trying to set up replication from Cloud SQL MySQL to BigQuery and I’m having an issue with tap-mysql not using state. The replication works fine except it always starts from scratch due to not picking up on the state the previous run generated. It does generate the state well when a run completes, but on the next run I always see the following two lines:

2025-03-20T13:59:43.766320Z [debug    ] Could not find state.json in /projects/.meltano/extractors/tap-mysql/state.json, skipping.

2025-03-20T13:59:43.793158Z [warning  ] No state was found, complete import.

and then every table says it requires a full resync. Even when I manually copy the state to the location in the first log line, it doesn’t pick it up. Any ideas?

The replication works fine except it always starts from scratch due to not picking up on the state the previous run generated. It does generate the state well when a run completes, but on the next run I always see the following two lines

What's the command you used? If you used

meltano elt

, you might need to pass

--state-id

.

I get a config error complaining about missing ssh_tunnel settings even when I have tunneling disabled.

I've updated the configuration schema, can you try again?

I’ve been using

meltano el

but it also happened when using

meltano run

. Passing

--state-id

did the trick, though! Thank you! Is there a way to not have to fetch the latest state and pass it? As for the config stuff, is the meltanolabs variant recommended over transferwise? If so, I think I have follow-up config questions.

Is there a way to not have to fetch the latest state and pass it?

You can use a custom state ID.

As for the config stuff, is the meltanolabs variant recommended over transferwise? If so, I think I have follow-up config questions.

I wouldn't say so unfortunately and I'd recommend sticking to transferwise, since I haven't had the time, nor do we use it internally, to be able to dedicate sufficient resources. I fixed the config stuff so you could at least get to use it further and report any issues.

This is great info, thanks! So basically if I always pass the same state_id to the el command, it will write and read from that state, yes? Meanwhile, this behavior is automatic for dbt run? Is there a recommended variant for target-bigquery?