I have an extraction task that normally extracts tens of tables using LOG_BASED replication strategy from tap-mysql to target-postgres. When troubleshooting, is there a way I can run the extraction of just a single table? Similarly if I needed to is there an easy way to trigger a full table sync (for just one table)?

You can use a meltano environment for this scenario! https://docs.meltano.com/concepts/environments create an environment called “testing” and override specific settings/config there and run it with

meltano --environment=testing run tap target

Specifically you’re suggesting changing the “select” fields to change what tables are being operated on? Will that do sane things to my state file?

it handles it appropriately - the environment name is prefixed to the state_id when you use meltano run https://docs.meltano.com/concepts/environments#state-id-suffix

But then when you revert back to the original pipeline that's set up for LOG_BASED it's going to try to resume from where it left off, no?

correct. I was assuming that testing would happen into a separate schema - but perhaps that was a bad assumption.

Hrm. Interesting. I thought the intent was to basically reload a single table, or update just one out of a larger group, in which case separate state data would be a problem. But maybe I misinterpreted... it happens 😉

that said, you can use

meltano state merge

to update the state id for the prod environment

@thomas_briggs yes that is what I would like to be able to do. I’ll take a look at how meltano state merge works

@mark_gordon This thread, among others, has some useful commentary.