I’m planning to merge two jobs together. I’m running tap-Mysql (with LOG-BASED replication) to target-postgres). I have job_ids:

job_a and job_b

. I’m going to select the

job_b

tables in

job_a

from now on. Can I just alter the

job.payload

row from

job_a

to include the most recent

job.payload

data from

job_b

? Would that trick Meltano into thinking that the jobs are the same job for the next run?

@matt_cooley That should work, but you can also use

--dump=state

on

meltano schedule run <name>

or

meltano elt ... --job_id=<name>

to dump the state for both pipelines to JSON files, which you can then manually combine/merge, after which you can pass the new state into a new pipeline using

--state=state.json

on

meltano elt

or

meltano schedule run

. That state will be used in that pipeline, instead of whatever was in the DB, and at the end of the run, the new combined + updated state will be stored on the job row in the DB for any future runs.

Ok cool. It sounds like you already built into the cli the exact thing that I’m trying to do. Might be a little tough with our dev setup, but I’ll definitely try using your method. Thanks!