Hi all, what happens if we have a tap / target combination that is currently running in our infrastructure, and then another job attempts to run for that same tap / target (exact same

meltano

command)? Both commands are sharing the same backend database. Would the second job fail to start because there is an existing pipeline running?

Depends on the tap and target honestly. Generally they'd both just run and work fine

Yeah, if you're using the same

meltano run

command, you'll indeed see an error with a message like:

Another '...' pipeline is already running

If that's not the desired behavior and you want to run multiple iterations of the pipeline, perhaps with different configurations, you should consider using the

--state-id-suffix

option to differentiate the two runs, or the

--force

to ignore the restriction and proceed with the pipeline: https://docs.meltano.com/reference/command-line-interface/#parameters-3