Hello folks, I am looking for some assistance with issues I have encountered while migrating our production meltano deployment from version

v2.3.0

to

v3.5.4

using the meltano docker images: • meltano/meltano:v3.5.4-python3.8 • meltano:v2.3.0-python3.8 Two issues that I noticed that resulted us in having to rollback the migration: 1. certain taps encountered the following error:

{"error": "Error: Setting value set via multiple environment variables:

The strange thing is I tested some of the taps that were throwing this error when using the production postgres state db, and they work locally when testing against a clean state local postgres db (i ran the extract with the old veresion and then ran again with the new version and this error does not appear locally). 2. No state was found for any of my meltano runs after migrating to the newer version. After the version upgrade was deployed all the meltano runs did a full refresh because it could not find a state for the run. Has anyone encountered these issues when migrating these issues from version 2 to 3?

Sounds related to https://github.com/meltano/meltano/issues/8787 TL;DR - try aligning `name`/`namespace` for your plugins, or use

meltano run

over

meltano [ el | elt ]

.

@Reuben (Matatika) Thank you for your response.

Regarding the state, do you know what state IDs exist currently?

Copy code

meltano state list

It might be that Meltano is now prepending the environment name to the generated state ID, i.e.

<environment>:<tap>-to-<target>

vs

<tap>-to-<target>

. Just a guess

I ran the suggested command

meltano state list

and I only see the expected state ids I explicitly ran with the

elt

command using the

--state-id

flag.

Ah OK, that makes sense.

meltano run

does not have an explicit

--state-id

option, so if you were unable to run

meltano [ el | elt ]

as per your first issue and tried

meltano run

instead, I don't think that would pick up state in the same way.

Hmmm if I use

run

that would also run a full refresh correct? I want to avoid running a full refresh when migrating to version 3.

Given there is no matching state ID based on the tap, target and state ID suffix, yes. https://docs.meltano.com/reference/command-line-interface#run You could experiment with copying your existing state with

meltano state copy <old> <tap>-to-<target>

to test incremental sync with

run

.

I will give copying state ids a try. Per meltano best practices is it preferred to use

run

or

el

?

I don't think there is a preferred approach; it depends on your use-case. If you are only ever running a tap to a target, probably

el

for simplicity (providing it works of course) - otherwise

run

(e.g. with

dbt

or other utilities, or potentially more than one tap/target pair).

Ok so I think I actually was getting the correct state id used for the taps I was ingesting from. But it does seem after upgrading from v2 to v3 the event log for the

Found state from..

message is not being logged anymore on v3. Is this expected. Here is an example log message I see when using v2:

{
  "event": "Found state from 2025-05-28 02:12:53.046742.",
  "level": "info",
  "timestamp": "2025-05-28T02:21:47.063538Z"
}

https://github.com/search?q=repo%3Ameltano%2Fmeltano+%22Found+state+from%22+language%3APython&type=code No code matches, but past issues reference similar logs so it was probably removed.

Thank you @Reuben (Matatika)

That log message was removed when state backends where introduced in v2.7.0: https://github.com/meltano/meltano/pull/6742/files#diff-a79aeb37ddc539a9bc2dee4d8cb764fb89c371a012e47a7e1686bf6a0e38d1df