https://linen.dev logo
#plugins-general
Title
# plugins-general
j

josh_lloyd

02/08/2023, 3:53 AM
I’m running a tap-mysql to target-snowflake in an ECS container with Dagster as the orchestrator. This is part of an attempt to update from meltano 1.x to meltano 2.14. The our stage environment seems to be running everything just fine but in our prod environment whenever I run any
meltano elt
the process will hang and then after a minute or two the process will get killed by the system with a message like
Copy code
/tmp/tmp5aexhp0z: line 1:   275 Killed                  meltano elt tap-mysql--foo target-snowflake --state-id=foo
The last thing logged by
--log-level=debug
before hanging and getting this message is:
Copy code
Creating engine '<meltano.core.project.Project object at 0x7fb04caa5f10>@postgresql://<redacted postgres connection string>?options=-csearch_path%3Dmeltano'
I’m gathering from contextual clues that ECS/docker is killing the process because the memory maxes out — I see the spike in memory in cloudwatch. I’m going to drop all the meltano tables to encourage it to rebuild them, but I wanted to post this here for posterity’s sake because I don’t think I saw anything like this mentioned anywhere (especially not in the migration guide)