Hello everyone, I'm having issue with my environment, it was running fine and then this morning I did a deploy that rebuild the meltano image and now we're getting this error:

code=1 message=ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject name=meltano

We're using meltano 2.15 and python 3.9.19. The image is run on ECS task through airflow. The changes I've made were to a .yml so I don't know why the rebuild is causing this. Anyone else faced a similar problem? I'm using tap-postgres, tap-mysql and meltano SDKs, all of them were failing.

Do you know if this is coming from Meltano or from one of your plugins? The underlying cause is surely the same as https://github.com/MeltanoLabs/target-snowflake/issues/206, so depending on where the numpy dependency is being pulled, it would require you to add a

numpy<2

constraint. See also https://github.com/numpy/numpy/issues/26710 for the issue reported by an Airflow maintainer.

We had an issue with NumPy 2.0.0 as an unpinned sub-dependency this morning. Nice to see it's making the rounds! FWIW we fixed by pinning the offending plugin to a known working version, as Edgar has said.

We're using poetry for one of our pipelines built using the meltano SDK that has this line in the poetry.lock file:

parquet = ["numpy (>=1.22)", "numpy (>=1.22,<1.25)", "pyarrow (>=13)"]

AFAIK

poetry.lock

isn't respected when `pip install`ing via a git URL.

This seems to be the issue, our pipelinewise-target-snowflakeis is from Transferwise and everytime meltano creates a new venv for a tap/target is setting our numpy to version 2.0.0

Same issue with me today and the

tap-bigquery

plugin. I can’t figure out it’s possible to tell meltano install to pin numpy<2.0.0 or if I need to fork the plugins and pin in myself by modifying the config.py. There’s got to be a better way to deal this though….

You can pin in the `pip_url`:

pip_url: git+<https://github.com/z3z1ma/target-bigquery.git|https://github.com/z3z1ma/target-bigquery.git> numpy<2

There is also this feature proposal which would prevent these kinds of issues from happening: https://github.com/meltano/meltano/issues/6416

Thanks Reuben! I didn’t know about that syntax for

pip_url

, you just saved my day!

Thanks Reuben! It solved my issue as well