Hey folks. We have an SDK tap with a `poetry.lock`...
# singer-tap-development
j
Hey folks. We have an SDK tap with a
poetry.lock
file locking the
singer-sdk
dependency to 0.3.8, and that lock file is committed to the repo. However, when we install the tap in our Meltano project (using a git repo link, if that matters), it ignores the lock file and uses 0.3.11. This happened to cause a breaking change. That’s not to say that there’s a breaking change in the sdk; we overrode
request_records
for a
RESTStream
, at our own risk. But I would have thought the lock file would have protected us from this. Is this a known issue, does
pip install
not know how to correctly install poetry apps? Or is there a setting we can add to our project files to fix this?
cc @ryan_bell
v
your pyproject.toml is also set to 0.3.8 right?
j
to
^0.3.7
. But I assume if poetry works similar to ruby bundler and npm, that the
^
only applies when doing
poetry update
e
pip only uses the
pyproject.toml
and ignores
poetry.lock
to resolve dependencies, so it installs any minor release equal or greater than
0.3.7
I guess it's the difference between a library and an application. At the moment, singer taps are treated like libraries and that's probably not the right thing to do. I hope that makes sense 😅
v
@edgar_ramirez_mondragon awesome! can you find the docs for this I was trying to find this to link to but you beat met o the answer 😄
https://pip.pypa.io/en/stable/reference/build-system/ is the closest I can find. Basically no reference to lockfiles at all because when you run
pip install
it doesn't reference lockfiles
e
So @julian_knight, in the case of the SDK, it's probably safest to pin the exact dependency in
pyproject.toml
, e.g.
singer-sdk = "0.3.8"
so pip always uses that version 😕 There's a few discussions going on in both the python and poetry communities around this topic: • Standardize around a lock file format: PEP 665 • Support building packages with pinned dependencies: https://github.com/python-poetry/poetry-core/pull/71
j
Thanks @edgar_ramirez_mondragon and @visch, that makes sense!
@edgar_ramirez_mondragon given that, should the SDK pin a specific version of itself in the cookie cutter output of
pyproject.toml
?
e
@julian_knight I think it should. I'm filing an issue as we chat 😅