I am finding `poetry run tap-xyz --discover --conf...
# singer-tap-development
a
I am finding
poetry run tap-xyz --discover --config config.json
is giving different results than
meltano invoke tap-xyz --discover --config config.json
. Any tips on how I can troubleshoot this?
something must be cached somewhere. By deleting the
.meltano
directory and running
meltano install
again, I am able to see the updates
Am I doing something wrong?
d
@adam_roderick Could the source code for tap-xyz inside
.meltano/extractors/tap-xyz/venv
have been outdated?
The only caching in
meltano invoke
is https://gitlab.com/meltano/meltano/-/issues/2627, which affects the discovered catalog when running the tap in sync mode, but shouldn’t make a difference when you explicitly run with
--discover
a
It probably was outdated source code in the
.meltano
directory. I'm trying to understand the development flow for tap development--should I delete the
.meltano
directory and rerun
.meltano install
anytime I want to test code changes through the meltano CLI?
d
If you’ve pointed
pip_url
directly at the dir, yes, because
pip install <pip_url>
will install the package inside
.meltano
. If you’ve installed in editable/dev mode (
pip_url: -e path/to/package
), it will be symlinked instead of copied over, so changes should be picked up automatically. That won’t work with Poetry-based projects though. When using the SDK to develop a new tap/target, we recommend omitting
pip_url
and pointing
executable
directly at the executable wherever you’re developing the tap, so that it will always use the latest version as well
@aaronsteers I can’t find the docs on that executable trick right now, can you help me out?
a
Perfect, thanks @douwe_maan
a
Hi, @adam_roderick. The trigger for Meltano to not cache is a null pip_url or a pip_url that starts with "-e". As Douwe mentioned, for SDK projects we recommend a null pip_url and setting the executable to the
tap-mysource.sh
script created by the cookie cutter. The cookie cutter should also contain a sample meltano.yml file which demonstrates this.
a
thank you!