How can I make `meltano.yml` refer my custom melt...
# troubleshooting
s
How can I make
meltano.yml
refer my custom meltano sdk code ? I have cloned it and made some changes to some default properties but I am unable to link my local installation to it 😞
when installing locally as
pip install sdk/
im getting
Copy code
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
meltano 3.0.0 requires python-dotenv<2.0.0,>=1.0.0, but you have python-dotenv 0.21.1 which is incompatible.
awscli 1.29.22 requires botocore==1.31.22, but you have botocore 1.29.161 which is incompatible.
aws-shell 0.2.2 requires prompt-toolkit<1.1.0,>=1.0.0, but you have prompt-toolkit 3.0.36 which is incompatible.
sqlalchemy-redshift 0.8.14 requires SQLAlchemy<2.0.0,>=0.9.2, but you have sqlalchemy 2.0.21 which is incompatible.
dbt-semantic-interfaces 0.2.0 requires jsonschema~=3.0, but you have jsonschema 4.19.1 which is incompatible.
meltanolabs-target-postgres 0.0.8.post1+47ec811 requires singer-sdk<0.32,>=0.28, but you have singer-sdk 0.32.0 which is incompatible.
meltanolabs-target-postgres 0.0.8.post1+47ec811 requires sqlalchemy<2, but you have sqlalchemy 2.0.21 which is incompatible.
v
Can you share your meltano.yml with what you've tried, it's definitely doable
s
@visch I am just referencing my tap and target in
meltano.yml
. I did not find any setting to set custom SDK path 😕
r
@silverbullet1 You can reference a plugin you are developing locally by setting
Copy code
pip_url: -e path/to/plugin
in your
meltano.yml
. You might also have to
Copy code
meltano install <type> <name>
after setting this.
When you create a new SDK plugin project, it creates a
meltano.yml
for you that references the subject plugin for this exact purpose. To test the plugin during development, you should be able to run
Copy code
meltano invoke <name>
from your project directory (provided you have a
meltano
executable available - see `pipx` installation).
s
@Reuben (Matatika) sorry for not being clear. I meant that i am making changes in core tap and base class itself in the Meltano SDK. And i want to use that custom sdk code in my custom plugin. I am already using custom plugins.
r
In that case, it's just a matter or adding your local clone of the SDK as an editable dependency of the plugin:
Copy code
poetry add -e path/to/sdk
Again, you might also have to
Copy code
meltano install <type> <name>
after adding this to test with Meltano. You could also try
Copy code
pip_url: -e path/to/plugin -e /path/to/sdk
, but no idea if that works.
s
pip_url: -e path/to/plugin -e /path/to/sdk
actually worked! thank you so much 🙂