https://linen.dev logo
Join Slack
Powered by
Hi, I'm having trouble upgrading from `v2` to `v3`...
# troubleshooting
d
Hi, I'm having trouble upgrading from 
v2
to 
v3
. My 
meltano.yml
looks like:
Copy code
version: 1
plugins:
  extractors:
  - name: tap-postgres
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-postgres.git>
  - name: tap-postgres-transferwise
    inherit_from: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
In v2 everything works fine:
Copy code
$ poetry add meltano@2.20.0 --extras psycopg2
$ poetry run meltano install

Installing 2 plugins...
Skipped installing extractor 'tap-postgres-transferwise'...
Installing extractor 'tap-postgres'...
Installed extractor 'tap-postgres'
Installed 1/2 plugins
Skipped installing 1/2 plugins
In v3 I get an error:
Copy code
$ poetry add meltano@3.0.0 --extras psycopg2
$ poetry run meltano install

Extractor 'tap-postgres' is not known to Meltano. Try running `meltano lock --update --all` to ensure your plugins are up to date.
(running the meltano lock command does not help)
This looked correct based on https://docs.meltano.com/guide/plugin-management/#multiple-variants, but maybe those docs are out-of-date?
It works if I switch to:
Copy code
plugins:
  extractors:
  - name: tap-postgres--meltanolabs
    inherit_from: tap-postgres
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-postgres.git>
  - name: tap-postgres--transferwise
    inherit_from: tap-postgres
    variant: transferwise
    pip_url: pipelinewise-tap-postgres
e
Hey @dean_morin! They might be, and since 
discovery.yml
was removed in 3.0, Meltano needs the plugin to be added to the project before in can be inherited. I think we need to revisit https://github.com/meltano/meltano/issues/6731, come up with and document guidelines both new and migrating users. Let me get a repro project set up to see if this can be worked around with some set of manual steps.
d
Oh I see, I've been manually editing the yml file, I'll try using the meltano cli and adding one at a time
Nm, seems like it's going to be more complicated than that, thanks for taking a look!
For context, I'm planning to switch from the 
transferwise
to 
meltanolabs
variant. I have a lot of postgres taps, and I want to start with some of the less important ones (so that we can see what differences there are and how we want to deal with them). The idea was to have one 
tap-postgres
that everything inherits from so that that I can set global secrets (e.g. the meltano metadata db uri) via a single environment variable that all of the other taps will inherit from.
v
Dean double posted 😕 https://meltano.slack.com/archives/C068YB1BMD5/p1725410727155789 I answered him here as well, there is a bug
Edgar linked to all the right stuff, my issue just adds a nice step by step workaround for your case
Open in Slack
PreviousNext
Powered by