https://linen.dev logo
Join Slack
Powered by
whats the preferred way to install two separate va...
# troubleshooting
d
whats the preferred way to install two separate variants of a plugin within the same meltano project?
looking to install two different variants of 
target-snowflake
I naively tried the following which works but you seem to lose all of the nice features such as settings discovery etc.:
Copy code
loaders:
  - name: target-snowflake
    variant: meltano
  - name: target-snowflake-pipelinewise
    pip_url: pipelinewise-target-snowflake==2.2.0
    namespace: target-snowflake-pipelinewise
    executable: target-snowflake
t
honestly I don’t know if this is a use case we’ve ever looked at. an older issue around aliasing comes to mind https://github.com/meltano/meltano/issues/3146 though
@edgar_ramirez_mondragon I feel like you have the quickest knowledge on something like this 😄
@david_wallace do you need the 
namespace
on the pipelinewise variant? That’s what generally indicates it’s a custom extractor
seems like having the name + pip_url + executable should work
d
let me see - I think I had a collision before
yeah removing namespace I get the error:
Copy code
#0 3.991 Loader 'target-snowflake-pipelinewise' is not known to Meltano
e
I think adding a lockfile manually might work. One sec…
t
interestingly I can get them both to install when they’re named the same, but I assume I’ll have a problem when I invoke
e
`meltano.yml`:
Copy code
loaders:
  - name: target-snowflake
    variant: meltano
  - name: target-snowflake-pipelinewise
    variant: pipelinewise
    pip_url: pipelinewise-target-snowflake==2.2.0
plugins/extractors/target-snowflake-pipelinewise--pipelinewise.lock
: <copy from https://hub.meltano.com/meltano/api/v1/plugins/loaders/target-snowflake--transferwise> and change the 
name
.
d
wow
ok trying this now
e
(you might need to set the 
executable
too)
t
This seems like a more niche use case, but I’m curious if there’s some UX that would make this easier? Woudl you open an issue @david_wallace when you get a chance?
d
weird - I still get:
Copy code
Loader 'target-snowflake-pipelinewise' is not known to Meltano
semi-related question: can environments inherit from other environments?
i think I can get where I need if thats possible
t
environments cannot inherit from others
d
booooo!
t
you absolute madman
😛
you want another layer of inheritance??
my first thought on this is that it seems like you should be using a separate git branch for the other variant, but that’s without knowing anything about your use case
d
well the issue is that we wanna run them side-by-side in production
e
I can confirm the lockfile approach works (tested on 2.5.0) 🤔
d
okay maybe i messed up
e
did you change the 
"name"
prop in the lockfile to 
target-snowflake-pipelinewise
?
a
semi-related question: can environments inherit from other environments?
@david_wallace - We've thought about this but apparently it wasn't yet logged as a feature request. Please feel free to add comments and upvote. 👍 https://github.com/meltano/meltano/issues/6730
Regarding the support of installing two variants of the same plugin, this definitely should be doable - albeit with some manual edits to the lock files and plugin definitions.
@david_wallace - Compiling the notes and suggestions from others in this thread, can you let us know if this approach works?: https://github.com/meltano/meltano/issues/6731#issuecomment-1239944284
5 Views
Open in Slack
PreviousNext
Powered by