visch
04/23/2021, 1:45 PMloaders:
82 - name: target-mssql
83 namespace: target_mssql
84 pip_url: ~/git/autoidm-target-mssql/target-mssql.sh
Which is wrong but the error message is not helpful meltano install loader target-mssql (states that /git/autoidm-target-mssql/target-mssql.sh doesn't exist. (issue 1)
Following the sdk cookie cutter here https://gitlab.com/meltano/singer-sdk/-/blob/main/cookiecutter/tap-template/%7B%7Bcookiecutter.library_name%7D%7D/meltano.yml I switched my loader to use executable
loaders:
82 - name: target-mssql
83 namespace: target_mssql
84 executable: ~/git/autoidm-target-mssql/target-mssql.sh
Which now does something odd
(.venv) visch@visch-ubuntu:~/git/oracle2mssql/oracle2mssql$ meltano install loader target-mssql
Installing 1 plugins...
Installing loader 'target-mssql'...
Installing loader 'target-mssql'...
Installs it twice? (issue 2)
Trying to install with git, repo as it was when I had this error (I've changed it since) https://gitlab.com/autoidm/autoidm-target-mssql/-/tree/1929c69b1e905bc4c39f6d80ff98000ec3f9ba6f https://gitlab.com/autoidm/autoidm-target-mssql
91 loaders:
92 - name: target-mssql
93 namespace: target_mssql
94 pip_url: git+<https://gitlab.com/autoidm/autoidm-target-mssql>
Meltano install target-mssql works for this case but my singer_sdk folder at the top level of the git repo doesn't come over. I'm sure I'm not following some convention or something. (Issue 3) , but this is easily fixed by moving my singer_sdk folder and changing some imports.
With BambooHR I went down the path of installing it as a custom-plugin https://meltano.com/docs/plugin-management.html#custom-plugins .
This is confusing, I know I can make it work but it took a while, and thought I'd share the experience. I'm sure there's a better way to do this but it's not clear what path to take. Singer-SDK recommends install meltano in the same directory. https://gitlab.com/DouweM/target-lunch-money has a good example as well.
My brain goes through this logic
1. Develop tap / target, do testing, etc.
2. When I want to test with data beyond test data I go to Meltano
3. I want to use my meltano repo that already has some extractors and targets written up (different directory). I want to pull in my target / tap I've been developing into that repo as I would any other source / target
4. Hack around until I make it work the way I want.
I'll end up just going with the singer-sdk / target-lunch-money method in the long term. But my natural tendency isn't to install meltano in the same directory, or copy my tap/target into my meltano directory manually.aaronsteers
04/23/2021, 2:30 PMvisch
04/23/2021, 2:31 PMvisch
04/26/2021, 5:17 PMaaronsteers
04/26/2021, 5:54 PM