I always (two times I've tried) have such a hard t...
# troubleshooting
v
I always (two times I've tried) have such a hard time getting locally developed taps/targets to run locally on meltano.
Copy code
loaders:
 82   - name: target-mssql
 83     namespace: target_mssql
 84     pip_url: ~/git/autoidm-target-mssql/target-mssql.sh
Which is wrong but the error message is not helpful
meltano install loader target-mssql
(states that
/git/autoidm-target-mssql/target-mssql.sh
doesn't exist. (issue 1) Following the sdk cookie cutter here https://gitlab.com/meltano/singer-sdk/-/blob/main/cookiecutter/tap-template/%7B%7Bcookiecutter.library_name%7D%7D/meltano.yml I switched my loader to use executable
Copy code
loaders:
 82   - name: target-mssql
 83     namespace: target_mssql
 84     executable: ~/git/autoidm-target-mssql/target-mssql.sh
Which now does something odd
Copy code
(.venv) visch@visch-ubuntu:~/git/oracle2mssql/oracle2mssql$ meltano install loader target-mssql
Installing 1 plugins...
Installing loader 'target-mssql'...
Installing loader 'target-mssql'...
Installs it twice? (issue 2) Trying to install with git, repo as it was when I had this error (I've changed it since) https://gitlab.com/autoidm/autoidm-target-mssql/-/tree/1929c69b1e905bc4c39f6d80ff98000ec3f9ba6f https://gitlab.com/autoidm/autoidm-target-mssql
Copy code
91   loaders:
 92   - name: target-mssql
 93     namespace: target_mssql
 94     pip_url: git+<https://gitlab.com/autoidm/autoidm-target-mssql>
Meltano install target-mssql works for this case but my singer_sdk folder at the top level of the git repo doesn't come over. I'm sure I'm not following some convention or something. (Issue 3) , but this is easily fixed by moving my singer_sdk folder and changing some imports. With BambooHR I went down the path of installing it as a custom-plugin https://meltano.com/docs/plugin-management.html#custom-plugins . This is confusing, I know I can make it work but it took a while, and thought I'd share the experience. I'm sure there's a better way to do this but it's not clear what path to take. Singer-SDK recommends install meltano in the same directory. https://gitlab.com/DouweM/target-lunch-money has a good example as well. My brain goes through this logic 1. Develop tap / target, do testing, etc. 2. When I want to test with data beyond test data I go to Meltano 3. I want to use my meltano repo that already has some extractors and targets written up (different directory). I want to pull in my target / tap I've been developing into that repo as I would any other source / target 4. Hack around until I make it work the way I want. I'll end up just going with the singer-sdk / target-lunch-money method in the long term. But my natural tendency isn't to install meltano in the same directory, or copy my tap/target into my meltano directory manually.
a
@visch - Thanks for this detailed feedback. This is an area we are working to improve. Just this morning we released an updated tutorial which gives more direct guidance on using custom SDK taps with Meltano: Meltano Tutorial - Create a Custom Extractor There’s still a lot more to do here, but wanted to let you know we hear you and are actively working on this. Thanks!
v
oooo 🙂 awesome I"ll take a look. Just wanted to share, glad you are on it!
Following the tutorial I have my target running through meltano properly! Thanks
a
Awesome! Glad to hear it! 👍