https://linen.dev logo
Join Slack
Powered by
Weird scenario I am coming across: I ran into a p...
# troubleshooting
j
Weird scenario I am coming across: I ran into a problem yesterday going from tap-mssql => target-jsonl (https://meltano.slack.com/archives/C01TCRBBJD7/p1687459077893099). Dan Norman was fantastic and now my problem is resolved. I reinstalled the plugin, but then if I try to execute 
meltano run tap-mssql-admin target-jsonl
(-admin inherits from tap-mssql) the old problem comes back and fails. I rename the plugin in the YAML to 
tap-mssql-admin2
, reran the line, and it proceeds to work. I then tried the following: 
meltano remove extractor tap-mssql-admin
(Restart Powershell) 
meltano add extractor tap-mssql-admin --inherit-from tap-mssql
(Configure select to look like this again)
Copy code
- name: tap-mssql-admin
    inherit_from: tap-mssql
    select:
    - Admin-*.*
The error continues to persist, but then again if I change the plugin name to be anything else then execute based on the new plugin name it works. I can't determine if this is a Powershell problem or something that I need to also clear from Meltano's side?
The only references to tap-mssql-admin outside the YAML when i look in files is a 
meltano.db
file, though I am unsure if it has anything to do with what I am seeing. You can see the other attempts I've made changing the plugin name (and having success from there) Out of curisoity, I removed the .DB file and tried re-running the same command. Meltano recreated the DB, but the Issue still persists unless I change the plugin name
My assumption is something to do with Windows at this point >_<
v
Lots of words here but I don't see actual Logs. Show us commands you ran, meltano.yml, and logs you're getting for failures
Realized my comment could be taken wrong, I mean this in the niceest way possible just if you want an answer that's what we'll need as right now all we can do is be a consultant for you, we're (community questions) better at answering specific issues on errors you're having
j
Hi Derek, no offence taken. Let me see what I can show
So this was the original YAML file
Copy code
version: 1
default_environment: dev
project_id: dad99788-9e54-4f81-8c1e-4f58c600a14d
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-mssql
    variant: buzzcutnorman
    pip_url: git+<https://github.com/BuzzCutNorman/tap-mssql.git>
    config:
      dialect: mssql
      driver_type: pyodbc
      host: 127.0.0.1,4433
      port: 1433
      user: abc
      password: xyz
      database: db
      sqlalchemy_eng_params:
        fast_executemany: 'True'
      sqlalchemy_url_query:
        driver: ODBC Driver 17 for SQL Server
        TrustServerCertificate: yes
  - name: tap-mssql-admin
    inherit_from: tap-mssql
    select:
    - Admin-*.*  
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
Running 
meltano run tap-mssql-admin target-jsonl
gave me a set of errors that was related to the Slack link I provided in the root of this thread. It has already been resolved and I reinstalled the package with 
meltano install extractor tap-mssql --clean
.
You can see the logs below where the error starts:
Copy code
2023-06-23T17:07:20.982910Z [info     ]                                cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2023-06-23T17:07:20.983446Z [info     ] Failed validating 'type' in schema['properties']['IsCancelled']: cmd_type=elb consumer=True name=target-jsonl producer=False stdio=stderr string_id=target-jsonl
2023-06-23 log.txt
Which replicated what I encountered yesterday. When I changed the name of the inherited tap to 
tap-mssql-admin2
, the extract works via 
meltano run tap-mssql-admin2 target-jsonl
Copy code
- name: tap-mssql-admin2
    inherit_from: tap-mssql
    select:
    - Admin-*.*  
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
The logs show that the block run is compelted and I see all the JSONL files under output
Copy code
2023-06-23T17:15:12.804770Z [info     ] 2023-06-23 13:15:12,631 | INFO     | tap-mssql            | Skipping deselected stream 'dbo-PostDeployment'. cmd_type=elb consumer=False name=tap-mssql-admin2 producer=True stdio=stderr string_id=tap-mssql-admin2
2023-06-23T17:15:12.805766Z [info     ] 2023-06-23 13:15:12,631 | INFO     | tap-mssql            | Skipping deselected stream 'dbo-PreDeployment'. cmd_type=elb consumer=False name=tap-mssql-admin2 producer=True stdio=stderr string_id=tap-mssql-admin2
2023-06-23T17:15:12.805766Z [info     ] 2023-06-23 13:15:12,631 | INFO     | tap-mssql            | Skipping deselected stream 'dbo-SuccessfulDeploymentLog'. cmd_type=elb consumer=False name=tap-mssql-admin2 producer=True stdio=stderr string_id=tap-mssql-admin2
2023-06-23T17:15:12.806767Z [info     ] 2023-06-23 13:15:12,631 | INFO     | tap-mssql            | Skipping deselected stream 'dbo-__RefactorLog'. cmd_type=elb consumer=False name=tap-mssql-admin2 producer=True stdio=stderr string_id=tap-mssql-admin2
2023-06-23T17:15:15.018507Z [info     ] Incremental state has been updated at 2023-06-23 17:15:15.018507.
2023-06-23T17:15:15.032509Z [info     ] Block run completed.           block_type=ExtractLoadBlocks err=None set_number=0 success=True
I tried the following actions as well 
meltano remove extractor tap-mssql-admin
(Restart Powershell) 
meltano add extractor tap-mssql-admin --inherit-from tap-mssql
(Configure select to look like the YAML) And the problem still persists.
It seems to me that somewhere or something is caching the old tap-mssql package I was running yesterday and its being run just when i have the name of the tap 
tap-mssql-admin
, any other name or configuration like 
tap-mssql-test
with the same select details works just fine.
v
I can't tell what problem you are currently having :( , what is your current issue you are having right now? What's the log output?
j
The current issue is that when I try to use the alias (for lack of a better word) of 
tap-mssql-admin
from that YAML file, it seems to be using an old set of code and breaking when trying to convert a boolean=>string. That was fixed and updated since with a clean install. I removed the inherited tap then added it back, but its still running into the same problem. When I change the name from 
tap-mssql-admin
to 
tap-mssql-admin2 (or anything else really)
everything works.
v
try 
meltano install --clean
j
If there's a different place where I can be looking for logs I can certainly provide!
v
Meltano install will upgrade existing pip packages if they are already installed so it's possible that's what's going on 🤷
Other way to be sure you're starting from a "fresh" slate is to remove 
.meltano/
if you still get the error after that you know it's not a cache issue
j
Ha, that's wild. I ran 
meltano install --clean
then 
meltano run tap-mssql-admin target-jsonl
, same problem. I then delete 
.meltano/
do the 
meltano install --clean
, then rerun 
meltano run tap-mssql-admin target-jsonl
and now it works
Thanks Derek, removing .meltano was the answer here
v
meltano install --clean
does the same thing so one working and not the other doesn't make sense, but I'm happy that you're working that's what matters most!
j
🪄 Magic 🪄 ?
v
It's never magic lol but happy melty time 🙂
j
I know 😉 . Thanks for helping
Open in Slack
PreviousNext
Powered by