https://linen.dev logo
Join Slack
Powered by
tap-mysql and tap-snowflake - not seeing any error...
# troubleshooting
a
tap-mysql and tap-snowflake - not seeing any errors at the same time not seeing any tables getting created and data population. Please advise
Copy code
version: 1
default_environment: dev
project_id: d3b480d0-865e-464f-9c5b-3bf5a208590b
plugins:
  extractors:
  - name: tap-mysql
    variant: transferwise
    pip_url: pipelinewise-tap-mysql
  loaders:
  - name: target-snowflake
    variant: transferwise
    pip_url: pipelinewise-target-snowflake
environments:
- name: dev
  config:
    plugins:
      extractors:
      - name: tap-mysql
        config:
          host: 127.0.0.1
          port: 3303
          user: root
          database: hr
        select:
        - employees.employee_id
        - employees.first_name
        - employees.last_name
        - employees.email
        - employees.phone_number
        - employees.job_id
        - employees.hire_date
        - locations.*
        metadata:
          employees:
            replication-method: INCREMENTAL
            replication-key: employee_id
          locations:
            replication-method: INCREMENTAL
            replication-key: location_id
      loaders:
      - name: target-snowflake
        config:
          account: cloverdev
          user: arjuntest
          dbname: arjuntest
          warehouse: ARJUNTEST_XSMALL
          file_format: arjuntest.public.meltano_local_test
          default_target_schema: PUBLIC
- name: staging
- name: prod
p
@arjunkumar_krishnankutty check out this thread https://meltano.slack.com/archives/C01TCRBBJD7/p1657115223238799 you might be seeing a similar issue related to a bug thats in the process of being fixed
You can also try running 
meltano invoke tap-mysql
to make sure records are being extracted to pin point the issue. And here are more troubleshooting tips in the docs https://docs.meltano.com/guide/integration#debug-mode
a
thank you for pointing it out, let me check
I am not using tap-mysql2 and I am not seeing any of those exceptions. When invoking, I am seeing:
Copy code
$ meltano invoke tap-mysql
2022-07-07T00:12:31.680027Z [info     ] Environment 'dev' is active
time=2022-07-06 17:12:32 name=tap_mysql level=INFO message=Server Parameters: version: 5.7.38-log, wait_timeout: 28800, innodb_lock_wait_timeout: 3600, max_allowed_packet: 4194304, interactive_timeout: 28800
time=2022-07-06 17:12:32 name=tap_mysql level=INFO message=Server SSL Parameters(blank means SSL is not active): [ssl_version: ], [ssl_cipher: ]
{"type": "STATE", "value": {"currently_syncing": null}}
I am seeing this stalled status for
Copy code
$ meltano invoke target-snowflake
2022-07-07T00:13:18.242175Z [info     ] Environment 'dev' is active
time=2022-07-06 17:13:20 name=target_snowflake level=INFO message=Getting catalog objects from table cache...
Copy code
2022-07-07T02:29:05.439119Z [debug    ] {"type": "STATE", "value": {"currently_syncing": null}} cmd_type=extractor name=tap-mysql (out) run_id=f00bf006-cf39-42f8-99f9-f21522effb1e state_id=2022-07-07T022904--tap-mysql--target-snowflake stdio=stdout
2022-07-07T02:29:06.500522Z [info     ] time=2022-07-06 19:29:06 name=target_snowflake level=INFO message=Getting catalog objects from table cache... cmd_type=loader name=target-snowflake run_id=f00bf006-cf39-42f8-99f9-f21522effb1e state_id=2022-07-07T022904--tap-mysql--target-snowflake stdio=stderr
2022-07-07T02:29:09.208863Z [info     ] time=2022-07-06 19:29:09 name=target_snowflake level=INFO message=Emitting state {"currently_syncing": null} cmd_type=loader name=target-snowflake run_id=f00bf006-cf39-42f8-99f9-f21522effb1e state_id=2022-07-07T022904--tap-mysql--target-snowflake stdio=stderr
2022-07-07T02:29:09.209130Z [debug    ] {"currently_syncing": null}    cmd_type=loader name=target-snowflake (out) run_id=f00bf006-cf39-42f8-99f9-f21522effb1e state_id=2022-07-07T022904--tap-mysql--target-snowflake stdio=stdout
p
I am not using tap-mysql2 and I am not seeing any of those exceptions.
tap-mysql2? It looks like youre using the pipelinewise-tap-mysql variant of tap-mysql, same as that other thread but it can definitely be a different issue youre running into. 
{"currently_syncing": null}
makes it look like its not finding any data, if you run 
meltano select tap-mysql --list
can you make sure youre entities are selected properly?
6 Views
Open in Slack
PreviousNext
Powered by