Morning at least for me wink I wanted to use target mysql in Meltano #troubleshooting

Morning! (at least for me :wink: ) I wanted to use...

marcin_wojciechowski

10/27/2023, 6:58 AM

Morning! (at least for me 😉 ) I wanted to use target-mysql in my project, however I'm receiving the following error when running the pipeline: ModuleNotFoundError: No module named 'MySQLdb' Has anyone encountered this same? Any quick fix?

Andy Carter

10/27/2023, 8:02 AM

Can you share the meltano command and a bit more of the stack trace?

marcin_wojciechowski

10/27/2023, 9:26 AM

So, my meltano file is the following:

Copy code

version: 1
default_environment: dev
project_id: 51325c1e-947a-451f-9e9a-0b44460adfe4
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-csv
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-csv.git>
    config:
      files:
      - entity: data
        path: ./data.csv
        keys:
        - ReportId
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
  - name: target-mysql
    variant: thkwag
    pip_url: thk-target-mysql
    config:
      database: SNGenerated
      default_target_schema: SNGen
      host: localhost
      user: loader
      port: '3306'

I run the meltano command:

meltano run tap-csv target-mysql

and I get a very long error. I think that the useful stack is:

marcin_wojciechowski

10/27/2023, 9:30 AM

Copy code

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.714775Z [info     ]   File "<string>", line 2, in create_engine cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.714775Z [info     ]   File "C:\Projects\meltano-test\.meltano\loaders\target-mysql\venv\Lib\site-packages\sqlalchemy\util\deprecations.py", line 375, in warned cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.715775Z [info     ]     return fn(*args, **kwargs) cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.715775Z [info     ]            ^^^^^^^^^^^^^^^^^^^ cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.716775Z [info     ]   File "C:\Projects\meltano-test\.meltano\loaders\target-mysql\venv\Lib\site-packages\sqlalchemy\engine\create.py", line 544, in create_engine cmd_type=elb consumer=True name=target-mysql 
producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.716775Z [info     ]     dbapi = dialect_cls.dbapi(**dbapi_args) cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.716775Z [info     ]             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.717775Z [info     ]   File "C:\Projects\meltano-test\.meltano\loaders\target-mysql\venv\Lib\site-packages\sqlalchemy\dialects\mysql\mysqldb.py", line 150, in dbapi cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.717775Z [info     ]     return __import__("MySQLdb") cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.717775Z [info     ]            ^^^^^^^^^^^^^^^^^^^^^ cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.718775Z [info     ] ModuleNotFoundError: No module named 'MySQLdb' cmd_type=elb consumer=True name=target-mysql producer=False stdio=stderr string_id=target-mysql
2023-10-27T09:27:51.781480Z [debug    ] Deleted configuration at C:\Projects\meltano-test\.meltano\run\tap-csv\tap.94ba3018-5f71-4c9a-b9f3-b904effa23c5.config.json
2023-10-27T09:27:51.782480Z [debug    ] Deleted configuration at C:\Projects\meltano-test\.meltano\run\target-mysql\target.76088e81-fac6-49c3-bee0-3ec47116e46b.config.json
2023-10-27T09:27:51.787480Z [error    ] [WinError 109] The pipe has been ended

marcin_wojciechowski

10/27/2023, 9:31 AM

btw.

meltano run tap-csv target-jsonl

works fine

Andy Carter

10/27/2023, 12:05 PM

Could you add

mysqlclient

to your pip_url line in the target and then reinstall your target? https://github.com/thkwag/target-mysql/issues/1

marcin_wojciechowski

10/27/2023, 1:48 PM

Hi, What I did is I downgraded my Python from v11 to v10 and it worked fine there

Andy Carter

10/27/2023, 2:39 PM

Might be worth adding that to the repo as an issue and your resolution

Open in Slack

Previous Next