Hi All, I just restarted my journey with meltano a...
# troubleshootingm
marcin_wojciechowski
09/25/2023, 3:50 PMHi All, I just restarted my journey with meltano and snowflake and I got the following error when I try to move some data with datetime columns. Any help would be appreciated:
NotImplementedError: Default TypeEngine.as_generic() heuristic method was unsuccessful for target_snowflake.snowflake_types.TIMESTAMP_NTZ. A custom as_generic() method must be implemented for this type class.
marcin_wojciechowski
09/26/2023, 12:34 PMAny suggestion on this? here's my config:
Copy code
version: 1
default_environment: dev
project_id: ************************
environments:
- name: dev
- name: staging
- name: prod
plugins:
extractors:
- name: tap-mysql
variant: transferwise
pip_url: pipelinewise-tap-mysql
config:
database: techlog
host: localhost
user: loader
metadata:
'*':
replication-method: FULL_TABLE
select:
- "techlog-defect.*"
loaders:
- name: target-snowflake
variant: meltanolabs
pip_url: meltanolabs-target-snowflake
config:
account: *********.eu-west-2.aws
database: DWH_SALESDEMO
schema: RAW_TECHLOG
user: MELTANO
warehouse: SALESDEMO_ELT
role: DATALOADER
default_target_schema: RAW_TECHLOG
file_format: DWH_SALESDEMO.RAW_TECHLOG.MELTANO_JSON
- name: target-jsonl
variant: andyh1203
pip_url: target-jsonlmarcin_wojciechowski
09/27/2023, 8:02 AMDid anyone succeeded in sending data from mysql to snowflake recently?
j
joshua_janicas
09/27/2023, 3:12 PMI know that I've had success a few weeks ago using target-meltano (meltano variant), but I was using MSSQL as a tap
m
marcin_wojciechowski
09/27/2023, 4:08 PMYeah, I just found the issue. I had a column with some date and time values but it was of TEXT type in MySQL. Looks like tap-snowflake detected that it's a datetime in text and struggled. Once I changed the MySQL type to DATETIME it all started to work again :)