luis_vallejo
06/26/2023, 1:01 PMfirst_col, second_col
value1, value2
And tries to transfer it to a target-snowflake, getting this error:
2023-06-20T15:29:16.986808Z [info ] snowflake.connector.errors.ProgrammingError: 000904 (42000): SQL compilation error: error line 1 at position 194 cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-20T15:29:16.987293Z [info ] invalid identifier 'SECOND_COL' cmd_type=elb consumer=True name=target-snowflake producer=False stdio=stderr string_id=target-snowflake
2023-06-20T15:29:17.253802Z [error ] Loader failed
2023-06-20T15:29:17.261494Z [error ] Block run completed.
It happens with both variants, meltano and transferwise.
Any help is welcome.user
06/26/2023, 4:27 PMmeltano invoke tap-csv > output.json
. Did the table exist in snowflake prior to this error and can you validate that the table structure is correct? You can also try removing .meltano/run/tap-csv/
in case theres a cached schema thats messing with your syncluis_vallejo
06/26/2023, 5:28 PMcreate or replace TABLE TESTDB.RAW.TEST (
FIRST_COL VARCHAR(16777216) NOT NULL,
" SECOND_COL" VARCHAR(16777216),
_SDC_RECEIVED_AT TIMESTAMP_NTZ(9),
_SDC_EXTRACTED_AT TIMESTAMP_NTZ(9),
_SDC_BATCHED_AT TIMESTAMP_NTZ(9),
_SDC_DELETED_AT TIMESTAMP_NTZ(9),
_SDC_SEQUENCE NUMBER(38,0),
_SDC_TABLE_VERSION NUMBER(38,0),
primary key (FIRST_COL)
);
The problem seems to be caused by the space in the header of the csv (first_col, second_col).
So removing the space, .meltano/run/tap-csv/ and the table on Snowflake, did the trick.
All is working now, thanks!!user
06/26/2023, 6:00 PM