Hey everyone wave I m using target snowflake to load data fr Meltano #troubleshooting

Hey everyone :wave: I'm using target-snowflake to ...

bassel

08/07/2023, 1:36 PM

Hey everyone 👋 I'm using target-snowflake to load data from mysql using tap-mysql to snowflake, it was running without any issue .. today I was running some tests (against the same data, nothing has been changed, same pipeline configs, same snowflake account) .. and now having this issue

IndexError: list index out of range cmd_type=loader name=target-snowflake

Just wondering if the recent changes to snowflake's repo maybe caused it thinkspin More detailed logs:

Copy code

2023-08-05T16:36:59.379356Z [info] self.connector.prepare_table() cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.379822Z [info] File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/connectors/sql.py", line 753, in prepare_table cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.380289Z [info] self.create_empty_table() cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.380753Z [info] File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/singer_sdk/connectors/sql.py", line 689, in create_empty_table cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.381227Z [info] self.to_sql_type(property_jsonschema), cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.381697Z [info] File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/target_snowflake/connector.py", line 264, in to_sql_type cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.382265Z [info] num_scale = SnowflakeConnector._get_numeric_scale(jsonschema_type) cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.382788Z [info] File "/project/.meltano/loaders/target-snowflake/venv/lib/python3.9/site-packages/target_snowflake/connector.py", line 242, in _get_numeric_scale cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.383249Z [info] precision = max(precision, len(str(jsonschema_type[attrib]).split(".")[1])) cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.383718Z [info] IndexError: list index out of range cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.427859Z [info] 2023-08-05 16:36:59,427 | INFO | snowflake.connector.connection | closed cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.428509Z [info] 2023-08-05 16:36:59,427 | INFO | snowflake.connector.connection | No async queries seem to be running, deleting session cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr
2023-08-05T16:36:59.452900Z [info] 2023-08-05 16:36:59,452 | INFO | snowflake.connector.connection | closed cmd_type=loader name=target-snowflake run_id=730b5b38-ebc7-463c-a4f6-be70a50b4ba1 state_id=mysql-to-snowflake stdio=stderr

edgar_ramirez_mondragon

08/07/2023, 1:39 PM

Well this does look like a bug in the target, in this line: https://github.com/MeltanoLabs/target-snowflake/blob/045b614c19422e8ff2210207bb40a414f3c13e95/target_snowflake/connector.py#L242 cc @pat_nadolny

bassel

08/07/2023, 1:42 PM

Never had it before (same data schema/values - never changed in the source 🤔 ) can I use in meltano's yaml file a specific/previous or stable version of target-snowflake? @edgar_ramirez_mondragon

user

08/07/2023, 1:49 PM

@bassel I'll check out that bug and get back to you. Yeah check out these docs for pinning your versions, its usually a good idea to pin all your plugins for stability

bassel

08/07/2023, 2:02 PM

@edgar_ramirez_mondragon @pat_nadolny Can you help / guide me in defining a stable version (it was working two ~ three weeks ago). Thanks a lot again.

edgar_ramirez_mondragon

08/07/2023, 2:08 PM

Sure @bassel, what does the

pip_url

of the plugin in

meltano.yml

look like at the moment?

bassel

08/07/2023, 2:09 PM

Copy code

- name: target-snowflake
    variant: meltanolabs
    pip_url: meltanolabs-target-snowflake

edgar_ramirez_mondragon

08/07/2023, 2:10 PM

Ok, you can pin it to `v0.4.2`:

Copy code

- name: target-snowflake
    variant: meltanolabs
    pip_url: meltanolabs-target-snowflake==0.4.2

bassel

08/07/2023, 2:10 PM

amazing! will try it now Thanks

bassel

08/07/2023, 4:04 PM

@edgar_ramirez_mondragon @pat_nadolny It worked .. thanks for the help gratitude thank you

Open in Slack

Previous Next