hello, i am having an issue with ingestion grindin...
# troubleshooting
m
hello, i am having an issue with ingestion grinding to glacial pace. i am using postgres tap and snowflake target both meltanolabs.
Copy code
December 05, 2024 at 14:21 (UTC-8:00)
{"consumer": true, "producer": false, "string_id": "target-snowflake", "cmd_type": "elb", "stdio": "stderr", "name": "target-snowflake", "event": "2024-12-05 22:21:40,649 | INFO | snowflake.connector.cursor | Number of results in first chunk: 1", "level": "info", "timestamp": "2024-12-05T22:21:40.650088Z"}
this kinda happened before but it wasn't an emergency as it is now; when i look at the state
Copy code
target state {\"bookmarks\": {\"public-attributes\": {\"starting_replication_value\": null}
i see that is doesn't have a
starting_replication_value
which makes me think it's starting from the beginning again? either way why is this writing only 1 value at a time to snowflake and why?
so i created a separate state-suffix-id for this flow and it has updated the
starting_replication_value
and in the partial state i see that it has been updating the
replication_key_value
that has been updated but it's been more that 8hours at this point for 25M rows