I’ve got a messy issue with an annoying API. The quickbase API allows me to filter records by a date, but not by a date-time. So even if I ask it to give me all records updated since 2022-08-31T080000, wit will give me all records updated since the beginning of 2022-08-31. This was mostly working for small tables when I was not explicitly setting

is_sorted=True

in my Stream class. I would get a few dupes in stream, but that would just cause a do-nothing update in Snowflake and everything was fine. However, without

is_sorted=True

, my tap couldn’t recover from errors, which is a problem for larger tables. So I set that flag in the Stream class and now I have what seems like a new problem where Meltano sees records that come before the last replication key value (which is the full date-time) and interprets it as out-of-sort-order and raises an error. Anyone have an elegant solution to work around this? I was thinking about trying to create a new column that was just the updated at date, without the time, but it’s not clear to me if that will work or just cause other issues.