got a weird issue when trying to use stateful stream partitioning. Partitioned stream works just fine when run from scratch, but when trying to run using a state file, I’m getting the following error a few partitions in:

ValueError: State file contains duplicate entries for partition: {state_partition_context}.
Matching state values were: [{'context': {'mid': 14709184, 'start': '2022-04-04', 'end': '2022-04-26'}, 'replication_key': 'transaction_time', 'replication_key_value': '2022-04-26T23:25:55+00:00'}, {'context': {'mid': 14709184, 'start': '2022-04-04', 'end': '2022-04-26'}, 'starting_replication_value': '2022-01-01', 'progress_markers': {'Note': 'Progress is not resumable if interrupted.', 'replication_key': 'transaction_time', 'replication_key_value': '2022-04-03T22:43:22+00:00'}}]

code in 🧵