paul_tiplady
04/15/2022, 7:37 PMtarget-bigquery
ignores state key deletions, and merges `Add`/`Change` operations (https://github.com/adswerve/target-bigquery/blob/master/target_bigquery/state.py#L33). It seems that it’s expecting STATE messages to contain partial updates that need to be merged into the target’s state. Is this assumption correct? I had thought (based on reading the spec) that you’d always get the full state in a STATE
message and so you just need to persist the latest object you receive.
The target does seem to have a configurable mode that just uses the latest state, which I think solves my problem. But I’ve spent many tens of hours debugging this issue and so I want to help fix the target to behave correctly if it really is non-conformant.paul_tiplady
04/15/2022, 8:45 PM