Does anyone have pointers on good implementations ...
# getting-starteda
avy_faingezicht
10/17/2023, 1:46 AMDoes anyone have pointers on good implementations of custom state management for incremental replication in the SDK? I am writing a tap on top of a clunky SQL interface that I can’t use the SQL Alchemy connection for, so I need to shoehorn in the replication key myself
avy_faingezicht
10/17/2023, 2:04 AMI see here that users are not expected to write any state management code. How does a tap apply the state variables that are written out by the default implementation, though?
m
mark_johnston
10/17/2023, 2:42 AMI had a go at this with my variant of tap-db2- it's very much a "work-in-progress" feature branch, but may help as I recall I needed to override some SDK functions to make incremental replication work properly: https://github.com/mjsqu/tap-db2/blob/86e3d6744df554828071a918eaaab61c655065bc/tap_db2/client.py#L190
a
avy_faingezicht
10/17/2023, 3:58 AMoh cool, so you just grab it from
get_starting_replication_key_valueavy_faingezicht
10/17/2023, 3:58 AMThank you for the pointer @mark_johnston!
2 Views