michael_cooper
05/20/2021, 10:11 PMreplication_key
, and this is partially in regards to Singer spec in general as well. Is a stream's replication_key
and key within an individual record that is used a means of bookmarking, or is a key you use to bookmark where a stream left off?
For example we have a hypothetical endpoint GET /api/orders?created_after='2020-01-13'
with a response of
{
"orders": [
{
"order_id": 1,
"customer_id": 2
},
{
"order_id": 2,
"customer_id": 55
}
]
}
Now we want to track where we left off since the last sync by tracking the last time we successfully queried and not have to due a full sync of all orders all the time. Since there is no key within an individual record, does that mean there is no technical replication_key
for this stream? Or is replication_key
arbitrary and just a way to bookmark where this individual stream left off?
For the SDK section, does the SDK utilize a stream's replication_key
in any functional way, or does it only use it for metadata?