https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
m

michael_cooper

05/20/2021, 10:11 PM
Question regarding a stream's
replication_key
, and this is partially in regards to Singer spec in general as well. Is a stream's
replication_key
and key within an individual record that is used a means of bookmarking, or is a key you use to bookmark where a stream left off? For example we have a hypothetical endpoint
GET /api/orders?created_after='2020-01-13'
with a response of
Copy code
{
  "orders": [
    {
      "order_id": 1,
      "customer_id": 2
    },
    {
      "order_id": 2,
      "customer_id": 55
    }
  ]
}
Now we want to track where we left off since the last sync by tracking the last time we successfully queried and not have to due a full sync of all orders all the time. Since there is no key within an individual record, does that mean there is no technical
replication_key
for this stream? Or is
replication_key
arbitrary and just a way to bookmark where this individual stream left off? For the SDK section, does the SDK utilize a stream's
replication_key
in any functional way, or does it only use it for metadata?