A question about parent-child streams: if a parent...
# singer-tap-development
s
A question about parent-child streams: if a parent stream is incremental, will the child stream only return records that were updated in the parent on each run? Or, do I have to explicitly set the child stream to be incremental replication as well?
e
Hi Steven! If the child stream should support incremental replication, you have to declare its replication key regardless of whether the parent stream is incremental or not. The only case that's slightly dangerous is when the parent replication key is not updated whenever a child item is updated. The child stream should be then forced to perform a full-table sync, even if in theory it supports incremental replication because you can't count on the parent being synced when it's needed. You declare this case by setting
ignore_parent_replication_key = true
on the child stream.
s
gotcha. What if the parent stream supports incremental but the child stream does not? For example the parent endpoint supports filtering by some date whereas the child endpoint does not, is it possible for the child stream to also be incremental in this case? I just want the child stream to return results that are incremental in the parent stream
the parent stream would pass an
id
of all updated/incremental records to the child stream in this case and the child stream would only grab those ids
e
is it possible for the child stream to also be incremental in this case? I just want the child stream to return results that are incremental in the parent stream
Yes, the child stream would behave almost incrementally in that case. You'd still get all the child records for each updated parent, but you won't get any child records for non-updated parents.
the parent stream would pass an
id
of all updated/incremental records to the child stream in this case and the child stream would only grab those ids (edited)
is that the parent or the child record ID?
s
is that the parent or the child record ID?
they share the same id in this case
e
I see, so let me know if I understand: The first endpoint returns a list of IDs and can be filtered by date. The second endpoint uses each of the IDs to retrieve details for that record. You pass the list of IDs via the child context dictionary, so only updated records are retrieved. That should work, AFAICT.