Is it possible to create parent-child streams that...
# singer-tap-development
h
Is it possible to create parent-child streams that share data for a given run? Suppose I wanted to query an endpoint for some data but wanted to split that data into two different outputs, in a non-trivial way that involves aggregation, is this what parent-child streams are for?
a
In theory, you can send any context from parents to children in the "get_child_context()" method, or by returning a tuple from get_records(), with the second item of the tuple being the additional context. (In that case though, you do want to make sure you override state_partitioning_context on the child stream so it doesn't use the entire context as a bookmark key. 🙂 ) Others may be able to add their own experiences here.
d
I’ve tried different approaches as described there: https://meltano.slack.com/archives/C01PKLU5D1R/p1677107164599079 Ended up with parent-child streams utilizing `get_child_context`/`get_records` and
state_partitioning_keys = []