I'm trying to create a child parent stream for the tap that I'm working on, Client Tether. The API returns the data in one large record formatted like this.

{
	data:[
		{
			"child_context": "...",
			"extra_stuff": "...",
			...
		}
		{
			"child_context": "...",
			"extra_stuff": "...",
			...
		}
	]
}

or the way it appears in the out.jsonl file

{"type":"RECORD","stream":"endpoint","record":{"data":[{"child_context":"id1","extra_stuff":"value"}{"child_context":"id2","extra_stuff":"value"}]}}

The issue I have is that the tap only notices and runs the first child_context but I need it to run for each one. Can anyone help me out?

Hey @chauncy wilson, have you taken a look at https://sdk.meltano.com/en/v0.37.0/classes/singer_sdk.Stream.html#singer_sdk.Stream.generate_child_contexts?

Yes, I have. I am using the generate child context method but it is only grabbing the first key available. This API just returns everything in one record instead of multiple records

Ok, so what does your implementation look like?

def get_child_context(self, record: dict, context: dict | None) -> dict | None:
        data_list = record['data']  # Ensure data is a list (empty list if not)
        for item in data_list :
            if isinstance(item, dict) and item.get("client_id"):  # Check if item is a dictionary and has "client_id"
                return {"client_id": item.get("client_id")}

note: I can't move the json record path forward in the client.py file because the different streams have different formatting returned

Oh, that's

get_child_context

, not

generate_child_contexts

I didn't realize that, thanks!

Awesome!