So I’m trying to develop a tap that has endpoints that needs information pulled from other endpoints. What is the “SingerSDK” way of going about that? For clarity, say I have an

ItemStream

and a

MerchantStream

. The

ItemStream

needs a

merchant_id

for each merchant but I don’t have a full list of

merchant_ids

until I run the

MerchantStream

.

Hi @michael_cooper! There is an issue for defining the API/spec around that type of complementary/subsequent requests: https://gitlab.com/meltano/sdk/-/issues/93. I'm leaning towards at least a

_make_request(self, method, ...)

member that devs can use (but should not need to override) and some other helper for chaining requests.

hacky solution with parent streams

MerchantStream

has a parent of

ItemStream

MerchantMerchantStream

has a parent of

MerchantStream

Well that’s unfortunate. That issue was opened 5 months ago, and this is an incredibly common issue I run into.

you could probably do some magic in prepare_request and override it yourself

Parent-Child streams may also help with this

I’m already overriding the

prepare_request()

due to needing to use XML.

Hi, @michael_cooper. Just want to clarify one point: does the API support (and are you wanting to design as) a parent-child relationship? (Specifically, I'm thinking, you would likely get the merchants stream first, and then for each merchant, pull their items.)

So you can pull the

merchant_id

from the context dict in

prepare_request_payload

or

prepare_request

So how does the

get_child_context()

work in regards to the child streams? If I make the call to

/api/merchants

and get a list of

merchant

records, does the child get a list of `merchant_id`s or does it get just one

merchant_id

with multiple instances of the child stream?

get_child_context()

gets called for every record in the parent stream, so if you output a record for each merchant in the response of

/api/merchants

and use said record to populate the child context with

{"merchant_id": record["merchant_id"]}

then the child stream will be synced for each context, so once for every

merchant_id