Is there a way with the SDK to define a kind of stream relationship that can generate records from chaining multiple requests? The API I'm dealing with has an endpoint that returns a list of entities, each with an

id

. There is a separate endpoint to get a set of entity features for a single entity by

id

, or multiple (paged) entities by query parameter

ids

as a comma-separated list. I want to merge the entity and entity features schemas, which means that I need to make an additional request to get a page of entity features for each page of entities. A parent-child stream doesn't feel optimal here given that it would be setup to fire off an entity features request for each entity, which forces a back-off implementation to deal with rate-limiting caused by improper use of the API. I want to leverage existing SDK features in order keep functionality like request authentication and error handling, which becomes much more of a pain to set up when manually invoking

requests

. I ended up instantiating a "secondary" stream inside my "primary" stream and calling

request_records

, but was wondering if there was a better way to do this. Hope I explained that well enough! 😅 https://github.com/Matatika/tap-spotify/blob/4d064c83dc862c9531c4a3ef6046352906f4d22b/tap_spotify/streams.py#L41-L76