Can a Child stream be partitioned?

It'd be great to have this baked into the SDK. It's probably a better way to think about Partitions, it's a superset of context or something along those lines instead of them being seperate

I tried blindly copying the approach from your clickup tap, but can't make it work for my custom tap. Where I'm stuck now is the "automagic" interpolation of the RESTStream's

path

property. It's no longer doing that after introducing the

_sync_children()

override. I was using that functionality (i.e

path = '/api/{parent_id}/subcomponent'

with the

parent_id

coming automatically from the

self.get_child_context() -> dict

But now the child stream is just making a request to

/api/{parent_id}/subcomponent

literally.

My guess is context doesnt' have

partition_id

for some reason. Maybe add a breakpoint at the setter for

path

might be easier to do it at

get_records

🤷 I'd at least that's where I'd look https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/streams/rest.py#L275 is the magic 🙂

Ok. Fixed the

path

interpolation. That was just me not paying attention to the subtleties of the

partitions = []

vs

partitions = None

semantics 😉 Now on to figuring out why the merging of the context is seemingly not happening ...

may the force be with you

Ha! Turned out that all the weirdness I was experiencing was because the source's REST API response behaviour has quietly been updated in the meantime as I was developing against it.

Ahhh did it end up working for you then!?

Nope. Lol. Parked it for later.

@robby_robinson https://github.com/meltano/sdk/issues/271 up vote this 😄