one thing i’m trying to do is create taps that follow a

parentstream

,

childstream

logic: think

table

then

table/1/columns

,

table/1/users

,

table/1/tags

. Is there a suggested way of tackling this? It seems like this is what

partition

is going for?

Hi, @stephen_bailey! Yes, that's correct. While we don't have a ton of examples (yet!), our Gitlab example in the 'samples' directory does implement partitions as a way of creating separate substreams per each git project or group. This was necessary because, as looks like is your case also, we can only pull those child entities on a per-parent basis, with REST endpoints such as

project/{project_id}/issues

. To get this functionality, you can override

partitions

in the stream class. On our development branch, you'll find a new partitions page with some new docs that are set to be merged in the 0.1.2 release on Monday.

Wow, this is a great implementation. the gitlab example really helped. really simplified the tap logic, well done!

Awesome - thanks @stephen_bailey for the feedback! If you have further suggestions, please do let us know. And once you get your tap up and running, we’d love to add it to our list of working samples, to help out others who might want to create similar functionality: docs/code_samples.md · development · meltano / Singer SDK