Hi Team, I am trying to use Meltano for our projects so that it can be 1 tap to fit all SAAS tools with the same authentication type, I see one of the ZOOM old Tap has a similar approach but it is on Singer, I know we have Meltano SDK now which is more robust. I want to understand a couple of concepts. • Is using Catalog.json and loading the catalog a better approach then defining Stream Classes for Each Stream. • Or do we have a way to use Stream Classes and achieve the Same, I am able to do it when all the endpoints are at level 1 and not child endpoints. I am stuck on 2 things ◦ How do we pass the Key or composite Key for the child endpoint ◦ How can I batch these child calls in spite of 1 parent at a time processing, like executing parallel. Thanks for the Time.

• Is using Catalog.json and loading the catalog a better approach then defining Stream Classes for Each Stream.

Not necessarily, and they are not mutually exclusive approaches. As a developer you could define stream classes for each stream, and as a user of the tap still use

--catalog catalog.json

to perform schema and metadata overrides. Though for your use case it seems like you need a generic stream class that is instantiated as many times as necessary given the entries in the catalog. Similar to what SQL taps do: https://github.com/meltano/sdk/blob/101ffc80720762c075455062ca4ea2ae76a680be/singer_sdk/tap_base.py#L686-L699

• How can I batch these child calls in spite of 1 parent at a time processing, like executing parallel.

That's a tougher implementation detail. The sdk doesn't have any builtin helpers for parallel stream processing at the moment.

@Edgar Ramírez (Arch.dev) - Thanks for the information, Is parallel stream in the pipeline for the future, do you have any example from the community that implemented this. If not , I will like to implement it but I am new to this can you point me to where to start.

See: • https://github.com/meltano/sdk/issues/183 • https://github.com/meltano/sdk/issues/2121 • https://github.com/meltano/sdk/pull/2295