Hey folks, diving deep into the Meltano SDK to bui...
# singer-tap-development
h
Hey folks, diving deep into the Meltano SDK to build a REST based Tap. If I have an API that is parameterised by a path parameter (e.g. app_id) is this a good candidate for using stream partitioning? It's not quite clear to me where it would be useful.
To put a bit more flesh on that - the app ID is a value you set to get data for a specific ID from this API. At first I thought I'd just have it as a config that I set at run time.
e
Hi @hawkar_mahmod! Yeah, using partitions would allow users of your tap to configure one or more app ids and the state for each app would be safely partitioned
h
Thanks @edgar_ramirez_mondragon! That makes sense. Is the idea that a partition is a implementation detail of the data source but we'd like to combine the data downstream. Is that the reason for their existence?
e
Exactly, the loader is not aware of the partitions and it sees a single stream