Join Slack
Powered by
Question: If I have a child stream that uses a par...
# singer-tap-development
e
elliot
06/23/2022, 12:01 PM
Question: If I have a child stream that uses a parameter (lets say
id
) in its query, is it possible to instead just provide a list of `id`’s to look at? Context: I have 2000+ `id`’s, but I only really want to get 10-20.
e
edgar_ramirez_mondragon
06/23/2022, 6:53 PM
Hi
@elliot
. I think what your need might be covered by the
partitions property
. Here's an example: 1. You declare the list of IDs as a tap setting:
https://github.com/MeltanoLabs/tap-stackexchange/blob/927669ca8f86db24a67b0f2986693bf3014e8d71/tap_stackexchange/tap.py#L38-L42
2. You override partitions to use the configured list:
https://github.com/MeltanoLabs/tap-stackexchange/blob/07a43004e2595ee97ea69e5b7f8b7f4f9a9d4ad6/tap_stackexchange/client.py#L138-L145
3. You get the tag directly from the context:
https://github.com/MeltanoLabs/tap-stackexchange/blob/07a43004e2595ee97ea69e5b7f8b7f4f9a9d4ad6/tap_stackexchange/streams.py#L118
OR You write your API path as a template and the ID will be replaced automatically:
https://github.com/MeltanoLabs/tap-stackexchange/blob/07a43004e2595ee97ea69e5b7f8b7f4f9a9d4ad6/tap_stackexchange/streams.py#L229
Open in Slack
Previous
Next