Meltano

If I have a endpoint /employees which accepts a parameter department and I have these list of departments coming in either from meltano config file or as a env variable(all departments as comma separated), how would I make the tap to get all departments mentioned in the config file and use them in  employees stream?

For now, I used post_process to filter out records that aren't in the list of departments. But, want to know if we can do multiple API calls with the param. Filtering out seems to be a bad thing since I'll be extracting too many records and then filtering. Direct API call with params will definitely make it faster

hi <@U06CEEDV9V2>. You could use <https://sdk.meltano.com/en/latest/classes/singer_sdk.Stream.html#singer_sdk.Stream.partitions|partitions> (see also <https://sdk.meltano.com/en/latest/partitioning.html|stream partitioning>) and the context argument of <https://sdk.meltano.com/en/latest/classes/singer_sdk.RESTStream.html#singer_sdk.RESTStream.get_url_params|get_url_params> to get those values. So, something like:

```class MyStream(RESTStream):
    @property
    def partitions(self):
        return [{"department": department} for department in self.config["departments"]]

    def get_url_params(self, context, next_page_token):
        # <https://api.example.com/employees?department=>&lt;dept. name&gt;
        return {
            "department": context["department"],
        }```
We don't have an example of this in the `partitions` API docs or in the collection of <https://sdk.meltano.com/en/latest/code_samples.html|code samples>, so if this works for you it'd be great to add your use case as an example :slightly_smiling_face:

Sure. Thank you. I'll try it out and add if it works

I was wondering about how to use partitions to achieve exactly this scenario just yesterday! Impeccable timing! I'll also give this one a try.

You can also see a real world example of this in `tap-github` where we go through a list of repositories passed in config, for instance