Hi, Is it possible to increase the time between requests? When I run my tap I'm getting after a while a

singer_sdk.exceptions.FatalAPIError

with code 400 and I think it is because the API is getting too many requests. I have been looking around in settings and environment variables but I couldn't find anything to increase time between requests.

Hi @marcos_palacios! I’ve had some success using the ratelimit package to limit/delay requests from the client side by overriding request_decorator and validate_response: • https://github.com/MeltanoLabs/tap-stackexchange/blob/bcd5b666fea2346597d6641b05c8924a5d206fe7/tap_stackexchange/client.py#L65 • https://github.com/MeltanoLabs/tap-stackexchange/blob/bcd5b666fea2346597d6641b05c8924a5d206fe7/tap_stackexchange/client.py#L77-L86 However, that package hasn’t fallen out of maintenance so I’m now trying out PyrateLimiter and seems to work similarly with a few changes: https://github.com/MeltanoLabs/tap-stackexchange/pull/106

In order to use the RESTStreams backoff and retry logic, you could add

400

as a non-fatal error code into your tap's

extra_retry_statuses

list .... https://sdk.meltano.com/en/latest/classes/singer_sdk.RESTStream.html#singer_sdk.RESTStream.extra_retry_statuses

Ah, that’s another option. Thanks @christoph!

Thanks @edgar_ramirez_mondragon and @christoph. I'll look into that right away!