Join Slack
Powered by
Does any one have a suggestion or best practice on...
# singer-tap-development
a
albert_m
04/15/2022, 3:48 PM
Does any one have a suggestion or best practice on api that has a rate-limiting? I think 500 hits per hour per endpoint
s
Stéphane Burwash
04/15/2022, 7:52 PM
I'm not sure if this is what you're looking for, but we use something called exponential backoff to automatically regulate how many calls we make:
https://bpaulino.com/entries/retrying-api-calls-with-exponential-backoff
e
edgar_ramirez_mondragon
04/16/2022, 1:58 AM
There are some implementations in the wild, like
https://github.com/MeltanoLabs/tap-stackexchange/blob/0bde20d6fd0d09fcdc7d87ef2f10d66c2b5da39d/tap_stackexchange/client.py#L29-L62
(disclaimer: it's my implementation 😅)
a
albert_m
04/16/2022, 4:39 AM
Thank you for sharing this. 😆
Open in Slack
Previous
Next