Hello! Had a question on what is recommended behavior when using backoff. I’m currently working with

tap-mixpanel

which has backoff configured with

max_tries=7,factor=3

. However, I’m hitting an error because the Mixpanel API has a limit of 60 queries per hour so when the rate limit is hit, it backs off for 10-30sec and then gives up. Since I know the limit would be for an entire hour is it recommended to fail like this, or should I reconfigure backoff to wait longer? If I should reconfigure, what should I change? cc: @aaronsteers what are your thoughts?

if you're lucky there's a RateLimit header https://github.com/AutoIDM/tap-clickup/blob/main/tap_clickup/client.py#L80

@visch The docs say the following:

If you exceed the rate limit, a 429 error will be returned.

However, I also see that if there is a 500 error the tap uses backoff. This is probably an issue 🙃 https://github.com/hotgluexyz/tap-mixpanel/blob/master/tap_mixpanel/client.py#L149

ha... sometimes apis do their own things 😕 That's nutty, I wonder if it's the API limit issue or if there is something up with the request? Could of course be a bug on mixpanel's side too. I've seen Google's api's 500 randomly, you're just supposed to retry them

Yeah, but it retries a few times and then hits the

max_tries

within a few mins