hey y’all :wave: have a question around some funky...
# troubleshooting
d
hey y’all 👋 have a question around some funky retry behavior I’m seeing from the
_request_with_backoff
method on the
RESTClient
🧵
I think I was under the impression from looking at this line that the backoff decorator on this method was there to handle backoff-able HTTP status codes (
429
etc.). I’ve noticed that a few of our taps that leverage this stream class are getting rate limited and immediately throwing a
RuntimeError
instead of retrying at all. Upon further inspection of the method, it seems like the only path to the backoff getting triggered is via
Timeout
,
ConnectionError
, and
TooManyRedirects
exceptions, since the method isn’t explicitly raising
HTTPError
anywhere based on the status code of the response.
am I missing an implementation detail here? Or is this method more of an abstract-style method that is meant to be overridden if you want to handle retryable errors like rate limiting
Oh wait - I think I just initially misread the
giveup
logic. It looks like it’s just giving up on anything client end. That makes more sense
so then it seems pretty clear that this method is intentionally not trying to handle things like rate limiting
f
@david_wallace a little late but just confirming what ya found. Right now it just covers client side/network issues. We’ve chatted about extending this to deal with rate limit scenarios though, and theres https://gitlab.com/meltano/sdk/-/issues/140 for that.