Hi, I am trying to create a tap that has a restrictive request limitation. The response is a 429 code with a "Retry-After" header containing the cool-off period before retrying. I found an example here but cannot figure out how to implement these examples. Any guidance would be appreciated!

hi @robby_robinson. You might need to leverage RESTStream.backoff_runtime with a function that returns the wait time from the response header (via the RetriableAPIError exception). Something like

def get_wait_time_from_response(exception):
    return exception.response.headers.get("Retry-After", 0)


class MyStream(RESTStream):
    extra_retry_statuses = [429]

    def backoff_wait_generator(self):
        return self.backoff_runtime(get_wait_time_from_response)

Hey Edgar, I think that I am having trouble following how the exception gets passed into the function. This is what I am getting when using the code:

extra_retry_statuses = [429]

def backoff_wait_generator(self):
	def get_wait_time_from_response(exception):
		print(exception)
		return 1

	return self.backoff_runtime(get_wait_time_from_response)

TypeError: backoff_runtime() takes 1 positional argument but 2 were given And setting the value in the args:

extra_retry_statuses = [429]

def backoff_wait_generator(self):
	def get_wait_time_from_response(exception):
		print(exception)
		return 1

	return self.backoff_runtime(value=get_wait_time_from_response)

File "..../virtualenvs/tap-adobe-umapi-QIkGfAcN-py3.8/lib/python3.8/site-packages/backoff/_common.py", line 43, in _next_wait seconds = value + jitter() TypeError: full_jitter() missing 1 required positional argument: 'value'

Ok, so the version of

backoff

the SDK is locked with at the moment doesn’t work properly with

backoff_runtime

. I logged https://github.com/meltano/sdk/issues/977

Thanks for looking into this!