I am developing a tap for an API that will return a HTTP 429 if the rate limit is exceeded. 1. Any suggestions how I might simulate a 429? I can do it with fiddler I suppose--is anything baked into the SDK 2. Is there a recommended practice using the SDK for handling 429 or other error codes?

cc @aaronsteers

I'm not sure what the goto library is, but there's certainly some kind of mocking framework. https://github.com/getsentry/responses#basics looks like it'd work https://realpython.com/testing-third-party-apis-with-mocks/ as well, Aj will be much better than me 🙂

No idea on 1), but for 2), what i did, i don't know if it's the recommended way, but i reimplemented _`_request_with_backoff`_ You can see the code here, including the code to retry to return false in giveup is what makes it retry with that code https://github.com/decentraland/tap-decentraland-api/blob/d878751c609d56713d7a34fb6268cb8fa55061b2/tap_decentraland_api/coingecko_mana_stream.py#L81

there are 2 open issues that im aware of around this

@pablo_seibelt - As @pnadolny points out, those two issues around rate limits and custom backoff/retry codes would get us closer to handling this in a more streamlined manner. Sounds like you have a path forward though. 👍

Yep, just suggesting the current hack that works in case it's useful to Adam, but i agree it'd be nice to have it in the framework, as always thanks for these great tools

Thanks for these ideas and the issues.

@adam_roderick The

RESTStream

has some error handling baked in: https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/streams/rest.py#L126 and retries: https://gitlab.com/meltano/sdk/-/blob/main/singer_sdk/streams/rest.py#L97

thanks I will take a look there!

On the example i sent, you can handle any errors by specifying the numbers in the giveup

https://requests-mock.readthedocs.io/en/latest/ saw while watching a build window scroll with Meltano