I am using the graph API for getting details of MS resources. However, I encounter name resolution errors 1% of the times:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='<http://login.microsoftonline.com|login.microsoftonline.com>', port=443): Max retries exceeded with url: /b5953ddd-dd80-4110-904b-e503716f0caf/oauth2/v2.0/token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7b99afdc5ed0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

To handle this I am keeping the

prepare_request

method in try except block and raising

RetriableAPIError

, but the retries aren't happening. Should I handle this in a different way? from the traceback the following exceptions happen: • requests.exceptions.ConnectionError • urllib3.exceptions.MaxRetryError • socket.gaierror

Yes, override

validate_reponse

if the default implementation doesn't meet your needs: https://sdk.meltano.com/en/v0.42.1/classes/singer_sdk.RESTStream.html#singer_sdk.RESTStream.validate_response

I don't think the code reaches that point. Since, this happens at the initial name resolution step for the auth code. We don't get a status code.

requests.exceptions.ConnectionError

is already retried by default. You can see that from the log you posted: >

Max retries exceeded with url: /b5953ddd-dd80-4110-904b-e503716f0caf/oauth2/v2.0/token

When meltano retries, I see the retry logs too. This seems to be the retry mechanism of requests library. I don't think we can configure the retries for the same via meltano.

I don't really understand why it wouldn't be working, unless you have something the breaks default behaviour: 1. The request is prepared - if you are manually making a separate request in here it won't be handled according to Meltano's retry logic 2. The request is made - any of these exceptions that occur as a result of the call to _request will be retried by default a. The request is sent - assuming this raises

requests.exceptions.ConnectionError

it should be retried by Meltano

I have added a response hook to log, handle and store errors. Other than that I am not overriding default behaviour. Also, this happens 1% of the times. If I get a 429, I see the retry logs with the number of times retry happened (default meltano logging). In this case I don't see the retry logs. The complete traceback:

Name resolution with the host being microsoftonline.com means to me you have a local network problem as well that would probably be easier to fix here. When I get these kinds of things and I can't ge tthe IT team to dive in, I either make a script to show it happening, or I just have my specefic code run with a different dns resolver (8.8.8.8) until I get my network team to come fix it. You can retry the stuff to try to work around the outages as well, 🤷 The issue you're having is that it's not following the validate_response flow because this is happening during an auth request which doesn't have the same retry flow (it really should, and I don't know if we have an SDK issue in for that) I know I have atleast one public tap that does the retries for you for auth here https://github.com/AutoIDM/tap-indeed specifically https://github.com/AutoIDM/tap-indeed/blob/main/tap_indeedsponsoredjobs/auth.py where we copied over the retry logic

Thanks, @visch. I suppose I have to use most of what's written after line 129, correct?

looks generally right in my 30 second glance!

Didn't spot the request was for auth, thanks Derek - sorry about that @Nir Diwakar (Nir)