has anyone ever run into

requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

issues when performing a meltano elt?

A full log would be helpful as we could tell if this is a tap thing or meltano thing https://stackoverflow.com/questions/1434451/what-does-connection-reset-by-peer-mean Smells like a proxy thing but impossible to tell without some more info

ah, I will rerun with the log level debug option.

I agree with Derek. Connection reset by peer is a generic TCP error that can be caused by a myriad of misbehaving network-layer components.

I just got this running a new-ish Hubspot tap. I only get it in certain environments though which is annoying. Most of stack overflow seems to suggest adding a

time.sleep(0.01) somewhere or just retrying. I don’t get this issue when I run with

--log-level=debug` I presume because it artificially adds this microscopic slow between calls due to printing out all the logs??

Connection Reset does mean something specefic, the server you're connecting to is dropping you. So you're probably connecting to fast or something along those lines. Retry seems like the best option to me, if that triggers lots of Retries (Like every request) then I'd add something to the request library to wait some amount of time between requests

My logs seem to indicate that it’s already retrying to some degree assuming with a backoff because I’m using the

singer-sdk=0.4.5

.

...
File "/opt/dagster/app/.meltano/extractors/tap-hubspot/venv/lib/python3.7/site-packages/singer_sdk/streams/rest.py", line 322, in request_records    
    resp = decorated_request(prepared_request, context)
File "/opt/dagster/app/.meltano/extractors/tap-hubspot/venv/lib/python3.7/site-packages/backoff/_sync.py", line 94, in retry
    ret = target(*args, **kwargs)
...

is it retrying because of the connection reset? I'd verify that, and if so then maybe you have to wait longer? It's all server side that does this

taking a deeper look, maybe it’s not on the retry it’s choking, because the sdk theoretically logs when it has to back off and I get no such log. So this error seems to go undetected by the sdk retry/backoff code

Right, so you probably have to tweak the retry settings to catch this, would be a good add to the main SDK too I think!

yeah, it’s not responding to a status code, that’s why it doesn’t trigger the retry. it’s just a plain ol’ exception so I’ll have to add a

try/except

somewhere

I think the Decorator on the backoff lib is accesible, you may just have to add the exception to the list of things to retry on

I tried adding

104

. it didn’t work because I don’t think it even registers as a

response.status_code

yup, you’re right. I’ll give that a go

https://requests.readthedocs.io/en/latest/_modules/requests/exceptions/#:~:text=class%20ConnectionError(RequestException)%3A Looks like 💰