Hi all - using the Google Analytics tap, running into what seems like excessive timeout issues. Daily pipeline runs will usually fail, with the odd successful run. Successful runs take quite a while: eg. 01:14 - 03:34 Failed runs can also take a while, which then seem to timeout: eg. 01:14 - 02:55 Failed runs dump a error in the elt.log, summarily: [error] Extraction failed | code=1 name=backoff message=Backing off _query_api(...) for 0.9s (socket.timeout: The read operation timed out) Does this behaviour seem correct? We are pulling the default set of reports - that is, we aren't customising the tap extraction at all. Simply setting the start date to 2022-01-01, which doesn't seem excessive. Would we have to continually move the start date forward, allowing the tap to pull less data and hopefully avoid timeouts? This seems like an anti-pattern. Any help would be appreciated, before digging into the source code!

Hi Zaki. We (and our customers) run many tap-google-analytics ELTs each day. Not all on massive sites, but some with lots of history that didn't have any issues. Our pipelines are running on Azure, wondering if yours are on a stable connection? On a separate note. It's almost time to write the GA4 tap in the SDK as Google are retiring universal analytics.

Hi @aaron_phethean - the same connection/host is used for many other taps without any issues (enterprise fibre), it's just the GA tap that stalls. Would you mind sharing the start_date used when configuring the taps? Our site is fairly large, which may be affecting things here. I think we'll have to inspect the tap manually for any bits of logic that may blow out with large datasets. Our

pages

table is around half a mill records.

Right. Our start dates are often only 6 months or a year. I can't recall whether that tap supports an incremental sync, but that would be the ideal situation. If your team doesn't want to dig into the code our guys could have a look at it for you. We are based in the UK - I think we have fairly reasonable UK rates. Cheers, Aaron