Testing follow up, thanks to @aaron_phethean , @edgar_ramirez_mondragon , @aaronsteers , and @michel_ebner (hope I didn't miss someone who pointed me places). Picked a few ideas from just about everywhere you all pointed me to, ended up with https://github.com/AutoIDM/tap-googleads/pull/19 , not as clean as I'd like yet but it does the job and is pretty readable

Aha so this is similar to https://github.com/MeltanoLabs/tap-github/issues/16 🤔

yes @edgar_ramirez_mondragon 😄 , I need to implement a max number of failures. In this case it's take the total number of clients that you're going to query, and then no more than let's say 25% can fail without failing the entire job. I also think one failure should really make the error code of the job be non zero (but not 1) maybe 1337 to indicate there was an issue with the job, but 🤷 I'd guess that will lead to some issues today with meltano and how ELT works

@visch - Just curious. Are you keeping track of the failed/skipped entries? Seems like a marker in STATE could be helpful here. If passed along in an extra STATE entry, those skipped/failed on one attempt could be selectively prioritized in a subsequent execution (or not, I guess). 🙂 🤷

Right now no, makes more sense then just failing on the same one over and over. Might help with the other issue that I need to solve which is failing on >25% of clients "partially" failing.