I’m curious what the best practices are for taps where the original API provider highly recommends OAuth. Do people setup a little fake site / route to do this? I’m not embedded in our engineering org at work so this generally would involve more work coordinating with them etc. Example - HubSpot (https://github.com/singer-io/tap-hubspot) a low-query-volume focused API (HAPIKEY) but I’m worried I’m going to hit that limit quickly with Meltano / Singer taps.

This is a good question. OAuth is really great for 3-legged oauth (end user -> you -> google), but frustrating for 2 legged oauth (when you are the end user). Many companies offer OAuth token generators / service accounts / personal API keys to allow users to build apps targeting their own account on that platform. HubSpot is one company though which doesn't offer an easy, production 2LO solution. You can use the HubSpot quickstart for (nontrivially) generating an access token, but sadly Hubspot access tokens only last for 6 hours and must be renewed thereafter via a refresh token. I know Meltano has an OAuth service built in, but I'm not well versed in it and am unsure if it can solve your use case. I've hit this same barrier recently when looking into creating an extractor so I'll let you know if I learn anything

this is a 5 ⭐ answer, thank you!

@srini_kadamati__apache_superset_ @zachary_wynegar The Meltano OAuth service (https://meltano.com/docs/settings.html#oauth-service) is currently only set up for Facebook Ads and Google Ads, but can be extended for other OAuth providers by making the appropriate changes here: https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/oauth/providers.py

@douwe_maan Ah gotcha, that's useful! Thanks for the explanation! One feature that might be worth exploring, down the line and after some further validation, is building a programmatic integration between the Meltano hosted OAuth service and Meltano self-hosted instances for reauthorizing access tokens as some OAuth provides, such as HubSpot, have very short lived access tokens requiring refresh after only a few hours. This would be a powerful tool for the community as, without it, there is no way (that I am aware of) to use production taps against APIs with short lived OAuth access tokens and no 2LO authentication options. I'm not sure how common this is, but HubSpot is one example

a programmatic integration between the Meltano hosted OAuth service and Meltano self-hosted instances for reauthorizing access tokens

That would definitely be useful, but it may be easier said than done since Meltano has no way of knowing whether a token has expired; only the tap that actually manages the connection does. There's some related discussion in https://gitlab.com/meltano/meltano/-/issues/1317 if you're interested, and we can definitely re-explore this at some point 🙂

Tganks for the issue reference, I'll check it out. I was thinking along the lines that the Meltano-hosted service would store refresh tokens, and reauthorize them when pinged by the self-hosted instance. It would be difficult to figure out the timing, however, as OAuth token timeouts are not standardized in the Singer spec. Anyways, I'll review the thread and leave any thoughts there!

@zachary_wynegar Feel free to create a new issue to address this particular detail!