https://github.com/NangoHQ/nango looks kind of interesting. I want something like this but as some kind of spec that multiple providers can implement. Then we could implement that spec for any tap that has an oauth dance required

Well, there is RFC9068 for the industry at large to agree on standard claims naming. The owner of the RFC (former Azure AD PM) did a neat write up on it: https://auth0.com/blog/how-the-jwt-profile-for-oauth-20-access-tokens-became-rfc9068/

Out of the (dozens+?) singer taps I've written I've only had to implement a JWT once, and as a tap we're only on the client side. Small sample size but generally that seems to be the case here. This isn't what I'm talking about but maybe you're making a different point and I'm missing it! My end goal (may not fit with nango directly) is to have a spec that defines the oauth dance. Today we handle this like (these are cases where we're being nice and giving a step by step, some taps don't even do this) 1. https://github.com/AutoIDM/tap-googleads#get-refresh-token 2. https://github.com/AutoIDM/tap-gmail#source-authentication-and-authorization 3. https://github.com/AutoIDM/tap-clickup#getting-an-api-token (Pretty common pattern, but no oauth dance, the spec here would literally just give you a description maybe in markdown format for how to do this, maybe even with helpful links) 4. https://github.com/AutoIDM/tap-zohosprints#setup-authentication These can be handled with a spec (some day we can name it) with something like filename

spec.specerton

for tap-googleds

inputs:
  - client_id
  - client_secret 
  - scope

dance_flow:
  get: <https://accounts.google.com/o/oauth2/v2/auth?response_type=code&client_id={inputs.clietn_id}&redirect_uri={specerton.uri}&scope={inputs.scope}&state=autoidm&access_type=offline&prompt=select_account&include_granted_scopes=true>
    await_response: true
    response_data: url_param("code")
  post: <https://www.googleapis.com/oauth2/v4/token?code={dance_flow[0].code}&client_id={inputs.client_id}&client_secret={inputs.client_secrett}&redirect_uri={specerton.uri}&grant_type=authorization_code>

Well, the majority of taps I've written are for HTTP APIs that don't even use OAuth, but instead use their own mechanisms. So, there's no "OAuth dance" at all. 😁 But for the ones that do use OAuth, I haven't come across one that doesn't encode their Access Tokens in the form of a JWT. In the context of a singer tap, the tap never really needs to worry about the claims inside the Access Tokens though, so there's probably not much overlap between the meltano ecosystem and RFC9068

Yeah as I was making that list there were 4-5 that were just manually generated API keys in the source app. Which honestly would be niceto have some standard way of documenting the steps somewhere

I believe what you are thinking of are all the OAuth 2 token grant flows that are not meant for server-to-server interactions. (e.g. auth code grant flow)

I wouldn't limit it to just oauth 2 token grant flows, and yes anytime we have to do a

dance

with another server where we are sending requests back and forth. The end goal for the tap side of all this is to just get that sweet refresh token in most cases.

Nice find! I was trying to find where Nango was keeping all their definitions.

The problem isn't finding the URLs really. The problem is something like: I have a front end service that allows folks to pull data from gmail service (Let's say gmail for now just to keep it simple but in reality you're talking about 10-20 services). To pull data from gmail I need a refresh token from each of my end users. To get that refresh token I have to have the users go through an auth flow of some kind. How do I define the oauth dance without creating 2 sources of truth for auth. The 2 sources of truth would be the tap, and the web service you create for gmail auth.

Interesting. I guess I have been lucky to only ever use the OAUTH2 client_credentials token grant flow (i.e. "a service account") ... Never really had to use any other OAUTH2 flows in a meltano tap. I see how a "service account" might be problematic in the GMail example and you might actually need every single user to agree to having their emails ingested. I wonder if this is a B2C vs B2B challenge?

It's just as hard in B2B, that's where I hit this most of the time

Rigth, I agree. 'Jira' doesn't seem to have support for client credentials grant flows. Azure AD is a piece of cake though. I use it a lot and never had to deviate from the client credentials grant flow in any azure ad cases.

At some point you have to get creds. service account, just means you give a client steps to do and then you have them pass over the creds in your UI. And you have a similar issue but maybe not as much worry about a refresh token

Yes. So basically the way I see it, there are two challenges: 1. Some providers don't have support for all grant types as specified in rfc6749, specifically the lack of client credentials grant support. 2. Meltano currently works well for client credentials grant flow, but not for authorization code grant.

Love this conversation. We deal with the 'dance' generically similar to Nango, but I really like the way they simplified adding the app. As an example, our google taps all use the same authorization + refresh endpoint in our service (which we call a proxy OAuth endpoint) https://github.com/Matatika/utility-gdrive/blob/master/utility_gdrive/auth.py#L8 Some others call it a proxy too: https://oauth2-proxy.github.io/oauth2-proxy/docs/configuration/oauth_provider/ I think Auth0 and Okta offer a service like this. Finally, I prefer to think of this as the consent flow. When it is a simple HTTP API it's almost a 'who cares' situation. Managing the credentials is fairly easy. When you have to get client consent and manage all those separately and refresh, it gets harder.

@aaron_phethean in your case you all had to develop similar "consent flows" for each of the apps right. So you had to develop flows for each tap/target that you wanted in a seperate part of your app (seperate meaning not in the tap/target). I think the "right" abstraction layer is in the tap/target itself but maybe I'm dreaming? Oauth proxies make sense in certain scenario's but the tap having any idea about where oauth is being proxied from seems a bit much but 🤷

Actually not entirely separate from the tap, the plugin.yaml indicates to the tap that it should use the proxy - (so Bearer token configuration works with the same tap) The consent flow does need a UI or something to get user sign in / scope consent, and our approach relies on the UI understanding these

oauth_credentials.

configs (something the meltano UI once did I think)

- name: oauth_credentials.authorization_url
    kind: hidden
    label: OAuth identity provider authorization endpoint used create and refresh tokens
    value: <https://oauth2.googleapis.com/token>
    required: true
  - name: oauth_credentials.scope
    kind: hidden
    label: OAuth scopes we need to request access to
    value: <https://www.googleapis.com/auth/drive.readonly>
    required: true
  - name: oauth_credentials.refresh_proxy_url
    kind: hidden
    label: Optional - will be called with 'oauth_credentials.refresh_token' to refresh the access token
  - name: oauth_credentials.refresh_proxy_url_auth
    kind: hidden
    label: Optional - Sets Authorization header on 'oauth_credentials.refresh_url' request
  - name: oauth_credentials.client_id
    kind: hidden
    label: Optional - OAuth Client ID used if refresh_proxy_url not supplied
    env: GDRIVE_CLIENT_ID
  - name: oauth_credentials.client_secret
    kind: hidden
    label: Optional - OAuth Client Secret used if refresh_proxy_url not supplied
    env: GDRIVE_CLIENT_SECRET
  - name: oauth_credentials.refresh_token
    kind: hidden
    label: OAuth Refresh Token
    env: GDRIVE_REFRESH_TOKEN
  - name: oauth_credentials.access_token
    kind: hidden
    label: Access Token
    env: GDRIVE_ACCESS_TOKEN
    required: true