Hi everyone, we are using the `tap-linkedin-ads` e...
# docker
c
Hi everyone, we are using the
tap-linkedin-ads
extractor (meltanolabs implementation) to obtain LinkedIn Ads data, and I am a little curious about how the token refresh is intended to be managed? When you obtain an OAuth 2.0 access token for Linkedin, you also are provided a refresh token in order to make a secondary call to refresh your access token. (The access token has an expiry of 60 days, and the refresh token has an expiry of 365 days.) I didn't see anything in the client.py where the refresh token is actively managed, so the access token would expire in 60 days by default, unless I am missing something? Is the tap able to manage this somehow? I also looked at the other versions for this same `tap-linkedin-ads`; the Stitch Data implementation seems to have a provision for that functionality in client.py, in two functions named
refresh_access_token
and
fetch_and_set_access_token
, but I am unsure how the new access token would be stored, as it would refresh the token and receive the new one, but it is self contained within the k8s pod and won't persist. https://hub.meltano.com/extractors/tap-linkedin-ads https://github.com/MeltanoLabs/tap-linkedin-ads https://github.com/singer-io/tap-linkedin-ads
v
normally you use the refresh token as a configuration value, and the tap will get a new access token every run (and refresh it as it needs) Sometimes there are weird things applications do where this doesn't work, but that's very rare. Looks to me like the meltanolabs version doesn't use a refresh token to get a new token (incorrect implementation so it seems but I hesitate to jump there as I don't know LinkedIn so there could be something odd) Looking at the stitch implementation they seem to correctly use the refresh token to get a new access token https://github.com/singer-io/tap-linkedin-ads/blob/master/tap_linkedin_ads/client.py#L236-L245
c
Ok thanks, but even in the Stitch implementation, I guess there would be some ongoing management regardless, as it will need the valid access token as an input to start with (we store it as a secret). • If the access token contained in the secret is already expired there isn't a way to auth and refresh • and if the functions work as intended to obtain a new access token via the refresh token, the new access token wouldn't persist outside of the run for the next time I wonder am I misunderstanding some part of how we would typically do this? I noticed in the Stitch readme it also mentioned the API requires a user to manually authenticate again within the 60 day window of the access token being valid.
v
Looks like there is some kind of weird Linkedin Ads thing https://github.com/singer-io/tap-linkedin-ads/blob/3e07935eedf2ca9a600018ac780470ec294410a2/tap_linkedin_ads/client.py#L179-L186 Looks like the access token needs to get regenerated every 2 months. What's not clear to me is why we wouldn't just generate a new acces token every time. But because there's 2 implementation that do it the same way I'd guess there's a Linkedin related reason here
Dove in as i couldn't help it, I have to go back to work but here's what I learned https://learn.microsoft.com/en-us/linkedin/shared/authentication/programmatic-refresh-tokens?context=linkedin%2Fcontext Really the taps should probably just have a refresh token as a configuration option, then you'd be good for a year before having to go get a new one setup. I'd guess historically this wasn't true which is why the access token stuff is there but I have no idea! Again I'd guess i'm wrong here, the refresh token might get reset or something and that's the core issue (storing the new refresh and access token)
c
Ok thanks Derek for your help!
h
wow, I also dove in as i couldn't help myself. its kinda wild that every year you have to get a new refresh token. i wonder if there's a way to avoid that (i.e. refresh the refresh token automatically)