pat_nadolny
07/14/2023, 6:21 PMtap-google-analytics
users 🔈. As most of you already know the Universal Analytics API is deprecated and stopped processing new data this month, although my understanding is that historical data will be available for at least another year. With that https://github.com/MeltanoLabs/tap-google-analytics/pull/142 the meltanolabs tap-google-analytics codebase to use the GA4 Data API. Huge shoutout to @connor_flynn @josue_sehnem @Henning Holgersen @Matt Menzenski @alexander_butler for their contributions!
How this is reflected on the hub is changing slightly though. When you search the hub you will now see both a GA4 plugin and a UA plugin. The original tap-google-analytics plugin that has been listed for years is now labeled as UA and the meltanolabs variant has its pip_url pinned to a pre-ga4 branch. Theres also a new tap-ga4
plugin page that represents the GA4 variants. The meltanolabs tap-ga4 variant is still tied to the tap-google-analytics repository but is listed on the hub with the tap-ga4
alias so with meltano you'd run it using meltano invoke tap-ga4
.
⚠️ Action Required ⚠️ - If you're using an unpinned pip_url and want to continue using UA you need to pin your pip_url to main_deprecated_ua
as shown in https://github.com/meltano/hub/blob/1c58814f48c7f08d23d179b6c17076efea0353f2/_data/meltano/extractors/tap-google-analytics/meltanolabs.yml#L18. If you want to switch to GA4 you will need to meltano add
the new tap-ga4
plugin and configure it as some settings have changed.edward_ryan
07/17/2023, 6:02 AMmindaugas_nizauskas
07/17/2023, 9:48 AMSkipping deselected stream 'user_acq_first_user_default_channel_group_report'
Also, when I execute meltano select tap-ga4 --list --all
I don't see this user_acq_first_user_default_channel_group_report
. Only the defaults that are initially created.
What do I miss and fail to understand?pat_nadolny
07/17/2023, 2:05 PMpat_nadolny
07/17/2023, 2:06 PMAlso, when I executeI wonder if this is related to the catalog file being cached. You could try deleting yourI don't see this...meltano select tap-ga4 --list --all
.meltano/run/tap-ga4/
directory to clear out at artifacts that might be getting cached. Then running the --list --all
command should show your streams and you can select them as needed.mindaugas_nizauskas
07/17/2023, 2:08 PMpat_nadolny
07/17/2023, 2:10 PMpat_nadolny
07/17/2023, 2:22 PMdima_anoshin
07/25/2023, 1:11 AMKeyError: 'refresh_token'
I did all the same to obtain Access and Refresh token. But it is not working.
What is the right way to get tokens?dima_anoshin
07/25/2023, 2:30 AMThe error message suggests that the Google Analytics 4 (GA4) tap in Meltano is unable to find the refresh_token in the configuration file.
The refresh_token is required for Meltano to gain authorization to your Google Analytics data. This token is usually obtained through the OAuth2 process. After you get an access token, you can use it to access Google APIs. When the access token expires, the application uses the refresh token to get a new one.
Based on the error, it seems like you've not properly set up the OAuth2 process or that the refresh_token is not included in the configuration file for the tap-ga4 plugin.
dima_anoshin
07/25/2023, 2:31 AMoauth_credentials.refresh_token [env: TAP_GA4_OAUTH_CREDENTIALS_REFRESH_TOKEN] current value:
dima_anoshin
07/25/2023, 2:35 AMdef _initialize_credentials(self):
if self.config.get("oauth_credentials"):
return GoogleCredentials(
access_token=self.config["oauth_credentials"]["access_token"],
refresh_token=self.config["oauth_credentials"]["refresh_token"],
client_id=self.config["oauth_credentials"]["client_id"],
client_secret=self.config["oauth_credentials"]["client_secret"],
token_expiry=None, # let the library refresh the token if it is expired
token_uri="<https://accounts.google.com/o/oauth2/token>",
user_agent="tap-google-analytics (via <http://singer.io|singer.io>)",
)
but in new:
def _initialize_credentials(self):
if self.config.get("oauth_credentials"):
return OAuthCredentials(
None,
refresh_token=self.config["refresh_token"],
client_id=self.config["client_id"],
client_secret=self.config["client_secret"],
token_uri="<https://accounts.google.com/o/oauth2/token>",
)
why tap-ga4
doesn't have the access_token
?
I see the method is changed to from GoogleCredentials
to
from google.oauth2.service_account import Credentials as OAuthCredentials
And If I check what kind arguments it has:
self,
signer,
service_account_email,
token_uri,
scopes=None,
default_scopes=None,
subject=None,
project_id=None,
quota_project_id=None,
additional_claims=None,
always_use_jwt_access=False,
universe_domain=_DEFAULT_UNIVERSE_DOMAIN,
trust_boundary=None,
I don't see refresh, access, client, secret.dima_anoshin
07/25/2023, 2:46 AMoauth_credentials.access_token
oauth_credentials.client_id
oauth_credentials.client_secret
oauth_credentials.refresh_token
it is not using the refresh_token
pat_nadolny
07/25/2023, 1:32 PMgit+<https://github.com/MeltanoLabs/tap-google-analytics.git@fix_oauth_creds>
and running meltano install extractor tap-ga4 --clean
.pat_nadolny
07/25/2023, 2:18 PMdima_anoshin
07/25/2023, 3:58 PMdima_anoshin
07/25/2023, 7:57 PM