https://linen.dev logo
Join Slack
Powered by
I’m setting the following config, following <the d...
# getting-started
m
I’m setting the following config, following the docs: • 
key_file_location
to 
''
• 
client_secrets
to the contents of a valid JSON secrets file • 
view_id
to the correct view ID • 
reports
to a path containing a valid JSON array reports specification • 
start_date
to 
2022-12-08
p
Hey @Matt Menzenski what does your config look like (secrets removed)? The error message makes it seem like the configs are missing or incorrect. Also I assume youre using the meltanolabs variant, right?
m
Copy code
$ meltano --environment=local config tap-google-analytics
2022-12-13T14:47:29.538681Z [info     ] Environment 'local' is active
{
  "key_file_location": "",
  "client_secrets": {
    "type": "service_account",
    "project_id": "<secret>",
    "private_key_id": "<secret>",
    "private_key": "-----BEGIN PRIVATE KEY-----\n<secret stuff>\n-----END PRIVATE KEY-----\n",
    "client_email": "<secret>",
    "client_id": "<secret>",
    "auth_uri": "<https://accounts.google.com/o/oauth2/auth>",
    "token_uri": "<https://oauth2.googleapis.com/token>",
    "auth_provider_x509_cert_url": "<https://www.googleapis.com/oauth2/v1/certs>",
    "client_x509_cert_url": "<secret>"
  },
  "view_id": <nine digit view id>,
  "reports": "./extract/google-analytics/report-definitions.json",
  "start_date": "2022-12-08"
}
yes the meltano labs variant:
Copy code
- name: tap-google-analytics
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-google-analytics.git>
I did minify+escape the client_services JSON file in order to produce the string that I set to the 
client_secrets
config parameter, maybe that’s incorrect?
Specifically I ran 
jq -R -s '.' < client_secrets.json
to get the string that I set to the client_secrets parameter.
p
I'm wondering if the empty 
key_file_location
is causing an issue. I use this tap and when I ran that command my output looks identical minus the 
key_file_location
setting
m
hmm. I removed it with
Copy code
$ meltano --environment=local config tap-google-analytics unset key_file_location
but then I get my usual error still:
Copy code
meltano --environment=local config tap-google-analytics test
2022-12-13T14:58:09.201555Z [info     ] Environment 'local' is active
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Plugin configuration is invalid
JSONSchema was: {'type': 'object', 'properties': {'start_date': {'type': ['string'], 'format': 'date-time', 'description': 'The earliest record date to sync'}, 'view_id': {'type': ['string'], 'description': 'Google Analytics View ID'}, 'client_secrets': {'type': ['object', 'null'], 'properties': {}, 'description': 'Google Analytics Client Secrets Dictionary'}, 'key_file_location': {'type': ['string', 'null'], 'description': 'File Path to Google Analytics Client Secrets'}, 'oauth_credentials': {'type': ['object', 'null'], 'properties': {'access_token': {'type': ['string', 'null'], 'description': 'Google Analytics Access Token'}, 'refresh_token': {'type': ['string', 'null'], 'description': 'Google Analytics Refresh Token'}, 'client_id': {'type': ['string', 'null'], 'description': 'Google Analytics Client ID'}, 'client_secret': {'type': ['string', 'null'], 'description': 'Google Analytics Client Secret'}}, 'description': 'Google Analytics OAuth Credentials'}, 'reports': {'type': ['string', 'null'], 'description': 'Google Analytics Reports Definition'}, 'end_date': {'type': ['string', 'null'], 'description': 'The last record date to sync'}, 'stream_maps': {'type': ['object', 'null'], 'properties': {}, 'description': 'Config object for stream maps capability. For more information check out [Stream Maps](<https://sdk.meltano.com/en/latest/stream_maps.html).>'}, 'stream_map_config': {'type': ['object', 'null'], 'properties': {}, 'description': 'User-defined config values to be used within map expressions.'}, 'flattening_enabled': {'type': ['boolean', 'null'], 'description': "'True' to enable schema flattening and automatically expand nested properties."}, 'flattening_max_depth': {'type': ['integer', 'null'], 'description': 'The max depth to flatten schemas.'}}, 'required': ['start_date', 'view_id']}
p
I tried breaking my client secrets and running the config/test command and it gave me the "Plugin configuration is invalid" error. If you run 
meltano --environment=local invoke tap-google-analytics
do you get any records?
m
no - but I get a clear error message!
Copy code
singer_sdk.exceptions.ConfigValidationError: Config validation failed: <number> is not of type 'string'
I believe I set that number within the 
meltano config tap-google-analytics set --interactive
command - should I have quoted it?
ahhhhhhhh it works now that I’ve quoted the view_id 🎉
thank you @pat_nadolny!
p
I wonder if config test could do a better job telling us whats wrong rather than just "config invalid" or even a note to suggest attempting to run with invoke to see if theres any helpful logs. What do you think would have been helpful in your experience?
m
the difference between 
config test
and 
invoke
wasn’t clear to me (still isn’t, to be honest). It seems like 
invoke
will actually return data while 
config test
is more of a “ping” ?
I think a note about trying with 
invoke
as well would be helpful. That made the difference for me today.
p
ok cool, I'm going to open an issue related to this and tag you on it. If you have thoughts, theyre welcomed and appreciated 😄
m
sounds good, thank you again!
p
Also yes to your assumption is right about the difference between the two commands. 
config test
is meant to confirm that the config is functional, I think by retrieving a single record then exiting. 
invoke
is for running a single tap directly, all singer records will log to the terminal which is helpful for debugging sometimes
Open in Slack
PreviousNext
Powered by