Hello. I’m really stumped on something. Trying to setup the

MeltanoLabs

0.0.4 Google Analytics extractor using the

client_secrets

env variable. I have set the

TAP_GOOGLE_ANALYTICS_CLIENT_SECRETS

and tried both setting it and not setting it explicitly in the

meltano.yml

. I can see the env variable set in Docker and it is serialized correctly. I can also get the connector to authenticate using the same information but using the

key_file_location

Hi @dennis_hume! I guess this is because the SDK doesn’t parse JSON objects when reading from environment variables 🤔. Though I would’ve imagined that Meltano correctly dumps it as an object. Can you take a look at the output of

meltano config tap-google-analytics

to confirm the

client_secrets

key is a JSON object and not a string?

It is a string there

Ok, so the problem seems to be on the side of Meltano. I’ll need a few more details. • Version of Meltano • Contents of

meltano.yml

or at least the definition of

tap-google-analytics

you got in there

We are on meltano, version 1.100.0. For the yaml, looks like:

- name: tap-google-analytics-xxx
    namespace: tap-google-analytics
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-google-analytics.git@v0.0.4>
    executable: tap-google-analytics
    capabilities:
    - state
    config:
      view_id: 'xxx'
      start_date: '2022-01-01'
      client_secrets: $TAP_GOOGLE_ANALYTICS_CLIENT_SECRETS
    load_schema: google_analytics_xxx

Ok, so the latest is

2.3.0

and we’ve shipped a few fixes to JSON object settings since

1.100.0

, so my first suggestion would be trying to update to the latest. Also, I see you’re using your own custom definition of

tap-google-analytics

. In the latest Meltano,

meltano add extractor tap-google-analytics

would get the definition with all the right settings and their kinds.

Thanks. We can try updating Meltano versions. Hopefully that will help

can you try:

- name: tap-google-analytics-xxx
    namespace: tap-google-analytics
    variant: meltanolabs
    pip_url: git+<https://github.com/MeltanoLabs/tap-google-analytics.git@v0.0.4>
    executable: tap-google-analytics
    capabilities:
    - state
    settings:
    - name: client_secrets
      kind: object
    config:
      view_id: 'xxx'
      start_date: '2022-01-01'
      client_secrets: $TAP_GOOGLE_ANALYTICS_CLIENT_SECRETS
    load_schema: google_analytics_xxx

Alright with that added, I now see it loading as a dict

For more on custom extractors you can read https://docs.meltano.com/tutorials/custom-extractor#add-the-plugin-to-your-meltano-project

Hello. Circling back on this, was sidetracked. I am able to correctly set the client secret as a dict from the env var however I don’t think it is handling the

private_key

correctly since it is inserting escape characters around the newlines. This escape character insertion does not happen when you just paste the private key as a string literal into the meltano.yml config file. Anyway to deal with this?

Hi @dennis_hume! I’m trying to reproduce this behavior. How are the newline characters represented in the evn var? e.g. an

\n

or

\\n

literal?

Worked on it some more, user error