Hi all, when trying to rename the stream name for the tap-googleads, I hit upon a specific error I am struggling to understand. So If I attempt to rename the ‘stream_adgroups’ stream like so:

stream_maps:
        stream_adgroups:
          __alias__: test

I see the following error:

StreamMapConfigError: Invalid key properties for 'test': [ad_group__id,ad_group__campaign,ad_group__status]. Property 'ad_group__id' was not detected in schema

Any help would be greatly appreciated

tap-googleads emits json but the key column is only present as a top-level field if you enable flattening of json on your target. This is my specific postgres config for use with this tap.

- name: target-postgres-flattening
    inherit_from: target-postgres
    config:
      data_flattening_max_level: 3
      add_record_metadata: true

What target are you using?

Target-bigquery

Do you have any flattening set on the target? https://github.com/z3z1ma/target-bigquery?tab=readme-ov-file#settings

Yes I do, like this

- name: target-bigquery
    variant: z3z1ma
    pip_url: git+<https://github.com/z3z1ma/target-bigquery.git>
    config:
      denormalized: true

Sorry I changed to flattening above

So when I configure the bigquery tap int he same way you do I see the same error, Are you renaming the streams on the tap also?

Can you do

meltano run tap-googleads target-jsonl

and see how the output files look for this stream?

I get the same error with jsonl

How about if you remove your stream maps block and try again?

Do you have an example of the above along with the rename of the streams?

No I don't do any

stream_maps

unfortunately.

- name: tap-googleads
    variant: matatika
    pip_url: 
      git+https://${ADO_TOKEN}@dev.azure.com/EmpiricPowerApps/AnalysisAndDataProjects/_git/tap-googleads@v0.7.0
    config:
      oauth_credentials.client_id: ${TAP_GOOGLEADS_CLIENT_ID}
      oauth_credentials.client_secret: ${TAP_GOOGLEADS_CLIENT_SECRET}
      oauth_credentials.refresh_token: ${TAP_GOOGLEADS_REFRESH_TOKEN}
      customer_id: ${TAP_GOOGLEADS_CUSTOMER_ID}
      login_customer_id: ${TAP_GOOGLEADS_LOGIN_CUSTOMER_ID}
      developer_token: ${TAP_GOOGLEADS_DEVELOPER_TOKEN}
      start_date: '2023-01-01T00:00:00Z'
    metadata:
      stream_adgroups:
        key-properties:
        - adGroup__id
        - adGroup__campaign
        - adGroup__status