Hello @alexander_butler, thank you for your amazing

target-bigqueryy

. I'm confuse on the enabled_flattening setting, does this setting use to flattened all nested records in the target table? I've set it to True but my target table still have field with Records Type. This is my config:

{
  "credentials_path": ".config/cred.json",
  "project": "xxxxx",
  "dataset": "meltano_ingestion",
  "location": "asia-southeast1",
  "denormalized": true,
  "method": "batch_job",
  "overwrite": true,
  "stream_maps": {
    "ads_insights": {
      "__alias__": "ads_insights__outbound_clicks"
    }
  },
  "flattening_enabled": true,
  "flattening_max_depth": 3
}

The flattening is a part of the sdk, so I believe the feature is fine. If you still see complex types, its probably because they are nested further than the max depth?

it's actually only one level. I'm not sure what I'm doing wrong here

Hi @akmal. Can you share the original schema of the stream you're trying to flatten? (it may be that the singer SDK is failing to flatten variant fields)

@edgar_ramirez_mondragon Here's the schema: ```{ "type": "SCHEMA", "stream": "ads_insights", "schema": { "type": [ "null", "object" ], "properties": { "unique_actions": { "type": [ "null", "array" ], "items": { "type": [ "null", "object" ], "properties": { "1d_click": { "type": [ "null", "number" ] }, "7d_click": { "type": [ "null", "number" ] }, "28d_click": { "type": [ "null", "number" ] }, "1d_view": { "type": [ "null", "number" ] }, "7d_view": { "type": [ "null", "number" ] }, "28d_view": { "type": [ "null", "number" ] }, "action_destination": { "type": [ "null", "string" ] }, "action_target_id": { "type": [ "null", "string" ] }, "action_type": { "type": [ "null", "string" ] }, "value": { "type": [ "null", "number" ] } } } }, "actions": { "type": [ "null", "array" ], "items": { "type": [ "null", "object" ], "properties": { "1d_click": { "type": [ "null", "number" ] }, "7d_click": { "type": [ "null", "number" ] }, "28d_click": { "type": [ "null", "number" ] }, "1d_view": { "type": [ "null", "number" ] }, "7d_view": { "type": [ "nul…

Ah, yes. You'd need to use a mapper plugin to accomplish this with a non-sdk tap: https://docs.meltano.com/guide/mappers/#standalone-mapper-plugins I do see your schema has arrays of objects, which I think can't be flattened by the singer SDK at the moment. Other flattening implementations create separate streams/tables for those case. Do log a feature request if that sounds like the feature you need

Great, thanks!