Hello! Perhaps this is the wrong channel for this sort of question, and feel free to redirect me. I’m using tap-shopify and using the target bigquery to load. In order to make the tap-shopify schema work, we’re having to formulate the schema (catalog) ‘by hand’ via the JSON schema. It turns out that we actually already have a schema definition for bigquery Hello! Perhaps this is the wrong channel for this sort of question, and feel free to redirect me. I’m using tap-shopify and using the target bigquery to load. In order to make the tap-shopify schema work, we’re having to formulate the schema (catalog) ‘by hand’ via the JSON schema. It turns out that we actually already have a schema definition for bigquery that we are happy with, but it’s not using JSON schema. My question is: Is there some way to have singer avoid loading the full schema into bigquery, as opposed to just loading pure JSON (so that then we can perform our own load step later) Ideally I’d love to avoid using json schema files in our project, while also having bigquery schema specifications floating around (https://cloud.google.com/bigquery/docs/schemas) that we are happy with, but it’s not the json schema/catalog. My question: Is there some way to have singer avoid loading the full schema into bigquery, as opposed to just loading pure JSON (so that then we can perform our own load step later) Ideally I’d love to avoid having to fill out a verbose json schema file

So you just want to load one column of raw_json, and then parse it from there with a BQ transformation?

Yes!

Hmm, I'm not sure. I don't think this would be possible without forking all your taps to force this behavior. But maybe it is. You might have better luck if you re-ask this question as "Is it possible for taps just to output records as 'raw json' for later transformations"

This is something that should be possible once we have https://gitlab.com/meltano/meltano/-/issues/2300: you'll be able to write a transformer modifies the SCHEMA and RECORD messages to nest the entire record coming from the tap into a single object property.