https://linen.dev logo
#plugins-general
Title
# plugins-general
c

charles_mcmillan

05/21/2021, 10:42 PM
Hello! Perhaps this is the wrong channel for this sort of question, and feel free to redirect me. I’m using tap-shopify and using the target bigquery to load. In order to make the tap-shopify schema work, we’re having to formulate the schema (catalog) ‘by hand’ via the JSON schema. It turns out that we actually already have a schema definition for bigquery Hello! Perhaps this is the wrong channel for this sort of question, and feel free to redirect me. I’m using tap-shopify and using the target bigquery to load. In order to make the tap-shopify schema work, we’re having to formulate the schema (catalog) ‘by hand’ via the JSON schema. It turns out that we actually already have a schema definition for bigquery that we are happy with, but it’s not using JSON schema. My question is: Is there some way to have singer avoid loading the full schema into bigquery, as opposed to just loading pure JSON (so that then we can perform our own load step later) Ideally I’d love to avoid using json schema files in our project, while also having bigquery schema specifications floating around (https://cloud.google.com/bigquery/docs/schemas) that we are happy with, but it’s not the json schema/catalog. My question: Is there some way to have singer avoid loading the full schema into bigquery, as opposed to just loading pure JSON (so that then we can perform our own load step later) Ideally I’d love to avoid having to fill out a verbose json schema file