https://linen.dev logo
Join Slack
Powered by
Hello! I'm trying to connect multiple shopify stor...
# troubleshooting
e
Hello! I'm trying to connect multiple shopify stores to big query tables. There's an issue with the tap schema map, so I'm storing locally in JSONs and then uploading to GBQ for now. I just finished setting up the pipeline, and I'm wondering if anyone has a clean way of cycling between my three stores by changing the shopify-tap without really manual code?
v
What's the issue with the tap schema map? I know there's some threads in slack about shopify https://github.com/AutoIDM/shopify2athena/blob/main/meltano.yml is some schema overrides. If you could share more details folks can help
e
Thanks for responding Derek! I took another deep dive into the issue. I tried the Shopify2Athena plugin but it didn't change anything. I think I've traced the issue down to the nested objects in the tap-Shopify response, which is incompatible with BigQuery. Any thoughts? See the log: 
[*debug*  ] *Invoking: [.meltano/extractors/tap-shopify/venv/bin/tap-shopify', '--config', '.meltano/run/tap-shopify/tap.6b8c52b4-6660-47fd-a07d-0cd3ae1260d3.config.json', '--catalog', '.meltano/run/tap-shopify/tap.properties.json', 'target-bigquery']*
usage: tap-shopify [-h] -c CONFIG [-s STATE] [-p PROPERTIES] [--catalog CATALOG] [-d]
tap-shopify: error: unrecognized arguments: target-bigquery
I've tried adjusting the tap.properties.json schema to declare datatypes: 1. For the nested objects that the code you shared addressed. 2. Removing empty properties 3. For 'duties' which need to be declared as a number. (If I leave it or declare it as something other than 'number', I get 
CRITICAL 'RECORD'
 The farthest I've gotten is now getting this error message: 
INFO Pushing state: {}
 and I also get an error saying the schema for 'duties' is incorrect 
.duties: data does not match {'type': ['number', 'null']}
 Any help would be really appreciated. šŸ™
v
Copy code
tap-shopify: error: unrecognized arguments: target-bigquery
Has nothing to do with mapping information. I think we should start with whatever issue you're having, please post the log. I'm confused reading through this. Please provide: ā€¢ meltano.yml ā€¢ relevant logs ā€¢ Any other releveant information And lets try to keep it to one of the issues so we can focus on fixing it šŸ˜„
get an error saying the schema for 'duties' is incorrect 
.duties: data does not match {'type': ['number', 'null']}
Is this from 
target-bigquery
? It's hard to know please follow this https://meltano.slack.com/archives/C01TCRBBJD7/p1676648672671689?thread_ts=1676497384.670299&cid=C01TCRBBJD7 and then someone can probably help!
e
Hey Derek! I've attached the logs and yml file. I've managed to build a pipeline by unpacking the target-jsonl loader. The issue could be the order line items contained within the line_items column, since there is a variable number of order line items per order. Thanks for supporting
elt.logmeltano.yml
Open in Slack
PreviousNext
Powered by