Hi everyone - we are currently using the

tap-spreadsheets-anywhere

(link to repo here) and the

target-bigquery

from Adswerve (link to repo here), and we are getting inconsistent schema definitions coming out of our

tap-spreadsheets-anywhere

which is leading to errors when we want to write those results to BigQuery. Is there any way to have a tap check what the schema is for the target and just use that when establishing it's schema definiton? Or is the only way to ensure consistency to just manually set the schema in the tap configuration? Thank you!

There’s no way for the tap to learn from the target, no. Setting the schema explicitly using https://meltano.com/docs/plugins.html#schema-extra would be my suggestion

That's what I figured. Just tough with such large tables and data sets, but if that's what we need to do that's what we'll do! Is there any way to split that out into separate files? Putting that all into one Meltano.yml file makes it pretty large.

It’s on the roadmap: https://gitlab.com/meltano/meltano/-/issues/2456 🙂