Hello Everyone, I am working on a project which needs to pull data from Big Query, I am using the tap-bigquery --variant meltanolabs everything is working as intended except for when dealing with Large tables I get Response Too large error. Does anyone have a work around for this besides doing a BQ Export job?

Hi @John Archer! tap-bigquery uses the official BQ sqlalchemy dialect and looking at its readme, it seems to support configuring a destination table as a query param, like

<http://my-project.my-dataset.my|my-project.my-dataset.my>_table

. However, tap-bigquery probably doesn't consider this option when constructing the sqlalchemy url. It's a matter of updating the tap config to accept a destination table reference and using that to construct the url. Do log an issue in the repo or even feel free to submit a PR!

@Edgar Ramírez (Arch.dev) thank you for the info