Hello! I'm using `target-bigqeury` (z3z1ma) to r...
# getting-started
r
Hello! I'm using
target-bigqeury
(z3z1ma) to receive data from MongoDB into BigQuery. I managed to send some collections to the target (not all), but some questions arose. If you could help me when you can, please, I would appreciate it: 1. How can I specify in
target-bigquery
some tables in BigQuery that should be partitioned by field X and clustered by Y, Z? 2. Why are two tables created in BigQuery: one with the execution time suffix, with data, and another without suffix and without data? Is a new table created with each load? (attached file) 3. I would like to confirm: normally there is no change in the MongoDB schema, but it can occur in case of an update. I am using denormalized: true. In case of a change, this can impact the load, correct? 4. The last error I got was "`ParseError: null is not allowed to be used as an element in a repeated field at processo.prioridade[0]`". Is it possible to handle this in stream-maps? Thanks!
e
1. That's probably better left to a downstream transformation stage, unless the target supports it natively, which I don't know 2. I guess it's a temp table used to hold the data before loading it to its final destination 3. Yes, the target _shoul_d update the table schema 4. Hard to know without knowing the field's schema (or without trying)
👍 1
r
Thanks again, @Edgar Ramírez (Arch.dev) I'll continue testing here. \o.
👍 1
e
Cool, let me know if I can help