casey
02/17/2021, 3:55 AMtap-postgres
with target-bigquery
. Both are "standard" flavors: transferwise and adswerve, respectively. But when I come upon a record that has an array type, I get the following error:
target-bigquery | CRITICAL 'type' or 'anyOf' are required fields in property: {'$ref': '#/definitions/sdc_recursive_string_array'}
target-bigquery | CRITICAL ['Traceback (most recent call last):\n', ' File "/home/chb/code/work/learning_equality/le-pipeline/meltano-projects/get-star
t/.meltano/loaders/target-bigquery/venv/lib/python3.7/site-packages/target_bigquery/__init__.py", line 93, in main\n for state in state_iterator:\n', ' Fi
le "/home/chb/code/work/learning_equality/le-pipeline/meltano-projects/get-start/.meltano/loaders/target-bigquery/venv/lib/python3.7/site-packages/target_bigq
uery/process.py", line 40, in process\n for s in handler.handle_record_message(msg):\n', ' File "/home/chb/code/work/learning_equality/le-pipeline/meltano
-projects/get-start/.meltano/loaders/target-bigquery/venv/lib/python3.7/site-packages/target_bigquery/processhandler.py", line 110, in handle_record_message\n
new_rec = filter_by_schema(schema, msg.record)\n', ' File "/home/chb/code/work/learning_equality/le-pipeline/meltano-projects/get-start/.meltano/loaders/
target-bigquery/venv/lib/python3.7/site-packages/target_bigquery/schema.py", line 75, in filter\n record[key]) # adswerve fix to match schema field name\n
', ' File "/home/chb/code/work/learning_equality/le-pipeline/meltano-projects/get-start/.meltano/loaders/target-bigquery/venv/lib/python3.7/site-packages/tar
get_bigquery/schema.py", line 84, in filter\n prop_type, _ = get_type(props)\n', ' File "/home/chb/code/work/learning_equality/le-pipeline/meltano-project
s/get-start/.meltano/loaders/target-bigquery/venv/lib/python3.7/site-packages/target_bigquery/schema.py", line 23, in get_type\n f"\'type\' or \'anyOf\' ar
e required fields in property: {property}"\n', "ValueError: 'type' or 'anyOf' are required fields in property: {'$ref': '#/definitions/sdc_recursive_string_ar
ray'}\n"]