Hello all I am having an issue with a tap I m using the <htt Meltano #plugins-general

Hello all, I am having an issue with a tap. I'm us...

emilie

04/11/2024, 6:44 AM

Hello all, I am having an issue with a tap. I'm using the tap-rest-api-msdk to pull data from an API to put into BigQuery. It runs fine when I write it to a local jsonl file with

meltano --log-level=debug run tap-rest-api-msdk target-jsonl

. I am getting a loader error when I run

meltano --log-level=debug run tap-rest-api-msdk target-bigquery

specifically:

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

. 🧵

emilie

04/11/2024, 6:45 AM

I have played around with the json settings for BigQuery but that didn't change the outcome. A redacted

meltano.yml

and logs are below. Any troubleshooting tips?

emilie

04/11/2024, 6:45 AM

meltano.yml

Copy code

version: 1
default_environment: dev
project_id: 5bcad1e7-2357-4653-a348-0cde84d60a90
environments:
- name: dev
- name: staging
- name: prod
plugins:
  extractors:
  - name: tap-rest-api-msdk
    variant: widen
    pip_url: tap-rest-api-msdk
    config:
      api_url: <https://website.com/api>
      streams:
      - name: customers
        path: /v1/customers
        primary_keys:
        - id
      username: <user_name>
      password: $TAP_REST_API_MSDK_PASSWORD
      auth_method: basic
  loaders:
  - name: target-jsonl
    variant: andyh1203
    pip_url: target-jsonl
  - name: target-bigquery
    variant: z3z1ma
    pip_url: git+<https://github.com/z3z1ma/target-bigquery.git>
    config:
      credentials_json: $GCP_CREDS
      dataset: <dataset_name>
      project: <project_name>

emilie

04/11/2024, 6:48 AM

Logs

Copy code

2024-04-11T06:31:50.210100Z [info     ] 2024-04-10 23:31:50,209 | INFO     | target-bigquery      | Using thread-based parallelism cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210325Z [info     ] 2024-04-10 23:31:50,210 | INFO     | target-bigquery      | Target 'target-bigquery' is listening for input from tap. cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210410Z [info     ] 2024-04-10 23:31:50,210 | INFO     | target-bigquery      | Initializing 'target-bigquery' target sink... cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210480Z [info     ] 2024-04-10 23:31:50,210 | INFO     | target-bigquery      | Initializing target sink for stream 'customer'... cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210545Z [info     ] Traceback (most recent call last): cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210608Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/bin/target-bigquery", line 8, in <module> cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210678Z [info     ]     sys.exit(TargetBigQuery.cli()) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210740Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__ cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210855Z [info     ]     return self.main(*args, **kwargs) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.210922Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211026Z [info     ]     rv = self.invoke(ctx)      cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211088Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211196Z [info     ]     return ctx.invoke(self.callback, **ctx.params) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211261Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211322Z [info     ]     return __callback(*args, **kwargs) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211381Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 578, in cli cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211730Z [info     ]     target.listen(file_input)  cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.211865Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/io_base.py", line 34, in listen cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212017Z [info     ]     self._process_lines(file_input) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212109Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 278, in _process_lines cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212192Z [info     ]     counter = super()._process_lines(file_input) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212260Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/io_base.py", line 78, in _process_lines cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212324Z [info     ]     self._process_schema_message(line_dict) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212385Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 378, in _process_schema_message cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212446Z [info     ]     _ = self.get_sink(         cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212514Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/target.py", line 487, in get_sink cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212625Z [info     ]     return self.add_sink(stream_name, schema, key_properties) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212693Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 240, in add_sink cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212757Z [info     ]     sink = sink_class(         cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212817Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/storage_write.py", line 295, in __init__ cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212878Z [info     ]     super().__init__(target, stream_name, schema, key_properties) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.212949Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/core.py", line 294, in __init__ cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213053Z [info     ]     self.client = bigquery_client_factory(self._credentials) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr

emilie

04/11/2024, 6:48 AM

Copy code

2024-04-11T06:31:50.213119Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/core.py", line 609, in bigquery_client_factory cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213179Z [info     ]     json.loads(creds.json), project=creds.project cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213240Z [info     ]   File "/Users/myth/.pyenv/versions/3.10.13/lib/python3.10/json/__init__.py", line 346, in loads cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213335Z [info     ]     return _default_decoder.decode(s) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213402Z [info     ]   File "/Users/myth/.pyenv/versions/3.10.13/lib/python3.10/json/decoder.py", line 337, in decode cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213464Z [info     ]     obj, end = self.raw_decode(s, idx=_w(s, 0).end()) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213523Z [info     ]   File "/Users/myth/.pyenv/versions/3.10.13/lib/python3.10/json/decoder.py", line 355, in raw_decode cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213581Z [info     ]     raise JSONDecodeError("Expecting value", s, err.value) from None cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213642Z [info     ] json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.274583Z [debug    ] Deleted configuration at /Users/myth/Documents/Github/meltano_website/.meltano/run/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/target.762bbb01-3589-465d-bb99-088807758ab0.config.json
2024-04-11T06:31:50.274804Z [debug    ] Deleted configuration at /Users/myth/Documents/Github/meltano_website/.meltano/run/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/tap.71c8fcae-6c3b-4b5a-922f-495302a6b8d0.config.json
2024-04-11T06:31:50.274860Z [error    ] Loading failed                 code=1 message=json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) name=meltano run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery
2024-04-11T06:31:50.363902Z [debug    ] ELT could not be completed: Loader failed.
For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag.
Note that you can also check the generated log file at '/Users/myth/Documents/Github/meltano_website/.meltano/logs/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/elt.log'.
For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging>
Traceback (most recent call last):
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 418, in _run_elt
    await _run_extract_load(log, elt_context, output_logger)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 491, in _run_extract_load
    await singer_runner.run(
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/core/runner/singer.py", line 221, in run
    await self.invoke(
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/core/runner/singer.py", line 198, in invoke
    raise RunnerError("Loader failed", {PluginType.LOADERS: target_code})
meltano.core.runner.RunnerError: Loader failed

emilie

04/11/2024, 6:48 AM

Copy code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 397, in _redirect_output
    yield
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 429, in _run_elt
    raise CliError(
meltano.cli.utils.CliError: ELT could not be completed: Loader failed.
For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag.
Note that you can also check the generated log file at '/Users/myth/Documents/Github/meltano_website/.meltano/logs/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/elt.log'.
For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging>
2024-04-11T06:31:50.364173Z [info     ] ELT could not be completed: Loader failed. cmd_type=elt name=meltano run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.364278Z [info     ] For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag. cmd_type=elt name=meltano run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.364363Z [info     ] Note that you can also check the generated log file at '/Users/myth/Documents/Github/meltano_website/.meltano/logs/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/elt.log'. cmd_type=elt name=meltano run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.364441Z [info     ] For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging> cmd_type=elt name=meltano run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.713435Z [debug    ] Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

ELT could not be completed: Loader failed.
For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag.
Note that you can also check the generated log file at '/Users/myth/Documents/Github/meltano_website/.meltano/logs/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/elt.log'.
For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging>
Traceback (most recent call last):
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 418, in _run_elt
    await _run_extract_load(log, elt_context, output_logger)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 491, in _run_extract_load
    await singer_runner.run(
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/core/runner/singer.py", line 221, in run
    await self.invoke(
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/core/runner/singer.py", line 198, in invoke
    raise RunnerError("Loader failed", {PluginType.LOADERS: target_code})
meltano.core.runner.RunnerError: Loader failed

emilie

04/11/2024, 6:49 AM

Copy code

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/__init__.py", line 102, in _run_cli
    cli(obj={"project": None})
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/cli.py", line 46, in main
    return super().main(*args, windows_expand_args=False, **kwargs)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/utils.py", line 632, in invoke
    super().invoke(ctx)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/utils.py", line 681, in invoke
    super().invoke(ctx)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/params.py", line 27, in decorate
    return func(*args, **kwargs)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/params.py", line 75, in decorate
    func(project, *args, **kwargs)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/core/utils/__init__.py", line 74, in wrapper
    return asyncio.run(func(*args, **kwargs))
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()

emilie

04/11/2024, 6:49 AM

Copy code

File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 216, in elt
    await _run_el_command(
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 300, in _run_el_command
    raise err
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 297, in _run_el_command
    await _run_job(tracker, project, job, session, context_builder, force=force)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 380, in _run_job
    await _run_elt(tracker, log, context_builder, output_logger)
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/elt.py", line 429, in _run_elt
    raise CliError(
meltano.cli.utils.CliError: ELT could not be completed: Loader failed.
For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag.
Note that you can also check the generated log file at '/Users/myth/Documents/Github/meltano_website/.meltano/logs/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/elt.log'.
For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/myth/Documents/Github/meltano_snowflake/meltano_dev/lib/python3.9/site-packages/meltano/cli/__init__.py", line 112, in _run_cli
    raise CliError(f"{troubleshooting_message}\n{err}") from err
meltano.cli.utils.CliError: Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

ELT could not be completed: Loader failed.
For more detailed log messages re-run the command using 'meltano --log-level=debug ...' CLI flag.
Note that you can also check the generated log file at '/Users/myth/Documents/Github/meltano_website/.meltano/logs/elt/2024-04-11T063147--tap-rest-api-msdk--target-bigquery/88007baa-ed25-4f64-83af-1c55a88dc3c5/elt.log'.
For more information on debugging and logging: <https://docs.meltano.com/reference/command-line-interface#debugging>

Reuben (Matatika)

04/11/2024, 10:15 AM

$GCP_CREDS

valid JSON? Sounds exactly like https://meltano.slack.com/archives/C069CQNHDNF/p1707505392731909?thread_ts=1706937113.387929&cid=C069CQNHDNF (same

target-bigquery

line number in the stack trace as well).

Copy code

2024-04-11T06:31:50.213119Z [info     ]   File "/Users/myth/Documents/Github/meltano_website/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/core.py", line 609, in bigquery_client_factory cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr
2024-04-11T06:31:50.213179Z [info     ]     json.loads(creds.json), project=creds.project cmd_type=loader name=target-bigquery run_id=88007baa-ed25-4f64-83af-1c55a88dc3c5 state_id=2024-04-11T063147--tap-rest-api-msdk--target-bigquery stdio=stderr

👍 1

Reuben (Matatika)

04/11/2024, 10:16 AM

This was the solution last time: https://meltano.slack.com/archives/C069CQNHDNF/p1707426769623459?thread_ts=1706937113.387929&cid=C069CQNHDNF

Open in Slack

Previous Next