I have a loader `target-bigquery` that is failing ...
# troubleshootinge
emilie
02/03/2024, 5:11 AMI have a loader
target-bigqueryRun invocation could not be completed as block failed: Loader failedemilie
02/03/2024, 5:12 AMI'm using Python 3.10.13
emilie
02/03/2024, 5:12 AMrelevant parts of the `meltano.yml`:
emilie
02/03/2024, 5:13 AM Copy code
loaders:
- name: target-jsonl
variant: andyh1203
pip_url: target-jsonl
- name: target-bigquery
variant: z3z1ma
pip_url: git+<https://github.com/z3z1ma/target-bigquery.git>
config:
credentials_json: ${{ secrets.EXTRACTOR_BIGQUERY__CREDS_JSON }}
dataset: <redacted>
project: <redacted>
upsert: trueemilie
02/03/2024, 5:15 AMrelevant parts of the logs
emilie
02/03/2024, 5:17 AM Copy code
2024-02-03T05:10:13.815748Z [error ] Loader failed
2024-02-03T05:10:13.815911Z [error ] Block run completed. block_type=ExtractLoadBlocks err=RunnerError('Loader failed') exit_codes={<PluginType.LOADERS: 'loaders'>: 1} set_number=0 success=False
2024-02-03T05:10:13.816828Z [debug ] Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.
Run invocation could not be completed as block failed: Loader failed
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/run.py:180 in _run_blocks │
│ │
│ 177 │ │ │ continue │
│ 178 │ │ │
│ 179 │ │ try: │
│ ❱ 180 │ │ │ await blk.run() │
│ 181 │ │ except RunnerError as err: │
│ 182 │ │ │ logger.error( │
│ 183 │ │ │ │ "Block run completed.", │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/core/block/extract_load.py:462 in run │
│ │
│ 459 │ │ │ # TODO: legacy `meltano elt` style logging should be deprecated │
│ 460 │ │ │ legacy_log_handler = self.output_logger.out("meltano", logger) │
│ 461 │ │ │ with legacy_log_handler.redirect_logging(): │
│ ❱ 462 │ │ │ │ await self.run_with_job() │
│ 463 │ │ │ │ return │
│ 464 │ │ else: │
│ 465 │ │ │ logger.warning( │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/core/block/extract_load.py:494 in run_with_job │
│ │
│ 491 │ │ │
│ 492 │ │ with closing(self.context.session) as session: │
│ 493 │ │ │ async with job.run(session): │
│ ❱ 494 │ │ │ │ await self.execute() │
│ 495 │ │
│ 496 │ async def terminate(self, graceful: bool = False) -> None: │
│ 497 │ │ """Terminate an in flight ExtractLoad execution, potentially disruptive. │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/core/block/extract_load.py:454 in execute │
│ │
│ 451 │ │ async with self._start_blocks(): │
│ 452 │ │ │ await self._link_io() │
│ 453 │ │ │ manager = ELBExecutionManager(self) │
│ ❱ 454 │ │ │ await manager.run() │
│ 455 │ │
│ 456 │ async def run(self) -> None: │
│ 457 │ │ """Run the ELT task.""" │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/core/block/extract_load.py:660 in run │
│ │
│ 657 │ │ any of the blocks exit with a non 0 exit code. │
│ 658 │ │ """ │
│ 659 │ │ await self._wait_for_process_completion(self.elb.head) │
│ ❱ 660 │ │ _check_exit_codes( │
│ 661 │ │ │ self._producer_code, │
│ 662 │ │ │ self._consumer_code, │
│ 663 │ │ │ self._intermediate_codes, │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/core/block/extract_load.py:831 in _check_exit_codes │
│ │
│ 828 │ │ raise RunnerError("Extractor failed", {PluginType.EXTRACTORS: producer_code}) │
│ 829 │ │
│ 830 │ if consumer_code: │
│ ❱ 831 │ │ raise RunnerError("Loader failed", {PluginType.LOADERS: consumer_code}) │
│ 832 │ │
│ 833 │ if failed_mappers := [ │
│ 834 │ │ {mapper_id: exit_code} │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯emilie
02/03/2024, 5:17 AM Copy code
RunnerError: Loader failed
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/__init__.py:102 in _run_cli │
│ │
│ 99 │ """ │
│ 100 │ try: │
│ 101 │ │ try: # noqa: WPS225, WPS505 │
│ ❱ 102 │ │ │ cli(obj={"project": None}) │
│ 103 │ │ except ProjectReadonly as err: │
│ 104 │ │ │ raise CliError( │
│ 105 │ │ │ │ f"The requested action could not be completed: {err}", │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/click/core.py:1157 in __call__ │
│ │
│ 1154 │ │
│ 1155 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │
│ 1156 │ │ """Alias for :meth:`main`.""" │
│ ❱ 1157 │ │ return self.main(*args, **kwargs) │
│ 1158 │
│ 1159 │
│ 1160 class Command(BaseCommand): │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/cli.py:46 in main │
│ │
│ 43 │ │ │ args: Positional arguments for the Click group. │
│ 44 │ │ │ kwargs: Keyword arguments for the Click group. │
│ 45 │ │ """ │
│ ❱ 46 │ │ return super().main(*args, windows_expand_args=False, **kwargs) │
│ 47 │
│ 48 │
│ 49 @click.group( │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/click/core.py:1078 in main │
│ │
│ 1075 │ │ try: │
│ 1076 │ │ │ try: │
│ 1077 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │
│ ❱ 1078 │ │ │ │ │ rv = self.invoke(ctx) │
│ 1079 │ │ │ │ │ if not standalone_mode: │
│ 1080 │ │ │ │ │ │ return rv │
│ 1081 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/utils.py:632 in invoke │
│ │
│ 629 │ │ enact_environment_behavior(self.environment_behavior, ctx) │
│ 630 │ │ if ctx.obj.get("tracker"): │
│ 631 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx)) │
│ ❱ 632 │ │ super().invoke(ctx) │
│ 633 │
│ 634 │
│ 635 class InstrumentedDefaultGroup(InstrumentedGroupMixin, DefaultGroup, DYMGroup): │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/click/core.py:1688 in invoke │
│ │emilie
02/03/2024, 5:18 AM Copy code
│ 1685 │ │ │ │ super().invoke(ctx) │
│ 1686 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │
│ 1687 │ │ │ │ with sub_ctx: │
│ ❱ 1688 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │
│ 1689 │ │ │
│ 1690 │ │ # In chain mode we create the contexts step by step, but after the │
│ 1691 │ │ # base command has been invoked. Because at that point we do not │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/utils.py:681 in invoke │
│ │
│ 678 │ │ if ctx.obj.get("tracker"): │
│ 679 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx)) │
│ 680 │ │ │ ctx.obj["tracker"].track_command_event(CliEvent.started) │
│ ❱ 681 │ │ super().invoke(ctx) │
│ 682 │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/click/core.py:1434 in invoke │
│ │
│ 1431 │ │ │ echo(style(message, fg="red"), err=True) │
│ 1432 │ │ │
│ 1433 │ │ if self.callback is not None: │
│ ❱ 1434 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ 1435 │ │
│ 1436 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]: │
│ 1437 │ │ """Return a list of completions for the incomplete value. Looks │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/click/core.py:783 in invoke │
│ │
│ 780 │ │ │
│ 781 │ │ with augment_usage_errors(__self): │
│ 782 │ │ │ with ctx: │
│ ❱ 783 │ │ │ │ return __callback(*args, **kwargs) │
│ 784 │ │
│ 785 │ def forward( │
│ 786 │ │ __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any # noqa: B902 │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/params.py:27 in decorate │
│ │
│ 24 │ │ if database_uri: │
│ 25 │ │ │ ProjectSettingsService.config_override["database_uri"] = database_uri │
│ 26 │ │ │
│ ❱ 27 │ │ return func(*args, **kwargs) │
│ 28 │ │
│ 29 │ return functools.update_wrapper(decorate, func) │
│ 30 │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/params.py:75 in decorate │
│ │
│ 72 │ │ │ │ except MigrationError as err: │
│ 73 │ │ │ │ │ raise CliError(str(err)) from err │
│ 74 │ │ │ │
│ ❱ 75 │ │ │ func(project, *args, **kwargs) │
│ 76 │ │ │
│ 77 │ │ return functools.update_wrapper(decorate, func) │
│ 78 │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/click/decorators.py:33 in new_func │
│ │
│ 30 │ """ │
│ 31 │ │
│ 32 │ def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R": │
│ ❱ 33 │ │ return f(get_current_context(), *args, **kwargs) │
│ 34 │ │
│ 35 │ return update_wrapper(new_func, f) │
│ 36 │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/core/utils/__init__.py:74 in wrapper │
│ │
│ 71 │ │
│ 72 │ @functools.wraps(func) │
│ 73 │ def wrapper(*args, **kwargs): │
│ ❱ 74 │ │ return asyncio.run(func(*args, **kwargs)) │
│ 75 │ │
│ 76 │ return wrapper │
│ 77 │
│ │emilie
02/03/2024, 5:18 AM Copy code
│ /Users/myth/.pyenv/versions/3.10.13/lib/python3.10/asyncio/runners.py:44 in run │
│ │
│ 41 │ │ events.set_event_loop(loop) │
│ 42 │ │ if debug is not None: │
│ 43 │ │ │ loop.set_debug(debug) │
│ ❱ 44 │ │ return loop.run_until_complete(main) │
│ 45 │ finally: │
│ 46 │ │ try: │
│ 47 │ │ │ _cancel_all_tasks(loop) │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/lib/python3.10/asyncio/base_events.py:649 in │
│ run_until_complete │
│ │
│ 646 │ │ if not future.done(): │
│ 647 │ │ │ raise RuntimeError('Event loop stopped before Future completed.') │
│ 648 │ │ │
│ ❱ 649 │ │ return future.result() │
│ 650 │ │
│ 651 │ def stop(self): │
│ 652 │ │ """Stop running the event loop. │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/run.py:149 in run │
│ │
│ 146 │ │ await _run_blocks(tracker, parsed_blocks, dry_run=dry_run) │
│ 147 │ except Exception as err: │
│ 148 │ │ tracker.track_command_event(CliEvent.failed) │
│ ❱ 149 │ │ raise err │
│ 150 │ tracker.track_command_event(CliEvent.completed) │
│ 151 │
│ 152 │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/run.py:146 in run │
│ │
│ 143 │ │ tracker.track_command_event(CliEvent.aborted) │
│ 144 │ │ raise CliError("Some ExtractLoadBlocks set failed validation.") │
│ 145 │ try: │
│ ❱ 146 │ │ await _run_blocks(tracker, parsed_blocks, dry_run=dry_run) │
│ 147 │ except Exception as err: │
│ 148 │ │ tracker.track_command_event(CliEvent.failed) │
│ 149 │ │ raise err │
│ │
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/run.py:192 in _run_blocks │
│ │
│ 189 │ │ │ ) │
│ 190 │ │ │ with tracker.with_contexts(tracking_ctx): │
│ 191 │ │ │ │ tracker.track_block_event(blk_name, BlockEvents.failed) │
│ ❱ 192 │ │ │ raise CliError( │
│ 193 │ │ │ │ f"Run invocation could not be completed as block failed: {err}", │
│ 194 │ │ │ ) from err │
│ 195 │ │ except Exception as bare_err: │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯emilie
02/03/2024, 5:18 AM Copy code
CliError: Run invocation could not be completed as block failed: Loader failed
The above exception was the direct cause of the following exception:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/myth/.pyenv/versions/3.10.13/envs/python_meltano_development_3_10/lib/python3.10/site-pac │
│ kages/meltano/cli/__init__.py:112 in _run_cli │
│ │
│ 109 │ │ except MeltanoError as err: │
│ 110 │ │ │ handle_meltano_error(err) │
│ 111 │ │ except Exception as err: │
│ ❱ 112 │ │ │ raise CliError(f"{troubleshooting_message}\n{err}") from err │
│ 113 │ except CliError as cli_error: │
│ 114 │ │ cli_error.print() │
│ 115 │ │ sys.exit(1) │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CliError: Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.
Run invocation could not be completed as block failed: Loader failed
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.
Run invocation could not be completed as block failed: Loader failedemilie
02/03/2024, 5:19 AMI am not sure how to interpret these logs to understand the failure. Can anyone help?
r
Reuben (Matatika)
02/05/2024, 11:30 AMSometimes plugin-specific errors can be obscured by or just hard to spot in the log output of
meltano run Copy code
meltano invoke <tap> > tap.out Copy code
cat tap.out | meltano invoke target-bigquerytarget-bigquery👍 1
e
emilie
02/08/2024, 8:15 PMThanks! I was able to get back to this. When I run the first command,
meltano invoke target-bigquery > tap.outemilie
02/08/2024, 8:17 PM Copy code
2024-02-08T20:13:17.333362Z [debug ] meltano 3.2.0, Darwin
2024-02-08T20:13:17.338399Z [debug ] /etc/localtime found
2024-02-08T20:13:17.338950Z [debug ] 1 found:
{'/etc/localtime is a symlink to': 'America/Phoenix'}
2024-02-08T20:13:17.341654Z [info ] Environment 'dev' is active
2024-02-08T20:13:17.350790Z [debug ] Creating DB engine for project at '/Users/myth/Documents/Github/meltano_something_local' with DB URI '<sqlite://Users/myth/Documents/Github/meltano_something_local/.meltano/meltano.db>'
2024-02-08T20:13:17.383299Z [debug ] Found plugin parent parent=target-bigquery plugin=target-bigquery source=<DefinitionSource.LOCKFILE: 4>
2024-02-08T20:13:17.509134Z [debug ] Created configuration at /Users/myth/Documents/Github/meltano_something_local/.meltano/run/target-bigquery/target.21390e56-74e1-43fc-a86c-21c396053845.config.json
2024-02-08T20:13:17.511253Z [debug ] Invoking: ['/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/bin/target-bigquery', '--config', '/Users/myth/Documents/Github/meltano_something_local/.meltano/run/target-bigquery/target.21390e56-74e1-43fc-a86c-21c396053845.config.json']
2024-02-08 13:13:18,239 | INFO | target-bigquery | Using thread-based parallelism
2024-02-08 13:13:18,239 | INFO | target-bigquery | Target 'target-bigquery' is listening for input from tap.r
Reuben (Matatika)
02/08/2024, 8:17 PMYou run that first command with the tap, not the target.
e
emilie
02/08/2024, 8:17 PMoh, thanks, let me try that.
👍 1
emilie
02/08/2024, 8:23 PMOkay, this is what I get from the second command
emilie
02/08/2024, 8:23 PM Copy code
2024-02-08T20:20:28.950449Z [debug ] meltano 3.2.0, Darwin
2024-02-08T20:20:28.955375Z [debug ] /etc/localtime found
2024-02-08T20:20:28.955857Z [debug ] 1 found:
{'/etc/localtime is a symlink to': 'America/Phoenix'}
2024-02-08T20:20:28.958386Z [info ] Environment 'dev' is active
2024-02-08T20:20:28.966637Z [debug ] Creating DB engine for project at '/Users/myth/Documents/Github/meltano_something_local' with DB URI '<sqlite://Users/myth/Documents/Github/meltano_something_local/.meltano/meltano.db>'
2024-02-08T20:20:28.992679Z [debug ] Found plugin parent parent=target-bigquery plugin=target-bigquery source=<DefinitionSource.LOCKFILE: 4>
2024-02-08T20:20:29.108409Z [debug ] Created configuration at /Users/myth/Documents/Github/meltano_something_local/.meltano/run/target-bigquery/target.ef721061-4c48-431a-afb3-bcd00c30c78f.config.json
2024-02-08T20:20:29.109960Z [debug ] Invoking: ['/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/bin/target-bigquery', '--config', '/Users/myth/Documents/Github/meltano_something_local/.meltano/run/target-bigquery/target.ef721061-4c48-431a-afb3-bcd00c30c78f.config.json']
2024-02-08 13:20:29,761 | INFO | target-bigquery | Using thread-based parallelism
2024-02-08 13:20:29,761 | INFO | target-bigquery | Target 'target-bigquery' is listening for input from tap.
2024-02-08 13:20:29,761 | INFO | target-bigquery | Initializing 'target-bigquery' target sink...
2024-02-08 13:20:29,761 | INFO | target-bigquery | Initializing target sink for stream 'somethingone-appointment'...
Traceback (most recent call last):
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/bin/target-bigquery", line 8, in <module>
sys.exit(TargetBigQuery.cli())
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 578, in cli
target.listen(file_input)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/io_base.py", line 34, in listen
self._process_lines(file_input)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 278, in _process_lines
counter = super()._process_lines(file_input)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/io_base.py", line 78, in _process_lines
self._process_schema_message(line_dict)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 378, in _process_schema_message
_ = self.get_sink(
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/target.py", line 487, in get_sink
return self.add_sink(stream_name, schema, key_properties)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/singer_sdk/target_base.py", line 240, in add_sink
sink = sink_class(
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/storage_write.py", line 295, in __init__
super().__init__(target, stream_name, schema, key_properties)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/core.py", line 294, in __init__
self.client = bigquery_client_factory(self._credentials)
File "/Users/myth/Documents/Github/meltano_something_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/core.py", line 609, in bigquery_client_factory
json.loads(creds.json), project=creds.project
File "/Users/myth/.pyenv/versions/3.10.13/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/Users/myth/.pyenv/versions/3.10.13/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/myth/.pyenv/versions/3.10.13/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
2024-02-08T20:20:29.829286Z [debug ] Deleted configuration at /Users/myth/Documents/Github/meltano_something_local/.meltano/run/target-bigquery/target.ef721061-4c48-431a-afb3-bcd00c30c78f.config.jsone
Edgar Ramírez (Arch.dev)
02/08/2024, 8:32 PMProbably worth visually inspecting the contents of
tap.out➕ 1
r
Reuben (Matatika)
02/08/2024, 8:45 PMIs this a custom tap? @emilie
e
emilie
02/08/2024, 8:46 PMNo, it is tap-snowflake
emilie
02/08/2024, 8:46 PMInvoking the tap works fine, as does using with target-jsonl
emilie
02/08/2024, 8:47 PMLooking at the results of
tap.outr
Reuben (Matatika)
02/08/2024, 9:12 PMAh, I think I see the problem. The target is trying to parse the value you provided for
credentials_json${{ secrets.EXTRACTOR_BIGQUERY__CREDS_JSON }}credentials_path👍 1
e
emilie
02/08/2024, 9:13 PMWow thanks for this. I have to do something else atm, but will update when I can.
emilie
02/09/2024, 7:02 PMYou were right! I set the
credentials_jsonmeltano config target-bigquery set --interactive👍 1
emilie
02/09/2024, 7:03 PMThank you for helping me on this. Were you able to work that out from this line error response?
emilie
02/09/2024, 7:03 PM Copy code
File "/Users/myth/Documents/Github/meltano_athena_local/.meltano/loaders/target-bigquery/venv/lib/python3.10/site-packages/target_bigquery/core.py", line 609, in bigquery_client_factory
json.loads(creds.json), project=creds.projectr
Reuben (Matatika)
02/09/2024, 7:05 PMYep, that's the one.
✅ 1
2 Views