Hi All I m having trouble getting any loader to work any hel Meltano #troubleshooting

Join Slack

Hi All — I’m having trouble getting any loader to ...

# troubleshooting

Tyler

08/01/2024, 11:46 PM

Hi All — I’m having trouble getting any loader to work any help would be greatly appreciated. Error log in thread.

Tyler

08/02/2024, 12:12 AM

Copy code

2024-08-01T23:29:01.587369Z [error    ] Loader failed
2024-08-01T23:29:01.588023Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Loader failed') exit_codes={<PluginType.LOADERS: 'loaders'>: 1} set_number=0 success=False
/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/structlog/stdlib.py:1098: UserWarning: Remove `format_exc_info` from your processor chain if you want pretty exceptions.
  ed = p(logger, meth_name, cast(EventDict, ed))
2024-08-01T23:29:01.592874Z [debug    ] Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Loader failed
Traceback (most recent call last):
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/run.py", line 222, in _run_blocks
    await blk.run()
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/block/extract_load.py", line 505, in run
    await self.run_with_job()
  File "/Users/admin/.pyenv/versions/3.11.0/lib/python3.11/contextlib.py", line 222, in __aexit__
    await self.gen.athrow(typ, value, traceback)
  File "/Users/admin/.pyenv/versions/3.11.0/lib/python3.11/contextlib.py", line 222, in __aexit__
    await self.gen.athrow(typ, value, traceback)
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/job/job.py", line 380, in _heartbeating
    yield
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/job/job.py", line 278, in run
    yield
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/block/extract_load.py", line 537, in run_with_job
    await self.execute()
  File "/Users/admin/.pyenv/versions/3.11.0/lib/python3.11/contextlib.py", line 222, in __aexit__
    await self.gen.athrow(typ, value, traceback)
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/block/extract_load.py", line 627, in _start_blocks
    yield
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/block/extract_load.py", line 497, in execute
    await manager.run()
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/block/extract_load.py", line 705, in run
    _check_exit_codes(
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/block/extract_load.py", line 877, in _check_exit_codes
    raise RunnerError("Loader failed", {PluginType.LOADERS: consumer_code})  # noqa: EM101
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
meltano.core.runner.RunnerError: Loader failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/__init__.py", line 103, in _run_cli
    cli(obj={"project": None})
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/cli.py", line 46, in main
    return super().main(*args, windows_expand_args=False, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/utils.py", line 662, in invoke
    super().invoke(ctx)  # type: ignore[misc]
    ^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/utils.py", line 711, in invoke
    super().invoke(ctx)
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/params.py", line 74, in decorate
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/params.py", line 164, in decorate
    func(project, *args, **kwargs)
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/core/utils/__init__.py", line 73, in wrapper
    return asyncio.run(func(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/lib/python3.11/asyncio/base_events.py", line 650, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/run.py", line 191, in run
    raise err
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/run.py", line 188, in run
    await _run_blocks(tracker, parsed_blocks, dry_run=dry_run)
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/run.py", line 234, in _run_blocks
    raise CliError(
meltano.cli.utils.CliError: Run invocation could not be completed as block failed: Loader failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/admin/.pyenv/versions/3.11.0/envs/meltano/lib/python3.11/site-packages/meltano/cli/__init__.py", line 113, in _run_cli
    raise CliError(f"{troubleshooting_message}\n{err}") from err  # noqa: EM102
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
meltano.cli.utils.CliError: Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Loader failed
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Loader failed

Edgar Ramírez (Arch.dev)

08/02/2024, 4:23 AM

Hi @Tyler. Are you using a custom extractor with these loaders?

Tyler

08/05/2024, 2:11 AM

@Edgar Ramírez (Arch.dev) Nope, the vanilla Shopify Extractor and Postgres Loader

Tyler

08/05/2024, 2:11 AM

Fresh env on py3.11.0

Edgar Ramírez (Arch.dev)

08/05/2024, 7:00 PM

And do you see the same error with other loaders? For example, target-jsonl? The above isn't the full log, so I'm trying to work out if something the tap emits makes the loader crash.

Tyler

08/06/2024, 6:48 PM

Yes

Tyler

08/06/2024, 6:48 PM

What’s the best means to generate the complete logfile?

Edgar Ramírez (Arch.dev)

08/06/2024, 8:21 PM

You might be able to see it under

.meltano/logs/elt/<state-id>/<run-id>

3 Views

Open in Slack

Previous Next