Hey all, I'm looking for those of us that have experience with the meltano postgres connectors! Im running into an error with

target-postgres

loader:

2025-02-09T06:54:37.214383Z [error    ] Block run completed.           block_type=ExtractLoadBlocks err=RunnerError('Loader failed') exit_codes={<PluginType.LOADERS: 'loaders'>: 1} set_number=0 success=False
Need help fixing this problem? Visit <http://melta.no/> for troubleshooting steps, or to
join our friendly Slack community.

Run invocation could not be completed as block failed: Loader failed

What doesn't make sense to me is that the config for both

tap-postgres

and

target-postgres

are the same but the loader seems to be failing. Has anyone run into this issue before? Happy to share more details!

What output do you get from

meltano invoke tap-postgres > tap.out

# this command
cat tap.out | meltano invoke target-postgres

?

When I run

meltano invoke tap-postgres > tap.out

the stream never ends (which I assume is the point of testing the connection). When I abort the connection and then run

cat tap.out | meltano invoke target-postgres

I run into the error:

Traceback (most recent call last):
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/bin/target-postgres", line 8, in <module>
    sys.exit(TargetPostgres.cli())
             ^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/plugin_base.py", line 92, in invoke
    return super().invoke(ctx)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 586, in invoke
    target.listen(file_input)
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/_singerlib/encoding/_base.py", line 53, in listen
    self._process_lines(file_input or self.default_input)
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 305, in _process_lines
    counter = super()._process_lines(file_input)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/_singerlib/encoding/_base.py", line 92, in _process_lines
    self._process_batch_message(line_dict)
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/target_base.py", line 484, in _process_batch_message
    sink.process_batch_files(
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/sinks/core.py", line 763, in process_batch_files
    context = {"records": [deserialize_json(line) for line in file]}
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/sinks/core.py", line 763, in <listcomp>
    context = {"records": [deserialize_json(line) for line in file]}
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/Documents/GitHub/etl/.meltano/loaders/target-postgres/venv/lib/python3.11/site-packages/singer_sdk/_singerlib/json.py", line 38, in deserialize_json
    return json.loads(  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/noamsiegel/.local/share/uv/python/cpython-3.11.6-macos-aarch64-none/lib/python3.11/json/__init__.py", line 341, in loads
    s = s.decode(detect_encoding(s), 'surrogatepass')
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Does this help?

Are you able to find the offending character in

tap.out

?

Stack trace indicates the target is processing

BATCH

messages, so it could be something to do with gzip: https://stackoverflow.com/a/44660123

Ah, what does your

meltano.yml

look like?