Hello, I have configured a pipeline 1.Source -tap-mongodb 2. Target- target-snowflake when i execute this

meltano elt tap-mongodb target-snowflake

I get the following error:

n = os.write(self._fileno, self._buffer)

BrokenPipeError: [Errno 32] Broken pipe

If you want help debugging, we would need more of the error than that.

how can i get

what have you tried so far?

I tried checking for access related issues, schema config. I tried dumping data from tap-mongodb - targetjsonl it worked fine.

So, I see that there is a warning message:

No columns discovered in the schema "ANALYSIS.MELTANO_ASHISH"

- does that mean anything to you? Otherwise, maybe it’s an issue with your config. It seems like you have enabled

add_metadata_columns

- is that correct? Does it work without that?

I tried disabling

add_metadata_columns

as well. it didn’t work. I got Brokenpipe error

The broken pipe error just means that the target failed for some reason, you need to look at the logs to see why it failed. Does the exception traceback without

add_metadata_columns

show the same

KeyError: 'properties'

error or something else?

Error looks something like this ```2022-05-02T143541.291593Z [info ] invalid identifier '_ID' cmd_type=loader job_id=mongo-to-snowflake name=target-snowflake run_id=528c60fa-5fc4-4836-b888-08ba97c5e8bd stdio=stderr 2022-05-02T143541.401337Z [error ] [Errno 32] Broken pipe Traceback (most recent call last): File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/logging/output_logger.py", line 127, in redirect_logging yield File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/cli/elt.py", line 234, in _redirect_output yield File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/cli/elt.py", line 248, in _run_elt await _run_extract_load(log, elt_context, output_logger) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/cli/elt.py", line 307, in _run_extract_load await singer_runner.run( File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/runner/singer.py", line 224, in run await self.invoke( File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/runner/singer.py", line 190, in invoke await p_target.stdin.wait_closed() File "/Users/ashishkumar/opt/anaconda3/lib/python3.9/asyncio/streams.py", line 359, in wait_closed await self._protocol._get_close_waiter(self) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/logging/utils.py", line 210, in capture_subprocess_output if not await _write_line_writer(writer, line): File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/logging/utils.py", line 179, in _write_line_writer await writer.wait_closed() File "/Users/ashishkumar/opt/anaconda3/lib/python3.9/asyncio/streams.py", line 359, in wait_closed await self._protocol._get_close_waiter(self) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/logging/utils.py", line 176, in _write_line_writer await writer.drain() File "/Users/ashishkumar/opt/anaconda3/lib/python3.9/asyncio/streams.py", line 387, in drain await self._protocol._drain_helper() File "/Users/ashishkumar/opt/anaconda3/lib/python3.9/asyncio/streams.py", line 197, in _drain_helper await waiter File "/Users/ashishkumar/opt/anaconda3/lib/python3.9/asyncio/unix_events.py", line 687, in _write_ready n = os.write(self._fileno, self._buffer) BrokenPipeError: [Errno 32] Broken pipe 2022-05-02T143541.443264Z [error ] Task exception was never retrieved future: <Task finished name='Task-10' coro=<capture_subprocess_output() done, defined at /Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/logging/utils.py:188> exception=BrokenPipeError(32, 'Broken pipe')> Traceback (most recent call last): File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/cli/params.py", line 23, in decorate return func(*args, **kwargs) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/cli/params.py", line 56, in decorate func(project, *args, **kwargs) File "/Users/ashishkumar/Desktop/ELT_MEL/meltano/lib/python3.9/site-packages/meltano/core/utils/_…

seems worth a try

Yes it is not supported.

I assume so, but I don’t know what the mechanism would be. I’d take a look at meltano mappers to see if there is one.

Sure do let me know if you get one.