Hi! I’m trying out the new `meltano run` command a...
# troubleshooting
m
Hi! I’m trying out the new
meltano run
command and I’m getting this error:
Copy code
'SubprocessStreamProtocol' object has no attribute '_closed'
• I don’t have environments set up currently • I’m using
target-postgres
(pipelinewise) and
tap-mysql
(singer) • Works fine with etl FWIW. I think it’s the target that’s failing but am not sure
f
@matt_cooley sorry bunch of us have been traveling, so little to the party. Any chance you can send along the full exception/log?
m
No problem! Hope you got to enjoy Lisbon 🙂 ```2022-03-23T035422.383009Z [debug ] Deleted configuration at /opt/meltano/.meltano/run/tap-mysql/tap.f3cb2165-e227-4ffb-9276-524d205a263e.config.json 2022-03-23T035422.383883Z [debug ] Deleted configuration at /opt/meltano/.meltano/run/target-postgres/target.4daa2906-118d-476b-a50d-562be5def8c3.config.json 2022-03-23T035422.384867Z [debug ] 'SubprocessStreamProtocol' object has no attribute '_closed' Traceback (most recent call last): File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/cli/__init__.py", line 47, in main cli(obj={"project": None}) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/click/core.py", line 829, in call return self.main(*args, **kwargs) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/cli/params.py", line 23, in decorate return func(*args, **kwargs) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/cli/params.py", line 56, in decorate func(project, *args, **kwargs) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/utils/__init__.py", line 71, in wrapper return run_async(func(*args, **kwargs)) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/utils/__init__.py", line 50, in run_async loop.run_until_complete(future) File "/usr/local/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete return future.result() File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/cli/run.py", line 84, in run await _run_blocks(parsed_blocks) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/cli/run.py", line 93, in _run_blocks await blk.run() File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/block/extract_load.py", line 406, in run await self.execute() File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/block/extract_load.py", line 396, in execute await manager.run() File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/block/extract_load.py", line 591, in run await self._wait_for_process_completion(self.elb.head) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/block/extract_load.py", line 670, in _wait_for_process_completion await self._handle_head_completed(current_head, start_idx) File "/home/analytics/.cache/pypoetry/virtualenvs/corsair-meltano-8vM4tV9J-py3.7/lib/python3.7/site-packages/meltano/core/block/extract_load.py", line 697, in _handle_head_completed await next_head.stdin.wait_cl…
f
Huh interesting, wonder if theres a race, looks like the target has already stopped or is in the process of stopping (based on that Deleted configuration at log line already being present), or something a long those lines. I pop'd this issue to take a look https://gitlab.com/meltano/meltano/-/issues/3347 and track it down. Can you tell if the target got any data at all ?
Well thats interesting, I couldn't reproduce this on 3.8 but noticed you where using 3.7. https://docs.python.org/3.7/library/asyncio-stream.html#asyncio.StreamWriter.wait_closed is present in 3.7 but seems to fail (with that that error).
I think this doesn't popup in
meltano elt
because its actually suppressing that exception as a workaround for supporting python 3.6 - https://gitlab.com/meltano/meltano/-/blob/master/src/meltano/core/runner/singer.py#L189
m
Interesting. Should I report this as a bug then? Or are you not planning on supporting this feature for 3.7?
f
Yep. We'll definitely get it fixed up and working with 3.7 (probably will make it out in next weeks release).
m
thanks so much!