Hi everyone, I'm trying to resolve a BrokenPipeErr...
# troubleshooting
d
Hi everyone, I'm trying to resolve a BrokenPipeError that happens without any associated error message. At least, I have noticed that the failure occurs directly after a specific collection (stream) from the source (
tap-mongodb
) is mentioned in the logs. Using the troubleshooting guide, I have established that the problem occurs in the loader (
target-postgres
). However, without an error message, I am at a loss for how to proceed further. Looking at the data from the specific collection did not reveal any obvious problem. Do you have any suggestions for further steps? Maybe there is a way to get at an error message? Of note: - I'm using the
transferwise
variants for tap and target. - Strangely, running the pipeline with only the specific collection selected works. Thanks in advance!
It seems this was due to running out of memory; setting flush_all_streams=true in the loader config solved the issue.
a
@dominik_ruegger Hi I am having this same issue. How did you set
flush_all_streams=true
in the config?