Hi all. I’m having a little trouble understanding what

parallelism

does in the transferwise targets. Is it a good way to speed up big EL jobs (currently from MySQL to Redsfhit).

The number of threads used to flush tables. 0 will create a thread for each stream, up to parallelism_max. -1 will create a thread for each CPU core. Any other positive number will create that number of threads, up to parallelism_max.

What is the “flush tables” step exactly? Right now my understanding is that: • the tap moves a cursor along the source, feeding rows into stdout • the target reads those from stdin until it reaches

max_batch_rows

(or the end of the data) and loads the data into the target Just trying to increase my understanding (and maybe speed up a big EL job I”m doing 🙂 )