https://linen.dev logo
#singer-tap-development
Title
# singer-tap-development
a

avy_faingezicht

10/19/2023, 11:55 PM
Related to the conversation above, does anyone have examples of adding parallelism/multiprocessing on the tap side of things? I have a script that uses multiprocessing today to retrieve 100s of thousands/millions of records per run, and pagination only allows me to get 1k at a time. Having a way to parallelize this in a tap would be awesome.