Is there any way to run table loading in parallel ...
# getting-started
a
Is there any way to run table loading in parallel . for example. If under tp-mysql i have 8 tables configured and target is also same. So, is there any way to run ingestion of 4 tables at once in parallel instead of running them one by one ?
j
At this time, doing this the way you’re thinking, no. In order to get things to run in parallel, you have a couple options: 1. use an external orchestrator to run each tap.entity individually and in parallel 2. Setup your
meltano.yml
to have multiple tap-mysql’s inheriting from the parent but each one with a different set of
select
patterns that target a single table. Then you can run each tap/elt job in parallel.