Anyone know if `meltano elt` is slated to gain map...
# troubleshooting
c
Anyone know if
meltano elt
is slated to gain mapper support in the future? I almost never use
meltano run
a
We likely will add it, but it isn't on our immediate roadmap. The two long-term items that would drive that: 1. We may refactor
meltano elt
as simply
meltano el
, at which time we'd also add mapper support. 2. We may create the option for localized and 'default' mappings per tap or target (versus today all mappings are globally available and not enabled by default). Either of these would likely bring with them the mapper support, but these won't be prioritized for a little while. The best near-term option would be to use
meltano run
and we can help out if you run into any issues when migrating. For instance,
meltano run
manages your state ID for you, but it does require a valid Meltano Environment to run.
c
Thanks for that info, AJ! I do have a Meltano Environment (don't use the internal option) and am specifying the state ID in my Airflow tasks. Only reason I'm using
elt
is to bypass running streams sequentially and run them in parallel instead 😄
a
Nice!
What degree of parallelism, out of curiosity?
c
My bad, parallelism concurrency is set to 15 for me.
a
Nice!
A common strategy is to break jobs into 3-4
select
groups, but that won't get you near 15.