Hi what’s the best way to change meltano orchestra...
# announcements
k
Hi what’s the best way to change meltano orchestrator to LocalExecutor rather than Sequential? I have a postgres instance connected to the airflow orchestrator already. I have tried changing it in airflow.cfg before running initdb but it kept changing it back to SequentialExecutor as well as having
AIRFLOW___CORE___EXECUTOR
env var
d
You can either directly set the
AIRFLOW__CORE__EXECUTOR
env var that Airflow is reading from, or run
meltano config airflow set core.executor LocalExecutor
to write
core.executor: LocalExecutor
in
meltano.yml
and have it show up in
meltano config airflow
, which is used to populate
airflow.cfg
!
This would be an example of a custom setting: https://meltano.com/docs/configuration.html#custom-settings
k
oh weird, I set the env var
AIRFLOW__CORE__EXECUTOR
so many times and it never worked…
meltano config airflow set core.executor LocalExecutor
while this worked however
d
@keat Hmm, that's odd! When I run
AIRFLOW__CORE__EXECUTOR=foo meltano --log-level=debug invoke airflow --help
, I see it getting passed along and picked up by Airflow (which understandably fails because that executor doesn't exist), and it works the same way when I put
AIRFLOW__CORE__EXECUTOR=foo
in
.env
. Can you verify that this is not working for you locally? Are you on the latest version?
e
@monika_rajput, @connor_flynn, and @jo_pearson here is a good thread on setting up executor