https://linen.dev logo
Join Slack
Powered by
Hello! I'm trying to deploy airflow webserver and ...
# troubleshooting
t
Hello! I'm trying to deploy airflow webserver and getting the below error
Copy code
/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py:525: DeprecationWarning: The sql_alchemy_conn option in [core] has been moved to the sql_alchemy_conn option in [database] - the old setting has been used, but please update your config.
  option = self._get_environment_variables(deprecated_key, deprecated_section, key, section)
Traceback (most recent call last):
  File "/project/.meltano/orchestrators/airflow/venv/bin/airflow", line 5, in <module>
    from airflow.__main__ import main
  File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/__init__.py", line 35, in <module>
    from airflow import settings
  File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/settings.py", line 35, in <module>
    from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf  # NOQA F401
  File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 1601, in <module>
    conf.validate()
  File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 314, in validate
    self._validate_config_dependencies()
  File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 408, in _validate_config_dependencies
    raise AirflowConfigException(f"error: cannot use sqlite with the {self.get('core', 'executor')}")
airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the LocalExecutor

Command `airflow --help` failed
I'm using 
postresql DB
and not 
sqlite
. Is there a new configuration that I'm missing? I'm using meltano 
v2.0.2.
Was getting same error on meltano 
v2.0.1
but it was working ok on meltano 
v1.105.0
s
👀 I second this problem!
d
So this is an airflow thing, did you config your executor to localexecutor instead of sequential?
sqllite only allows one connection at a time (one of the major benefits of setting up an external db is multi connection capability needed for other executors)
if you wanna use local, gotta set up a real db
otherwise
Copy code
meltano config airflow set core executor SequentialExecutor
t
That's what I have
Copy code
- name: "AIRFLOW__CORE__SQL_ALCHEMY_CONN"
  value: "postgresql://:@:5432/airflow?sslmode=disable"
- name: AIRFLOW__CORE__EXECUTOR
  value: LocalExecutor
p
Idk if its helpful or not but theres also https://docs.meltano.com/guide/production#airflow-orchestrator which looks like it touches on the same topic
t
👆that's what I followed, as I mentioned it worked on 
v1.105.0
. Probably something changed on 
v2
d
so at the top of the traceback im noticing a depreciation of that env variable lets try setting this
Copy code
meltano config airflow set database.sql_alchemy_conn yourpostgresuri
Looking at my airflow variables we set 
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
not 
AIRFLOW__CORE
t
ahh that might be the issue. I'll try that
Thank you @devon_seitz! Switching to
Copy code
- name: "AIRFLOW__DATABASE__SQL_ALCHEMY_CONN"
  value: "postgresql://:@:5432/airflow?sslmode=disable"
- name: AIRFLOW__CORE__EXECUTOR
  value: LocalExecutor
worked
d
😄 awesome!
s
Would this be in the changelog for 2.0? I'd love to include it in my PR
d
its likely that meltano2 uses a newer version of airflow2 instead of meltano1 using airflow~1ish
i am not familiar with how poetry works but thats where i would look for the change (airflow version)
s
Yeah my next step is upgrading the airflow version to 2.0, I'm just scared shitless of breaking our production DB
Sadly this fix does not seem to have worked for me 😞 Here is a copy of my logs: ```15/06/2022, 113218 [2m2022-06-15T153218.287237Z[0m [[32m[1mdebug [0m] [1mDeleted configuration at /project/.meltano/run/airflow/airflow.cfg[0m airflow-scheduler 15/06/2022, 113218 [2m2022-06-15T153218.287889Z[0m [[31m[1merror [0m] [1mTraceback (most recent call last): airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/bin/airflow", line 25, in <module> airflow-scheduler 15/06/2022, 113218 from airflow.configuration import conf airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/__init__.py", line 31, in <module> airflow-scheduler 15/06/2022, 113218 from airflow.utils.log.logging_mixin import LoggingMixin airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/utils/__init__.py", line 24, in <module> airflow-scheduler 15/06/2022, 113218 from .decorators import apply_defaults as _apply_defaults airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/utils/decorators.py", line 36, in <module> airflow-scheduler 15/06/2022, 113218 from airflow import settings airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/settings.py", line 38, in <module> airflow-scheduler 15/06/2022, 113218 from airflow.configuration import conf, AIRFLOW_HOME, WEBSERVER_CONFIG # NOQA F401 airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 737, in <module> airflow-scheduler 15/06/2022, 113218 conf.read(AIRFLOW_CONFIG) airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 425, in read airflow-scheduler 15/06/2022, 113218 self._validate() airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 217, in _validate airflow-scheduler 15/06/2022, 113218 self._validate_config_dependencies() airflow-scheduler 15/06/2022, 113218 File "/project/.meltano/orchestrators/airflow/venv/lib/python3.8/site-packages/airflow/configuration.py", line 249, in _validate_config_dependencies airflow-scheduler 15/06/2022, 113218 raise AirflowConfigException( airflow-scheduler 15/06/2022, 113218 airflow.exceptions.AirflowConfigException: error: cannot use sqlite with the LocalExecutor airflow-scheduler 15/06/2022, 113218 [0m airflow-scheduler 15/06/2022, 113218 [2m2022-06-15T153218.289815Z[0m [[32m[1mdebug [0m] [1mCommand 
airflow --help
failed[0m airflow-scheduler 15/06/2022, 113218 Traceback (most recent call last): airflow-scheduler 15/06/2022, 113218 File "/usr/local/lib/python3.8/site-packages/meltano/cli/__init__.py", line 55, in _run_cli airflow-scheduler 15/06/2022, 113218 cli(obj={"project": None}) airflow-scheduler 15/06/2022, 113218 File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in call airflow-scheduler 15/06/2022, 113218 return self.main(*args, **kwargs) airflow-scheduler 15/06/2022, 113218 File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main airflow-scheduler 15/06/2022, 113218 rv = self.invoke(ctx) airflow-scheduler 15/06/2022, 113218 File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke airflow-scheduler 15/06/2022, 113218 return _process_result(sub_ctx.command.invoke(sub_ctx)) airflow-scheduler 15/06/2022, 113218 File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke airflow-scheduler 15/06/2022, 113218 return ctx.invoke(self.callback, **ctx.params) airflow-scheduler 15/06…
Would this have to do with the fact that I'm using airflow 1.10.15?
t
probably, I'm using the latest version 
v2.3.2
.
Copy code
orchestrators:
  - name: airflow
    variant: apache
    pip_url: psycopg2 apache-airflow
s
Alright, this is going to be interesting
t
But, we're not in PROD yet. Just making sure everything works before we move to prod
s
My realisation since starting DE is that I hate PROD 🤣 it's a cursed land where nothing works
k
@finn_frotscher
n
Bumping this problem again - we were previously running airflow 1.10.14 in prod and attempted to upgrade to the most recent version of meltano. I’m trying to get a combo of meltano and airflow versions running locally that works again, but continue to get this same error. Have any other folks found workarounds for this? I’ve tried updating to Airflow 2.3.2 per the linked issue’s suggestion, but the error still seems to be happening
s
Depending on how much you care about your stored airflow data, my suggestion would be to simply drop the airflow.db and reinstall with new files
n
Thanks - I noticed your comment about that and tried that approach, but either I didn’t do it right or it didn’t work. Did you just get rid of the old data, or did you also reinstall airflow itself?
s
Both; delete airflow.db + reinstall airflow
n
and did you do that airflow reinstall manually via pip or “via meltano”?>
s
meltano
n
perfect, thanks! I’ll give that a shot
s
meltano install orchestrator airflow --clean
Keep me updated on the results 😄 good luck
n
Once I get it running successfully locally - then I’ll have to figure out how to make that work in prod too. Fun times!
Thanks again for your help @Stéphane Burwash! I’ve left the full post-mortem on the github issue for future searchers here: https://github.com/meltano/meltano/issues/6209
s
Glad I could help!
3 Views
Open in Slack
PreviousNext
Powered by