#3350 Jobs canceled via Airflow may not be properly aborted in logs Issue created by MeltyBot Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/3428 Originally created by @aaronsteers on 2022-04-21 155642 --- Following from a slack conversation, it seems Airflow is killing jobs too quickly or in a way that does not allow Meltano to mark jobs as canceled/failed. This is leaving job logs in a state that may require users to use

--force

when trying to rerun. More info in slack thread: https://meltano.slack.com/archives/C01TCRBBJD7/p1650393919971169?thread_ts=1650299838.276379&cid=C01TCRBBJD7

First, we should probably check Airflow wrappers to see if the behavior of its job canceling (SIGTERM) is not properly shutting down meltano is an exception or the norm. Perhaps there are specific conditions that cause the job abort to not be logged? As noted:

> My concern is when a task is stopped in airflow the metlano is not shutting down properly

meltano/meltano