Hi! I'm facing an issue with the graceful terminat...
# troubleshooting
b
Hi! I'm facing an issue with the graceful termination introduced in the SDK starting v0.47 with this PR. What I except to happen is: SIGTERM is received, data stops being read from the tap, target drains all sinks, state is emitted, voila. What I find happens is that SIGINT (or ctrl+c) does behave that way, but sending SIGTERM does not drain the sinks and no state emits. The easiest way I have to prove is that with SIGTERM (
pkill -2 -f meltano
), I can see the following log messages from this code path: •
target-snowflake *Received termination signal 2, draining all sinks...*
target-snowflake *Received termination signal 15, draining all sinks...*
With SIGTERM (
pkill -15 -f meltano
) the two log messages above don't show up; instead, I only see: •
meltano      *Attempting graceful termination of current block*
meltano      *Terminating ExtractLoadBlocks*
meltano      *Run completed*
No state, nada. I suspect that the reason for that might have to do with
_handling_sigterm
in the Meltano source. I have no other clue.
So far I've determined the following: 1.
SIGTERM
gets treated differently than
SIGINT
(I've solved this by setting
STOPSIGNAL SIGINT
in my Dockerfile) 2. The
SIGTERM
code path terminates before the plugins terminate gracefully due to the exception raised here
Opened an issue for this
e
Thanks for opening an issue @Boris Ralichkov! This is something that I definitely need to revisiting on Meltano's end, I'll take a look.
❤️ 1
👀 1