Hi Which component is responsible for clearing batch files after being loaded by target? I didn't find anything in meltano code (after a quick skim) and SDK documentation doesn't mention it

(assuming

target-snowflake

) https://hub.meltano.com/loaders/target-snowflake/#clean_up_batch_files-setting

ok, so it's not handled in any reusable way. Thanks for the pointer!

Not at the moment, I don't think enough of the more popular targets support

BATCH

messages for it to have made its way into the SDK.

or is it the other way around? 😉 targets don't support it because of poor handling in sdk.

Not my place to answer that! 😅 It does look like it's supported for SQL targets out the box actually: https://github.com/meltano/sdk/blob/5a46d250e0fd997560d3b3e5928977314e5947c0/singer_sdk/sinks/sql.py#L264-L280 True batch implementations are platform-specific though.

I transferred this to the SDK: https://github.com/meltano/sdk/issues/3064

targets don't support it because of poor handling in sdk.

That's a big part of it. It's tough to keep the targets from deviating from the upstream and then it's also hard to pull reference implementations back into the SDK.