Hi, anyone know how to bypass or include Content-MD5 when using Meltano (version 3.7.0) with MinIO?

meltano run tap-rest-api-msdk target-jsonl

It fails with:

An error occurred (MissingContentMD5) when calling the DeleteObjects operation: Missing required header for this request: Content-MD5

Seems like Meltano can’t delete the lock file in MinIO due to this.

https://hub.meltano.com/extractors/tap-rest-api-msdk/#headers-setting Maybe

- name: tap-rest-api-msdk
    variant: widen
    pip_url: tap-rest-api-msdk
    config:
      headers:
        Content-MD5: <value>

?

thank you, let me try it

@Elliott Gao if I understand correctly you're using S3-compatible MinIO as the state backend? FWIW I think that would be solved by bumping MinIO: https://github.com/minio/minio/issues/20845.

yeah, I am using S3-compatible MinIO, I will ask if our clients want to bump minio

Otherwise you can pin boto3 to `<1.36`: https://github.com/meltano/meltano/pull/9035. So, something like

uv tool install --with 'boto3<1.36' 'meltano[s3]'

, depending of course on how you actually installed Meltano.

@Edgar Ramírez (Arch.dev) thank you so much for your advice. I'll def try it out.