Does iceberg always require

add_record_metadata: true

to take care of snapshots? If I have those extra

_sdc_

columns, I get

No schema for record field

error for these columns: "_sdc_deleted_at" and "_sdc_table_version" (when there are null values in those columns)

I'm not familiar with iceberg or target-iceberg, but https://github.com/meltano/sdk/blob/3ea0667b982af3efe514185e865951c5bd8c7251/singer_sdk/sinks/core.py#L167-L168 should ensure those are present in the schema.

@Edgar Ramírez (Arch.dev) Thanks. Yes they are present, by default. I'm wondering if it is safe to remove those columns. Maybe someone with the experience of iceberg table format would be able to confirm if these columns are responsible for the

snapshot

feature.

I think you're the maintainer of target-iceberg? If so, you probably know best whether the snapshot implementation in the target would benefit from using those columns 😅

@Edgar Ramírez (Arch.dev) I'm one of the maintainers. Unfortunately, I'm still learning more and more about iceberg and the other table formats. Maybe I can find a work around soon.