My target-snowflake (transferwise) fails because a few columns in the source table have been removed. Is there an easy fix to this? The only workaround I can think of is to use the mapper functionality to add the fields.

in tap schema or catalog, you can mention default values?

My tap does automatic schema discovery (it reads excel files 😱), there isn’t any configuration to force.

Maybe just fork the tap and add defaults?

I mentioned this in Office Hours, but throwing it in here for discoverability: A quick fix is to use a Snowflake FILE FORMAT that has

ERROR_ON_COLUMN_COUNT_MISMATCH=FALSE

(docs). I have a few pipelines where there’s a similar mismatch issue that I want to ignore, so in my project I have two base snowflake targets, one uses a more strict/normal file format and one uses a file format that allows mismatches.

Just tested this now, works great! So far I have not been able to fool it.