Hey folks, I have a mystery involving archiving to S3 using the target-snowflake plugin. I could use your help!
In the configuration for target-snowflake, there is a setting called:
archive_load_files. This setting is a boolean.
However, from its description: "_When incremental replication is used, the archived files will also have the following S3 metadata keys - incremental-key, incremental-key-min and incremental-key-max"._
That this value, archive_load_files, has a value of a dictionary with the above metadata keys inside is
represented in the code, here.
The problem is, because these key:values are missing, a default of None gets set for 'tap' and the whole job fails. Is this config value supposed to be a boolean? or a dictionary?
e.g.
archive_load_files: true
tap:
incremental-key:
incremental-key-min:
incremental-key-max: