CSV - best of both worlds (tap-csv & tap-spreadsheets_anywhere) I have csv files with some fixed columns and some ephemeral columns. This means that some columns exist in some csv files, but not others. However, I want to load all of the csv files into the same table in postgres.

target-postgres

(transferwise), in conjunction with

tap-csv

, let's me do this today using with the

data_flattening_max_level

config setting. Each time I extract a new csv, target-postgres will add any columns that are missing. Unfortunately, tap-csv does not seem to support state. If I add a new file to the

path:

folder, tap-csv will extract all files, not just the new one. State support is also called out as missing here: Implement State Capability On the other hand, while

tap-spreadsheets-anywhere

properly supports state, the new columns are not automatically created for some reason. (Not sure why, it seems that capability is part of target-postgres, not the extractor...). I'd like to see one extractor with both features, but I'm wondering which one would be better to improve? tap-csv seems like the best place, but there are some other rich capabilities in tap-spreadsheets-anywhere that myself and others would certainly benefit from. Any thoughts on combining the features of spreadsheets-anywhere into tap-csv?