Meltano

:wave:  Hi all. I want to find all the Sheets documents in a Google Drive folder and extract the data from them. Then periodically check for new Sheets documents that are added to the folder and extract data from those.
`tap-google-sheets` has a `spreadsheet_id` configuration so a pipeline with using this tap can only load from a single Sheets document at a time.
I’m thinking I will create an Airflow DAG to:
1. fetch any new Sheets and their `spreadsheet_id` (i might need to create a Google Drive extractor plugin to do this? or maybe just a script run by Airflow…)
2. invoke my Meltano pipeline that invokes the `tap-google-sheets` pipeline for each new `spreadsheet_id` 
Thank you :slightly_smiling_face:

I can certainly see a possible future of forking the tap and creating a variant which can accept a pointer to a Google drive folder and "discover" all spreadsheets in that directory. This use case seems common enough that I'm sure others would appreciate the capability.

Your approach actually sounds better though especially as a first iteration, since you dodge having to tackle Singer-specific design work such as catalog generation and customizing selection logic. :raised_hands: