Hello everyone! We are working on Meltano over Google App Engine and we want to export the generated csv files via "target-csv" to Cloud Storage. Could you point me to the "best practice" to achieve this?

@rocco_melillo If you wanted to store the CSV in S3 (or something compatible), you could use https://github.com/transferwise/pipelinewise-target-s3-csv, but otherwise the best practice would be to use

meltano elt

with

target-csv

to write the CSV file to the local filesystem, and then configure your orchestrator (e.g. Airflow) to add an extra step after

meltano elt

that will grab that file and upload it to your desired destination. That means you'd want to add a custom Airflow DAG generator if you're currently using the default one (https://gitlab.com/meltano/files-airflow/-/blob/master/bundle/orchestrate/dags/meltano.py).

Thank you for your kind reply. I'll take a look to the link and I'll follow your suggestion. 😀