Hi, I need to bidirectional sync two CSV or Google...
# getting-started
f
Hi, I need to bidirectional sync two CSV or Google Sheets data. I am using tap-csv and target-csv for testing purposes and successfully moving data from source to destination CSVs files. But i have no idea how to make it bidirectional? If there is any change in source or destination file. It should be sync both sides. How to achieve this or i need to test it on Google Sheets?
c
if you aren’t using Airflow and sticking strictly to Meltano I’d try solving this by having both CSVs as a tap and a different named CSV as a target, and then upon completion have another process rename the target CSV to the original CSV. You’d still need some sort of external process to check for changes and this assumes only one CSV would contain changes at any given time - if both CSVs contain changes that need to be resolved, I don’t think the tap/target architecture will work for you
1
💯 1
f
Instead of using AirFlow, is it possible to achieve this if we create custom tap/targets and use them along with existing connectors using Meltano SDK/CDK? Or If we use AirFlow for this purpose, Is it possible to use cron jobs instead of AirFlow?
c
you can certainly use cron, no reason why you couldn’t