hello everyone, is there a way to run a pipeline continuously? (orchestration guide only mentions running a pipeline using "--interval @daily") edit: by continuously, I meant running the pipeline in a loop instead of using --interval

Hi @sai_shobith_l no there isn't. Seems like a pretty common use case though doesn't it. A typical sync pattern I've seen many times is to sync, then restart immediately if there were items processed, otherwise sleep for awhile. ‘meltano run’ only runs, no logic / processing. All orchestration is handled externally to meltano run. The tap could do this continuous behaviour - but I've not seen any that do this.

Relevant Singer proposal: https://github.com/MeltanoLabs/Singer-Working-Group/issues/7. Feel free to comment, upvote, etc. 🙂

Thanks for info!