Refactoring an existing project into Meltano. We have a few “augmentation” steps, pulling in data from other APIs when new data comes in through the main EL pipeline. I think I’d like to model these as separate EL steps, where the “augmentation E” only grabs those rows that are new from the earlier main pipeline. How should I do this selecting? Hypothetical options: 1. the naive thing, just do API calls in a custom extractor as part of the main pipeline, no augmentation EL 2. pass state from the main EL to the augmentation EL -> this might be something? 3. let the augmentation EL figure out what entities are new -> this doesn’t seem very idiomatic, but I guess we could build in some queries before it gets going to decide which new rows to augment Am I approaching this from the wrong angle? Is this not usually an EL job?

I'd have to sit down and map out what your problem is, why you need "augmentation EL" etc etc and see what the right solution is. Would take a bit I guess is my point to understand what you need

I guess that was just context. Simplified, can either of the following be done: • pass output from one Meltano task as input to another • should a Meltano task consult a database for example to figure out what to fetch and what not to