best practice question: how do you organise your meltano projects? one project per data source? one project per instance of a source?

Hi, @ben_marengo. Generally people use a single project for an entire set of sources and transforms. Basically the entire data platform code (one end-to-end ELT pipeline) can be in the same project comfortably. When you get to the point where you want a downstream platform ingesting the first project as a dependency, you can create a new project which consumes the output of the first (along with any other data sources relevant to the second project). Does this help at all?

hmm interesting. i notice .meltano (and therefore the tap/target venvs) is dockerignored. i think this means they will auto-upgrade every pipeline run. is it possible to fix the versions in the meltano yaml or somewhere?

@ben_marengo http://meltano.com/docs/plugin-management.html#pinning-a-plugin-to-a-specific-version 🙂