Trying to get us using environments and wondering ...
# getting-started
q
Trying to get us using environments and wondering about best practices for configuring them. Looking at the meltano-squared repo under data/environments, it seems like there's a lot of repetition. E.g. there's a yml file that's nearly identical in userdev.meltano.yml & in prod.meltano.yml describing all the same extractors and loaders with just slightly different configs. Is that repetition pretty standard? Is there a DRY-er way to do it? Would love any resources people have found helpfull or other examples of how you're all using environments!
I guess my main use case for a dev env is to perform a quick check when we make changes to the prod meltano.yml and make sure it doesnt crash meltano and that the changed pipeline runs without error. So if I end up having to maintain two separate yml files for the different envs, and they're mostly not inheriting the overall structure from each other... I'm not sure that that solves that problem very well
v
I keep everything in one file 🤷 I think that solves most of what you're asking , and it all depends on what you're after. Sometimes you might have large teams that change things and you want them to work as independently as possible
q
Ok so the thing I was missing here was a simpler example, I think. Having walked through our own file and figured out exactly which bits we'd need to break out into the environments section, this makes a lot more sense. We just have about 25 lines that we'll need under environments. The meltano-squared example had such a massive environments section that I was like "are we gonna have to pretty much reproduce our whole yml file twice, once for dev and once for prod? and if so, how does that help us test the prod yml section?"
v
You can split it all up which is amazing, but you don't have to which is also great. So you have the flexibility but you don't have to override things
q
Totally