<#9504 feat: make PluginInvoker use manifest for e...
# meltano-repo-activity
g
#9504 feat: make PluginInvoker use manifest for env var resolution Pull request opened by cjohnhanson on <!date^1757954097^{date_short}|2025-09-15T16:34:57Z> ## Summary Updates PluginInvoker to read environment variables from the compiled manifest instead of expanding them at runtime. This is the first step toward using the manifest as the single source of truth for configuration (#7271). ## Changes • Add
src/meltano/core/manifest/loader.py
with functions to load/compile manifests • Update
PluginInvoker.env()
to check manifest first, fall back to original logic if unavailable • Auto-compile manifest when missing (same behavior as
meltano compile
) • Log warning when manifest is stale (meltano.yml newer than manifest) ## Context The manifest already contains all resolved environment variables after compilation. Reading from it directly avoids the complex runtime expansion logic in PluginInvoker. Without this change, env var resolution happens every time a plugin is invoked, requiring multiple passes of variable expansion. With this change, we read pre-resolved values from the manifest when available. ## Notes • Fully backwards compatible - falls back to original behavior if manifest unavailable • Auto-compilation might be unexpected during
meltano invoke
, but follows the pattern of other Meltano commands • One test required
project_function
fixture instead of
project
due to class-scoped isolation ## Test plan • Added 13 unit tests for manifest loader • Added 4 tests for PluginInvoker manifest integration • Created integration test for end-to-end validation • Verified no regressions in existing tests • Pre-commit hooks pass Fixes #7271 ## Summary by Sourcery Use the compiled Meltano manifest as the primary source for resolving environment variables in PluginInvoker, with lazy loading, auto-compilation of missing or stale manifests, and fallback to existing expansion logic when needed. New Features: • Introduce a manifest loader module that locates, loads, checks staleness, and compiles project manifests • Update PluginInvoker.env to fetch pre-resolved environment variables from the compiled manifest, auto-compiling the manifest if it’s missing Enhancements: • Log warnings when the manifest is stale or a plugin is not found in the manifest • Implement lazy loading of manifest data in PluginInvoker and graceful fallback to legacy env var expansion CI: • Add a new "meltano-invoker-manifest" integration test job to the GitHub Actions workflow Documentation: • Provide an example library with documentation and scripts illustrating PluginInvoker manifest usage Tests: • Add 13 unit tests for the manifest loader and 4 integration tests for PluginInvoker manifest support • Include an end-to-end integration test example project demonstrating manifest-based env resolution meltano/meltano