Does Meltano support having

!include

statements in the meltano.yml file? Or any other way of bringing in other YAML files into the main config?

@garret_cree check out https://docs.meltano.com/concepts/project#multiple-yaml-files

Ah, thank you! I was looking for a way to put a stream_mapper in a separate yml file and then bring it in with a generic !include , but it looks like that's not one of the supported files. I wanted a way to keep column names closer to the code that uses them. Re: this thread

If I understand correctly, that is supported. See my project https://github.com/meltano/squared/tree/main/data I put mappers in their own yaml https://github.com/meltano/squared/blob/main/data/extract/mappers.meltano.yml

We do similar: we have individual YAML files in a

meltano-yml

directory

meltano-yml
├── environments
│   ├── ca-prod.meltano.yml
│   ├── ca-staging.meltano.yml
│   ├── dev.meltano.yml
│   ├── prod.meltano.yml
│   ├── staging.meltano.yml
│   └── system.meltano.yml
├── extractors
│   ├── tap-campaign-monitor.meltano.yml
│   ├── tap-github.meltano.yml
│   ├── tap-google-analytics-v4.meltano.yml
│   ├── tap-google-analytics.meltano.yml
│   ├── tap-jira.meltano.yml
│   ├── tap-mailchimp.meltano.yml
│   ├── tap-meltano.meltano.yml
│   ├── tap-mongodb.meltano.yml
│   ├── tap-pendo.meltano.yml
│   ├── tap-quickbooks.meltano.yml
│   ├── tap-salesforce.meltano.yml
│   ├── tap-spreadsheets-anywhere.meltano.yml
│   ├── tap-tableau.meltano.yml
│   └── tap-zendesk.meltano.yml
├── loaders
│   ├── target-jsonl.meltano.yml
│   ├── target-postgres.meltano.yml
│   └── target-redshift.meltano.yml
└── utilities
    ├── dbt-postgres.meltano.yml
    ├── dbt-redshift.meltano.yml
    ├── great_expectations.meltano.yml
    └── sqlfluff.meltano.yml

5 directories, 27 files

And then include them all at the beginning of our

meltano.yml

file:

version: 1
include_paths:
  - ./meltano-yml/environments/ca-prod.meltano.yml
  - ./meltano-yml/environments/ca-staging.meltano.yml
  - ./meltano-yml/environments/dev.meltano.yml
  - ./meltano-yml/environments/prod.meltano.yml
  - ./meltano-yml/environments/staging.meltano.yml
  - ./meltano-yml/environments/system.meltano.yml
  - ./meltano-yml/extractors/tap-campaign-monitor.meltano.yml
  - ./meltano-yml/extractors/tap-github.meltano.yml
  - ./meltano-yml/extractors/tap-google-analytics-v4.meltano.yml
  - ./meltano-yml/extractors/tap-google-analytics.meltano.yml
  - ./meltano-yml/extractors/tap-jira.meltano.yml
  - ./meltano-yml/extractors/tap-mailchimp.meltano.yml
  - ./meltano-yml/extractors/tap-meltano.meltano.yml
  - ./meltano-yml/extractors/tap-mongodb.meltano.yml
  - ./meltano-yml/extractors/tap-pendo.meltano.yml
  - ./meltano-yml/extractors/tap-quickbooks.meltano.yml
  - ./meltano-yml/extractors/tap-salesforce.meltano.yml
  - ./meltano-yml/extractors/tap-spreadsheets-anywhere.meltano.yml
  - ./meltano-yml/extractors/tap-tableau.meltano.yml
  - ./meltano-yml/extractors/tap-zendesk.meltano.yml
  - ./meltano-yml/loaders/target-jsonl.meltano.yml
  - ./meltano-yml/loaders/target-postgres.meltano.yml
  - ./meltano-yml/loaders/target-redshift.meltano.yml
  - ./meltano-yml/utilities/dbt-postgres.meltano.yml
  - ./meltano-yml/utilities/dbt-redshift.meltano.yml
  - ./meltano-yml/utilities/great_expectations.meltano.yml
  - ./meltano-yml/utilities/sqlfluff.meltano.yml
default_environment: dev

it should be! https://docs.meltano.com/concepts/project#multiple-yaml-files

oh wow how did I mis this ?? facepalm trek