GitHub
08/25/2025, 8:37 PMmy_stream.prop.* should imply selecting the top-level property my_stream.prop
Issue created by edgarrmondragon
### Meltano Version
3.5.4
### Python Version
3.12
### Bug scope
CLI (options, error messages, logging, etc.)
### Operating System
NA
### Description
Configuring select with:
select:
- my_stream.prop.*
selects all nested sub-properties of my_stream.prop, but fails to select the parent property itself. This catalog output causes the SDK schema pruning algorithm to drop prop and all of its nested properties from the schema. The recommended workaround is to explicitly select `prop`:
select:
- my_stream.prop
- my_stream.prop.*
Ideally users would not need to do this and it's arguably bad UX that they need to.
One way we might be able to address this is by generating MetadataRule instances for these special cases. We already generate two rules for a single pattern in some cases:
meltano/src/meltano/core/plugin/singer/catalog.py
Lines 188 to 208 in</meltano/meltano/commit/3b7a3f900644636ea44df3d5213aa2abe91d1859|3b7a3f9>
| if selected or not prop_pattern or prop_pattern == "*": |
| ------------------------------------------------------- |
| rules.append( |
| MetadataRule( |
| tap_stream_id=pattern.stream_pattern, |
| breadcrumb=[], |
| key="selected", |
| value=selected, |
| ), |
| ) |
| |
| if prop_pattern: |
| props = prop_pattern.split(".") |
| |
| rules.append( |
| MetadataRule( |
| tap_stream_id=pattern.stream_pattern, |
| breadcrumb=property_breadcrumb(props), |
| key="selected", |
| value=selected, |
| ), |
| ) |
For example, the pattern my_stream.prop.* generates the rules
MetadataRule("my_stream", [], "selected", value=True)
MetadataRule("my_stream", ["properties", "prop", "properties", "*"], "selected", value=True)
but we could patch so it also generates
MetadataRule("my_stream", ["properties", "prop"], "selected", value=True)
Related:
• #3024
• meltano/sdk#2687
• https://gitlab.com/meltano/sdk/-/issues/285#note_750983833
### Code
No response
### Link to Slack/Linen
No response
meltano/meltanoGitHub
08/25/2025, 8:37 PM