#3490 perf: :zap: Speed up method `MetadataMapping.resolve_selection` by 72% Pull request opened by codeflash-ai[bot] on <!date^1771366173^{date_short}|2026-02-17T22:09:33Z> 📄 72% (0.72x) speedup for

MetadataMapping.resolveselection

in

singersdk/singerlib/catalog.py

⏱️ Runtime :

3.10 milliseconds

→

1.81 milliseconds

(best of

250

runs) 📝 Explanation and details The optimized code achieves a 71% speedup (3.10ms → 1.81ms) by introducing memoization to eliminate redundant recursive computations in the

resolve_selection()

method. What changed: The optimization replaces the recursive

_breadcrumb_is_selected()

method with an inline

compute()

function that uses a cache dictionary. When resolving selection for deeply nested breadcrumb hierarchies, the original code repeatedly computed the same parent breadcrumb selections multiple times. Why it's faster: In the original implementation, each breadcrumb resolution required traversing up the parent chain recursively. For nested structures like

('properties', 'parent', 'properties', 'child')

, the parent breadcrumb

('properties', 'parent')

would be evaluated multiple times if multiple children existed. The line profiler shows

_breadcrumb_is_selected()

was called 15,492 times with 38.6ms total time in the original version, but only once (1μs) in the optimized version. With memoization, each breadcrumb's selection is computed exactly once and cached. Subsequent lookups are instant dictionary retrievals. This is particularly effective for: • Deep nesting: Tests with 10-level depth show 360% speedup (156μs → 34.0μs) • Large flat structures: 1000 properties show 74.3% speedup (401μs → 230μs) • Complex parent-child relationships: 50 parents with 10 children each show 118% speedup (310μs → 142μs) How it impacts workloads: Schema resolution with nested properties (common in JSON schemas) benefits most. The test results show consistent speedups across all scale tests, with the largest gains in deeply nested structures. The optimization maintains identical correctness (all tests pass with same results) while dramatically reducing redundant work. Trade-offs: Small overhead (~100-200ns) for trivial cases with 1-2 breadcrumbs due to cache initialization, but this is negligible compared to the massive gains for realistic workloads with dozens or hundreds of properties. ✅ Correctness verification report: | Test | Status | | ----------------------------- | --------------- | | ⚙️ Existing Unit Tests | ✅ 16 Passed | | 🌀 Generated Regression Tests | ✅ 125 Passed | | ⏪ Replay Tests | 🔘 None Found | | 🔎 Concolic Coverage Tests | 🔘 None Found | | 📊 Tests Coverage | 100.0% | ⚙️ Click to see Existing Unit Tests | Test File::Test Function | Original ⏱️ | Optimized ⏱️ | Speedup | | ------------------------------------------------ | ----------- | ------------ | ------- | | singerlib/test_catalog.py::test_metadata_mapping | 7.83μs | 6.83μs | 14.6%✅ | 🌀 Click to see Generated Regression Tests # Import the real classes from the module under test. from future import annotations from singer_sdk.singerlib.catalog import Metadata, MetadataMapping, StreamMetadata def test_resolve_selection_on_empty_mapping_returns_empty_selectionmask(): # Create an empty MetadataMapping instance (no breadcrumbs added). mm = MetadataMapping() # Call resolve_selection which should return a SelectionMask instance. codeflash_output = mm.resolve_selection() selection = codeflash_output # 583ns -> 791ns (26.3% slower) def test_stream_level_selected_and_child_inherits_selected_by_default(): # Stream-level metadata set to selected_by_default=True (using StreamMetadata) mm = MetadataMapping() stream_md = StreamMetadata() # construct a real StreamMetadata stream_md.selected_by_default = True # set selected_by_default on the stream mm[()] = stream_md # assign to the empty breadcrumb (stream-level) # Add an explicit property breadcrumb with no explicit selected value. child_breadcrumb = ("properties", "child") mm[child_breadcrumb] = Metadata() # child metadata with defaults (None) codeflash_output = mm.resolve_selection() selection = codeflash_output # 3.92μs -> 3.62μs (8.06% faster) sel_dict = dict(selection) def test_parent_false_blocks_child_even_if_child_selected_true(): # Parent (stream) explicitly deselected mm = MetadataMapping() mm[()] = StreamMetadata() mm[()].selected = False # parent deselected # Child explicitly selected=True but should be blocked by parent=False child = ("properties", "a") child_md = Metadata() child_md.selected = True mm[child] = child_md codeflash_output = mm.resolve_selection() selection = codeflash_output # 1.79μs -> 1.83μs (2.24% slower) sel = dict(selection) def test_unsupported_inclusion_ignores_selected_true_and_returns_false(): # Unsupported inclusion should always return False regardless of selected=True mm = MetadataMapping() b = ("properties", "unsupported_field") md = Metadata() # Use the real InclusionType enum on the real Metadata class md.inclusion = Metadata.InclusionType.UNSUPPORTED md.selected = True # input tries to select it mm[b] = md codeflash_output = mm.resolve_selection() selection = codeflash_output # 2.75μs -> 2.71μs (1.55% faster) sel = dict(selection) def test_automatic_inclusion_ignores_selected_false_and_returns_true(): # AUTOMATIC inclusion should return True even if selected is False mm = MetadataMapping() b = ("properties", "auto_field") md = Metadata() md.inclusion = Metadata.InclusionType.AUTOMATIC md.selected = False # input tries to deselect mm[b] = md codeflash_output = mm.resolve_selection() selection = codeflash_output # 2.54μs -> 2.58μs (1.63% slower) sel = dict(selection) def test_selected_by_default_used_when_selected_is_none_and_parent_true(): # Parent is selected True, child has selected_by_default=True but selected=None mm = MetadataMapping() mm[()] = StreamMetadata() mm[()].selected = True # parent selected True child = ("properties", "child_default") md = Metadata() md.selected = None md.selected_by_default = True mm[child] = md codeflash_output = mm.resolve_selection() selection = codeflash_output # 1.92μs -> 1.92μs (0.000% faster) sel = dict(selection) def test_fallback_to_parent_or_false_when_metadata_omitted(): # Parent omitted (defaults to no entry), but mapping contains a property-only metadata. # If the property metadata omits selected and selected_by_default, fallback to parent_value # which for a missing parent (no metadata entries at all) would be True only in absence # of any metadata, but since we have entries (the property itself), parent resolution # tries to consult parent breadcrumb which is empty tuple and will be created with default. mm = MetadataMapping() # Add a property breadcrumb with no selection info prop = ("properties", "p") mm[prop] = Metadata() # no selected / selected_by_default set # Also add an unrelated breadcrumb to ensure "self" is not empty (so default True is not used) mm["properties", "dummy"] = Metadata() codeflash_output = mm.resolve_selection() selection = codeflash_output # 3.25μs -> 2.92μs (11.4% faster) sel = dict(selection) def test_nested_parent_chain_blocks_descendants(): # Test deeper breadcrumb chains: ('properties','parent','properties','child') mm = MetadataMapping() parent = ("properties", "parent") child = ("properties", "parent", "properties", "child") # Parent explicitly deselected parent_md = Metadata() parent_md.selected = False mm[parent] = par… meltano/sdk