<#3693 feat: `try_cast` helper for stream map expr...
# meltano-repo-activity
g
#3693 feat: `try_cast` helper for stream map expressions Issue created by ReubenFrankel Feature scope Inline mapping (stream maps, flattening, etc.) Description Problem Stream map expressions are evaluated with
simpleeval
, which has no `try`/`except` and no
lambda
. This makes safe type coercion of dirty upstream data awkward:
int(value)
,
float(value)
,
uuid.UUID(value)
, etc. raise on malformed input, and there is no way to catch that inside an expression — so a single bad record can error the whole stream instead of being coerced to a sensible default. There's no clean workaround today. Registering a module doesn't help (
simpleeval
blocks module objects), and there's no way to wrap the call in `try`/`except`. Proposal Register a
try_cast
helper in the stream map function namespace that coerces a value and falls back to a default on failure, instead of raising: def try_cast(value, caster, default=None, exceptions=(ValueError, TypeError)): """Coerce value via caster(value); on failure return default instead of raising.""" try: return caster(value) except exceptions: return default The key point that makes this work under `simpleeval`: pass the callable plus its argument (callables are values), and let the helper — which is ordinary Python, not sandboxed — perform the `try`/`except`. This sidesteps the missing-
lambda
limitation. Example usage stream_maps: my_stream: amount: try_cast(amount, float, default=0.0) # garbage -> 0.0, not a crash count: try_cast(count, int) # unparseable -> None It also composes into
__filter__
as an existence check: __filter__: try_cast(str(value), int) is not None Design considerations • Narrow exception scope by default. Defaulting to
(ValueError, TypeError)
rather than bare
Exception
keeps genuine config bugs (e.g. a misspelled field name raising `KeyError`/`NameError`) loud instead of silently returning
default
on every row. •
default=None
, not
False
.
A
False
default breaks the common
... is not None
idiom (
False is not None
is
True
). • Caster name resolution. For
try_cast(value, int)
to work with a bare caster name, the caster must be resolvable in the expression context. To stay robust across
simpleeval
configurations, either register intended casters in both the names and functions maps, or accept the caster as a string label (e.g.
try_cast(value, "int")
) resolved internally against a whitelist — the latter also lets the SDK constrain which casts are permitted. --- Follows this Slack discussion - would go hand-in-hand with #3694. 🤖 Generated with Claude meltano/sdk