chris_evans
12/01/2022, 10:35 PM__else__
behaviour for inline stream maps https://sdk.meltano.com/en/latest/stream_maps.html#else-behavior-currently-limited-to-null-assignment?
It would be nice to apply a transformation across properties of a stream dynamically. Specifically, I would like to cast all properties of a stream to txt w/o knowing the properties before hand (dealing w/ some messy schema detection from tap-csv
where it is easier to manage typing at a later step).
Currently, I mutate the catalog before passing it to the elt command for any given stream. However, I would like to move to using the run command and mappings w/o having to introspect/mutate the catalog.aaronsteers
12/01/2022, 11:19 PM____else____
clause down the road.
However, now we're leaning towards a proposal to launch revamp "V2" stream map spec which would have each transformation as a complex object, and would allow for more expressivity overall.
https://github.com/meltano/sdk/discussions/1054
Specifically, I would like to cast all properties of a stream to txt w/o knowing the properties before handFrom the syntax in the link above, this would be possible with something like this:
- apply-to: "my_csv_stream.*"
description: Coerce all to string
new-value: str(self)
new-type: string
chris_evans
12/01/2022, 11:29 PM