Hi all, Are there any plans to expand the capabil...
# troubleshooting
c
Hi all, Are there any plans to expand the capabilities of the
__else__
behaviour for inline stream maps https://sdk.meltano.com/en/latest/stream_maps.html#else-behavior-currently-limited-to-null-assignment? It would be nice to apply a transformation across properties of a stream dynamically. Specifically, I would like to cast all properties of a stream to txt w/o knowing the properties before hand (dealing w/ some messy schema detection from
tap-csv
where it is easier to manage typing at a later step). Currently, I mutate the catalog before passing it to the elt command for any given stream. However, I would like to move to using the run command and mappings w/o having to introspect/mutate the catalog.
a
Hi, @chris_evans! It's a complex answer. Yes, there are plans to expand this. Originally, we expected we might just have more complex expressions allowed in the
____else____
clause down the road. However, now we're leaning towards a proposal to launch revamp "V2" stream map spec which would have each transformation as a complex object, and would allow for more expressivity overall. https://github.com/meltano/sdk/discussions/1054
Specifically, I would like to cast all properties of a stream to txt w/o knowing the properties before hand
From the syntax in the link above, this would be possible with something like this:
Copy code
- apply-to: "my_csv_stream.*"
  description: Coerce all to string
  new-value: str(self)
  new-type: string
c
Thanks for sharing this @aaronsteers. Liking the added expressivity in the proposal