Is anyone familiar with these? https://sdk.meltano.com/en/latest/stream_maps.html The documentation only provides an example for aliasing the stream, but it mentions property aliases are also available. What's the syntax for those? I'm assuming that would be for renaming a column? I'm trying to add some logic to clean up column names, hoping I can provide some kind of function for it to run. Would it be something like this? If so, any way to supply a non-built-in function?

stream_maps:
  customers:
    customer_id:
       __alias__: id

Edit: I think the path I need might be standalone mapper plugins? It says they are not compatible with

meltano elt

though. I use

meltano el

currently

Hi @Daniel Luo! Can you confirm is my recent comment on GitHub is helpful for your use case?

Hi Edgar, thanks, that is quite helpful. So it seems like the syntax I was trying to apply isn't valid. That said, the example in #2545 is still not supported, even with the glob expressions. What he's trying to do sounds very similar to my use case. I don't want to have to manually define all of my mappings. Some tables can have over a hundred columns. We need a programmatic way to just apply some column name transformation. In his example, he wants all column names to be in lowercase. You wouldn't want to manually list that for all column names that you have across all of your streams. For me, I want to make everything snake_case and have some way of replacing invalid characters. The latter isn't as important since it's a rare scenario that we can probably account for by hard coding the mapping in the config.

Gotcha, that makes sense. I just realized you created https://github.com/meltano/sdk/issues/2490 😅

Ha, you just reminded me. I forgot myself. Been going back and forth and just came back to this issue

Upvoted issue I'm facing same challenge 🙂