Meltano

Can I pull the same stream twice, with different columns names and renaming the output?

<@U06CBV487CJ> Renaming a stream is not currently possible, but if you're loading the data into a database/warehouse, you can override the target schema by overriding the `TAP_FOO__LOAD_SCHEMA` env var on your `meltano elt`  run: <https://meltano.com/docs/plugins.html#load-schema-extra>. On each `meltano elt` run, you can then use `--select=column`  (or `TAP_FOO__SELECT=['column']` ) to select a different subset of attributes.

If you'd like to split a stream up into two tables, I recommend using `dbt` , though.

Once we have <https://gitlab.com/meltano/meltano/-/issues/2300>, it will be pretty easy to implement a Python transformer to rename a stream or split it up into separate tables, but I don't expect to get to that for at least another month.

We're also planning to let streams be renamed using a directive in `meltano.yml` : <https://gitlab.com/meltano/meltano/-/issues/2299>, but splitting up a stream into 2 tables isn't currently in scope, although it could be added :slightly_smiling_face:

I have a table in postgressql that has a column of type hstore and it has commas in it so it's messing me up.  I was thinking maybe I could just bring that column over by itself and join back to the main table

Messing you up how? Is it not making it out of the source or into the destination correctly?

Unless splitting out that hstore column is something the tap you're using supports natively (which it probably doesn't), Meltano/Singer is not going to be of much help here I'm afraid. It sounds like a typical task for a transformer like dbt :slightly_smiling_face: