Hi! I have a question regarding using dbt as transformer in meltano. I know one could configure a source schema where the dbt code reads from and a target schema for it to write in the same database. However, is it possible for dbt to read from one database and after transformation write data into a different database? Thank you in advance!

I do this right now by going 1.

meltano elt tap-data target-db2

2.

dbt run

3.

meltano elt tap-db2 target-differentdb

Probably better ways but this is where I"m at currently. The updated meltano run functionality should be able to make this one command which is exciting

Hi! Thank you for answering. But I am afraid I did not get how your solution works in a ELT pipeline with hourly schedule? Under folder transform

profiles.yml

I guess one could only specify one database as dbt target?

I transfer the data from one database to another in step #3 If they are the same database types there's normally better ways right inside of the DB. Like in MSSQL there's linked servers

Aha! So you have a second elt to transfer the data from one db to another, I see, thank you! I thought about this solution, but was hoping there is a better way

What's the Database you are transferring to / from. DBT isn't made to do those types of transfers and they advise against it

Thank you!

I'd probably just use a different schema and bypass all the step #3 things in that case

Haha this is the current setting and we had the idea of separating the sensitive data and clean data in two different databases, which is why I came across this problem

😂 good luck

😅 thanks