Question on the order of ELT through meltano. So l...
# announcements
m
Question on the order of ELT through meltano. So lets say I have a tap that has a field where the value is an array, and I want to break that up into a separate table using dbt. Does the pipeline take the output from the tap, transform it with the dbt code I wrote, and then send the result to the target? Or does meltano send the tap output to the target, runs the transformation, and then also send that output to the target as well?
d
@michael_cooper Since dbt runs inside the database, it only runs after the data has been extracted and loaded in the destination, which is why it's ELT, not ETL 🙂 So Meltano runs the tap, then the target, and then runs DBT, passing along some env vars so that it knows where to find the loaded data.
m
Okay, thank you!
e
AFAIK at least datamill's postgres and redshift targets do automatic "denesting" of array fields into separate tables