or can you write some python scripts to do some tr...
# announcements
g
or can you write some python scripts to do some transformations before sending the data further down
d
Hi @gnaffle! Meltano currently only supports dbt (https://www.getdbt.com/, open source) for transformation, which means that it's built for ELT rather than ETL, since transformation takes place inside the loading database, rather than in between the E and L steps. As a consequence, dbt transformations are written in SQL and act on a live database, rather than being Python scripts acting on raw extracted records before they're loaded into the database. I'm very interested in exploring the ETL direction more, though, as I mentioned here: https://meltano.slack.com/archives/CFG3C3C66/p1597719041499500 and in the linked Hacker News thread. I'll create some issues around that direction today. If you can share a bit more detail about the kind of transformations you intend to apply, I'll gladly help you figure out whether dbt would be a good fit, or running Python scripts after loading the data (as @adam_roderick suggested in https://meltano.slack.com/archives/CFG3C3C66/p1597845357003500?thread_ts=1597831066.001600&cid=CFG3C3C66), or whether you'll really need transformation in between extraction and loading, which Meltano doesn't support yet.