I'm just starting with ELT in general and I like Meltano Unix-like system a lot. My qyestion is a general one tho: ETL/ELT in itself is a generic term for data extraction transformation and loading. The way I understand it, I could use to load data to, for example, Google Calendar over its API. It's not much different then loading data to Google Sheet. they are both CRUD-REST-APIs. Yet nobody seems to be doing that. There's plenty extractors for various systems, but only a handful loaders for DBs, data file and spreadsheets formats. Somehow it's assumed that the whole use of ETL/ELT is building data warehouse solely for the purpose of BI. But I don't see that ingrained in the technology. Am I missing something, or is it everybody else? ;)

Hi and welcome @raphael_krupinski!

ETL/ELT in itself is a generic term for data extraction transformation and loading.

ELT is a bit less generic than ETL, being that it places the data store/warehouse at the center and almost all transformations happen with SQL inside of it. Targets like GSheets, CSV, S3, etc. are also used frequently to debug or as staging area to later load to the warehouse.

The way I understand it, I could use to load data to, for example, Google Calendar over its API. It's not much different then loading data to Google Sheet. they are both CRUD-REST-APIs. Yet nobody seems to be doing that.

There's a few people doing that and most commonly calling it

Reverse ETL

🤷‍♂️. It's not as widespread and people at companies that sell such services probably have opinions as to why it's not as easy to commoditize/open-source as E(xtract)L(oad). The biggest challenge of matching ELT with these target APIs, is that every source is gonna be different so the SQL transformations that massage the data in the right format, for say Google Calendar, are always gonna be ad-hoc. That said, we certainly see potential, and we have been calling them SaaS targets or as the P(ublish) in ELTP. An example is target-lunch-money, an example of which used with Meltano you can see here: https://gitlab.com/DouweM/investing-to-lunch-money #C01RKUVUG4S is a good place to discuss the particulars of SaaS target development in more detail

Thank you very much for an in-depth answer. The way I'd see it would be a two sided process with the P from the data warehouse. So ELTP would be a good description for it. Data warehouse wouldn't be the "ultimate target" for the data but actually the single source of truth. Great to know that there isn't any technological or architectural obstacles against doing it this way Than you again👍