Meltano (UI) with https://github.com/ets/tap-smart-csv and one of the `target-postgres`es is not that many steps removed from this tool, which looks very nice and is making the rounds on Hacker News: https://www.dropbase.io, https://news.ycombinator.com/item?id=24189582 I’d love to see Meltano UI develop into that direction for simple transformations over Singer tap stream/entity schema and record JSON, so that we can do ETL as well as dbt-backed ELT. We’d probably start with a way of specifying transformations in

meltano.yml

, similar in spirit to the

select

,

schema

, and

metadata

extra’s, and/or by pointing at a Python file that can process each Singer message. Building a DropBase-style UI over that would be on the horizon too, once we’ve brought the Entity Selection interface back (https://gitlab.com/meltano/meltano/-/issues/2002) and add interfaces for metadata rules and schema overrides.

Looks like an interesting project! The idea of turning a CSV/JSON file into a live database reminds me of this project: https://docs.datasette.io/en/stable/ That one uses sqlite rather than Postgres though

I've commented on Hacker News with some additional thoughts: https://news.ycombinator.com/edit?id=24194916