Does anyone know if there are Singer taps for Trin...
# random
c
Does anyone know if there are Singer taps for Trino? I checked and couldn't find anything in the usual places.
e
There isn't one afaict, but an MR that's currently in the works will make really easy to create one (or might even work out-of-the-box 😄)
a
Hi, @carolus_holman! As @edgar_ramirez_mondragon points out, we've got a WIP database stream type coming soon. That implementation uses SQLAlchemy behind the scenes (in turn powered by DB API 2.0), with the expectation that developers would only need to override parts of the process for which a custom approach is significantly faster/better than the default SQLAlchemy approach. Is your use case expected to be high volume? If not, I'd be curious if our generic tap-sqlalchemy might work for your case - and/or if you could fork it and add Trino-specific optimizations as needed. The SQLAlchemy tap should still be considered pre-release, but you can try it out if you wanted. It should give generic extract ability for any installed sqlalchemy drivers on your system.
(Would need to be paired with something like sqlalchemy-trino · PyPI)
e
@aaronsteers We should add a guide to tap-sqlalchemy on installing those "dialects" or maybe add them as extras to
pyproject.toml
c
Thanks @aaronsteers For the amount of data, that would be case by case specific. Thinking about how Trino/Presto works, I wanted to ask more of a "Is this the Right Tool?". I am thinking of using Meltano as part of a Stack which includes Trino/dbt/Snowflake. Maybe we can discuss - Thanks
a
Yeah, sounds super interesting and I'm happy to chat more