Meltano

Built a tap this week for <http://Gong.io|Gong.io>, which will automatically transcribe calls and do some analytics on top of them. Their API is really just a couple endpoints, with the `calls` endpoint having a bunch of nested tables, like "participants", "topics", "analytics", etc.

As a developer, I could implement this mega-table as a single stream (truer to the interface and an "EL" workflow), or I could create more sensible child tables along the lines of the "multistream" approach described in this issue (<https://gitlab.com/meltano/sdk/-/issues/167>). I'm curious if anyone has a philosophical opinion on the approach?

I went with the "mega table" approach FWIW, <https://github.com/immuta/tap-gong/blob/main/tap_gong/streams/calls.py>

The way i'd do it is create a huge table with the tap and then transform as needed with dbt

<@U06CC4FA5PV> Curious – how did you get access to their API? I thought it was closed access only. Are their API docs public?

it's hidden back there, for sure. but if you are an admin, you can generate the api key and get access to their docs, which are behind a login wall: <https://app.gong.io/settings/api/documentation#overview>

<@U06C5F8FU3G> that's my instinct as well because it mirrors the "replicate then transform" most closely. i can see two arguments: 1) some databases make it hard to work with nested json (i.e. redshift), and 2) it means every developer who wants to transform has to hand-roll their own dbt transforms. it's actually a good case imo for the meltano tap + `--transform tap-based-package` approach

Everybody's needs are different so everyone having to do their own dbt transforms (with a sensible default) seems like a great idea to me

But i agree about redshift, having used both that one and Snowflake

json handling was one reason we switched!