Linen
Join Slack
Hi All, I was wondering if <Arch.Dev> uses Meltano under the hood, or it has its own protocol.
# getting-started
s
Hi All, I was wondering if Arch.Dev uses Meltano under the hood, or it has its own protocol.
e
Hi @Siddu Hussain! It uses Meltano under the hood 🙂
s
Hi @Edgar RamĂ­rez (Arch.dev) , I am trying to implement Meltano for my company. Can you help me with any tutorials you know which can help me to set up an end to end system.
Similarly I will like to know if you have experience with zoom tap I feel Zoom has bad design of API and it’s taking us forever to import data for 10k users
e
Can you help me with any tutorials you know which can help me to set up an end to end system.
It depends a lot on the experience and the stack already in use by your team: • https://medium.com/data-manypets/how-to-run-meltano-in-a-container-on-google-cloud-composer-860783d0575c • https://gist.github.com/iEv0lv3/00880b4f578f94a9c12ad3c6294eb02e • https://docs.meltano.com/guide/production/ I also know of people running it on Argo Workflows.
I don't have experience with the Zoom tap, though. Is there a hierarchical relationship between streams that's causing the slowness?
s
@Edgar RamĂ­rez (Arch.dev) - yes you are on point ,it has hierarchical calls which is making this impossible to implement
Can we pull all meta in 1 pipeline and store the user and meeting ID in the state or something and create a new pipeline to run for 1000 users and extract data. if we have passed a list of items likes {"USERID1","MEETINGID1"} is using state a right field or you recommend doing this a different a way
@Edgar RamĂ­rez (Arch.dev) - how are you doing. Any suggestion on the use case above
e
There are hardly any workarounds for badly designed APIs, I'd have to dig into the endpoints in question
Open in Slack
PreviousNext
Powered by

Meltano

Meltano community

Powered by