https://linen.dev logo
Join Slack
Powered by
Hello! Has anyone used the Meltano Development AP...
# getting-started
a
Hello! Has anyone used the Meltano Development API. Can you show some examples? The documentation page has not been completed. I want to create an internal WEB API server to receive data from different taps (like one server for getting any RAW data)
e
The documentation page has not been completed.
Hey @alex_dimov! Which page would that be?
a
Hi @alex_dimov for your ‘internal web api’ is your plan to create a custom target for that?
a
@Edgar Ramírez (Arch.dev) sorry, a lost your message. Here is link https://docs.meltano.com/contribute/api/
@aaron_phethean Mine was that my server has an endpoint for example /tap-cloud-watch-response which accepts a request body like
Copy code
{
     username: 'user1',
     password: 'user2',
     startDate: "11-10-2024"
}
And it returns just an array of data. If this is not possible, then it writes the data to a file and my server retrieves the data from the file and returns it to the client that called this endpoint. The fact is that we have a tap that runs every 15 minutes to synchronize data But also in some places we need to show realTime data from third-party products and it would be stupid to describe all the logic for obtaining logic, etc. in one more place
e
@alex_dimov oh that page is certainly stale 😬. The API was removed from Meltano in 3.0 along with the UI.
🫠 1
a
Maybe you can recommend something for my goal?
s
Hi @alex_dimov @Edgar Ramírez (Arch.dev) Even I am interested in API, do you have any recommendations ?
👀 1
a
We haven't found anything yet
a
Hi @alex_dimov - I may have misunderstood your API requirement. @Sneha Tripathi happy to chat about an API if you want to DM me Matatika sits on top of a meltano project and can launch jobs via an API: https://www.matatika.com/docs/api/resources/jobs Our api can also be used to fetch data: https://www.matatika.com/docs/api/resources/datasets#view-the-data-of-a-dataset
a
It looks like something diff. I want to create server, that will have 100 taps. Then my other micro services will make requsts like { tap: “slack” stream: “userAudit” credentials: { domain : “” clientId: “”, clientSecret: “” }, params: {startDate: “2023.03.03”} } And I will expect response the same as meltano returns when it returns data for targets. That’s what I want to have
a
Yeah, that’s what I gathered. The issue can arise due to the spin up time. So you also need a webhook or some mechanism to come back later. Minting a new url to check is another good model. It’s not a big development to our platform to be honest.
a
no, I need the data asap. I'm going to show them right away on the UI. Sync data will be downloaded in the background. The fact is that we have integration with, for example, 5 products from which we get events, and at the time of creating the integration I need to get a list of organizations and allow users to select events from which organizations they want to see. In order not to create separate queries with the syntax and conversion of the model to our format, I want to use meltano tap, which is already described and which for all 5 products can already return models converted to our format.
Open in Slack
PreviousNext
Powered by