Join Slack
Powered by
Hi! has anyone been dealing with crawlers that can...
# announcements
d
dav
02/06/2021, 3:43 PM
Hi! has anyone been dealing with crawlers that can periodically mirror data 1:1 from a GraphQL API to a local clone? basically I just want a way to periodically mirror data from a graphql server-api to a local disk cache.
d
douwe_maan
02/08/2021, 4:24 PM
There isn't currently a Singer tap that will work with any arbitrary GraphQL API, but if you've got the Python skills you can write a custom tap that uses (for example)
https://github.com/graphql-python/gql
to pull data from your API and output it in accordance with the Singer spec, which should then work beautifully in Meltano and any arbitrary target (loader). The new Singer SDK we're working on (
https://gitlab.com/meltano/singer-sdk
,
https://gitlab.com/meltano/meltano/-/issues/2401
) will have some convenience classes to make it easy to write a tap for GraphQL APIs:
https://gitlab.com/meltano/singer-sdk/-/blob/development/singer_sdk/streams/graphql.py
, but that won't be ready for a little while.
d
dav
02/09/2021, 7:39 AM
thanks!
6
Views
Open in Slack
Previous
Next