https://meltano.com/ logo
#announcements
Title
# announcements
w

wide-lifeguard-13477

02/06/2021, 3:43 PM
Hi! has anyone been dealing with crawlers that can periodically mirror data 1:1 from a GraphQL API to a local clone? basically I just want a way to periodically mirror data from a graphql server-api to a local disk cache.
1
r

ripe-musician-59933

02/08/2021, 4:24 PM
There isn't currently a Singer tap that will work with any arbitrary GraphQL API, but if you've got the Python skills you can write a custom tap that uses (for example) https://github.com/graphql-python/gql to pull data from your API and output it in accordance with the Singer spec, which should then work beautifully in Meltano and any arbitrary target (loader). The new Singer SDK we're working on (https://gitlab.com/meltano/singer-sdk, https://gitlab.com/meltano/meltano/-/issues/2401) will have some convenience classes to make it easy to write a tap for GraphQL APIs: https://gitlab.com/meltano/singer-sdk/-/blob/development/singer_sdk/streams/graphql.py, but that won't be ready for a little while.
w

wide-lifeguard-13477

02/09/2021, 7:39 AM
thanks!