Channels
jobs_gigs
infra-deployment
data-news
singer-tap-development
-tap-salesforce-sdk
introductions
office-hours-planning
thanks
windows
singer-target-development
demo-day
troubleshooting
hub
best-practices
singer-taps
singer-targets
plugins-dbt
motorsports
gaming
shramp
discussion
bookclub
company-handbook
documentation
music
provider-status
meltano-alerts
90-day-fiance
hub-repo-activity
show-us-what-you-got
tidbyt
meltano-squared
fitness
loc-texas
pets-of-meltano
c04c499jwjc-target-bigquery-dev-chatter
singer-ecosystem-activity
c04b3ud8pge-github-mds-in-a-box-project
analyze-athena
3dprinting
hack-day
c04cla0s4g0-postgres-tap-target-sdk
weather
stack-overflow-activity
devcontainers
labs
meltano-plugin-development
c04bg3978nt-snowflake-tap-target-sdk
arc-browser-users
gpt-and-llm-development
product
pair-programming
loc-mexico-city
c04npl29ktq-evidence-extension-dev
meltano-org-contribution-activity
vendor-content
guild-singer-ecosystem
c05cnuf699b-file-tap-target-sdk
cli
announcements
model-specs
engineering-chatter
webapp
random
getting-started
docker
plugins-general
meltano-repo-activity
contributing
Powered by
#announcements
Title
# announcements
w
wide-lifeguard-13477
02/06/2021, 3:43 PM
Hi! has anyone been dealing with crawlers that can periodically mirror data 1:1 from a GraphQL API to a local clone? basically I just want a way to periodically mirror data from a graphql server-api to a local disk cache.
✅ 1
r
ripe-musician-59933
02/08/2021, 4:24 PM
There isn't currently a Singer tap that will work with any arbitrary GraphQL API, but if you've got the Python skills you can write a custom tap that uses (for example)
https://github.com/graphql-python/gql
to pull data from your API and output it in accordance with the Singer spec, which should then work beautifully in Meltano and any arbitrary target (loader). The new Singer SDK we're working on (
https://gitlab.com/meltano/singer-sdk
,
https://gitlab.com/meltano/meltano/-/issues/2401
) will have some convenience classes to make it easy to write a tap for GraphQL APIs:
https://gitlab.com/meltano/singer-sdk/-/blob/development/singer_sdk/streams/graphql.py
, but that won't be ready for a little while.
w
wide-lifeguard-13477
02/09/2021, 7:39 AM
thanks!
Post