Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
874 views
in Technique[技术] by (71.8m points)

extract - Tool for periodic mirroring of full data-set from GraphQL endpoint?

I'm looking for a tool that can make a clone of data exposed on a GraphQL API.

Basically something that can run periodically and recurively copy the raw data reponse to disk, making use of connection based pagination & cursors to ensure consistency of progress of the mirrored content.

Assuming this would be a runner that extracts data 24/7, it will either have to rewrite/transform already copied data, or even better apply updates in a more event sourced way to make it easier to provide diff-sets of changes in the source API data.

question from:https://stackoverflow.com/questions/66055462/tool-for-periodic-mirroring-of-full-data-set-from-graphql-endpoint

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

I'm not aware of any such tool. I'm not sure such a tool will exist, because

  1. retrieving data from GraphQL requires only the thinnest of layers over the existing GraphQL libraries, which are quite feature-rich
  2. the transformation/writing will likely be part of a different tool. I'm sure several tools for this already exist. The simplest example I could think of is Git. Getting a diff is as simple as running git diff after overwriting an existing version-controlled file.

A simple example of retrieving the data is adapted from the graphql-request Documentation Quickstart

import { request, gql } from 'graphql-request'
import { writeFile } from 'fs/promises'

const query = gql`
  {
    Movie(title: "Inception") {
      releaseDate
      actors {
        name
      }
    }
  }
`

request('https://api.graph.cool/simple/v1/movies', query)
  .then((data) => writeFile('data.json', data))

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...