The Graph Network In Depth - Part 1

This post was originally published in October of 2019 and has been updated to reflect changes to the protocol design that have been adopted in the 1+ years since the original post was published.

Introduction

When we first introduced The Graph in July of 2018, we shared our vision of building a decentralized indexing protocol for Web3. The team has been hard at work and today I’m excited to share the design of the first version of The Graph Network in detail. The Graph Network is core infrastructure for Web3—a necessary component for delivering decentralized applications with consumer-grade performance.

For this post, I assume some prior familiarity with The Graph. If you’ve never heard of The Graph, a great place to start is our announcement post, our docs, or several great posts from our community.

This is the first part of a two part post exploring the design of The Graph Network. You can jump to part two here.

Full-Stack Decentralization

The mission of The Graph is to enable internet applications that are entirely powered by public infrastructure.

Full-stack decentralization will enable applications that are robust to business failures and rent seeking and also facilitate an unprecedented level of interoperability. Users and developers will be able to know that software they invest time and money into can’t suddenly disappear.

To reach this vision of fully decentralized applications (dApps), it’s critical that we shift from a paradigm of businesses paying for the ongoing storage, compute, and other services required to keep an application running, toward users directly paying networks of decentralized service providers for granular usage of these resources.

Today, most “decentralized” applications only adopt such a model in the bottom layer of the stack—the blockchain—where users pay for transactions that modify application state. The rest of the stack continues to be operated by centralized businesses and is subject to arbitrary failures and rent seeking.

What is The Graph Network

The Graph Network decentralizes the API and query layer of the internet application stack. For the first time it will be possible to efficiently query blockchain data without relying on a centralized service provider.

Today, developers can run a Graph Node on their own infrastructure, or they can build on our hosted service. In The Graph Network, any Indexer will be able to stake Graph Tokens (GRT) to participate in the network and earn rewards for indexing subgraphs and fees for serving queries on those subgraphs.

Consumers will be able to query this diverse set of Indexers by paying for their metered usage, proving a model where the laws of supply and demand sustain the services provided by the protocol.

Protocol Roles

These are the roles that interact with the system, the behaviors they must engage in for the protocol to function correctly, and what incentives motivate them.

• Consumers. Consumers pay Indexers for queries. These will typically be end users but could also be web services or middleware that integrate with The Graph.
• Indexers. Indexers are the node operators of The Graph. They are motivated by earning financial rewards.
• Curators. Curators use GRT to signal what subgraphs are valuable to index. These will typically be developers and share their motivations but could also be end users supporting a valuable service they rely upon or a persona that is purely financially motivated.
• Delegators. Delegators put GRT at stake on behalf of an Indexer in order to earn a portion of indexer rewards and fees, without having to personally run a Graph Node. They are financially motivated.

Usage

Developers

One notable difference is in how developers deploy subgraphs. Rather than deploying to a local or hosted Graph Node, they will deploy their subgraph to a registry hosted on Ethereum and deposit a stake of GRT to curate that subgraph. This serves as a signal to Indexers that this subgraph should be indexed.

End Users

For end users, the major difference is that rather than interacting with centralized APIs that are subsidized, they will need to begin paying to query a decentralized network of Indexers. This will be done via a query engine running on their machine—either in the browser, as an extension, or embedded in the dApp.

The query engine allows the user to safely query the vast amounts of data stored on The Graph without having to personally do the work to compute and store that data. The query engine also acts as a trading engine, making decisions such as which Indexers to do business with or how much to pay, based on the dApp being used or the user’s preferences.

For the query engine to provide a good user experience, it will need the capability to automatically sign micropayment transactions on behalf of users rather than prompting them for every transaction that needs signing. We’re working with several state channel teams building on Ethereum to make sure that the wallets and functionality they ship meets the needs of metered usage protocols like The Graph. In the meantime, we will host a gateway that subsidizes queries on behalf of users.

Indexers

Indexers will be able to join The Graph by staking GRT and running a version of Graph Node.

They will also want to run an indexer agent that programmatically monitors their resource usage, sets prices, and decides which subgraphs to index. The indexer agent will be pluggable, and we expect that node operators will experiment with their own pricing models and strategies to gain a competitive edge in the marketplace over other Indexers.

Curators and Delegators

Curators and delegators will curate and delegate via Graph Explorer. Users will be able to curate and delegate through the Graph Explorer, that will be a fully decentralized application that requires a dApp-enabled browser with an Ethereum wallet.

Architecture

The Graph Network includes smart contracts that run on Ethereum combined with a variety of additional services and clients that operate off-chain.

Query Market

The query market serves a similar purpose to an API in a traditional cloud-based application—efficiently serving data required by a front end running on a user’s device. The key difference is that whereas a traditional API is operated by a single economic entity that users have no say over, the query market comprises a decentralized network of Indexers, all competing to provide the best service at the best price.

This redundancy means that even if a single Indexer goes offline, as long as there exists demand for querying a dataset, other Indexers will be incentivized to absorb the additional work.

Transactions in the query market are priced based on the bandwidth and compute required to process the query.

Let’s take a look at what a typical flow might look like for a consumer interacting with the query market.

• Service Discovery. The consumer asks The Graph which Indexers have the data they are interested in.

• Indexer Selection. The consumer selects an Indexer to transact with based on which they deem most likely to provide the highest quality service at the best price. This involves weighing the utility of performance, freshness of data, economic security, as well as subjective measures of Indexer reputation.

Indexers express their prices using a cost model which a Consumer can use to produce a GRT-denominated cost based on the features of a query and variables provided by the Indexer such as database statistics, the USD/GRT price, or a flag to indicate whether the Indexer is throttling queries. Thus, pricing is set between an Indexer and a Consumer, rather than at the network level as is the case in a smart contract platform like Ethereum.

• Query + Conditional Micropayment. The consumer sends the Indexer a query along with a conditional micropayment that specifies how much they are willing to pay for compute and bandwidth.

The query specifies which subgraph to query, as of what block, and what data is expected to be available.

• Response + Attestation. If the Indexer accepts the price offered by the consumer, then they process the query and respond with the resulting data, as well as an attestation that this response is correct. Providing this attestation unlocks the conditional micropayment.

The attestation is produced deterministically and is uniquely attributable to the Indexer for the purposes of verification and dispute resolution elsewhere in the protocol.

A single decentralized application querying The Graph may use multiple subgraphs indexed by different Indexers and in that case would go through the above flow for each subgraph being queried.

Graph Tokens

To support the functioning of the query market, the protocol introduces a native utility token: Graph Tokens (GRT).

Graph Tokens have two primary uses that are indispensable to the functioning of the protocol:

• Indexer Staking. Indexers deposit Graph Tokens to be discoverable in the query market and to provide economic security for the work they are performing.
• Curator Signaling. Curators deposit Graph Tokens in a curation market, where they are rewarded for correctly predicting which subgraphs will be valuable to the network.

Consumers are encouraged to hold ETH or the stable coin of their choice in their wallets. Payments will be settled, however, in GRT to ensure a common unit of account across the protocol.

In addition to the uses outlined above, having a native utility token enables us to incentivize certain behaviors that benefit the network as a whole, such as the indexing of new subgraphs, through new token issuance.

Continue Reading

At this point we’ve walked through the most important high level concepts in The Graph. There are, however, numerous secondary concepts and mechanisms which support an efficient query market, a performant protocol and a good experience for dapp developers.

In part two we’ll be continuing with an exploration of these components, including the various usages of the Graph Token, such as indexer staking mechanics, curation markets and indexer rewards.

We’ll also be covering our micropayments infrastructure, how we verify query responses and how end users will interact with The Graph via an “explorer” decentralized application.

See you there!