blockworks-foundation
/
sbv2-solana
mirror of https://github.com/blockworks-foundation/sbv2-solana.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
sbv2-solana/website/docs/feed/architecture.mdx

204 lines
13 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
sidebar_position: 1
slug: .
title: Architecture
---
# Data Feed Architecture
import MarkdownImage from "/src/components/MarkdownImage";
import { Box, Typography, Grid } from "@mui/material";
import Link from "@docusaurus/Link";
An aggregator or data feed is what on-chain developers use when building smart contracts. A data feed is a collection of jobs that get aggregated to produce a single, deterministic result. Typically the first task in a job will fetch external data with subsequent tasks responsible for parsing the response and transforming the value into a single data type, like an integer or decimal.
When an oracle is assigned to process a data feed update, the oracle executes the defined jobs, computes the weighted median of the job responses, and publishes the result on-chain. If sufficient oracles respond, the on-chain program computes the final result as the median of the assigned oracle responses.
Data feeds published on Solana are public and there is no mechanism to prevent other users from reading and consuming the data. Because of this, Switchboard, by default, treats feeds as public utilities allowing anyone to contribute. This is by design as data feeds should be community controlled. If a program is relying on an oracle and the lease expires, any user is allowed to extend the lease, push on a crank, and keep the feed updating, but only if the feed config allows it. Switchboard envisions data feeds being community governed by the protocols supporting them. As a feed grows in popularity and is used across protocols, the feed maintenance cost can be spread across the protocols to reduce the economic burden on a single entity.
## Configuration
<Grid container spacing={3}>
<Grid item md={4} sm={12} order={{ xs: 2, sm: 1 }}>
<ul>
<li>
<b>Aggregator: </b>Contains the data feed configuration, dictating how
data feed updates get requested, updated, and resolved on-chain.
</li>
<li>
<b>Job Account: </b>Stores the blueprints for how data is fetched
off-chain for a particular data source.
</li>
<li>
<b>Permission Account: </b>Permits a data feed to join an oracle queue.
</li>
<li>
<b>Lease Contract: </b>Pre-funded escrow contract to reward oracles for
their work.
</li>
<li>
<b>Crank: </b>Optional, owned by the queue and allows a data feed to be
updated at a regular interval.
</li>
<li>
<b>History Buffer: </b>Optional, allows a feed to store the last N
values.
</li>
</ul>
</Grid>
<Grid item md={8} sx={12} order={{ xs: 1, sm: 2 }}>
<MarkdownImage
img="/img/feeds/Aggregator_Accounts.png"
sx={{
display: "flex",
}}
/>
</Grid>
</Grid>
<hr />
:::tip
See [/idl/accounts/AggregatorAccountData](/idl/accounts/AggregatorAccountData) for the full list of an AggregatorAccount's configuration parameters.
:::
## Job Definitions
An Aggregator Account stores a collection of Job Account public keys along with the hashes of the job definitions. This is to prevent malicious RPC nodes from providing incorrect task definitions to oracles before fulfillment.
A Job Account is a collection of [Switchboard Tasks](/api/tasks) that get executed by an oracle sequentially. Each Job Account typically corresponds to a single data source. A data feed requires at least one job account and at most 16 job accounts. Switchboard Job Accounts can be used to source data from:
- HTTP endpoints, public or private$^{[1]}$
- Websockets
- On-Chain data from Solana, Ethereum, etc
- Anchor programs
- JupiterSwap
- Uniswap
- SushiSwap
- Saber
- ... and more
$^{[1]}$ Endpoints requiring an API key require a [Private Queue](../queue/private-queues.mdx) to prevent leaking the API key on-chain
### Job Weights
A data feed can assign job weights to a job account which will be used when the oracle calculates the median across the job responses. This is useful to weight data sources by some metric such as liquidity or a reliability score.
It is **strongly** recommended to utilize job weights as _not all data sources are created equally_.
:::info
Currently the only way to set a job weight is to remove and re-add the job account to a feed.
:::
### Lease Contract
The LeaseContract is a pre-funded escrow account to reward oracles for fulfilling update request. The LeaseContract has a pre-specified `lease.withdrawAuthority` which is the only wallet allowed to withdraw funds from the lease escrow. Any user is able to contribute to a LeaseContract and keep the feed updating.
When a new openRound is successfully requested for a data feed, the user who requested it is transferred `queue.reward` tokens from the feeds LeaseContract. This is to incentivize users and crank turners to keep feeds updating based on a feeds config.
When a data feed result is accepted on-chain by a batch of oracles, the oracle rewards, as specified by `queue.reward`, are automatically deducted from the `lease.escrow` and transferred to an `oracle.tokenAccount`.
## Requesting Updates
A feed is updated when someone calls `aggregatorOpenRound` on-chain. If openRound is called before `aggregator.minUpdateDelaySeconds` have elapsed, the openRound call will fail and the user will forfeit their transaction fees. If successful, the user is rewarded for keeping the feed updating.
### Periodic Updates
Any data feed permitted to request updates on a queue is also permitted to join a queue's existing Crank, `aggregator.crankPubkey`. A Crank is the scheduling mechanism behind feeds that allow them to be periodically updated. The Crank is a buffer account that stores a collection of aggregator public keys, ordered by their next available update, with some level of jitter added to prevent a predictable oracle allocation cycle
When a feeds Lease Contract is low on funds, it is automatically removed from the crank and must be manually repushed upon refunding the LeaseContract.
A feed can set `aggregator.disableCrank` to prevent being pushed onto a Crank and draining it's lease.
## Data Feed Cost
Each data feed update cost can be calculated by the following equation:
$T_{costPerUpdate}=(1 + numSuccess) × T_{queueReward}$
where,
- `T` is the raw token amount in base units (_Ex: lamports or satoshis_)
- _`+1`_ is to reward the update requester for keeping the feed updating
- `numSuccess` is the number of successful oracle responses, which will always be between `[aggregator.minOracleResults, aggregator.oracleRequestBatchSize]`
- `queue.reward` is the queue's set oracle reward
If an update round fails to receive `minOracleResults`, only the update requester receives funds from the lease escrow.
### Variance Threshold
A feed can set an `aggregator.varianceThreshold` to instruct an oracle to skip reporting a value on-chain if the percentage change between the current result and the `aggregator.previousConfirmedRoundResult` is not exceeded. This is a cost saving tool to conserve lease cost during low volatility.
A feeds `aggregator.forceReportPeriod` is the compliment and instructs an oracle to always report a result if `aggregator.forceReportPeriod` seconds have elapsed since the last successful confirmed round. This can be thought of as the maximum allowable staleness for a feed.
The two settings above can greatly increase the lifespan of a feed's lease but also makes it difficult to estimate the remaining time on a lease.
Check out [@switchboard-xyz/lease-observer](https://github.com/switchboard-xyz/switchboard-v2/tree/main/packages/lease-observer) to get PagerDuty alerts when a lease crosses a low balance threshold.
## History Buffer
A history buffer account stores a set number of accepted results for an aggregator, and given Solana’s maximum account size of 10MB, the maximum number of samples a single history buffer can support is ~350,000 samples. An aggregator can only have a single history buffer associated with it.
A history buffer has a static account size when it is initialized, equal to: `12 Bytes + (28 Bytes × Num Samples)`. Each time an aggregator value is updated on-chain, the associated history buffer is shifted to the right, and the last value is dropped.
This feature allows Switchboard tasks to parse a history buffer and perform a set of calculations, such as the TwapTask. This allows feeds to reference other feeds and perform complex calculations based on historical samples.
## Update Lifecycle
Let's walk through what the feed update lifecycle looks like.
### Update Request
- Any user calls [aggregatorOpenRound](/idl/instructions/aggregatorOpenRound), either manually or via a crank turn
- sbv2 program checks if `aggregator.minUpdateDelaySeconds` have passed since the last openRound call
- sbv2 program checks if a LeaseContract has enough funds to reward the oracles for the next round
- sbv2 program assigns the next `aggregator.oracleRequestBatchSize` oracles to the update request and emits an [AggregatorOpenRoundEvent](/idl/events/AggregatorOpenRoundEvent)
### Oracle Execution
- Oracle watches the chain for an [AggregatorOpenRoundEvent](/idl/events/AggregatorOpenRoundEvent) with the oracle's public key assigned to the update
- Oracle fetches the feed and job account definitions from its RPC Provider
- Oracle verifies the job account definitions match the feeds `aggregator.jobHashes`
- Oracle executes the job definitions in parallel
- When an oracle receives `aggregator.minJobResults`, it calculates the weighted median based on the feeds `aggregator.jobWeights`. Note, this is not enforced on-chain and is purely up to the oracle to respect
- If a feed has configured a `aggregator.varianceThreshold` and `aggregator.forceReportPeriod` has not elapsed, the oracle calculates the percentage change between its calculated result and the previous confirmed round. If it does not exceed the feeds `aggregator.varianceThreshold`, the oracle drops the update request and waits for new update request
- If a feeds configuration dictate a new on-chain result, the oracle submits an [aggregatorSaveResult](https://docs.switchboard.xyz/idl/instructions/aggregatorSaveResult) transaction
### Oracle Consensus
- sbv2 program waits for `aggregator.minOracleResults` to be submitted by the assigned oracles
- When sufficient oracle responses, the sbv2 program computes the accepted value from the median of the oracle responses
- If a feed has a history buffer account, the accepted result is pushed onto the buffer
- Oracles that responded within `queue.varianceToleranceMultiplier` are rewarded `queue.reward` from the feed's LeaseContract
- If `queue.slashingEnabled`, oracles that responded outside the `queue.varianceToleranceMultiplier` are slashed `queue.reward` tokens from it's `oracle.tokenAccount` and transferred to the feed's `lease.escrow`
- If additional oracle responses are submitted after a value has been accepted, the median is recalculated based on the new response set, oracle rewards are redistributed, and the history buffer value is updated
## Data Feed Composability
Data feeds may reference other data feeds and build upon each other. It is **_strongly_** recommended that you own any feed that you reference in case of downstream impacts out of your control. While anyone can extend another feeds lease, a lease owner can always withdraw any lease funds and prevent future updates.
As an example, you could construct the following feed definition:
- Create a Switchboard feed that sources SOL/USD prices from a variety of exchanges, each weighted by their 7d volume, along with a history buffer
- Create a Switchboard feed that uses an OracleTask to fetch the Pyth SOL/USD price every 10 seconds, along with a history buffer
- Create a Switchboard feed that uses an OracleTask to fetch the Chainlink SOL/USD price every 10 seconds, along with a history buffer
- Finally, create a Switchboard feed that calculates the 1min TWAP of each source above and returns the median of the results
This is just a small window into how Switchboard feeds can build on each other and let the downstream consumer configure their feeds to meet their own use cases.
## More Information
- [/api/tasks](/api/tasks)
- [/idl/accounts/AggregatorAccountData](/idl/accounts/AggregatorAccountData)
- [/idl/accounts/CrankAccountData](/idl/accounts/CrankAccountData)
- [/idl/accounts/AggregatorHistoryBuffer](/idl/accounts/AggregatorHistoryBuffer)
- [/idl/accounts/PermissionAccountData](/idl/accounts/PermissionAccountData)
- [/idl/accounts/JobAccountData](/idl/accounts/JobAccountData)
- [/idl/accounts/LeaseAccountData](/idl/accounts/LeaseAccountData)
- [feed-parser Typescript Example](https://github.com/switchboard-xyz/switchboard-v2/tree/main/packages/feed-parser)
- [feed-walkthrough Typescript Example](https://github.com/switchboard-xyz/switchboard-v2/tree/main/packages/feed-walkthrough)
Reference in New Issue View Git Blame Copy Permalink