376 lines
8.0 KiB
Plaintext
376 lines
8.0 KiB
Plaintext
---
|
|
sidebar_position: 10
|
|
title: Private Queues
|
|
---
|
|
|
|
A private queue is any Oracle Queue not controlled by the Switchboard DAO.
|
|
|
|
Switchboard is architected to route off-chain data to an on-chain account. A publisher is responsible for building the job definition, which defines the task(s) the oracles must perform to fetch and transform external data. Sometimes a publisher may wish to bring private data on-chain using an API key which poses a set of challenges. In order for the oracle to retrieve the data, they need access to the publisher's API key. Blockchains are public so there is no easy way to conceal the API key on-chain.
|
|
|
|
Switchboard provides the ability to create your own queue with your own set of oracles, allowing the oracles access to your API key so they can resolve the private endpoints.
|
|
|
|
## Variable Expansion
|
|
|
|
Oracles can be provided a `configs.json` file to store various configurations needed to execute job definitions. If an oracle encounters a job definition with a variable, it will parse the `configs.json` and embed the value in the job definition. API keys should be specified in the config under the Job Account public key to prevent a malicous feed from leaking an API key.
|
|
|
|
```json title="configs.json"
|
|
{
|
|
"jobVariables": {
|
|
// Pubkey of the Job account for which this variable expansion applies
|
|
"HtB62K71H49RJbATYpmB6UCMBXLK6G3Q5JtGveTMR8Mt": {
|
|
"VARIABLE_NAME": "abc123"
|
|
},
|
|
// Global variable expansion that applies to any Job account. SEE CAUTION BELOW
|
|
"*": {
|
|
"GLOBAL_VARIABLE_NAME": "abc123"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
:::danger
|
|
|
|
**_Wildcard variables should only be used for testing purposes and never in production._**
|
|
|
|
Private queue's should **_ALWAYS_** have `unpermissionedFeedsEnabled` set to false. Allowing unpermitted feeds could result in a malicious actor creating a job definition that leaks your sensitive API keys.
|
|
|
|
:::
|
|
|
|
## Example
|
|
|
|
We'll be using [commodities-api](https://www.commodities-api.com) to resolve our data for this example. You will need to signup for an account to get a `COMMODITIES_API_KEY`.
|
|
|
|
### Create a Queue
|
|
|
|
First, we will need to create our own queue. The following command will create a queue with a single oracle and crank.
|
|
|
|
```bash
|
|
sbv2 queue:create \
|
|
--name "Private Queue" \
|
|
--keypair ../payer-keypair.json \
|
|
--authority ../payer-keypair.json \
|
|
--numOracles 1 \
|
|
--reward 0 \
|
|
--outputFile "Private_Queue.json"
|
|
```
|
|
|
|
### Start Oracle
|
|
|
|
Create a docker-compose file, replacing `ORACLE_KEY`, `RPC_URL`, and `PAYER_KEYPAIR` with the appropriate values.
|
|
|
|
```yml title="docker-compose.yml"
|
|
version: "3.3"
|
|
services:
|
|
oracle:
|
|
image: "switchboardlabs/node:dev-v2-07-18-22"
|
|
network_mode: host
|
|
restart: always
|
|
secrets:
|
|
- PAYER_SECRETS
|
|
environment:
|
|
- LIVE=1
|
|
- CLUSTER=devnet
|
|
- HEARTBEAT_INTERVAL=30 # Seconds
|
|
- ORACLE_KEY=${ORACLE_KEY}
|
|
- RPC_URL=${RPC_URL}
|
|
volumes:
|
|
- ./configs.json:/configs.json
|
|
secrets:
|
|
PAYER_SECRETS:
|
|
file: ${PAYER_KEYPAIR}
|
|
```
|
|
|
|
We need to embed the commodities-api key in our oracle's configs.json file.
|
|
|
|
```json title="configs.json"
|
|
{
|
|
"jobVariables": {
|
|
// Pubkey of the OracleJob account for which this variable expansion applies or *.
|
|
"*": {
|
|
"COMMODITIES_API_KEY": "YOUR_API_KEY_HERE"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Start the oracle
|
|
|
|
```bash
|
|
docker-compose up
|
|
```
|
|
|
|
### Create WHEAT Aggregator
|
|
|
|
Looking at their docs, we'll need to fetch data from the following endpoint and take the inverse. The aggregator definition will look like this:
|
|
|
|
```json title="Wheat.json"
|
|
{
|
|
"name": "WHEAT",
|
|
"metadata": "",
|
|
"oracleRequestBatchSize": 1,
|
|
"minOracleResults": 1,
|
|
"minJobResults": 1,
|
|
"minUpdateDelaySeconds": 900,
|
|
"jobs": [
|
|
{
|
|
"name": "commodities-api WHEAT",
|
|
"tasks": [
|
|
{
|
|
"httpTask": {
|
|
"url": "https://www.commodities-api.com/api/latest?access_key=${COMMODITIES_API_KEY}&base=USD&symbols=WHEAT"
|
|
}
|
|
},
|
|
{
|
|
"jsonParseTask": {
|
|
"path": "$.data.rates.WHEAT"
|
|
}
|
|
},
|
|
{
|
|
"powTask": {
|
|
"scalar": -1
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
Now we need to create an aggregator and add this job definition to it.
|
|
|
|
```bash
|
|
sbv2 aggregator:create:json wheat.json \
|
|
--keypair ../payer-keypair.json \
|
|
--queueKey QUEUE_KEY_OUTPUTTED_ABOVE \
|
|
--outputFile Wheat_Aggregator.json
|
|
```
|
|
|
|
Now we can request an update from our running oracle
|
|
|
|
```bash
|
|
sbv2 aggregator:update AGGREGATORKEY --keypair PAYERKEYPAIR
|
|
```
|
|
|
|
We should see the oracle respond to the job and update the on-chain value
|
|
|
|
```bash
|
|
sbv2 aggregator:print AGGREGATORKEY
|
|
```
|
|
|
|
### Other Aggregator Definitions
|
|
|
|
<details>
|
|
|
|
<summary>
|
|
|
|
#### Crude
|
|
|
|
</summary>
|
|
|
|
```json title="Crude.json"
|
|
{
|
|
"name": "Crude WTIOIL",
|
|
"metadata": "",
|
|
"oracleRequestBatchSize": 1,
|
|
"minOracleResults": 1,
|
|
"minJobResults": 1,
|
|
"minUpdateDelaySeconds": 900,
|
|
"jobs": [
|
|
{
|
|
"name": "commodities-api WTIOIL",
|
|
"tasks": [
|
|
{
|
|
"httpTask": {
|
|
"url": "https://www.commodities-api.com/api/latest?access_key=${COMMODITIES_API_KEY}&base=USD&symbols=WTIOIL"
|
|
}
|
|
},
|
|
{
|
|
"jsonParseTask": {
|
|
"path": "$.data.rates.WTIOIL"
|
|
}
|
|
},
|
|
{
|
|
"powTask": {
|
|
"scalar": -1
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</details>
|
|
|
|
<details>
|
|
|
|
<summary>
|
|
|
|
#### Gold
|
|
|
|
</summary>
|
|
|
|
```json title="Gold.json"
|
|
{
|
|
"name": "Gold",
|
|
"metadata": "",
|
|
"oracleRequestBatchSize": 1,
|
|
"minOracleResults": 1,
|
|
"minJobResults": 1,
|
|
"minUpdateDelaySeconds": 900,
|
|
"jobs": [
|
|
{
|
|
"name": "commodities-api XAU",
|
|
"tasks": [
|
|
{
|
|
"httpTask": {
|
|
"url": "https://www.commodities-api.com/api/latest?access_key=${COMMODITIES_API_KEY}&base=USD&symbols=XAU"
|
|
}
|
|
},
|
|
{
|
|
"jsonParseTask": {
|
|
"path": "$.data.rates.XAU"
|
|
}
|
|
},
|
|
{
|
|
"powTask": {
|
|
"scalar": -1
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</details>
|
|
|
|
<details>
|
|
|
|
<summary>
|
|
|
|
#### Silver
|
|
|
|
</summary>
|
|
|
|
```json title="Silver.json"
|
|
{
|
|
"name": "Silver",
|
|
"metadata": "",
|
|
"oracleRequestBatchSize": 1,
|
|
"minOracleResults": 1,
|
|
"minJobResults": 1,
|
|
"minUpdateDelaySeconds": 900,
|
|
"jobs": [
|
|
{
|
|
"name": "commodities-api XAG",
|
|
"tasks": [
|
|
{
|
|
"httpTask": {
|
|
"url": "https://www.commodities-api.com/api/latest?access_key=${COMMODITIES_API_KEY}&base=USD&symbols=XAG"
|
|
}
|
|
},
|
|
{
|
|
"jsonParseTask": {
|
|
"path": "$.data.rates.XAG"
|
|
}
|
|
},
|
|
{
|
|
"powTask": {
|
|
"scalar": -1
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</details>
|
|
|
|
<details>
|
|
|
|
<summary>
|
|
|
|
#### Nickel
|
|
|
|
</summary>
|
|
|
|
```json title="Nickel.json"
|
|
{
|
|
"name": "Nickel",
|
|
"metadata": "",
|
|
"oracleRequestBatchSize": 1,
|
|
"minOracleResults": 1,
|
|
"minJobResults": 1,
|
|
"minUpdateDelaySeconds": 900,
|
|
"jobs": [
|
|
{
|
|
"name": "commodities-api NI",
|
|
"tasks": [
|
|
{
|
|
"httpTask": {
|
|
"url": "https://www.commodities-api.com/api/latest?access_key=${COMMODITIES_API_KEY}&base=USD&symbols=NI"
|
|
}
|
|
},
|
|
{
|
|
"jsonParseTask": {
|
|
"path": "$.data.rates.NI"
|
|
}
|
|
},
|
|
{
|
|
"powTask": {
|
|
"scalar": -1
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</details>
|
|
|
|
<details>
|
|
|
|
<summary>
|
|
|
|
#### Coffee
|
|
|
|
</summary>
|
|
|
|
```json title="Coffee.json"
|
|
{
|
|
"name": "Coffee",
|
|
"metadata": "",
|
|
"oracleRequestBatchSize": 1,
|
|
"minOracleResults": 1,
|
|
"minJobResults": 1,
|
|
"minUpdateDelaySeconds": 900,
|
|
"jobs": [
|
|
{
|
|
"name": "commodities-api COFFEE",
|
|
"tasks": [
|
|
{
|
|
"httpTask": {
|
|
"url": "https://www.commodities-api.com/api/latest?access_key=${COMMODITIES_API_KEY}&base=USD&symbols=COFFEE"
|
|
}
|
|
},
|
|
{
|
|
"jsonParseTask": {
|
|
"path": "$.data.rates.COFFEE"
|
|
}
|
|
},
|
|
{
|
|
"powTask": {
|
|
"scalar": -1
|
|
}
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</details>
|