certusone
/
wormhole
mirror of https://github.com/certusone/wormhole.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
wormhole/docs/query_proxy.md

10 KiB
Raw Permalink Blame History

Operating the Wormhole Queries Proxy Server

The Wormhole queries proxy server (sometimes referred to as the CCQ proxy) is a server that listens on a REST endpoint for Wormhole query requests. It validates those requests and forwards them to the guardian CCQ P2P network for processing by the guardians. It then accumulates the responses from the guardians, verifies quorum and forwards the response to the client.

Building the Proxy Server

The proxy server runs as another instance of the guardiand process, similar to the spy. It is built exactly the same as the spy, and requires the same dependencies. Please see the Operations Guide for details on how to build guardiand.

Deploying the Proxy Server

The proxy server can be deployed just like the spy, including potentially running in a container. Note that it requires a public IP address to listen for REST requests, and it needs to be able to reach the guardian P2P network.

The proxy is not particularly resource intensive, so should run successfully on a reasonable size VM.

Configuring the Proxy Server

There are two main parts to configuring the proxy server. The first is setting up the command line arguments, which generally will not change after initial setup. The second part of the configuration is the permissions file, which will change as the requirements of integrators change.

Proxy Server Command Line Arguments

The following is a sample command line for running the proxy server in mainnet.

wormhole $build/bin/guardiand query-server \
      --env "mainnet" \
      --nodeKey /home/ccq/data/ccq_server.nodeKey \
      --permFile "/home/ccq/data/ccq_server.perms.json" \
      --signerKey "/home/ccq/data/ccq_server.signerKey" \
      --listenAddr "[::]:8080" \
      --ethRPC https://eth.drpc.org \
      --ethContract "0x98f3c9e6E3fAce36bAAd05FE09d375Ef1464288B" \
      --logLevel=info \
      --telemetryLokiURL $LOKI_URL \
      --telemetryNodeName "Mainnet CCQ server 1" \
      --promRemoteURL $PROM_URL
  • The env can be mainnet, testnet or devnet.
  • The nodeKey should point to the file containing the P2P key. The first time the proxy runs, if the file does not exist, it will be created. You can look in the proxy server logs to get the generated key.
  • The permFile is the JSON permissions file, which is documented below.
  • The signerKey should point to an armored file containing a key that will be used to sign requests received from integrators who are configured to support auto signing and opt not to sign a request. Please see below for how to generate this file.
  • The listenAddr specifies the port on which the proxy listens for REST requests.
  • The ethRPC and ethContract are used to read the wormhole guardian set on start up. The address above is for mainnet. If you are running in testnet, you should point to Holesky and use 0xa10f2eF61dE1f19f586ab8B6F2EbA89bACE63F7a. (You can confirm these addresses here.) Note that using a public endpoint should be fine, since the proxy only does a single read of the guardian set.
  • The telemetryLokiURL, telemetryNodeName and promRemoteURL are used for telemetry purposes and the values will be provided by Wormhole Foundation personnel if appropriate.

Optional Parameters

  • The gossipAdvertiseAddress argument allows you to specify an external IP to advertize on P2P (use if behind a NAT or running in k8s).
  • The monitorPeers flag will cause the proxy server to periodically check its connectivity to the P2P bootstrap peers, and attempt to reconnect if necessary.
  • The allowAnything flag enables defining users with the allowAnything flag set to true. This is only allowed in testnet and devnet.

Creating the Signing Key File

Do the following to create the signing key file. Note that the block-type must exactly match what is specified below, but the desc can be anything you want.

wormhole$ build/bin/guardiand keygen --desc "Your CCQ proxy server" --block-type "CCQ SERVER SIGNING KEY" /home/ccq/data/ccq_server.signerKey

Guardian Support for a New Proxy Server

The Queries P2P network is permissioned. The guardians will ignore P2P traffic from sources that are not in their configuration. Additionally, they will only honor query requests signed using a key in their configured list. Before you can begin publishing requests from your proxy, you must get a quorum (preferably all) of the guardians to add your values for the following to their configurations:

  • P2P key (the value from nodeKey file, logged on proxy start up).
  • The public key associated with the signing key. See the signerKey file.

Please work with foundation personnel to get your proxy server added to the guardian configurations.

Permissions Configuration

The file specified by the permFile parameter contains JSON that defines the set of allowed queries users, along with the sets of requests they are allowed to make.

File Format

The simplest file would look something like this

{
  "permissions": [
    {
      "userName": "Monitor",
      "apiKey": "insert_generated_api_key_here",
      "allowUnsigned": true,
      "allowedCalls": [
        {
          "ethCall": {
            "note:": "Name of WETH on Ethereum",
            "chain": 2,
            "contractAddress": "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2",
            "call": "0x06fdde03"
          }
        }
      ]
    }
  ]
}

This creates a single user called "Monitor", who will use the specified API key (more on API keys below). This user is allowed to submit unsigned requests (which will be signed using the configured signing key).

This sample user is only allowed to make a single ethCall request on Ethereum (Wormhole chain ID 2), which allows them to call the name method on the contract that resides at 0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2. The call parameter is the first four bytes of the hash of the ABI encoded function call to be allowed.

A given user can have any number of allowed calls (at least one), but they can only make calls that are configured here.

Supported Call Types

The proxy server supports all of the query types supported by the Wormhole Queries protocol. For details on those calls, please see the Wormhole Queries Whitepaper.

The following are the EVM call types, all of which require the chain, contractAddress and call arguments.

  • ethCall
  • ethCallByTimestamp
  • ethCallWithFinality

The following are the Solana call types. Both require the chain parameter plus the extra parameter listed below.

  • solAccount, requires the account parameter.
  • solPDA, requires the programAddress parameter.

The Solana account and and program address can be expressed as either a 32 byte hex string starting with "0x" or as a base 58 value.

Creating New API Keys

Each user must have an API key. These keys only have meaning to the proxy server. They are not passed to the guardians. The proxy requires that a key be present in each query request, and that the specified key exists in the permissions file. Beyond that, the API keys have no special meaning. They can be generated using a site like this.

Updating the Permissions File

The proxy server monitors the permissions file for changes. Whenever a change is detected, it reads the file, validates it, and if it passes validation, switches to the new version. Care should be taken when editing the file while the proxy server is running, because as soon as you save the file, the changes will be picked up (whether they are logically complete or not).

The allowAnything flag

If this flag is specified for a user, then that user may make any call on any supported chain, without restriction. This flag is only allowed if the allowAnything command line argument is specified. If this flag is specified, then allowedCalls must not be specified.

{
  "permissions": [
    {
      "userName": "Monitor",
      "apiKey": "insert_generated_api_key_here",
      "allowUnsigned": true,
      "allowedAnything": true
    }
  ]
}

Telemetry

The proxy server provides two types of telemetry data, logs and metrics.

Logging

The proxy server uses the same logging mechanism as the guardian. It will write to a local file, but can also be configured to publish logs to Grafana using the Loki protocol. If you will be running your proxy server in mainnet, you should contact foundation personnel about getting a Grafana ID to be used for logging and use it to set the --telemetryLokiURL command line argument.

If you set the log level to info, the proxy server logs information on all incoming requests and output bound responses. This can be helpful for determining when requests reach quorum, but may be too chatty as the level of queries traffic grows. If that is the case, you can set the log level to warn.

Metrics

The proxy server uses Prometheus to track various activity and can publish them to Grafana. If you will be running your proxy server in mainnet, you should contact foundation personnel about getting a Grafana ID and use it to set the --promRemoteURL command line argument.

For the set of available metrics, see here.

Troubleshooting

P2P Health

If you think you are having trouble with your access to the P2P network, you can add --monitorPeers to the command line arguments, which will cause the proxy server to periodically check its connectivity to the P2P bootstrap peers, and attempt to reconnect if necessary.

Invalid Requests

If the proxy server determines that a request is invalid, it does the following:

  • Logs an error message using the user name (not the API Key).
  • Increments the appropriate Prometheus metric.
  • Sends a failure response to the user.

Note that if the proxy server thinks a request is valid, but the guardians do not, the guardians silently drop the request, so it will look like a timeout. This is to avoid a denial of service attack on the guardians. This can happen if the proxy server is not properly permissioned on the guardians.

Logging Request Detail.

If a given integrator is reporting problems with their queries, you may find it useful to add the following to their permissions config (at the same level as the API Key, etc).

"logResponses": true,

This will cause the proxy server to log every response received for that user, along with the number of responses and how many are still needed to meet quorum.