Protocol Documentation

Table of Contents

compact_formats.proto

Top

ChainMetadata

ChainMetadata represents information about the state of the chain as of a given block.

FieldTypeLabelDescription
saplingCommitmentTreeSize uint32

the size of the Sapling note commitment tree as of the end of this block

orchardCommitmentTreeSize uint32

the size of the Orchard note commitment tree as of the end of this block

CompactBlock

CompactBlock is a packaging of ONLY the data from a block that's needed to:

1. Detect a payment to your shielded Sapling address

2. Detect a spend of your shielded Sapling notes

3. Update your witnesses to generate new Sapling spend proofs.

FieldTypeLabelDescription
protoVersion uint32

the version of this wire format, for storage

height uint64

the height of this block

hash bytes

the ID (hash) of this block, same as in block explorers

prevHash bytes

the ID (hash) of this block's predecessor

time uint32

Unix epoch time when the block was mined

header bytes

(hash, prevHash, and time) OR (full header)

vtx CompactTx repeated

zero or more compact transactions from this block

chainMetadata ChainMetadata

information about the state of the chain as of this block

CompactOrchardAction

https://github.com/zcash/zips/blob/main/zip-0225.rst#orchard-action-description-orchardaction

(but not all fields are needed)

FieldTypeLabelDescription
nullifier bytes

[32] The nullifier of the input note

cmx bytes

[32] The x-coordinate of the note commitment for the output note

ephemeralKey bytes

[32] An encoding of an ephemeral Pallas public key

ciphertext bytes

[52] The first 52 bytes of the encCiphertext field

CompactSaplingOutput

output encodes the `cmu` field, `ephemeralKey` field, and a 52-byte prefix of the

`encCiphertext` field of a Sapling Output Description. These fields are described in

section 7.4 of the Zcash protocol spec:

https://zips.z.cash/protocol/protocol.pdf#outputencodingandconsensus

Total size is 116 bytes.

FieldTypeLabelDescription
cmu bytes

note commitment u-coordinate

ephemeralKey bytes

ephemeral public key

ciphertext bytes

first 52 bytes of ciphertext

CompactSaplingSpend

CompactSaplingSpend is a Sapling Spend Description as described in 7.3 of the Zcash

protocol specification.

FieldTypeLabelDescription
nf bytes

nullifier (see the Zcash protocol specification)

CompactTx

CompactTx contains the minimum information for a wallet to know if this transaction

is relevant to it (either pays to it or spends from it) via shielded elements

only. This message will not encode a transparent-to-transparent transaction.

FieldTypeLabelDescription
index uint64

Index and hash will allow the receiver to call out to chain explorers or other data structures to retrieve more information about this transaction. the index within the full block

hash bytes

the ID (hash) of this transaction, same as in block explorers

fee uint32

The transaction fee: present if server can provide. In the case of a stateless server and a transaction with transparent inputs, this will be unset because the calculation requires reference to prior transactions. If there are no transparent inputs, the fee will be calculable as: valueBalanceSapling + valueBalanceOrchard + sum(vPubNew) - sum(vPubOld) - sum(tOut)

spends CompactSaplingSpend repeated

outputs CompactSaplingOutput repeated

actions CompactOrchardAction repeated

darkside.proto

Top

DarksideBlock

A block is a hex-encoded string.

FieldTypeLabelDescription
block string

DarksideBlocksURL

DarksideBlocksURL is typically something like:

https://raw.githubusercontent.com/zcash-hackworks/darksidewalletd-test-data/master/basic-reorg/before-reorg.txt

FieldTypeLabelDescription
url string

DarksideEmptyBlocks

FieldTypeLabelDescription
height int32

nonce int32

count int32

DarksideHeight

FieldTypeLabelDescription
height int32

DarksideMetaState

FieldTypeLabelDescription
saplingActivation int32

branchID string

chainName string

startSaplingCommitmentTreeSize uint32

startOrchardCommitmentTreeSize uint32

DarksideSubtreeRoots

FieldTypeLabelDescription
shieldedProtocol ShieldedProtocol

startIndex uint32

subtreeRoots SubtreeRoot repeated

DarksideTransactionsURL

DarksideTransactionsURL refers to an HTTP source that contains a list

of hex-encoded transactions, one per line, that are to be associated

with the given height (fake-mined into the block at that height)

FieldTypeLabelDescription
height int32

url string

DarksideStreamer

Darksidewalletd maintains two staging areas, blocks and transactions. The

Stage*() gRPCs add items to the staging area; ApplyStaged() "applies" everything

in the staging area to the working (operational) state that the mock zcashd

serves; transactions are placed into their corresponding blocks (by height).

Method NameRequest TypeResponse TypeDescription
Reset DarksideMetaState Empty

Reset reverts all darksidewalletd state (active block range, latest height, staged blocks and transactions) and lightwalletd state (cache) to empty, the same as the initial state. This occurs synchronously and instantaneously; no reorg happens in lightwalletd. This is good to do before each independent test so that no state leaks from one test to another. Also sets (some of) the values returned by GetLightdInfo(). The Sapling activation height specified here must be where the block range starts.

StageBlocksStream DarksideBlock stream Empty

StageBlocksStream accepts a list of blocks and saves them into the blocks staging area until ApplyStaged() is called; there is no immediate effect on the mock zcashd. Blocks are hex-encoded. Order is important, see ApplyStaged.

StageBlocks DarksideBlocksURL Empty

StageBlocks is the same as StageBlocksStream() except the blocks are fetched from the given URL. Blocks are one per line, hex-encoded (not JSON).

StageBlocksCreate DarksideEmptyBlocks Empty

StageBlocksCreate is like the previous two, except it creates 'count' empty blocks at consecutive heights starting at height 'height'. The 'nonce' is part of the header, so it contributes to the block hash; this lets you create identical blocks (same transactions and height), but with different hashes.

StageTransactionsStream RawTransaction stream Empty

StageTransactionsStream stores the given transaction-height pairs in the staging area until ApplyStaged() is called. Note that these transactions are not returned by the production GetTransaction() gRPC until they appear in a "mined" block (contained in the active blockchain presented by the mock zcashd).

StageTransactions DarksideTransactionsURL Empty

StageTransactions is the same except the transactions are fetched from the given url. They are all staged into the block at the given height. Staging transactions to different heights requires multiple calls.

ApplyStaged DarksideHeight Empty

ApplyStaged iterates the list of blocks that were staged by the StageBlocks*() gRPCs, in the order they were staged, and "merges" each into the active, working blocks list that the mock zcashd is presenting to lightwalletd. Even as each block is applied, the active list can't have gaps; if the active block range is 1000-1006, and the staged block range is 1003-1004, the resulting range is 1000-1004, with 1000-1002 unchanged, blocks 1003-1004 from the new range, and 1005-1006 dropped. After merging all blocks, ApplyStaged() appends staged transactions (in the order received) into each one's corresponding (by height) block The staging area is then cleared. The argument specifies the latest block height that mock zcashd reports (i.e. what's returned by GetLatestBlock). Note that ApplyStaged() can also be used to simply advance the latest block height presented by mock zcashd. That is, there doesn't need to be anything in the staging area.

GetIncomingTransactions Empty RawTransaction stream

Calls to the production gRPC SendTransaction() store the transaction in a separate area (not the staging area); this method returns all transactions in this separate area, which is then cleared. The height returned with each transaction is -1 (invalid) since these transactions haven't been mined yet. The intention is that the transactions returned here can then, for example, be given to StageTransactions() to get them "mined" into a specified block on the next ApplyStaged().

ClearIncomingTransactions Empty Empty

Clear the incoming transaction pool.

AddAddressUtxo GetAddressUtxosReply Empty

Add a GetAddressUtxosReply entry to be returned by GetAddressUtxos(). There is no staging or applying for these, very simple.

ClearAddressUtxo Empty Empty

Clear the list of GetAddressUtxos entries (can't fail)

AddTreeState TreeState Empty

Adds a GetTreeState to the tree state cache

RemoveTreeState BlockID Empty

Removes a GetTreeState for the given height from cache if present (can't fail)

ClearAllTreeStates Empty Empty

Clear the list of GetTreeStates entries (can't fail)

SetSubtreeRoots DarksideSubtreeRoots Empty

Sets the subtree roots cache (for GetSubtreeRoots), replacing any existing entries

Stop Empty Empty

Stop causes the server to shut down cleanly.

service.proto

Top

Address

FieldTypeLabelDescription
address string

AddressList

FieldTypeLabelDescription
addresses string repeated

Balance

FieldTypeLabelDescription
valueZat int64

BlockID

A BlockID message contains identifiers to select a block: a height or a

hash. Specification by hash is not implemented, but may be in the future.

FieldTypeLabelDescription
height uint64

hash bytes

BlockRange

BlockRange specifies a series of blocks from start to end inclusive.

Both BlockIDs must be heights; specification by hash is not yet supported.

FieldTypeLabelDescription
start BlockID

end BlockID

ChainSpec

Chainspec is a placeholder to allow specification of a particular chain fork.

Duration

Duration is currently used only for testing, so that the Ping rpc

can simulate a delay, to create many simultaneous connections. Units

are microseconds.

FieldTypeLabelDescription
intervalUs int64

Empty

Empty is for gRPCs that take no arguments, currently only GetLightdInfo.

Exclude

FieldTypeLabelDescription
txid bytes repeated

GetAddressUtxosArg

Results are sorted by height, which makes it easy to issue another

request that picks up from where the previous left off.

FieldTypeLabelDescription
addresses string repeated

startHeight uint64

maxEntries uint32

zero means unlimited

GetAddressUtxosReply

FieldTypeLabelDescription
address string

txid bytes

index int32

script bytes

valueZat int64

height uint64

GetAddressUtxosReplyList

FieldTypeLabelDescription
addressUtxos GetAddressUtxosReply repeated

GetSubtreeRootsArg

FieldTypeLabelDescription
startIndex uint32

Index identifying where to start returning subtree roots

shieldedProtocol ShieldedProtocol

Shielded protocol to return subtree roots for

maxEntries uint32

Maximum number of entries to return, or 0 for all entries.

LightdInfo

LightdInfo returns various information about this lightwalletd instance

and the state of the blockchain.

FieldTypeLabelDescription
version string

vendor string

taddrSupport bool

true

chainName string

either "main" or "test"

saplingActivationHeight uint64

depends on mainnet or testnet

consensusBranchId string

protocol identifier, see consensus/upgrades.cpp

blockHeight uint64

latest block on the best chain

gitCommit string

branch string

buildDate string

buildUser string

estimatedHeight uint64

less than tip height if zcashd is syncing

zcashdBuild string

example: "v4.1.1-877212414"

zcashdSubversion string

example: "/MagicBean:4.1.1/"

donationAddress string

Zcash donation UA address

PingResponse

PingResponse is used to indicate concurrency, how many Ping rpcs

are executing upon entry and upon exit (after the delay).

This rpc is used for testing only.

FieldTypeLabelDescription
entry int64

exit int64

RawTransaction

RawTransaction contains the complete transaction data. It also includes the

height for the block in which the transaction was included in the main

chain, if any (as detailed below).

FieldTypeLabelDescription
data bytes

The serialized representation of the Zcash transaction.

height uint64

The height at which the transaction is mined, or a sentinel value. Due to an error in the original protobuf definition, it is necessary to reinterpret the result of the `getrawtransaction` RPC call. Zcashd will return the int64 value `-1` for the height of transactions that appear in the block index, but which are not mined in the main chain. Here, the height field of `RawTransaction` was erroneously created as a `uint64`, and as such we must map the response from the zcashd RPC API to be representable within this space. Additionally, the `height` field will be absent for transactions in the mempool, resulting in the default value of `0` being set. Therefore, the meanings of the `height` field of the `RawTransaction` type are as follows: * height 0: the transaction is in the mempool * height 0xffffffffffffffff: the transaction has been mined on a fork that is not currently the main chain * any other height: the transaction has been mined in the main chain at the given height

SendResponse

A SendResponse encodes an error code and a string. It is currently used

only by SendTransaction(). If error code is zero, the operation was

successful; if non-zero, it and the message specify the failure.

FieldTypeLabelDescription
errorCode int32

errorMessage string

SubtreeRoot

FieldTypeLabelDescription
rootHash bytes

The 32-byte Merkle root of the subtree.

completingBlockHash bytes

The hash of the block that completed this subtree.

completingBlockHeight uint64

The height of the block that completed this subtree in the main chain.

TransparentAddressBlockFilter

TransparentAddressBlockFilter restricts the results to the given address

or block range.

FieldTypeLabelDescription
address string

t-address

range BlockRange

start, end heights

TreeState

The TreeState is derived from the Zcash z_gettreestate rpc.

FieldTypeLabelDescription
network string

"main" or "test"

height uint64

block height

hash string

block id

time uint32

Unix epoch time when the block was mined

saplingTree string

sapling commitment tree state

orchardTree string

orchard commitment tree state

TxFilter

A TxFilter contains the information needed to identify a particular

transaction: either a block and an index, or a direct transaction hash.

Currently, only specification by hash is supported.

FieldTypeLabelDescription
block BlockID

block identifier, height or hash

index uint64

index within the block

hash bytes

transaction ID (hash, txid)

ShieldedProtocol

NameNumberDescription
sapling 0

orchard 1

CompactTxStreamer

Method NameRequest TypeResponse TypeDescription
GetLatestBlock ChainSpec BlockID

Return the height of the tip of the best chain

GetBlock BlockID CompactBlock

Return the compact block corresponding to the given block identifier

GetBlockNullifiers BlockID CompactBlock

Same as GetBlock except actions contain only nullifiers

GetBlockRange BlockRange CompactBlock stream

Return a list of consecutive compact blocks

GetBlockRangeNullifiers BlockRange CompactBlock stream

Same as GetBlockRange except actions contain only nullifiers

GetTransaction TxFilter RawTransaction

Return the requested full (not compact) transaction (as from zcashd)

SendTransaction RawTransaction SendResponse

Submit the given transaction to the Zcash network

GetTaddressTxids TransparentAddressBlockFilter RawTransaction stream

Return the transactions corresponding to the given t-address within the given block range NB - this method is misnamed, it returns transactions, not transaction IDs.

GetTaddressBalance AddressList Balance

GetTaddressBalanceStream Address stream Balance

GetMempoolTx Exclude CompactTx stream

Return the compact transactions currently in the mempool; the results can be a few seconds out of date. If the Exclude list is empty, return all transactions; otherwise return all *except* those in the Exclude list (if any); this allows the client to avoid receiving transactions that it already has (from an earlier call to this rpc). The transaction IDs in the Exclude list can be shortened to any number of bytes to make the request more bandwidth-efficient; if two or more transactions in the mempool match a shortened txid, they are all sent (none is excluded). Transactions in the exclude list that don't exist in the mempool are ignored.

GetMempoolStream Empty RawTransaction stream

Return a stream of current Mempool transactions. This will keep the output stream open while there are mempool transactions. It will close the returned stream when a new block is mined.

GetTreeState BlockID TreeState

GetTreeState returns the note commitment tree state corresponding to the given block. See section 3.7 of the Zcash protocol specification. It returns several other useful values also (even though they can be obtained using GetBlock). The block can be specified by either height or hash.

GetLatestTreeState Empty TreeState

GetSubtreeRoots GetSubtreeRootsArg SubtreeRoot stream

Returns a stream of information about roots of subtrees of the note commitment tree for the specified shielded protocol (Sapling or Orchard).

GetAddressUtxos GetAddressUtxosArg GetAddressUtxosReplyList

GetAddressUtxosStream GetAddressUtxosArg GetAddressUtxosReply stream

GetLightdInfo Empty LightdInfo

Return information about this lightwalletd instance and the blockchain

Ping Duration PingResponse

Testing-only, requires lightwalletd --ping-very-insecure (do not enable in production)

Scalar Value Types

.proto TypeNotesC++JavaPythonGoC#PHPRuby
double double double float float64 double float Float
float float float float float32 float float Float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead. int32 int int int32 int integer Bignum or Fixnum (as required)
int64 Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead. int64 long int/long int64 long integer/string Bignum
uint32 Uses variable-length encoding. uint32 int int/long uint32 uint integer Bignum or Fixnum (as required)
uint64 Uses variable-length encoding. uint64 long int/long uint64 ulong integer/string Bignum or Fixnum (as required)
sint32 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s. int32 int int int32 int integer Bignum or Fixnum (as required)
sint64 Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s. int64 long int/long int64 long integer/string Bignum
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int uint32 uint integer Bignum or Fixnum (as required)
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long uint64 ulong integer/string Bignum
sfixed32 Always four bytes. int32 int int int32 int integer Bignum or Fixnum (as required)
sfixed64 Always eight bytes. int64 long int/long int64 long integer/string Bignum
bool bool boolean boolean bool bool boolean TrueClass/FalseClass
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode string string string String (UTF-8)
bytes May contain any arbitrary sequence of bytes. string ByteString str []byte ByteString string String (ASCII-8BIT)