|
||
---|---|---|
.github/workflows | ||
ci | ||
examples | ||
yellowstone-grpc-client | ||
yellowstone-grpc-client-nodejs | ||
yellowstone-grpc-geyser | ||
yellowstone-grpc-proto | ||
.editorconfig | ||
.gitignore | ||
CHANGELOG.md | ||
Cargo.lock | ||
Cargo.toml | ||
LICENSE | ||
LICENSING.md | ||
Makefile | ||
README.md | ||
deny.toml | ||
rust-toolchain.toml | ||
rustfmt.toml |
README.md
Yellowstone Dragon's Mouth - a Geyser based gRPC interface for Solana
This repo contains a fully functional gRPC interface for Solana. It is built around Solana's Geyser interface. In this repo we have the plugin as well as sample clients for multiple languages.
It provides the ability to get slots, blocks, transactions, and account update notifications over a standardised path.
For additional documentation, please see: https://docs.triton.one/rpc-pool/grpc-subscriptions
Known bugs
Block reconstruction inside gRPC plugin based on information provided by BlockMeta, unfortunately number of entries for blocks generated on validators always equal to zero. These blocks always will have zero entries. See issue on GitHub: https://github.com/solana-labs/solana/issues/33823
Validator
$ solana-validator --geyser-plugin-config yellowstone-grpc-geyser/config.json
Plugin config check
cargo-fmt && cargo run --bin config-check -- --config yellowstone-grpc-geyser/config.json
Block reconstruction
Geyser interface on block update do not provide detailed information about transactions and accounts updates. To provide this information with block message we need to collect all messages and expect specified order. By default if we failed to reconstruct full block we log error message and increase invalid_full_blocks_total
counter in prometheus metrics. If you want to panic on invalid reconstruction you can change option block_fail_action
in config to panic
(default value is log
).
Filters for streamed data
Please check yellowstone-grpc-proto/proto/geyser.proto for details.
commitment
— commitment level:processed
/confirmed
/finalized
accounts_data_slice
— array of objects{ offset: uint64, length: uint64 }
, allow to receive only required data from accountsping
— optional boolean field. Some cloud providers (like Cloudflare, Fly.io) close the stream if client doesn't send anything during some time. As workaroud you can send same filter every N seconds, but this would be not optimal since you need to keep this filter. Instead, you can send subscribe request withping
field set totrue
and ignore rest of the fields in the request. Since we sentPing
message every 15s from the server, you can send subscribe request withping
as reply and receivePong
message.
Slots
filter_by_commitment
— by default slots sent for all commitment levels, but with this filter you can receive only selected commitment level
Account
Accounts can be filtered by:
account
— acount Pubkey, match to any Pubkey from the arrayowner
— account owner Pubkey, match to any Pubkey from the arrayfilters
— same asgetProgramAccounts
filters, array ofdataSize
orMemcmp
(bytes, base58, base64 are supported)
If all fields are empty then all accounts are broadcasted. Otherwise fields works as logical AND
and values in arrays as logical OR
(except values in filters
that works as logical AND
).
Transactions
vote
— enable/disable broadcastvote
transactionsfailed
— enable/disable broadcastfailed
transactionssignature
— match only specified transactionaccount_include
— filter transactions that use any account from the listaccount_exclude
— opposite toaccount_include
account_required
— require all accounts from the list to be used in transaction
If all fields are empty then all transactions are broadcasted. Otherwise fields works as logical AND
and values in arrays as logical OR
.
Entries
Currently we do not have filters for the entries, all entries broadcasted.
Blocks
account_include
— filter transactions and accounts that use any account from the listinclude_transactions
— include all transactionsinclude_accounts
— include all accounts updatesinclude_entries
— include all entries
Blocks meta
Same as Blocks
but without transactions
, accounts
and entries. Currently we do not have filters for block meta, all messages are broadcasted.
Limit filters
It's possible to add limits for filters in the config. If filters
field is omitted then filters doesn't have any limits.
"grpc": {
"filters": {
"accounts": {
"max": 1,
"any": false,
"account_max": 10,
"account_reject": ["TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"],
"owner_max": 10,
"owner_reject": ["11111111111111111111111111111111"]
},
"slots": {
"max": 1
},
"transactions": {
"max": 1,
"any": false,
"account_include_max": 10,
"account_include_reject": ["TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"],
"account_exclude_max": 10,
"account_required_max": 10
},
"blocks": {
"max": 1,
"account_include_max": 10,
"account_include_any": false,
"account_include_reject": ["TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"],
"include_transactions": true,
"include_accounts" : false,
"include_entries" : false
},
"blocks_meta": {
"max": 1
},
"entry": {
"max": 1
}
}
}
Unary gRPC methods
Ping
GetLatestBlockhash
GetBlockHeight
GetSlot
IsBlockhashValid
GetVersion
Examples
NOTE: Some load balancers will terminate gRPC connections if there are no messages sent from the client for a period of time.
In order to mitigate this you need to send a message periodically. The ping
field in the SubscribeRequest is used for this purpose.
The gRPC server already sends pings to the client, so you can simply reply with a ping and your connection will remain open.
You can see in the rust example how to reply to the ping from the server with the client.
Projects based on Geyser gRPC
- https://github.com/rpcpool/yellowstone-grpc-kafka — forward gRPC stream to Kafka, dedup, read stream from Kafka with gRPC server