docs: transactions fees/confirmation and deploying programs (#28895)
docs: adds and updates
This commit is contained in:
Nick Frostbutter
GitHub
parent
12de7637f1
commit
d8da242e5e
|
|
@ -156,6 +156,16 @@ module.exports = {
|
|||
id: "developing/lookup-tables",
|
||||
label: "Address Lookup Tables",
|
||||
},
|
||||
{
|
||||
type: "doc",
|
||||
id: "developing/intro/transaction_fees",
|
||||
label: "Transaction Fees",
|
||||
},
|
||||
{
|
||||
type: "doc",
|
||||
id: "developing/transaction_confirmation",
|
||||
label: "Transaction Confirmation",
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -1,12 +1,33 @@
|
|||
---
|
||||
title: Fork Generation
|
||||
description: "A fork is created when validators do not agree on a newly produced block. Using a consensus algorithm validators vote on which will be finalized."
|
||||
---
|
||||
|
||||
This section describes how forks naturally occur as a consequence of [leader rotation](leader-rotation.md).
|
||||
The Solana protocol doesn’t wait for all validators to agree on a newly produced block before the next block is produced. Because of that, it’s quite common for two different blocks to be chained to the same parent block. In those situations, we call each conflicting chain a [“fork.”](./fork-generation.md)
|
||||
|
||||
Solana validators need to vote on one of these forks and reach agreement on which one to use through a consensus algorithm (that is beyond the scope of this article). The main point you need to remember is that when there are competing forks, only one fork will be finalized by the cluster and the abandoned blocks in competing forks are all discarded.
|
||||
|
||||
This section describes how forks naturally occur as a consequence of [leader rotation](./leader-rotation.md).
|
||||
|
||||
## Overview
|
||||
|
||||
Nodes take turns being leader and generating the PoH that encodes state changes. The cluster can tolerate loss of connection to any leader by synthesizing what the leader _**would**_ have generated had it been connected but not ingesting any state changes. The possible number of forks is thereby limited to a "there/not-there" skip list of forks that may arise on leader rotation slot boundaries. At any given slot, only a single leader's transactions will be accepted.
|
||||
Nodes take turns being [leader](./../terminology.md#leader) and generating the PoH that encodes state changes. The cluster can tolerate loss of connection to any leader by synthesizing what the leader _**would**_ have generated had it been connected but not ingesting any state changes.
|
||||
|
||||
The possible number of forks is thereby limited to a "there/not-there" skip list of forks that may arise on leader rotation slot boundaries. At any given slot, only a single leader's transactions will be accepted.
|
||||
|
||||
### Forking example
|
||||
|
||||
The table below illustrates what competing forks could look like. Time progresses from left to right and each slot is assigned to a validator that temporarily becomes the cluster “leader” and may produce a block for that slot.
|
||||
|
||||
In this example, the leader for slot 3 chose to chain its “Block 3” directly to “Block 1” and in doing so skipped “Block 2”. Similarly, the leader for slot 5 chose to chain “Block 5” directly to “Block 3” and skipped “Block 4”.
|
||||
|
||||
> Note that across different forks, the block produced for a given slot is _always_ the same because producing two different blocks for the same slot is a slashable offense. So the conflicting forks above can be distinguished from each other by which slots they have _skipped_.
|
||||
|
||||
| | Slot 1 | Slot 2 | Slot 3 | Slot 4 | Slot 5 |
|
||||
| ------ | ------- | ------- | ------- | ------- | ------- |
|
||||
| Fork 1 | Block 1 | | Block 3 | | Block 5 |
|
||||
| Fork 2 | Block 1 | | Block 3 | Block 4 | |
|
||||
| Fork 3 | Block 1 | Block 2 | | | |
|
||||
|
||||
## Message Flow
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,87 @@
|
|||
---
|
||||
title: Transaction Fees
|
||||
description: "Transaction fees are the small fees paid to process instructions on the network. These fees are based on computation and an optional prioritization fee."
|
||||
keywords: "instruction fee, processing fee, storage fee, low fee blockchain, gas, gwei, cheap network, affordable blockchain"
|
||||
---
|
||||
|
||||
The small fees paid to process [instructions](./terminology.md#instruction) on the Solana blockchain are known as "_transaction fees_".
|
||||
|
||||
As each transaction (which contains one or more instructions) is sent through the network, it gets processed by the current leader validation-client. Once confirmed as a global state transaction, this _transaction fee_ is paid to the network to help support the economic design of the Solana blockchain.
|
||||
|
||||
> NOTE: Transactions fees are different from the blockchain's data storage fee called [rent](./rent.md)
|
||||
|
||||
### Transaction Fee Calculation
|
||||
|
||||
Currently, the amount of resources consumed by a transaction do not impact fees in any way. This is because the runtime imposes a small cap on the amount of resources that transaction instructions can use, not to mention that the size of transactions is limited as well. So right now, transaction fees are solely determined by the number of signatures that need to be verified in a transaction. The only limit on the number of signatures in a transaction is the max size of transaction itself. Each signature (64 bytes) in a transaction (max 1232 bytes) must reference a unique public key (32 bytes) so a single transaction could contain as many as 12 signatures (not sure why you would do that). The fee per transaction signature can be fetched with the `solana` cli:
|
||||
|
||||
```bash
|
||||
$ solana fees
|
||||
Blockhash: 8eULQbYYp67o5tGF2gxACnBCKAE39TetbYYMGTx3iBFc
|
||||
Lamports per signature: 5000
|
||||
Last valid block height: 94236543
|
||||
```
|
||||
|
||||
The `solana` cli `fees` subcommand calls the `getFees` RPC API method to retrieve the above output information, so your application can call that method directly as well:
|
||||
|
||||
```bash
|
||||
$ curl http://api.mainnet-beta.solana.com -H "Content-Type: application/json" -d '
|
||||
{"jsonrpc":"2.0","id":1, "method":"getFees"}
|
||||
'
|
||||
|
||||
# RESULT (lastValidSlot removed since it's inaccurate)
|
||||
{
|
||||
"jsonrpc": "2.0",
|
||||
"result": {
|
||||
"context": {
|
||||
"slot": 106818885
|
||||
},
|
||||
"value": {
|
||||
"blockhash": "78e3YBCMXJBiPD1HpyVtVfFzZFPG6nUycnQcyNMSUQzB",
|
||||
"feeCalculator": {
|
||||
"lamportsPerSignature": 5000
|
||||
},
|
||||
"lastValidBlockHeight": 96137823
|
||||
}
|
||||
},
|
||||
"id": 1
|
||||
}
|
||||
```
|
||||
|
||||
### Fee Determinism
|
||||
|
||||
It's important to keep in mind that fee rates (such as `lamports_per_signature`) are subject to change from block to block (though that hasn't happened in the full history of the `mainnet-beta` cluster). Despite the fact that fees can fluctuate, fees for a transaction can still be calculated deterministically when creating (and before signing) a transaction. This determinism comes from the fact that fees are applied using the rates from the block whose blockhash matches the `recent_blockhash` field in a transaction. Blockhashes can only be referenced by a transaction for a few minutes before they expire.
|
||||
|
||||
Transactions with expired blockhashes will be ignored and dropped by the cluster, so it's important to understand how expiration actually works. Before transactions are added to a block and during block validation, [each transaction's recent blockhash is checked](https://github.com/solana-labs/solana/blob/647aa926673e3df4443d8b3d9e3f759e8ca2c44b/runtime/src/bank.rs#L3482) to ensure it hasn't expired yet. The max age of a transaction's blockhash is only 150 blocks. This means that if no slots are skipped in between, the blockhash for block 100 would be usable by transactions processed in blocks 101 to 252, inclusive (during block 101 the age of block 100 is "0" and during block 252 its age is "150"). However, it's important to remember that slots may be skipped and that age checks use "block height" _not_ "slot height". Since slots are skipped occasionally, the actual age of a blockhash can be a bit longer than 150 slots. At the time of writing, slot times are about 500ms and skip rate is about 5% so the expected lifetime of a transaction which uses the most recent blockhash is about 1min 19s.
|
||||
|
||||
### Fee Collection
|
||||
|
||||
Transactions are required to have at least one account which has signed the transaction and is writable. Writable signer accounts are serialized first in the list of transaction accounts and the first of these accounts is always used as the "fee payer".
|
||||
|
||||
Before any transaction instructions are processed, the fee payer account balance will be deducted to pay for transaction fees. If the fee payer balance is not sufficient to cover transaction fees, the transaction will be dropped by the cluster. If the balance was sufficient, the fees will be deducted whether the transaction is processed successfully or not. In fact, if any of the transaction instructions return an error or violate runtime restrictions, all account changes _except_ the transaction fee deduction will be rolled back.
|
||||
|
||||
### Fee Distribution
|
||||
|
||||
Transaction fees are partially burned and the remaining fees are collected by the validator that produced the block that the corresponding transactions were included in. The transaction fee burn rate was initialized as 50% when inflation rewards were enabled at the beginning of 2021 and has not changed so far. These fees incentivize a validator to process as many transactions as possible during its slots in the leader schedule. Collected fees are deposited in the validator's account (listed in the leader schedule for the current slot) after processing all of the transactions included in a block.
|
||||
|
||||
## Upcoming Changes
|
||||
|
||||
### Transaction wide compute budget
|
||||
|
||||
As of version 1.8 of the Solana protocol, the maximum compute budget for transactions is assessed on a per instruction basis. This has allowed for flexibility in protocol design to squeeze out more compute by splitting up operations across multiple instructions but this workaround has skewed the distribution of compute consumption across different transactions. To keep transaction fees properly priced, the maximum compute budget will instead be assessed over the entire transaction. This change is likely to be released in version 1.9 of the Solana protocol and is gated on the following feature switch:
|
||||
|
||||
```bash
|
||||
$ ~/Workspace/solana (master branch) > cargo run --bin solana -- feature status 5ekBxc8itEnPv4NzGJtr8BVVQLNMQuLMNQQj7pHoLNZ9 --url mainnet-beta
|
||||
5ekBxc8itEnPv4NzGJtr8BVVQLNMQuLMNQQj7pHoLNZ9 | inactive | transaction wide compute cap
|
||||
```
|
||||
|
||||
This adjustment could negatively impact the usability of some protocols which have relied on the compute budget being reset for each instruction in a transaction. For this reason, this compute budget change will not be enabled until a new mechanism for increasing total transaction compute budget is added. This mechanism is described below...
|
||||
|
||||
### Request increased compute budget
|
||||
|
||||
As protocols have gotten more complex, the [default compute budget of 200,000 compute units](https://github.com/solana-labs/solana/blob/647aa926673e3df4443d8b3d9e3f759e8ca2c44b/sdk/src/compute_budget.rs#L105) has become a common pain-point for developers. Developers have gotten creative in working around this limitation by breaking up operations across multiple instructions and/or transactions. But in order to properly address this issue, a new program instruction will be added to request additional compute units from the runtime (up to a max of 1 million compute units). To request more compute, [create a `RequestUnits` instruction which invokes the new `Compute Budget` program](https://github.com/solana-labs/solana/blob/647aa926673e3df4443d8b3d9e3f759e8ca2c44b/sdk/src/compute_budget.rs#L44) and insert it at the beginning of a transaction. This new program will be released along with the transaction-wide compute budget change described above and is gated on the same feature switch. There are currently no increased transaction fees for using this feature, however that is likely to change.
|
||||
|
||||
Note that adding a `RequestUnits` compute budget instruction will take up 39 extra bytes in a serialized transaction. That breaks down into 32 bytes for the compute budget program id, 1 byte for program id index, 1 byte for empty ix account vec len, 1 byte for data vec len, and 4 bytes for the requested compute.
|
||||
|
||||
### Calculate transaction fees with RPC API
|
||||
|
||||
In order to simplify fee calculation for developers, a new `getFeeForMessage` RPC API is planned to be released in v1.9 of the Solana protocol. This new method accepts a blockhash along with an encoded transaction message and will return the amount of fees that would be deducted if the transaction message is signed, sent, and processed by the cluster. Full documentation can be found on the "edge" version of the Solana docs here: [https://edge.docs.solana.com/developing/clients/jsonrpc-api#getfeeformessage](https://edge.docs.solana.com/developing/clients/jsonrpc-api#getfeeformessage)
|
||||
|
|
@ -1,24 +1,143 @@
|
|||
---
|
||||
title: "Deploying Programs"
|
||||
description: "Deploying on-chain programs can be done using the Solana CLI using the Upgradable BPF loader to upload the compiled byte-code to the Solana blockchain."
|
||||
---
|
||||
|
||||
|
||||
Solana on-chain programs (otherwise known as "smart contracts") are stored in "executable" accounts on Solana. These accounts are identical to any other account but with the exception of:
|
||||
|
||||
As shown in the diagram above, a program author creates a program, compiles it
|
||||
to an ELF shared object containing SBF bytecode, and uploads it to the Solana
|
||||
cluster with a special _deploy_ transaction. The cluster makes it available to
|
||||
clients via a _program ID_. The program ID is an _address_ specified when
|
||||
deploying and is used to reference the program in subsequent transactions.
|
||||
- having the "executable" flag enabled, and
|
||||
- the owner being assigned to a BPF loader
|
||||
|
||||
Upon a successful deployment the account that holds the program is marked
|
||||
executable. If the program is marked "final", its account data become permanently
|
||||
immutable. If any changes are required to the finalized program (features, patches,
|
||||
etc...) the new program must be deployed to a new program ID.
|
||||
Besides those exceptions, they are governed by the same runtime rules as non-executable accounts, hold SOL tokens for rent fees, and store a data buffer which is managed by the BPF loader program. The latest BPF loader is called the "Upgradeable BPF Loader".
|
||||
|
||||
If a program is upgradeable, the account that holds the program is marked
|
||||
executable, but it is possible to redeploy a new shared object to the same
|
||||
program ID, provided that the program's upgrade authority signs the transaction.
|
||||
## Overview of the Upgradeable BPF Loader
|
||||
|
||||
The Solana command line interface supports deploying programs, for more
|
||||
information see the [`deploy`](cli/usage.md#deploy-program) command line usage
|
||||
documentation.
|
||||
### State accounts
|
||||
|
||||
The Upgradeable BPF loader program supports three different types of state accounts:
|
||||
|
||||
1. [Program account](https://github.com/solana-labs/solana/blob/master/sdk/program/src/bpf_loader_upgradeable.rs#L34): This is the main account of an on-chain program and its address is commonly referred to as a "program id." Program id's are what transaction instructions reference in order to invoke a program. Program accounts are immutable once deployed, so you can think of them as a proxy account to the byte-code and state stored in other accounts.
|
||||
2. [Program data account](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/bpf_loader_upgradeable.rs#L39): This account is what stores the executable byte-code of an on-chain program. When a program is upgraded, this account's data is updated with new byte-code. In addition to byte-code, program data accounts are also responsible for storing the slot when it was last modified and the address of the sole account authorized to modify the account (this address can be cleared to make a program immutable).
|
||||
3. [Buffer accounts](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/bpf_loader_upgradeable.rs#L27): These accounts temporarily store byte-code while a program is being actively deployed through a series of transactions. They also each store the address of the sole account which is authorized to do writes.
|
||||
|
||||
### Instructions
|
||||
|
||||
The state accounts listed above can only be modified with one of the following instructions supported by the Upgradeable BPF Loader program:
|
||||
|
||||
1. [Initialize buffer](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/loader_upgradeable_instruction.rs#L21): Creates a buffer account and stores an authority address which is allowed to modify the buffer.
|
||||
2. [Write](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/loader_upgradeable_instruction.rs#L28): Writes byte-code at a specified byte offset inside a buffer account. Writes are processed in small chunks due to a limitation of Solana transactions having a maximum serialized size of 1232 bytes.
|
||||
3. [Deploy](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/loader_upgradeable_instruction.rs#L77): Creates both a program account and a program data account. It fills the program data account by copying the byte-code stored in a buffer account. If the byte-code is valid, the program account will be set as executable, allowing it to be invoked. If the byte-code is invalid, the instruction will fail and all changes are reverted.
|
||||
4. [Upgrade](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/loader_upgradeable_instruction.rs#L102): Fills an existing program data account by copying executable byte-code from a buffer account. Similar to the deploy instruction, it will only succeed if the byte-code is valid.
|
||||
5. [Set authority](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/loader_upgradeable_instruction.rs#L114): Updates the authority of a program data or buffer account if the account's current authority has signed the transaction being processed. If the authority is deleted without replacement, it can never be set to a new address and the account can never be closed.
|
||||
6. [Close](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/sdk/program/src/loader_upgradeable_instruction.rs#L127): Clears the data of a program data account or buffer account and reclaims the SOL used for the rent exemption deposit.
|
||||
|
||||
## How `solana program deploy` works
|
||||
|
||||
Deploying a program on Solana requires hundreds, if not thousands of transactions, due to the max size limit of 1232 bytes for Solana transactions. The Solana CLI takes care of this rapid firing of transactions with the `solana program deploy` subcommand. The process can be broken down into the following 3 phases:
|
||||
|
||||
1. [Buffer initialization](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/cli/src/program.rs#L2113): First, the CLI sends a transaction which [creates a buffer account](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/cli/src/program.rs#L1903) large enough for the byte-code being deployed. It also invokes the [initialize buffer instruction](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/programs/bpf_loader/src/lib.rs#L320) to set the buffer authority to restrict writes to the deployer's chosen address.
|
||||
2. [Buffer writes](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/cli/src/program.rs#L2129): Once the buffer account is initialized, the CLI [breaks up the program byte-code](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/cli/src/program.rs#L1940) into ~1KB chunks and [sends transactions at a rate of 100 transactions per second](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/client/src/tpu_client.rs#L133) to write each chunk with [the write buffer instruction](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/programs/bpf_loader/src/lib.rs#L334). These transactions are sent directly to the current leader's transaction processing (TPU) port and are processed in parallel with each other. Once all transactions have been sent, the CLI [polls the RPC API with batches of transaction signatures](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/client/src/tpu_client.rs#L216) to ensure that every write was successful and confirmed.
|
||||
3. [Finalization](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/cli/src/program.rs#L1807): Once writes are completed, the CLI [sends a final transaction](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/cli/src/program.rs#L2150) to either [deploy a new program](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/programs/bpf_loader/src/lib.rs#L362) or [upgrade an existing program](https://github.com/solana-labs/solana/blob/7409d9d2687fba21078a745842c25df805cdf105/programs/bpf_loader/src/lib.rs#L513). In either case, the byte-code written to the buffer account will be copied into a program data account and verified.
|
||||
|
||||
## Reclaim rent from program accounts
|
||||
|
||||
The storage of data on the Solana blockchain requires the payment of [rent](./../intro/rent.md), including for the byte-code for on-chain programs. Therefore as you deploy more or larger programs, the amount of rent paid to remain rent-exempt will also become larger.
|
||||
|
||||
Using the current rent cost model configuration, a rent-exempt account requires a deposit of ~0.7 SOL per 100KB stored. These costs can have an outsized impact on developers who deploy their own programs since [program accounts](./../programming-model/accounts.md#executable) are among the largest we typically see on Solana.
|
||||
|
||||
#### Example of how much data is used for programs
|
||||
|
||||
As a data point of the number of accounts and potential data stored on-chain, below is the distribution of the largest accounts (at least 100KB) at slot `103,089,804` on `mainnet-beta` by assigned on-chain program:
|
||||
|
||||
1. **Serum Dex v3**: 1798 accounts
|
||||
2. **Metaplex Candy Machine**: 1089 accounts
|
||||
3. **Serum Dex v2**: 864 accounts
|
||||
4. **Upgradeable BPF Program Loader**: 824 accounts
|
||||
5. **BPF Program Loader v2**: 191 accounts
|
||||
6. **BPF Program Loader v1**: 150 accounts
|
||||
|
||||
> _Note: this data was pulled with a modified `solana-ledger-tool` built from this branch: [https://github.com/jstarry/solana/tree/large-account-stats](https://github.com/jstarry/solana/tree/large-account-stats)_
|
||||
|
||||
### Reclaiming buffer accounts
|
||||
|
||||
Buffer accounts are used by the Upgradeable BPF loader to temporarily store byte-code that is in the process of being deployed on-chain. This temporary buffer is required when upgrading programs because the currently deployed program's byte-code cannot be affected by an in-progress upgrade.
|
||||
|
||||
Unfortunately, deploys fail occasionally and instead of reusing the buffer account, developers might retry their deployment with a new buffer and not realize that they stored a good chunk of SOL in a forgotten buffer account from an earlier deploy.
|
||||
|
||||
> As of slot `103,089,804` on `mainnet-beta` there are 276 abandoned buffer accounts that could be reclaimed!
|
||||
|
||||
Developers can check if they own any abandoned buffer accounts by using the Solana CLI:
|
||||
|
||||
```bash
|
||||
solana program show --buffers --keypair ~/.config/solana/MY_KEYPAIR.json
|
||||
|
||||
Buffer Address | Authority | Balance
|
||||
9vXW2c3qo6DrLHa1Pkya4Mw2BWZSRYs9aoyoP3g85wCA | 2nr1bHFT86W9tGnyvmYW4vcHKsQB3sVQfnddasz4kExM | 3.41076888 SOL
|
||||
```
|
||||
|
||||
And they can close those buffers to reclaim the SOL balance with the following command:
|
||||
|
||||
```bash
|
||||
solana program close --buffers --keypair ~/.config/solana/MY_KEYPAIR.json
|
||||
```
|
||||
|
||||
#### Fetch the owners of buffer accounts via RPC API
|
||||
|
||||
The owners of all abandoned program deploy buffer accounts can be fetched via the RPC API:
|
||||
|
||||
```bash
|
||||
curl http://api.mainnet-beta.solana.com -H "Content-Type: application/json" \
|
||||
--data-binary @- << EOF | jq --raw-output '.result | .[] | .account.data[0]'
|
||||
{
|
||||
"jsonrpc":"2.0", "id":1, "method":"getProgramAccounts",
|
||||
"params":[
|
||||
"BPFLoaderUpgradeab1e11111111111111111111111",
|
||||
{
|
||||
"dataSlice": {"offset": 5, "length": 32},
|
||||
"filters": [{"memcmp": {"offset": 0, "bytes": "2UzHM"}}],
|
||||
"encoding": "base64"
|
||||
}
|
||||
]
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
After re-encoding the base64 encoded keys into base58 and grouping by key, we see some accounts have over 10 buffer accounts they could close, yikes!
|
||||
|
||||
```bash
|
||||
'BE3G2F5jKygsSNbPFKHHTxvKpuFXSumASeGweLcei6G3' => 10 buffer accounts
|
||||
'EsQ179Q8ESroBnnmTDmWEV4rZLkRc3yck32PqMxypE5z' => 10 buffer accounts
|
||||
'6KXtB89kAgzW7ApFzqhBg5tgnVinzP4NSXVqMAWnXcHs' => 12 buffer accounts
|
||||
'FinVobfi4tbdMdfN9jhzUuDVqGXfcFnRGX57xHcTWLfW' => 15 buffer accounts
|
||||
'TESAinbTL2eBLkWqyGA82y1RS6kArHvuYWfkL9dKkbs' => 42 buffer accounts
|
||||
```
|
||||
|
||||
### Reclaiming program data accounts
|
||||
|
||||
You may now realize that program data accounts (the accounts that store the executable byte-code for an on-chain program) can also be closed.
|
||||
|
||||
> **Note:** This does _not_ mean that _program accounts_ can be closed (those are immutable and can never be reclaimed, but it's fine they're pretty small). It's also important to keep in mind that once program data accounts are deleted, they can never be recreated for an existing program. Therefor, the corresponding program (and its program id) for any closed program data account is effectively disabled forever and may not be re-deployed
|
||||
|
||||
While it would be uncommon for developers to need to close program data accounts since they can be rewritten during upgrades, one potential scenario is that since program data accounts can't be _resized_. You may wish to deploy your program at a new address to accommodate larger executables.
|
||||
|
||||
The ability to reclaim program data account rent deposits also makes testing and experimentation on the `mainnet-beta` cluster a lot less costly since you could reclaim everything except the transaction fees and a small amount of rent for the program account. Lastly, this could help developers recover most of their funds if they mistakenly deploy a program at an unintended address or on the wrong cluster.
|
||||
|
||||
To view the programs which are owned by your wallet address, you can run:
|
||||
|
||||
```bash
|
||||
solana -V # must be 1.7.11 or higher!
|
||||
solana program show --programs --keypair ~/.config/solana/MY_KEYPAIR.json
|
||||
|
||||
Program Id | Slot | Authority | Balance
|
||||
CN5x9WEusU6pNH66G22SnspVx4cogWLqMfmb85Z3GW7N | 53796672 | 2nr1bHFT86W9tGnyvmYW4vcHKsQB3sVQfnddasz4kExM | 0.54397272 SOL
|
||||
```
|
||||
|
||||
To close those program data accounts and reclaim their SOL balance, you can run:
|
||||
|
||||
```bash
|
||||
solana program close --programs --keypair ~/.config/solana/MY_KEYPAIR.json
|
||||
```
|
||||
|
||||
You might be concerned about this feature allowing malicious actors to close a program in a way that negatively impacts end users. While this is a valid concern in general, closing program data accounts doesn't make this any more exploitable than was already possible.
|
||||
|
||||
Even without the ability to close a program data account, any upgradeable program could be upgraded to a no-op implementation and then have its upgrade authority cleared to make it immutable forever. This new feature for closing program data accounts merely adds the ability to reclaim the rent deposit, disabling a program was already technically possible.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,199 @@
|
|||
---
|
||||
title: "Transaction Confirmation"
|
||||
---
|
||||
|
||||
into to transaction confirmation?
|
||||
|
||||
Problems relating to [transaction confirmation](./transaction_confirmation.md) are common with many newer developers while building applications. This article aims to boost the overall understanding of the confirmation mechanism used on the Solana blockchain, including some recommended best practices.
|
||||
|
||||
## Brief background on transactions
|
||||
|
||||
Let’s first make sure we’re all on the same page and thinking about the same things...
|
||||
|
||||
### What is a transaction?
|
||||
|
||||
Transactions consist of two components: a [message](./../terminology.md#message) and a [list of signatures](./../terminology.md#signature). The transaction message is where the magic happens and at a high level it consists of three components:
|
||||
|
||||
- a **list of instructions** to invoke,
|
||||
- a **list of accounts** to load, and
|
||||
- a **“recent blockhash.”**
|
||||
|
||||
In this article, we’re going to be focusing a lot on a transaction’s [recent blockhash](./../terminology.md#blockhash) because it plays a big role in transaction confirmation.
|
||||
|
||||
### Transaction lifecycle refresher
|
||||
|
||||
Below is a high level view of the lifecycle of a transaction. This article will touch on everything except steps 1 and 4.
|
||||
|
||||
1. Create a list of instructions along with the list of accounts that instructions need to read and write
|
||||
2. Fetch a recent blockhash and use it to prepare a transaction message
|
||||
3. Simulate the transaction to ensure it behaves as expected
|
||||
4. Prompt user to sign the prepared transaction message with their private key
|
||||
5. Send the transaction to an RPC node which attempts to forward it to the current block producer
|
||||
6. Hope that a block producer validates and commits the transaction into their produced block
|
||||
7. Confirm the transaction has either been included in a block or detect when it has expired
|
||||
|
||||
## What is a Blockhash?
|
||||
|
||||
A [“blockhash”](./../terminology.md#blockhash) refers to the last Proof of History (PoH) hash for a [“slot”](./../terminology.md#slot) (description below). Since Solana uses PoH as a trusted clock, a transaction’s recent blockhash can be thought of as a **timestamp**.
|
||||
|
||||
### Proof of History refresher
|
||||
|
||||
Solana’s Proof of History mechanism uses a very long chain of recursive SHA-256 hashes to build a trusted clock. The “history” part of the name comes from the fact that block producers hash transaction id’s into the stream to record which transactions were processed in their block.
|
||||
|
||||
[PoH hash calculation](https://github.com/solana-labs/solana/blob/9488a73f5252ad0d7ea830a0b456d9aa4bfbb7c1/entry/src/poh.rs#L82): `next_hash = hash(prev_hash, hash(transaction_ids))`
|
||||
|
||||
PoH can be used as a trusted clock because each hash must be produced sequentially. Each produced block contains a blockhash and a list of hash checkpoints called “ticks” so that validators can verify the full chain of hashes in parallel and prove that some amount of time has actually passed. The stream of hashes can be broken up into the following time units:
|
||||
|
||||
# Transaction Expiration
|
||||
|
||||
By default, all Solana transactions will expire if not committed to a block in a certain amount of time. The **vast majority** of transaction confirmation issues are related to how RPC nodes and validators detect and handle **expired** transactions. A solid understanding of how transaction expiration works should help you diagnose the bulk of your transaction confirmation issues.
|
||||
|
||||
## How does transaction expiration work?
|
||||
|
||||
Each transaction includes a “recent blockhash” which is used as a PoH clock timestamp and expires when that blockhash is no longer “recent” enough. More concretely, Solana validators look up the corresponding slot number for each transaction’s blockhash that they wish to process in a block. If the validator [can’t find a slot number for the blockhash](https://github.com/solana-labs/solana/blob/9488a73f5252ad0d7ea830a0b456d9aa4bfbb7c1/runtime/src/bank.rs#L3687) or if the looked up slot number is more than 151 slots lower than the slot number of the block being processed, the transaction will be rejected.
|
||||
|
||||
Slots are configured to last about [400ms](https://github.com/solana-labs/solana/blob/47b938e617b77eb3fc171f19aae62222503098d7/sdk/program/src/clock.rs#L12) but often fluctuate between 400ms and 600ms, so a given blockhash can only be used by transactions for about 60 to 90 seconds.
|
||||
|
||||
Transaction has expired pseudocode: `currentBankSlot > slotForTxRecentBlockhash + 151`
|
||||
|
||||
Transaction not expired pseudocode: `currentBankSlot - slotForTxRecentBlockhash < 152`
|
||||
|
||||
### Example of transaction expiration
|
||||
|
||||
Let’s walk through a quick example:
|
||||
|
||||
1. A validator is producing a new block for slot #1000
|
||||
2. The validator receives a transaction with recent blockhash `1234...` from a user
|
||||
3. The validator checks the `1234...` blockhash against the list of recent blockhashes leading up to its new block and discovers that it was the blockhash for slot #849
|
||||
4. Since slot #849 is exactly 151 slots lower than slot #1000, the transaction hasn’t expired yet and can still be processed!
|
||||
5. But wait, before actually processing the transaction, the validator finished the block for slot #1000 and starts producing the block for slot #1001 (validators get to produce blocks for 4 consecutive slots).
|
||||
6. The validator checks the same transaction again and finds that it’s now too old and drops it because it’s now 152 slots lower than the current slot :(
|
||||
|
||||
## Why do transactions expire?
|
||||
|
||||
There’s a very good reason for this actually, it’s to help validators avoid processing the same transaction twice.
|
||||
|
||||
A naive brute force approach to prevent double processing could be to check every new transaction against the blockchain’s entire transaction history. But by having transactions expire after a short amount of time, validators only need to check if a new transaction is in a relatively small set of _recently_ processed transactions.
|
||||
|
||||
### Other blockchains
|
||||
|
||||
Solana’s approach of prevent double processing is quite different from other blockchains. For example, Ethereum tracks a counter (nonce) for each transaction sender and will only process transactions that use the next valid nonce.
|
||||
|
||||
Ethereum’s approach is simple for validators to implement, but it can be problematic for users. Many people have encountered situations when their Ethereum transactions got stuck in a _pending_ state for a long time and all the later transactions, which used higher nonce values, were blocked from processing.
|
||||
|
||||
### Advantages on Solana
|
||||
|
||||
There are a few advantages to Solana’s approach:
|
||||
|
||||
1. A single fee payer can submit multiple transactions at the same time that are allowed to be processed in any order. This might happen if you’re using multiple applications at the same time.
|
||||
2. If a transaction doesn’t get committed to a block and expires, users can try again knowing that their previous transaction won’t ever be processed.
|
||||
|
||||
By not using counters, the Solana wallet experience may be easier for users to understand because they can get to success, failure, or expiration states quickly and avoid annoying pending states.
|
||||
|
||||
### Disadvantages on Solana
|
||||
|
||||
Of course there are some disadvantages too:
|
||||
|
||||
1. Validators have to actively track a set of all processed transaction id’s to prevent double processing.
|
||||
2. If the expiration time period is too short, users might not be able to submit their transaction before it expires.
|
||||
|
||||
These disadvantages highlight a tradeoff in how transaction expiration is configured. If the expiration time of a transaction is increased, validators need to use more memory to track more transactions. If expiration time is decreased, users don’t have enough time to submit their transaction.
|
||||
|
||||
Currently, Solana clusters require that transactions use blockhashes that are no more than [151 slots](https://github.com/solana-labs/solana/blob/9488a73f5252ad0d7ea830a0b456d9aa4bfbb7c1/sdk/program/src/clock.rs#L65) old.
|
||||
|
||||
> This [Github issue](https://github.com/solana-labs/solana/issues/23582) contains some calculations that estimate that mainnet-beta validators need about 150MB of memory to track transactions.
|
||||
> This could be slimmed down in the future if necessary without decreasing expiration time as I’ve detailed in that issue.
|
||||
|
||||
## Transaction confirmation tips
|
||||
|
||||
As mentioned before, blockhashes expire after a time period of only 151 slots which can pass as quickly as **one minute** when slots are processed within the target time of 400ms.
|
||||
|
||||
One minute is not a lot of time considering that a client needs to fetch a recent blockhash, wait for the user to sign, and finally hope that the broadcasted transaction reaches a leader that is willing to accept it. Let’s go through some tips to help avoid confirmation failures due to transaction expiration!
|
||||
|
||||
### Fetch blockhashes with the appropriate commitment level
|
||||
|
||||
Given the short expiration time frame, it’s imperative that clients help users create transactions with blockhash that is as recent as possible.
|
||||
|
||||
When fetching blockhashes, the current recommended RPC API is called [`getLatestBlockhash`](./clients/jsonrpc-api#getlatestblockhash). By default, this API uses the `"finalized"` commitment level to return the most recently finalized block’s blockhash. However, you can override this behavior by [setting the `commitment` parameter](./clients/jsonrpc-api#configuring-state-commitment) to a different commitment level.
|
||||
|
||||
**Recommendation**
|
||||
|
||||
The `"confirmed"` commitment level should almost always be used for RPC requests because it’s usually only a few slots behind the `"processed"` commitment and has a very low chance of belonging to a dropped [fork](./../cluster/fork-generation.md).
|
||||
|
||||
But feel free to consider the other options:
|
||||
|
||||
- Choosing `"processed"` will let you fetch the most recent blockhash compared to other commitment levels and therefore gives you the most time to prepare and process a transaction. But due to the prevalence of forking in the Solana protocol, roughly 5% of blocks don’t end up being finalized by the cluster so there’s a real chance that your transaction uses a blockhash that belongs to a dropped fork. Transactions that use blockhashes for abandoned blocks won’t ever be considered recent by any blocks that are in the finalized blockchain.
|
||||
- Using the default commitment level `"finalized"` will eliminate any risk that the blockhash you choose will belong to a dropped fork. The tradeoff is that there is typically at least a 32 slot difference between the most recent confirmed block and the most recent finalized block. This tradeoff is pretty severe and effectively reduces the expiration of your transactions by about 13 seconds but this could be even more during unstable cluster conditions.
|
||||
|
||||
### Use an appropriate preflight commitment level
|
||||
|
||||
If your transaction uses a blockhash that was fetched from one RPC node then you send, or simulate, that transaction with a different RPC node, you could run into issues due to one node lagging behind the other.
|
||||
|
||||
When RPC nodes receive a `sendTransaction` request, they will attempt to determine the expiration block of your transaction using the most recent finalized block or with the block selected by the `preflightCommitment` parameter. A **VERY** common issue is that a received transaction’s blockhash was produced after the block used to calculate the expiration for that transaction. If an RPC node can’t determine when your transaction expires, it will only forward your transaction **one time** and then will **drop** the transaction.
|
||||
|
||||
Similarly, when RPC nodes receive a `simulateTransaction` request, they will simulate your transaction using the most recent finalized block or with the block selected by the `preflightCommitment` parameter. If the block chosen for simulation is older than the block used for your transaction’s blockhash, the simulation will fail with the dreaded “blockhash not found” error.
|
||||
|
||||
**Recommendation**
|
||||
|
||||
Even if you use `skipPreflight`, **ALWAYS** set the `preflightCommitment` parameter to the same commitment level used to fetch your transaction’s blockhash for both `sendTransaction` and `simulateTransaction` requests.
|
||||
|
||||
### Be wary of lagging RPC nodes when sending transactions
|
||||
|
||||
When your application uses an RPC pool service or when the RPC endpoint differs between creating a transaction and sending a transaction, you need to be wary of situations where one RPC node is lagging behind the other. For example, if you fetch a transaction blockhash from one RPC node then you send that transaction to a second RPC node for forwarding or simulation, the second RPC node might be lagging behind the first.
|
||||
|
||||
**Recommendation**
|
||||
|
||||
For `sendTransaction` requests, clients should keep resending a transaction to a RPC node on a frequent interval so that if an RPC node is slightly lagging behind the cluster, it will eventually catch up and detect your transaction’s expiration properly.
|
||||
|
||||
For `simulateTransaction` requests, clients should use the [`replaceRecentBlockhash`](./clients/jsonrpc-api#simulatetransaction) parameter to tell the RPC node to replace the simulated transaction’s blockhash with a blockhash that will always be valid for simulation.
|
||||
|
||||
### Avoid reusing stale blockhashes
|
||||
|
||||
Even if your application has fetched a very recent blockhash, be sure that you’re not reusing that blockhash in transactions for too long. The ideal scenario is that a recent blockhash is fetched right before a user signs their transaction.
|
||||
|
||||
**Recommendation for applications**
|
||||
|
||||
Poll for new recent blockhashes on a frequent basis to ensure that whenever a user triggers an action that creates a transaction, your application already has a fresh blockhash that’s ready to go.
|
||||
|
||||
**Recommendation for wallets**
|
||||
|
||||
Poll for new recent blockhashes on a frequent basis and replace a transaction’s recent blockhash right before they sign the transaction to ensure the blockhash is as fresh as possible.
|
||||
|
||||
### Use healthy RPC nodes when fetching blockhashes
|
||||
|
||||
By fetching the latest blockhash with the `"confirmed"` commitment level from an RPC node, it’s going to respond with the blockhash for the latest confirmed block that it’s aware of. Solana’s block propagation protocol prioritizes sending blocks to staked nodes so RPC nodes naturally lag about a block behind the rest of the cluster. They also have to do more work to handle application requests and can lag a lot more under heavy user traffic.
|
||||
|
||||
Lagging RPC nodes can therefore respond to blockhash requests with blockhashes that were confirmed by the cluster quite awhile ago. By default, a lagging RPC node detects that it is more than 150 slots behind the cluster will stop responding to requests, but just before hitting that threshold they can still return a blockhash that is just about to expire.
|
||||
|
||||
**Recommendation**
|
||||
|
||||
Monitor the health of your RPC nodes to ensure that they have an up-to-date view of the cluster state with one of the following methods:
|
||||
|
||||
1. Fetch your RPC node’s highest processed slot by using the [`getSlot`](./clients/jsonrpc-api#getslot) RPC API with the `"processed"` commitment level and then call the [`getMaxShredInsertSlot](./clients/jsonrpc-api#getmaxshredinsertslot) RPC API to get the highest slot that your RPC node has received a “shred” of a block for. If the difference between these responses is very large, the cluster is producing blocks far ahead of what the RPC node has processed.
|
||||
2. Call the `getLatestBlockhash` RPC API with the `"confirmed"` commitment level on a few different RPC API nodes and use the blockhash from the node that returns the highest slot for its [context slot](./clients/jsonrpc-api#rpcresponse-structure).
|
||||
|
||||
### Wait long enough for expiration
|
||||
|
||||
**Recommendation**
|
||||
|
||||
When calling [`getLatestBlockhash`](./clients/jsonrpc-api#getlatestblockhash) RPC API to get a recent blockhash for your transaction, take note of the `"lastValidBlockHeight"` in the response.
|
||||
|
||||
Then, poll the [`getBlockHeight`](./clients/jsonrpc-api#getblockheight) RPC API with the “confirmed” commitment level until it returns a block height greater than the previously returned last valid block height.
|
||||
|
||||
### Consider using “durable” transactions
|
||||
|
||||
Sometimes transaction expiration issues are really hard to avoid (e.g. offline signing, cluster instability). If the previous tips are still not sufficient for your use-case, you can switch to using durable transactions (they just require a bit of setup).
|
||||
|
||||
To start using durable transactions, a user first needs to submit a transaction that [invokes instructions that create a special on-chain “nonce” account](https://docs.rs/solana-program/latest/solana_program/system_instruction/fn.create_nonce_account.html) and stores a “durable blockhash” inside of it. At any point in the future (as long as the nonce account hasn’t been used yet), the user can create a durable transaction by following these 2 rules:
|
||||
|
||||
1. The instruction list must start with an [“advance nonce” system instruction](https://docs.rs/solana-program/latest/solana_program/system_instruction/fn.advance_nonce_account.html) which loads their on-chain nonce account
|
||||
2. The transaction’s blockhash must be equal to the durable blockhash stored by the on-chain nonce account
|
||||
|
||||
Here’s how these transactions are processed by the Solana runtime:
|
||||
|
||||
1. If the transaction’s blockhash is no longer “recent”, the runtime checks if the transaction’s instruction list begins with an “advance nonce” system instruction
|
||||
2. If so, it then loads the nonce account specified by the “advance nonce” instruction
|
||||
3. Then it checks that the stored durable blockhash matches the transaction’s blockhash
|
||||
4. Lastly it makes sure to advance the nonce account’s stored blockhash to the latest recent blockhash to ensure that the same transaction can never be processed again
|
||||
|
||||
For more details about how these durable transactions work, you can read the [original proposal](./../implemented-proposals/durable-tx-nonces.md) and [check out an example](./clients/javascript-reference#nonceaccount) in the Solana docs.
|
||||
|
|
@ -11,7 +11,7 @@ As each transaction (which contains one or more instructions) is sent through th
|
|||
> **NOTE:** Transaction fees are different from [account rent](./terminology.md#rent)!
|
||||
> While transaction fees are paid to process instructions on the Solana network, rent is paid to store data on the blockchain.
|
||||
|
||||
<!-- > You can learn more about rent here: [What is rent?](./developing/intro/rent.md) -->
|
||||
> You can learn more about rent here: [What is rent?](./developing/intro/rent.md)
|
||||
|
||||
## Why pay transaction fees?
|
||||
|
||||
|
|
@ -66,3 +66,13 @@ Recently, Solana has introduced an optional fee called the "_[prioritization fee
|
|||
The prioritization fee is calculated by multiplying the requested maximum _compute units_ by the compute-unit price (specified in increments of 0.000001 lamports per compute unit) rounded up to the nearest lamport.
|
||||
|
||||
You can read more about the [compute budget instruction](./developing/programming-model/runtime.md#compute-budget) here.
|
||||
|
||||
## Fee Collection
|
||||
|
||||
Transactions are required to have at least one account which has signed the transaction and is writable. Writable signer accounts are serialized first in the list of transaction accounts and the first of these accounts is always used as the "fee payer".
|
||||
|
||||
Before any transaction instructions are processed, the fee payer account balance will be deducted to pay for transaction fees. If the fee payer balance is not sufficient to cover transaction fees, the transaction will be dropped by the cluster. If the balance was sufficient, the fees will be deducted whether the transaction is processed successfully or not. In fact, if any of the transaction instructions return an error or violate runtime restrictions, all account changes _except_ the transaction fee deduction will be rolled back.
|
||||
|
||||
## Fee Distribution
|
||||
|
||||
Transaction fees are partially burned and the remaining fees are collected by the validator that produced the block that the corresponding transactions were included in. The transaction fee burn rate was initialized as 50% when inflation rewards were enabled at the beginning of 2021 and has not changed so far. These fees incentivize a validator to process as many transactions as possible during its slots in the leader schedule. Collected fees are deposited in the validator's account (listed in the leader schedule for the current slot) after processing all of the transactions included in a block.
|
||||
|
|
|
|||
Loading…
Reference in New Issue