Go to file
Alessandro Decina 0f41719918
direct mapping: misc fixes (#32649)
* transaction_context: update make_data_mut comment

* bpf_loader: cpi: pass SerializeAccountMetadata to CallerAccount::from*

We now have a way to provide CallerAccount with trusted values coming
from our internal serialization code and not from untrusted vm space

* bpf_loader: direct_mapping: enforce account info pointers to be immutable

When direct mapping is enabled, we might need to update account data
memory regions across CPI calls. Since the only way we have to retrieve
the regions is based on their vm addresses, we enforce vm addresses to
be stable.  Accounts can still be mutated and resized of course, but it
must be done in place.

This also locks all other AccountInfo pointers, since there's no legitimate
reason to make them point to anything else.

* bpf_loader: cpi: access ref_to_len_in_vm through VmValue

Direct mapping needs to translate vm values at each access since
permissions of the underlying memory might have changed.

* direct mapping: improve memory permission tracking across CPI calls

Ensure that the data and realloc regions of an account always track the
account's permissions. In order to do this, we also need to split
realloc regions in their own self contained regions, where before we
had:

[account fields][account data][account realloc + more account fields + next account fields][next account data][...]

we now have:

[account fields][account data][account realloc][more account fields + next account fields][next account data][...]

Tested in TEST_[FORBID|ALLOW]_WRITE_AFTER_OWNERSHIP_CHANGE*

Additionally when direct mapping is on, we must update all perms at once before
doing account data updates. Otherwise, updating an account might write into
another account whose perms we haven't updated yet. Tested in
TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE.

* bpf_loader: serialization: address review comment don't return vm_addr from push_account_region

* bpf_loader: rename push_account_region to push_account_data_region

* cpi: fix slow edge case zeroing extra account capacity after shrinking an account

When returning from CPI we need to zero all the account memory up to the
original length only if we know we're potentially dealing with uninitialized
memory.

When we know that the spare capacity has deterministic content, we only need to
zero new_len..prev_len.

This fixes a slow edge case that was triggerable by the following scenario:

- load a large account (say 10MB) into the vm
- shrink to 10 bytes - would memset 10..10MB
- shrink to 9 bytes - would memset 9..10MB
- shrink to 8 bytes - would memset 8..10MB
- ...

Now instead in the scenario above the following will happen:

- load a large account (say 10MB) into the vm
- shrink to 10 bytes - memsets 10..10MB
- shrink to 9 bytes - memsets 9..10
- shrink to 8 bytes - memset 8..9
- ...

* bpf_loader: add account_data_region_memory_state helper

Shared between serialization and CPI to figure out the MemoryState of an
account.

* cpi: direct_mapping: error out if ref_to_len_in_vm points to account memory

If ref_to_len_in_vm is allowed to be in account memory, calles could mutate it,
essentially letting callees directly mutate callers memory.

* bpf_loader: direct_mapping: map AccessViolation -> InstructionError

Return the proper ReadonlyDataModified / ExecutableDataModified /
ExternalAccountDataModified depending on where the violation occurs

* bpf_loader: cpi: remove unnecessary infallible slice::get call
2023-08-30 16:57:24 +07:00
.buildkite ci: stop spl downstream project test temporarily (#32610) 2023-07-24 19:49:42 +00:00
.github CI: Add back some downstream SPL jobs (#32986) 2023-08-25 12:35:08 +02:00
account-decoder stake: Rename `StakeStateWithFlags` -> `StakeStateV2` (#32795) 2023-08-11 01:07:21 +02:00
accounts-bench Use update_accounts_hash() in AccountsHashVerifier (#32830) 2023-08-14 13:11:23 -04:00
accounts-cluster-bench Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
accounts-db preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
banking-bench Remove improper uses of &Arc<Bank> (#32802) 2023-08-18 16:46:34 -05:00
banks-client ci: update cargo.toml (#31934) 2023-06-02 15:08:07 +00:00
banks-interface banks-client: Add `get_fee_for_message` (#31703) 2023-05-18 23:31:02 +02:00
banks-server banks-server: Support versioned transactions (#33005) 2023-08-28 23:08:41 +02:00
bench-streamer uses Duration type instead of untyped ..._ms: u64 (#30971) 2023-03-31 15:42:49 +00:00
bench-tps BenchTPS: Allow a fixed compute-unit-price (#32775) 2023-08-15 15:20:04 -07:00
bloom Fix - Upcoming `arithmetic_side_effects` lints (#33000) 2023-08-29 20:58:53 +02:00
bucket_map return file name as u128 for disk buckets (#33044) 2023-08-29 08:16:54 -07:00
cd
ci ci: remove --ignore RUSTSEC-2022-0093 (#33019) 2023-08-29 05:58:50 +00:00
clap-utils Refactor - Demotes `Arc` to `Rc`. (#32982) 2023-08-25 00:54:06 +02:00
clap-v3-utils Refactor - Demotes `Arc` to `Rc`. (#32982) 2023-08-25 00:54:06 +02:00
cli preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
cli-config chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
cli-output preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
client client: Speed up send-and-confirm-parallel (#33032) 2023-08-29 18:29:10 +02:00
client-test Remove improper uses of &Arc<Bank> (#32802) 2023-08-18 16:46:34 -05:00
connection-cache improve quic-client packet send metrics (#32864) 2023-08-22 21:07:53 -07:00
core preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
cost-model sdk: Limited Borsh 0.9 support (Pubkey and helpers) (#32511) 2023-08-02 23:15:24 +02:00
docs docs: update discord links (#32935) 2023-08-22 17:17:58 +00:00
dos preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
download-utils Renames SnapshotType to SnapshotKind (#32906) 2023-08-19 19:55:29 -04:00
entry Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
faucet preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
frozen-abi Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
genesis stake: Rename `StakeStateWithFlags` -> `StakeStateV2` (#32795) 2023-08-11 01:07:21 +02:00
genesis-utils Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
geyser-plugin-interface Update discord invite links (#30909) 2023-03-29 09:14:22 -07:00
geyser-plugin-manager Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
gossip preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
install feat(solana-install): subcommand to list installed versions (#32823) 2023-08-18 14:58:17 -05:00
keygen preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
ledger preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
ledger-tool General cleanup (#32980) 2023-08-24 21:44:19 +02:00
local-cluster preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
log-analyzer chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
logger chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
measure ci: deflake `Measurement` tests (#31812) 2023-05-25 12:46:46 -06:00
memory-management chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
merkle-root-bench Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
merkle-tree General cleanup (#32980) 2023-08-24 21:44:19 +02:00
metrics General cleanup (#32980) 2023-08-24 21:44:19 +02:00
multinode-demo bench-tps: rename tx_count arg to tx-count for consistency (#31213) 2023-04-17 10:59:39 -07:00
net Adjust mmap and FD count for net scripts (#31728) 2023-05-19 18:13:11 +00:00
net-shaper Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
net-utils Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
notifier chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
perf Fix - Upcoming `arithmetic_side_effects` lints (#33000) 2023-08-29 20:58:53 +02:00
poh Get slot info along tpu leader (#32942) 2023-08-27 03:13:41 -07:00
poh-bench Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
program-runtime direct mapping: misc fixes (#32649) 2023-08-30 16:57:24 +07:00
program-test banks-server: Support versioned transactions (#33005) 2023-08-28 23:08:41 +02:00
programs direct mapping: misc fixes (#32649) 2023-08-30 16:57:24 +07:00
pubsub-client ci: bump tungstenite and tokio-tungstenite to 0.20.0 (#32996) 2023-08-28 11:20:00 +08:00
quic-client improve quic-client packet send metrics (#32864) 2023-08-22 21:07:53 -07:00
rayon-threadlimit chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
rbpf-cli rbpf-cli: fix deprecation notice (#31743) 2023-05-22 04:47:05 -05:00
remote-wallet Refactor - Demotes `Arc` to `Rc`. (#32982) 2023-08-25 00:54:06 +02:00
rpc preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
rpc-client rpc-client: Encapsulate `set_message_for_confirmed_transactions` (#32941) 2023-08-23 11:46:17 +02:00
rpc-client-api adds tpu_quic field to RpcContactInfo (#31310) 2023-04-22 20:18:58 +00:00
rpc-client-nonce-utils chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
rpc-test separates out connection-cache metrics for different protocols (#31803) 2023-05-25 14:48:22 +00:00
runtime preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
scripts Update Solang Solidity compiler to v0.3.2 (#32975) 2023-08-25 12:56:27 +01:00
sdk direct mapping: misc fixes (#32649) 2023-08-30 16:57:24 +07:00
send-transaction-service preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
stake-accounts Refactor - Demotes `Arc` to `Rc`. (#32982) 2023-08-25 00:54:06 +02:00
storage-bigtable General cleanup (#32980) 2023-08-24 21:44:19 +02:00
storage-proto General cleanup (#32980) 2023-08-24 21:44:19 +02:00
streamer preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
system-test Update queried metric to determine tps in system tests (#31531) 2023-05-08 10:37:23 -05:00
test-validator Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
thin-client removes redundant NewConnectionConfig trait (#31979) 2023-06-06 21:28:29 +00:00
tokens preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
tpu-client client: Speed up send-and-confirm-parallel (#33032) 2023-08-29 18:29:10 +02:00
transaction-dos Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
transaction-status preliminaries for bumping nightly to 2023-08-25 (#33047) 2023-08-29 23:05:35 +00:00
turbine revises turbine epoch stakes for shreds propagation (#32743) 2023-08-28 21:31:51 +00:00
udp-client removes redundant NewConnectionConfig trait (#31979) 2023-06-06 21:28:29 +00:00
upload-perf chore: workspace inheritance (#29893) 2023-02-23 22:01:54 +08:00
validator General cleanup (#32980) 2023-08-24 21:44:19 +02:00
version ci: test-checks.sh all sbf code & use nightly only (#30602) 2023-03-13 14:28:34 +09:00
watchtower Add --active-stake-alert-threshold to watchtower (#31858) 2023-05-29 23:33:11 +00:00
web3.js Add link to new web3.js repo (#30096) 2023-02-27 10:23:54 -06:00
zk-keygen [zk-token-sdk] Make `ElGamalKeypair` fields private (#32190) 2023-06-22 10:28:13 +09:00
zk-token-sdk Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871) 2023-08-21 19:11:21 +02:00
.clippy.toml
.codecov.yml Add .codecov.yml with checks disabled (#30666) 2023-03-10 20:11:22 +09:00
.gitignore
.mergify.yml Add Mergify rules for v1.16 backport (#31875) 2023-05-30 15:07:05 -05:00
.travis.yml chore: remove Explorer build scripts from the Monorepo (#30262) 2023-02-10 17:25:32 -06:00
CHANGELOG.md Add changelog (#26894) 2023-07-28 17:09:35 -05:00
CONTRIBUTING.md contributing: Is your PR easy to say "yes" to? (#31059) 2023-04-06 08:43:22 -04:00
Cargo.lock direct mapping: misc fixes (#32649) 2023-08-30 16:57:24 +07:00
Cargo.toml build(deps): bump url from 2.4.0 to 2.4.1 (#33052) 2023-08-29 13:15:18 +00:00
LICENSE Correct LICENSE to conform with https://apache.org/licenses/LICENSE-2.0 (#31860) 2023-05-31 03:06:42 +00:00
README.md
RELEASE.md
SECURITY.md SECURITY.md: Require exploit PoC for submission consideration (#31113) 2023-04-11 19:32:48 -06:00
cargo
cargo-build-bpf
cargo-build-sbf
cargo-test-bpf
cargo-test-sbf
fetch-perf-libs.sh
fetch-spl.sh Upgrade to token-2022 0.6.0 (#30353) 2023-03-10 10:32:19 +09:00
nextest.toml ci: make test_batched_range_proof_u256 use all threads in nextest (#32627) 2023-07-26 15:42:17 +08:00
run.sh
rust-toolchain.toml Upgrades Rust to 1.69.0 (#31336) 2023-04-25 10:22:38 -04:00
rustfmt.toml
test-abi.sh
vercel.json

README.md

Solana

Solana crate Solana documentation Build status codecov

Building

1. Install rustc, cargo and rustfmt.

$ curl https://sh.rustup.rs -sSf | sh
$ source $HOME/.cargo/env
$ rustup component add rustfmt

When building the master branch, please make sure you are using the latest stable rust version by running:

$ rustup update

When building a specific release branch, you should check the rust version in ci/rust-version.sh and if necessary, install that version by running:

$ rustup install VERSION

Note that if this is not the latest rust version on your machine, cargo commands may require an override in order to use the correct version.

On Linux systems you may need to install libssl-dev, pkg-config, zlib1g-dev, protobuf etc.

On Ubuntu:

$ sudo apt-get update
$ sudo apt-get install libssl-dev libudev-dev pkg-config zlib1g-dev llvm clang cmake make libprotobuf-dev protobuf-compiler

On Fedora:

$ sudo dnf install openssl-devel systemd-devel pkg-config zlib-devel llvm clang cmake make protobuf-devel protobuf-compiler perl-core

2. Download the source code.

$ git clone https://github.com/solana-labs/solana.git
$ cd solana

3. Build.

$ ./cargo build

Testing

Run the test suite:

$ ./cargo test

Starting a local testnet

Start your own testnet locally, instructions are in the online docs.

Accessing the remote development cluster

  • devnet - stable public cluster for development accessible via devnet.solana.com. Runs 24/7. Learn more about the public clusters

Benchmarking

First, install the nightly build of rustc. cargo bench requires the use of the unstable features only available in the nightly build.

$ rustup install nightly

Run the benchmarks:

$ cargo +nightly bench

Release Process

The release process for this project is described here.

Code coverage

To generate code coverage statistics:

$ scripts/coverage.sh
$ open target/cov/lcov-local/index.html

Why coverage? While most see coverage as a code quality metric, we see it primarily as a developer productivity metric. When a developer makes a change to the codebase, presumably it's a solution to some problem. Our unit-test suite is how we encode the set of problems the codebase solves. Running the test suite should indicate that your change didn't infringe on anyone else's solutions. Adding a test protects your solution from future changes. Say you don't understand why a line of code exists, try deleting it and running the unit-tests. The nearest test failure should tell you what problem was solved by that code. If no test fails, go ahead and submit a Pull Request that asks, "what problem is solved by this code?" On the other hand, if a test does fail and you can think of a better way to solve the same problem, a Pull Request with your solution would most certainly be welcome! Likewise, if rewriting a test can better communicate what code it's protecting, please send us that patch!

Disclaimer

All claims, content, designs, algorithms, estimates, roadmaps, specifications, and performance measurements described in this project are done with the Solana Labs, Inc. (“SL”) good faith efforts. It is up to the reader to check and validate their accuracy and truthfulness. Furthermore, nothing in this project constitutes a solicitation for investment.

Any content produced by SL or developer resources that SL provides are for educational and inspirational purposes only. SL does not encourage, induce or sanction the deployment, integration or use of any such applications (including the code comprising the Solana blockchain protocol) in violation of applicable laws or regulations and hereby prohibits any such deployment, integration or use. This includes the use of any such applications by the reader (a) in violation of export control or sanctions laws of the United States or any other applicable jurisdiction, (b) if the reader is located in or ordinarily resident in a country or territory subject to comprehensive sanctions administered by the U.S. Office of Foreign Assets Control (OFAC), or (c) if the reader is or is working on behalf of a Specially Designated National (SDN) or a person subject to similar blocking or denied party prohibitions.

The reader should be aware that U.S. export control and sanctions laws prohibit U.S. persons (and other persons that are subject to such laws) from transacting with persons in certain countries and territories or that are on the SDN list. Accordingly, there is a risk to individuals that other persons using any of the code contained in this repo, or a derivation thereof, may be sanctioned persons and that transactions with such persons would be a violation of U.S. export controls and sanctions law.