117a194b73
* AccountSharedData: make data_mut() private This ensures that the inner Vec is never handed out. This is in preparation of enforcing that the capacity of the inner vec never shrinks, which is required for direct mapping. * Adds the feature bpf_account_data_direct_mapping. * Remaps EbpfError::AccessViolation into InstructionError::ReadonlyDataModified. * WIP: Memory regions for each instruction account in create_vm(). * Fix serialization benches, run both copy and !copy variants * rbpf-cli: fix build * BorrowedAccount: ensure that account capacity is never reduced Accounts can be directly mapped in address space. Their capacity can't be reduced mid transaction as that would create holes in vm address space that point to invalid host memory. * bpf_load: run serialization tests for both copy and !copy account data * bpf_loader: add Serializer::write_account * fix lints * BorrowedAccount: make_data_mut is host only * Fix unused import warning * Fix lints * cpi: add explicit direct_mapping arg to update_(callee|caller)_account * cpi: rename account_data_or_only_realloc_padding to serialized_data * cpi: add CallerAccount::original_data_len comment * cpi: add update_callee_account direct_mapping test * cpi: add test_update_caller_account_data_direct_mapping and fix bug We used to have a bug in zeroing data when shrinking account, where we zeroed the spare account capacity but not the realloc padding. * cpi: add tests for mutated readonly accounts * cpi: update_caller_account doesn't need to change .serialized_data when direct_mapping is on * cpi: update_caller_account: ensure that account capacity is always enough Introduce a better way to ensure that account capacity never goes below what might be mapped in memory regions. * cpi: zero account capacity using the newly introduced BorrowedAccount::spare_data_capacity_mut() Before we were using BorrowedAccount::get_data_mut() to get the base pointer to the account data, then we were slicing the spare capacity from it. Calling get_data_mut() doesn't work if an account has been closed tho, since the current program doesn't own the account anymore and therefore get_data_mut() errors out. * bpf_loader: fix same lint for the umpteenth time * bpf_loader: map AccessViolation to ReadonlyDataModified only for account region violations * programs/sbf: realloc: add test for large write after realloc Add a test that after a realloc does a large write that spans the original account length and the realloc area. This ensures that memory mapping works correctly across the boundary. * programs/sbf: run test_program_sbf_realloc with both direct_mapping on and off By default test banks test with all features on. This ensures we keep testing the existing code until the new feature is enabled. * bpf_loader: tweak memcmp syscall Split the actual memcmp code in a separate function. Remove check indexing the slices since the slices are guaranteed to have the correct length by construction. * bpf_loader: tweak the memset syscall Use slice::fill, which is effectively memset. * bpf_loader: syscalls: update mem syscalls to work with non contiguous memory With direct mapping enabled, accounts can now span multiple memory regions. * fix lint, rebase mem_ops * Implement CoW for writable accounts * Fix CI * Move CoW to the MemoryMapping level * Update after rbpf API change * Fix merge screwup * Add create_vm macro. Fix benches. * cpi: simplify update_caller_account Simplify the logic to update a caller's memory region when a callee causes an account data pointer to change (eg during CoW) * benches/bpf_loader: move serialization out of create_vm bench * benches/bpf_loader: don't copy accounts when direct mapping is on * Fix review nits * bpf_loader: mem_ops: handle u64 overflow in MemoryChunkIterator::new When starting at u64::MAX, the chunk iterator would always return the empty sequence (None on the first next()) call, instead of returning a memory access violation. Use checked instead of saturating arithmetic to detect the condition and error out. This commit also adds more tests around boundary conditions. * Fix loader-v3 tests: data_mut => data_as_mut_slice * Fix CI * bpf_loader: fix tuner bench: account must be writable With direct mapping on, invalid writes are caught early meaning the tuner would fail on the first store and not consume the whole budget like the benchmark expects. --------- Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net> |
||
---|---|---|
.buildkite | ||
.github | ||
account-decoder | ||
accounts-bench | ||
accounts-cluster-bench | ||
banking-bench | ||
banks-client | ||
banks-interface | ||
banks-server | ||
bench-streamer | ||
bench-tps | ||
bloom | ||
bucket_map | ||
cd | ||
ci | ||
clap-utils | ||
clap-v3-utils | ||
cli | ||
cli-config | ||
cli-output | ||
client | ||
client-test | ||
connection-cache | ||
core | ||
docs | ||
dos | ||
download-utils | ||
entry | ||
faucet | ||
frozen-abi | ||
genesis | ||
genesis-utils | ||
geyser-plugin-interface | ||
geyser-plugin-manager | ||
gossip | ||
install | ||
keygen | ||
ledger | ||
ledger-tool | ||
local-cluster | ||
log-analyzer | ||
logger | ||
measure | ||
memory-management | ||
merkle-root-bench | ||
merkle-tree | ||
metrics | ||
multinode-demo | ||
net | ||
net-shaper | ||
net-utils | ||
notifier | ||
perf | ||
poh | ||
poh-bench | ||
program-runtime | ||
program-test | ||
programs | ||
pubsub-client | ||
quic-client | ||
rayon-threadlimit | ||
rbpf-cli | ||
remote-wallet | ||
rpc | ||
rpc-client | ||
rpc-client-api | ||
rpc-client-nonce-utils | ||
rpc-test | ||
runtime | ||
scripts | ||
sdk | ||
send-transaction-service | ||
stake-accounts | ||
storage-bigtable | ||
storage-proto | ||
streamer | ||
sys-tuner | ||
system-test | ||
test-validator | ||
thin-client | ||
tokens | ||
tpu-client | ||
transaction-dos | ||
transaction-status | ||
udp-client | ||
upload-perf | ||
validator | ||
version | ||
watchtower | ||
web3.js | ||
zk-token-sdk | ||
.clippy.toml | ||
.codecov.yml | ||
.gitignore | ||
.mergify.yml | ||
.travis.yml | ||
CONTRIBUTING.md | ||
Cargo.lock | ||
Cargo.toml | ||
LICENSE | ||
README.md | ||
RELEASE.md | ||
SECURITY.md | ||
cargo | ||
cargo-build-bpf | ||
cargo-build-sbf | ||
cargo-test-bpf | ||
cargo-test-sbf | ||
fetch-perf-libs.sh | ||
fetch-spl.sh | ||
nextest.toml | ||
run.sh | ||
rust-toolchain.toml | ||
rustfmt.toml | ||
test-abi.sh | ||
vercel.json |
README.md
Building
1. Install rustc, cargo and rustfmt.
$ curl https://sh.rustup.rs -sSf | sh
$ source $HOME/.cargo/env
$ rustup component add rustfmt
When building the master branch, please make sure you are using the latest stable rust version by running:
$ rustup update
When building a specific release branch, you should check the rust version in ci/rust-version.sh
and if necessary, install that version by running:
$ rustup install VERSION
Note that if this is not the latest rust version on your machine, cargo commands may require an override in order to use the correct version.
On Linux systems you may need to install libssl-dev, pkg-config, zlib1g-dev, protobuf etc.
On Ubuntu:
$ sudo apt-get update
$ sudo apt-get install libssl-dev libudev-dev pkg-config zlib1g-dev llvm clang cmake make libprotobuf-dev protobuf-compiler
On Fedora:
$ sudo dnf install openssl-devel systemd-devel pkg-config zlib-devel llvm clang cmake make protobuf-devel protobuf-compiler perl-core
2. Download the source code.
$ git clone https://github.com/solana-labs/solana.git
$ cd solana
3. Build.
$ ./cargo build
Testing
Run the test suite:
$ ./cargo test
Starting a local testnet
Start your own testnet locally, instructions are in the online docs.
Accessing the remote development cluster
devnet
- stable public cluster for development accessible via devnet.solana.com. Runs 24/7. Learn more about the public clusters
Benchmarking
First, install the nightly build of rustc. cargo bench
requires the use of the
unstable features only available in the nightly build.
$ rustup install nightly
Run the benchmarks:
$ cargo +nightly bench
Release Process
The release process for this project is described here.
Code coverage
To generate code coverage statistics:
$ scripts/coverage.sh
$ open target/cov/lcov-local/index.html
Why coverage? While most see coverage as a code quality metric, we see it primarily as a developer productivity metric. When a developer makes a change to the codebase, presumably it's a solution to some problem. Our unit-test suite is how we encode the set of problems the codebase solves. Running the test suite should indicate that your change didn't infringe on anyone else's solutions. Adding a test protects your solution from future changes. Say you don't understand why a line of code exists, try deleting it and running the unit-tests. The nearest test failure should tell you what problem was solved by that code. If no test fails, go ahead and submit a Pull Request that asks, "what problem is solved by this code?" On the other hand, if a test does fail and you can think of a better way to solve the same problem, a Pull Request with your solution would most certainly be welcome! Likewise, if rewriting a test can better communicate what code it's protecting, please send us that patch!
Disclaimer
All claims, content, designs, algorithms, estimates, roadmaps, specifications, and performance measurements described in this project are done with the Solana Labs, Inc. (“SL”) good faith efforts. It is up to the reader to check and validate their accuracy and truthfulness. Furthermore, nothing in this project constitutes a solicitation for investment.
Any content produced by SL or developer resources that SL provides are for educational and inspirational purposes only. SL does not encourage, induce or sanction the deployment, integration or use of any such applications (including the code comprising the Solana blockchain protocol) in violation of applicable laws or regulations and hereby prohibits any such deployment, integration or use. This includes the use of any such applications by the reader (a) in violation of export control or sanctions laws of the United States or any other applicable jurisdiction, (b) if the reader is located in or ordinarily resident in a country or territory subject to comprehensive sanctions administered by the U.S. Office of Foreign Assets Control (OFAC), or (c) if the reader is or is working on behalf of a Specially Designated National (SDN) or a person subject to similar blocking or denied party prohibitions.
The reader should be aware that U.S. export control and sanctions laws prohibit U.S. persons (and other persons that are subject to such laws) from transacting with persons in certain countries and territories or that are on the SDN list. Accordingly, there is a risk to individuals that other persons using any of the code contained in this repo, or a derivation thereof, may be sanctioned persons and that transactions with such persons would be a violation of U.S. export controls and sanctions law.