test-validator: Fix upgradeable programs at genesis (#3692)
#### Problem
As noted in
https://solana.stackexchange.com/questions/17478/solana-localnet-error-while-upgrading-a-program-loaded-at-genesis-using-solan,
an upgradeable program loaded at genesis is not actually upgradeable,
failing with `instruction changed executable accounts data`. This is
because the program-data account is incorrectly set as "executable",
which it isn't really.
#### Summary of changes
Don't set the program-data account as executable. This is the only place
in the repo where we were doing this incorrectly.
(cherry picked from commit 73f54a05ac38228b1d86504041a17efcb46d6468)
Co-authored-by: Jon C <me@jonc.dev>
* test-validator: Add flag to clone feature set from a cluster
#### Problem
Program devs run into issues when testing, where everything works great
with a solana-test-validator, and then fails once they deploy to a real
cluster. Most times, this happens because the test validator enables all
features by default, and their program depended on a new feature without
their knowledge.
#### Summary of changes
To make local development much easier, add a `--clone-feature-set` flag
to solana-test-validator to easily mimic the functionality on the
cluster targeted with `--url`.
* Add changelog entry
* use rate limit on connectings
use rate limit on connectings; missing file
* Change connection rate limit to 8/min instead of 4/s
* Addressed some feedback from Trent
* removed some comments
* fix test failures which are opening connections more frequently
* moved the flag up
* turn off rate limiting to debug CI
* Fix CI test failures
* differentiate of the two throttling cases in stats: across connections or per ip addr
* fmt issues
* Addressed some feedback from Trent
* Added unit tests
Cleanup connection cache rate limiter if exceeding certain threshold
missing files
CONNECITON_RATE_LIMITER_CLEANUP_THRESHOLD to 100_000
clippy issue
clippy issue
sort crates
* revert Cargo.lock changes
* Addressed some feedback from Pankaj
RuntimeConfig doesn't use anything SVM specific and logically belongs
in program runtime rather than SVM. This change moves the definition
of RuntimeConfig struct from the SVM crate to program-runtime and
adjusts `use` statements accordingly.
* allow pedantic invalid cast lint
* allow lint with false-positive triggered by `test-case` crate
* nightly `fmt` correction
* adapt to rust layout changes
* remove dubious test
* Use transmute instead of pointer cast and de/ref when check_aligned is false.
* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.
* bump rust nightly to 2023-08-25
* Upgrades Rust to 1.72.0
---------
Co-authored-by: Trent Nelson <trent@solana.com>
Changes:
marshallpierce/rust-base64@v0.13.1...v0.21.0
`base64::{encode,decode}` are now deprecated in favor of an API that
explicitly selects an `Engine`. Migrated all calls to the new API.
* Enables JIT everywhere by default.
* Removes the CLI argument "no-bpf-jit" and its plumbering through out the validator code base.
* Removes with_jit bpf_loader variants.
* Removes the to_builtin!() macro.
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well. It is somewhat
confusing to call it `poh_verify`.
* Move AdminRpcRequestMetadataPostInit to solana-core
* Move AdminRpcRequestMetadataPostInit write to just before wait_for_supermajority
* Pass AdminRpcRequestMetadataPostInit in TestValidatorGenesis
* Fixup local-cluster
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
tpu-client/tpu_connection_cache is refactored out the module and moved to connection-cache/connection_cache and the logic in client/connection_cache is consolidated to connection-cache/connection_cache as well. client/connection_cache only has a thin wrapper which forward calls to connection-cache/connection_cache and deal with constructions of quic/udp connection cache for clients using them both.2.
The TpuConnection is refactored to ClientConnection to make it generic and functions renamed to be proper for other workflows. eg. tpu_addr -> server_addr, send_transaction --> send_data and etc...
The enum dispatch is removed so that we can make the bulk of code of quic and udp agnostic of each other. The client is possible to load quic or udp only into its runtime.
The generic type parameter in the tpu-client/tpu_connection_cache is removed in order to create both quic and udp connection cache and use the object to send transactions with multiple branching when sending data. The generic type parameters and associated types are dropped in other types in order to make the trait "object safe" for this purpose.
I have annotated the code explaining the reasoning and the refactoring source -> destination.
There is no functional changes
bench-tps has been performed for rpc-client, thin-client and tpu-client. And it is found the performance number largely match the ones before the refactoring.
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
* Clean up files in the old account_path before trasitioning to the new run path
* try_exist and accounts_dir removing extra
* sync rmdir, is_dir check
* handle the account_path not deletable case
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
mergify[bot]
steviez
Andrew Fitzgerald
Kevin Heavey
Yihau Chen
Jon C
Tyera
Joe C
Lijun Wang
Joe C
Dmitri Makarov
Pankaj Garg
Brooks
Joe C
Alexander Meißner
behzad nouri
Jeff Biseda
Illia Bobyr
cavemanloverboy
Tyera
Yihau Chen
Michael Vines
skrrb
Kevin Ji
Will Hickey
joeaba
Xiang Zhu