Commit Graph

4395 Commits

Author SHA1 Message Date
godmodegalactus c1040b0129
Add mango patches to jito 2024-01-10 15:58:34 +01:00
buffalu 165a87482d Backport Jito Patch (#408)
Backport clone derivation (416) (#418)

backports #430: update jito-programs (#432)

Backport #446 to v1.17 (#448)

[JIT-1661] Faster Autosnapshot (#445)

v1.17: Backport #449 (#451)

backports #419: add upsert to accountoverrides (#421)

backport 428 runtime-plugin (#458)

[JIT-1713] Fix bundle's blockspace preallocation (Backport to 1.17) (#499)

Backport #500 to v1.17 (#507)

Backport #520: add priority fees to mev claim (#525)
2024-01-09 17:05:26 -06:00
Alexander Meißner 5787b48b92
v1.17 - Cooperative Program Loading (#34487)
* Disables verification-less reloading.

* Refactors LoadedPrograms::extract() to use a mutable parameter instead of returning the list of missing entries.

* cargo fmt

* Refactors LoadedPrograms::extract() to use a mutable parameter instead of returning a LoadedProgramsForTxBatch.

* Adds explicit SecondLevel structure to LoadedPrograms.

* Adds cooperative_loading_task.

* Implements cooperative loading in the bank.

* Fixes fork graph in tests.

* Adds LoadingTaskWaiter.
2023-12-16 14:27:17 +01:00
mergify[bot] e191d8d1ca
v1.17: Refactor - `ExtractedPrograms` (backport of #34205) (#34351)
Refactor - `ExtractedPrograms` (#34205)

* Puts ExtractedPrograms into Arc<Mutex<>>.

* Merges ExtractedPrograms::missing and ExtractedPrograms::unloaded.

* Unifies missing entry insertion in LoadedPrograms::extract().

(cherry picked from commit 4ee5078e5f)

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-12-14 17:53:37 +01:00
mergify[bot] ca34005ad5
v1.17: Set root slot/epoch in cache constructor (backport of #34447) (#34460) 2023-12-14 07:15:56 -08:00
mergify[bot] 4abd18f6cd
v1.17: Skip shrink until startup verification is complete (backport of #34209) (#34227) 2023-11-30 17:58:40 -05:00
mergify[bot] ff1d9a6292
v1.17: Output BankHashDetails file when leader drops its' own block (backport of #34256) (#34275)
Output BankHashDetails file when leader drops its' own block (#34256)

Currently, the file is generated when a node drops a block that was
produced by another node. However, it would also be beneficial to see
the account state when a node drops its' own block.

Output the file in this additional failure codepath

(cherry picked from commit 935e06f8f1)

Co-authored-by: steviez <steven@solana.com>
2023-11-30 16:36:16 -06:00
mergify[bot] 61caae6b2b
v1.17: ancient shrink on its own cadence (backport of #33712) (#34060)
ancient shrink on its own cadence (#33712)

(cherry picked from commit d948e5bf69)

Co-authored-by: Jeff Washington (jwash) <jeff.washington@solana.com>
2023-11-17 16:09:13 -06:00
mergify[bot] cf1d9d4c45
v1.17: Fix - `Bank::compute_active_feature_set()` and `Bank::apply_feature_activations()` (backport of #34124) (#34136)
Fix - `Bank::compute_active_feature_set()` and `Bank::apply_feature_activations()` (#34124)

* Moves modification of feature accounts from Bank::compute_active_feature_set() into Bank::apply_feature_activations().

* Renames allow_new_activations and newly_activated to include_pending and pending.

* Fix test_compute_active_feature_set.

(cherry picked from commit 6b8545061f)

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-11-17 15:59:55 +01:00
mergify[bot] 956eacb23e
v1.17: Feature - Epoch boundary recompilation phase (backport of #33477) (#34003)
Feature - Epoch boundary recompilation phase (#33477)

* Adds LoadedPrograms::upcoming_environments.

* Moves LoadedPrograms::prune_feature_set_transition() into LoadedPrograms::prune().

* Adds parameter recompile to Bank::load_program().

* Sets latest_root_slot/epoch and environments in Bank::finish_init().

* Removes FEATURES_AFFECTING_RBPF list.

* Adjusts test_feature_activation_loaded_programs_recompilation_phase().

(cherry picked from commit a9509f56b7)

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-11-09 18:37:05 +01:00
mergify[bot] 97795881ce
v1.17: Burn fees collected into invalid accounts (backport of #33887) (#33951)
* Burn fees collected into invalid accounts (#33887)

* refactor: create bank::fee_distribution module

* feature: add checks to fee distribution

* refactor: move Bank::deposit fn into test_utils

* feedback

* feedback 2

* add datapoints

* change to datapoint_warn

* typo

(cherry picked from commit ebe8afb0c3)

# Conflicts:
#	runtime/src/bank.rs
#	runtime/src/bank/tests.rs

* resolve conflicts

---------

Co-authored-by: Justin Starry <justin.starry@icloud.com>
Co-authored-by: Justin Starry <justin@solana.com>
2023-11-07 00:49:34 +01:00
mergify[bot] ae7d7427be
v1.17: revert programify feature gate (backport of #33897) (#33905)
revert programify feature gate (#33897)

(cherry picked from commit ecd674bd92)

Co-authored-by: Joe C <joe.caulfield@solana.com>
2023-10-28 00:24:31 +02:00
mergify[bot] bfab9bb3ae
v1.17: Use program cache fork graph in extract() (backport of #33806) (#33862)
Use program cache fork graph in extract() (#33806)

* Use program cache fork graph instead of WorkingSlot trait

* Fix deadlocked tests

* keep WorkingSlot trait for now

(cherry picked from commit 78c31aa6b8)

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2023-10-25 16:03:39 +00:00
mergify[bot] 22f44d0518
v1.17: Initialize fork graph in program cache during bank_forks creation (backport of #33810) (#33817)
* Initialize fork graph in program cache during bank_forks creation (#33810)

* Initialize fork graph in program cache during bank_forks creation

* rename BankForks::new to BankForks::new_rw_arc

* fix compilation

* no need to set fork_graph on insert()

* fix partition tests

(cherry picked from commit 9d42cd7efe)

# Conflicts:
#	core/src/repair/repair_service.rs
#	core/tests/epoch_accounts_hash.rs
#	ledger/src/blockstore_processor.rs
#	rpc/src/rpc_service.rs
#	runtime/src/bank/tests.rs

* fix merge conflicts

---------

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2023-10-23 18:28:40 +00:00
mergify[bot] cc4b71a8d7
v1.17: Set a global fork graph in program cache (backport of #33776) (#33809)
Set a global fork graph in program cache (#33776)

* Set a global fork graph in program cache

* fix deadlock

* review feedback

(cherry picked from commit 59cb3b57ee)

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2023-10-21 13:21:57 -07:00
mergify[bot] 9021d224be
v1.17: Bump solana_rbpf to v0.8.0 (backport of #33679) (#33793)
* updated inner_call for hashing fn to follow generic approach (#33128)

* updated inner_call for hashing fn to follow generic approach

* different hash compute budget values for all digests

* fixed conflicts

* reverted changes to compute_budget.rs and added 3method to trait to get compute budget values

* updated type for result fn for HasherImpl

* using Hash directly in result fn, got rid of HASH_BYTES and removed comment form compute_budget

* updated import statement

* cargo fmt -all

* removed unused import and reference related warning

* oops forgot semicolon

* removed trailing white space

(cherry picked from commit a60d1857bf)

* Bump solana_rbpf to v0.8.0 (#33679)

* Bumps solana_rbpf to v0.8.0

* Adjustments:
Replaces declare_syscall!() with declare_builtin_function!().
Removes Config::encrypt_runtime_environment.
Simplifies error propagation.

(cherry picked from commit a5c7c999e2)

---------

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-10-21 14:14:23 +02:00
mergify[bot] e76dd77e45
v1.17: Fix non-determinism in account_hash_ignore_slot on genesis (backport of #33692) (#33720)
Fix non-determinism in account_hash_ignore_slot on genesis (#33692)

(cherry picked from commit 69495f4c13)

Co-authored-by: Jeff Washington (jwash) <jeff.washington@solana.com>
2023-10-19 08:56:43 -07:00
mergify[bot] 02f9c97446
v1.17: Refactor - LoadedPrograms part 2 (backport of #33694) (#33695)
* Refactor - LoadedPrograms part 2 (#33694)

(cherry picked from commit a3f85aba21)

# Conflicts:
#	runtime/src/bank/tests.rs

* fix merge conflicts

---------

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
Co-authored-by: Pankaj Garg <pankaj@solana.com>
2023-10-13 22:00:35 +00:00
mergify[bot] 10b00df3d1
v1.17: bank: do not remove trailing 0 bytes from return data (backport of #33639) (#33685)
bank: do not remove trailing 0 bytes from return data (#33639)

This is creating havoc for Solang, as the return data is borsh encoded
and therefore `u64` values like 0x100 get truncated.

(cherry picked from commit 47511999bb)

Co-authored-by: Sean Young <sean@mess.org>
2023-10-13 20:22:34 +01:00
mergify[bot] c96223b69a
v1.17: Prevent delay_visibility_of_program_deployment feature from cache pruning (backport of #33689) (#33691)
Prevent delay_visibility_of_program_deployment feature from cache pruning (#33689)

(cherry picked from commit fd9297775a)

Co-authored-by: Pankaj Garg <pankaj@solana.com>
2023-10-13 15:24:18 +00:00
mergify[bot] e1278e2127
v1.17: Adds `--no-skip-initial-accounts-db-clean` *hidden* CLI flag (backport of #33664) (#33677)
Adds `--no-skip-initial-accounts-db-clean` *hidden* CLI flag (#33664)

(cherry picked from commit 452fd5d384)

Co-authored-by: Brooks <brooks@solana.com>
2023-10-12 16:50:38 -04:00
mergify[bot] eceba1c241
v1.17: Update hashes per tick with feature gates (backport of #33600) (#33661)
Update hashes per tick with feature gates (#33600)

* Update hashes per tick with feature gates

(cherry picked from commit b36d051b51)

Co-authored-by: Brennan <brennan.watt@solana.com>
2023-10-11 12:26:12 -07:00
mergify[bot] c534100799
v1.17: update feature gate no-op program id (backport of #33535) (#33540)
update feature gate no-op program id (#33535)

(cherry picked from commit 93d8bcca4b)

Co-authored-by: Joe C <jcaulfield135@gmail.com>
2023-10-06 16:24:55 -05:00
mergify[bot] 05bbaf3f8e
v1.17: Removes unnecessary borrow (backport of #33539) (#33541)
(cherry picked from commit e0091d6995)

Co-authored-by: Brooks <brooks@solana.com>
2023-10-05 17:57:17 -04:00
mergify[bot] 9318394bb5
v1.17: Bank: Add function to replace empty account with upgradeable program on feature activation (backport of #32783) (#33527)
Bank: Add function to replace empty account with upgradeable program on feature activation (#32783)

* replace program account

* modify for all cases

* remove non-data swap

* address tests & conditional feedback

* get the rent involved

* mix in owner & executable

* feature-related cases

* stripped back to feature-specific case only

* added feature

* address initial feedback

* added more lamport checks

* condense tests

* using test_case

* add fail cases to tests

* more cleanup

* add verifiably built program

* update program account state

* cleaned up serializing logic

* use full word capitalization

* rename old & new to dst & src

* swap src and dst in parameters

* add warnings and errors

* rename feature to programify

* test suite description clarity

* remove strings from datapoints

* spell out source and destination

* more verbose comments in account replace functions

* move lamport calculation

* swap lamport check for state check

* move replace functions to helper module

* make replace_account methods fallible

* refactor error handling

* add test for source program state

(cherry picked from commit 25460f76e7)

Co-authored-by: Joe C <jcaulfield135@gmail.com>
2023-10-05 18:28:30 +02:00
mergify[bot] b90f08f966
v1.17: Deprecate bz2/gzip/none/tar snapshot compression types (backport of #33484) (#33492)
Deprecate bz2/gzip/none/tar snapshot compression types (#33484)

These options are now disallowed on the command line for
solana-validator and solana-ledger-tool, which effectively means no more
snapshots will be created with this types in normal usecases. However,
support for reading the deprecated types is still in place.

(cherry picked from commit 73e9e6dd70)

Co-authored-by: steviez <steven@solana.com>
2023-10-03 15:18:42 -05:00
mergify[bot] f9c87c9d57
v1.17: Refactor - `LoadedPrograms` (backport of #33482) (#33491)
Refactor - `LoadedPrograms` (#33482)

* Adds type ProgramRuntimeEnvironment.

* Moves LoadedPrograms::remove_expired_entries() into LoadedPrograms::prune().

* Adds Stats::prunes_environment and renames Stats::prunes_orphan and Stats::prunes_expired.

* Adds LoadedPrograms::latest_root_epoch.

* Typo fix, authored-by: Dmitri Makarov <dmakarov@users.noreply.github.com>

(cherry picked from commit 8033be333e)

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-10-03 21:01:06 +02:00
Tao Zhu cc4e9283db
reduce WARN logging to only necessary scenario (#33408) 2023-09-28 09:53:17 -05:00
steviez 5b9a167c51
Add uniform start/stop log lines for background hash verification (#33441)
Add uniform start/stop log lines for background hash verification
2023-09-28 15:13:51 +02:00
Alexander Meißner 7ff797bcef
Refactor - Remove parameter `feature_set` from `load_program_from_bytes()` (#33395)
Replaces parameter feature_set with delay_visibility_of_program_deployment in load_program_from_bytes().
2023-09-25 17:16:31 +02:00
ananas-block 997aa0a3f8
Feat(syscall): add altbn128 g1 & g2 compression (#32870)
* solana-program - altbn128: add g1 & g2 compression

still fixing tests for point of infinity

feat: proof compression syscall working

add rust test to ci

remove prints

added c test

added sycall pricing

* fixed ci checks

* refactored altbn128 and compression
2023-09-25 05:43:34 -07:00
Trent Nelson 1840fd7ab3
Feature - better error codes for tx lamport check (#33343)
Replaces `TransactionError::InstructionError(0, InstructionError::UnbalancedInstruction)` with `TransactionError::UnbalancedTransaction`.

Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2023-09-22 23:58:05 +00:00
Tao Zhu a2ad820309
compute_budget heap_size does not have to be optional (#33313) 2023-09-21 09:24:47 -05:00
Tyera bca41edf20
Make active stake consistent in split (#33295)
* Add feature gate

* Add helper fn

* Require split destination to be rent-exempt if it is active

* Update cli to prefund split accounts

* cli: require rent param with sign-only

* Update tokens to prefund split accounts

* Update split tests with sysvar accounts

* Fix test_split_to_account_with_rent_exempt_reserve

* Fix test_staked_split_destination_minimum_balance

* Fix test_split_more_than_staked

* Fix test_split_minimum_stake_delegation and remove misleading StakeState::Initialized case

* Fix test_split_from_larger_sized_account

* Add test for pre-/post-activation behavior splitting some or all of stake account

* Assert active stake

* Fix runtime test

* Ignore stake-pool downstream

* Review comments

* Feature gate sysvar reads
2023-09-20 06:00:51 +00:00
Pankaj Garg f50342a790
Split vote related code from runtime to its own crate (#32882)
* Move vote related code to its own crate

* Update imports in code and tests

* update programs/sbf/Cargo.lock

* fix check errors

* update abi_digest

* rebase fixes

* fixes after rebase
2023-09-19 10:46:37 -07:00
Brooks 889d6c655e
Moves accounts-db benches to accounts-db crate (#33306) 2023-09-19 13:09:42 -04:00
Alexander Meißner bc38ef27d8
Feature - Enable Program-Runtime-v2 and Loader-v4 (#33294)
* Adds a new feature `enable_program_runtime_v2_and_loader_v4`.

* Adds a feature gated builtin entry for the solana_loader_v4_program.
2023-09-19 16:29:52 +02:00
Brooks bc2b372762
Adds DCOU to verify_snapshot_archive() (#33298) 2023-09-18 19:11:28 -04:00
Tao Zhu 8b8a21a52f
cleanup feature: enable request heap frame instruction #30076 (#33243)
* cleanup feature: enable request heap frame instruction #30076

* update sbf tests

* removed out dated comments and test
2023-09-18 16:06:24 -05:00
steviez 9e11ae6275
Make program owners a const array instead of Vec<_> (#33275)
The program owners pubkeys are constant, no need to reconstruct the
Vec<Pubkey> and Vec<&Pubkey> each time this function runs (every time we
execute transactions).
2023-09-18 10:59:03 -05:00
behzad nouri c1090d3959
moves new_warmup_cooldown_rate_epoch outside iterators and for loops (#33259)
Recalculating new_warmup_cooldown_rate_epoch for each item is redundant
and wasteful and instead can be done only once outside the iterators and
for loops.
Also NewWarmupCooldownRateEpoch is unnecessary and verbose and is
removed in this commit.
2023-09-15 00:06:34 +00:00
Christian Kamm 525e59f01a
sdk, programs/bpf_loader: add sol_remaining_compute_units syscall (#31640)
bpf_loader: add sol_remaining_compute_units syscall

Co-authored-by: jonch <9093549+jon-chuang@users.noreply.github.com>
2023-09-13 08:57:08 -06:00
Pankaj Garg 5562f79cc5
Do not unnecessarily re-verify unloaded program (#32722)
* Do not unnecessarily re-verify unloaded program

* clippy fixes

* new unit test for extract

* fixes after rebase

* fixes after rebase

* address review comments

* fix clippy
2023-09-13 06:25:56 -07:00
Alexander Meißner bbb57be0a5
Refactor - Move interfaces of address-lookup-table into the program SDK (#33165)
* Adds a module `address_lookup_table` to the SDK.

* Adds a module `address_lookup_table::instruction` to the SDK.

* Adds a module `address_lookup_table::error` to the SDK.

* Adds a module `address_lookup_table::state` to the SDK.

* Moves AddressLookupTable into SDK as well.

* Moves AddressLookupTableAccount into address_lookup_table.

* Adds deprecation messages.

* Disentangles dependencies across cargo files.
2023-09-11 21:10:40 +02:00
Tao Zhu 4f4ce69f5f
purge duplicated bank prioritization fee from cache (#33062)
* purge duplicated bank prioritization fee from cache

* add test for purge dup bank

* Added metrics counts to monitor anomalies

* fix a flaky test
2023-09-11 10:08:55 -05:00
Pankaj Garg 377b6003a8
Check loader-v4::id() as a valid program owner (#33167) 2023-09-06 11:54:18 -07:00
Brooks a80819b029
PrunedBanksRequestHandler::handle_request() is only pub with dcou (#33155) 2023-09-06 17:02:52 +00:00
behzad nouri 88ee8f5820
replaces once_cell::sync::OnceCell with std::sync::OnceLock (#33140)
std::sync::OnceLock has become stable since rust 1.70.0 and there is no
longer a need for an external crate dependency.
2023-09-06 16:46:51 +00:00
Brooks 9e156f88f4
Removes invariant `is_serialized_with_abs` param (#33154) 2023-09-06 12:00:18 -04:00
Brooks d921b9a44e
Adds metrics when purging banks with the same slot (#33153) 2023-09-06 11:38:30 -04:00