Commit Graph

529 Commits

Author SHA1 Message Date
Brooks 93f5b514fa
Adds StartingSnapshotStorages to AccountsHashVerifier (#58) 2024-03-04 16:32:51 -05:00
Pankaj Garg 46b9586630
SVM: Move SVM code to its own crate folder (#35119) 2024-02-06 16:06:32 -08:00
Pankaj Garg 3cf5dd2afb
SVM: Move RuntimeConfig to svm folder (#35085) 2024-02-05 13:49:36 -08:00
Brooks e1260a9604
Removes unused parameters from AccountsHashVerifier::new() (#34976) 2024-01-26 21:52:05 +00:00
Brooks b8144979dc
clippy: core lints (#34629)
```
warning: writing `&Vec` instead of `&[_]` involves a new object where a slice will do
   --> core/src/banking_stage/consumer.rs:153:29
    |
153 |         packets_to_process: &Vec<Arc<ImmutableDeserializedPacket>>,
    |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: change this to: `&[Arc<ImmutableDeserializedPacket>]`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#ptr_arg
    = note: `#[warn(clippy::ptr_arg)]` on by default

warning: `solana-core` (lib) generated 1 warning
```

```
warning: unit tests in doctest are not executed
  --> core/tests/fork-selection.rs:15:5
   |
15 |   //! #[test]
   |  _____^
16 | | //! #[ignore]
17 | | //! fn test_all_partitions() {
   | |__________________________^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#test_attr_in_doctest
   = note: `#[warn(clippy::test_attr_in_doctest)]` on by default

warning: `solana-core` (test "fork-selection") generated 1 warning
```
2024-01-04 17:02:56 +00:00
Nick Frostbutter fc2a8794be
[docs] updated readme and fix links (#34565)
* feat: updated readme

* fix: updated links

* fix: proposal links

* fix: more links

* fix: json-rpc links

* fix: more links

* fix: zk links

* fix: managing forks

* fix: links for deprecated methods
2024-01-03 09:06:06 -05:00
Ford 057ad2ab47
Fix typo in comments in core (#34594) 2023-12-26 16:13:17 -05:00
Brooks 6b76ec1de0
Uses Bank::new_for_tests() instead of new_for_tests_with_config() (#34587) 2023-12-26 12:54:14 -05:00
GoodDaisy 03386cc7b9
Fix typos (#34459)
* Fix typos

* Fix typos

* fix typo
2023-12-21 13:06:00 -07:00
Brooks 0b6d939e21
Removes epoch_accounts_hash featurization (#34417) 2023-12-13 15:30:39 -05:00
Lucas Steuernagel 1877fdb273
Use BankForks on tests - Part 4 (#34271)
* Use BankForks on tests - Part 4

* Ensure the correct slot is set
2023-12-06 13:32:04 -03:00
steviez 73815aee51
Move and rename ledger services from core to ledger (#33947)
These services currently live in core/; however, they operate on the
ledger. Mores so, these two services operate on the blockstore only,
and not necessarily the entire ledger. So, it makes sense to move these
services out of core and into ledger. We've recently been doing similar
changes with breaking things out into individual crates in order to
reduce the scope of core.

So, this change moves the services from core/ to ledger/, and replaces
ledger with blockstore.
2023-11-08 11:58:31 -06:00
Pankaj Garg 78c31aa6b8
Use program cache fork graph in extract() (#33806)
* Use program cache fork graph instead of WorkingSlot trait

* Fix deadlocked tests

* keep WorkingSlot trait for now
2023-10-25 06:04:38 -07:00
Jeff Washington (jwash) b0b4e1f0c0
remove IncludeSlotInHash after feature activation on mnb (#33816)
* remove IncludeSlotInHash after feature activation on mnb

* fix compile errors

* compile errors

* fix tests

* fix test results
2023-10-23 15:12:02 -07:00
Pankaj Garg 9d42cd7efe
Initialize fork graph in program cache during bank_forks creation (#33810)
* Initialize fork graph in program cache during bank_forks creation

* rename BankForks::new to BankForks::new_rw_arc

* fix compilation

* no need to set fork_graph on insert()

* fix partition tests
2023-10-23 09:32:41 -07:00
Ryo Onodera 5a963529a8
Add BankWithScheduler for upcoming scheduler code (#33704)
* Add BankWithScheduler for upcoming scheduler code

* Remove too confusing insert_without_scheduler()

* Add doc comment as a bonus

* Simplify BankForks::banks()

* Add derive(Debug) on BankWithScheduler
2023-10-21 15:56:43 +09:00
Brooks 452fd5d384
Adds `--no-skip-initial-accounts-db-clean` *hidden* CLI flag (#33664) 2023-10-12 13:32:40 -04:00
Ryo Onodera 1704789247
Define tick related helper test methods (#33537)
* Define tick related helper methods

* dcou VoteSimulator

* blacklist ledger-tool for dcou

* fix dcou ci...

* github
2023-10-10 09:23:18 +09:00
Brooks c8d545c501
Uses stable `u64::next_multiple_of()` (#33549) 2023-10-06 13:45:14 -04:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
Brooks a8c1dabf6a
Renames AccountsPackageType to AccountsPackageKind (#32908) 2023-08-21 13:00:00 -04:00
Brooks f9bc3cec24
Renames SnapshotType to SnapshotKind (#32906) 2023-08-19 19:55:29 -04:00
steviez a4c8cc3ce0
Remove improper uses of &Arc<Bank> (#32802)
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy

This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
2023-08-18 16:46:34 -05:00
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
Pankaj Garg ef8d3206d7
Move bank specific code out of epoch_accounts_hash/utils.rs (#32623)
* Move bank specific code out of epoch_accounts_hash/utils.rs

* cleanup
2023-07-25 14:12:20 -07:00
Pankaj Garg aba637d5d9
Split snapshot_utils.rs into snapshot_bank_utils.rs (#32612) 2023-07-24 16:31:03 -07:00
Brooks 979913e1d5
Returns Self instead of Result<Self> for AccountsPackage::new_for_snapshot() (#32545) 2023-07-20 12:52:31 -04:00
Brooks 1663a7091a
Uses checked math in EAH tests (#32483) 2023-07-13 18:06:11 -04:00
steviez 77b587aa4d
Add constant for disabled snapshot interval (#32236)
Slot::MAX was used to specify that a type of snapshots should not be
created; define a constant to be that value and reference the constant
to have a single point of edit.
2023-06-26 12:26:56 -05:00
Brooks bfaf073ecf
Tests get snapshot archive format from snapshot config (#32209) 2023-06-20 15:44:03 -04:00
Brooks 8596e00549
Revert "EpochAccountsHash tests no longer ignore shutdown errors (#31883)" (#32018)
This reverts commit 2fc1dc1bf6.
2023-06-07 16:44:46 -04:00
Illia Bobyr 4353ac6797
Pass Arc<AtomicBool> by value, not by reference. (#31916)
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it.  Just adds an extra layer of
indirection.

Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.

This change focuses specifically on `Arc<AtomicBool>`.  There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner.  But it would make the change even larger.
2023-06-01 17:25:48 -07:00
steviez debe794987
Replace improper &Arc<...> with Arc<...> in Bank and Accounts (#31892)
The callstack updated in this PR passed an &Arc<...> down only to have
the bottom level clone the reference. Thus, we are giving shared
ownership so the reference is a bit redundant and arguably obscures the
intention to clone further down the callstack.
2023-05-31 12:36:44 -05:00
Brooks 2fc1dc1bf6
EpochAccountsHash tests no longer ignore shutdown errors (#31883) 2023-05-31 09:11:06 -04:00
Brooks c569a37713
Handle errors when sending an accounts package during shutdown (#31874) 2023-05-30 17:42:33 -04:00
Brooks 8a3f446db0
Removes MAX_BANK_SNAPSHOTS_TO_RETAIN constant (#31827) 2023-05-25 17:13:28 -04:00
Brooks 8da91d8670
Test must flush accounts cache before calculating accounts hash from storages (#31752) 2023-05-23 09:39:20 -04:00
Trent Nelson ad67fd5be5
validator: remove optional remote accounts hash consistency check (#31279) 2023-05-16 14:23:13 -06:00
Jeff Washington (jwash) 122b05b9f5
pass include_slot_in_hash through hash calcs to allow rehashing if hash is not stored (#31579)
* pass include_slot_in_hash through hash calcs to allow rehashing

* tests use each include_slot_in_hash value

* move include_slot_in_hash

* typo

* reorder struct init

* spelling is hard
2023-05-11 13:23:29 -07:00
Brooks 93087324e3
Uses AccountsBackgroundService::setup_bank_drop_callback() in tests (#31598) 2023-05-11 12:40:03 -04:00
Brooks 90e1b00fb5
Uses get_bank_snapshot_dir() in `core/tests/snapshot.rs` (#31434) 2023-05-01 22:57:59 +00:00
Xiang Zhu 0a2e897f16
Clean up the outdated SnapshotPackage snapshot_links field (#31360)
* Remove snapshot_links

* Change the function name from snapshot_dir to bank_snapshot_dir

* Format fix

* Fix test_concurrent_snapshot_packaging

* Fix clippy error

* Fix nits

* Fix nits 2nd try

* Use get_bank_snapshots_dir

* Use slot_dir

* Revert "Use get_bank_snapshots_dir" because get_bank_snapshots_dir is private to crate

This reverts commit 1ed9b3b2c8e84689a918beee7159f63c56500a96.
2023-05-01 11:24:59 -07:00
Ryo Onodera a30830d7a9
ci: treewide: deny used_underscore_binding (#31319)
* Enforce used_underscore_binding

* Fix all

* Work around for cfg()-ed code...

* ci....

* Make clipply fixes more pleasant

* Clone exit signal while intentionally shadowing

* Use more verbose code to avoid any #[allow(...)]s
2023-04-27 10:10:16 +09:00
Xiang Zhu f3e94ca73c
AHV processes the snapshot dirs in place (#30978)
* AHV processes the snapshot dirs in place

Let account pacakge use the snapshot dir, so AHV computes the accounts hash and turns the pre snapshot dir into a post snapshot dir

* fix status cache path to maintain the archive layout for the in-place snapshot dir archiving

* fix test_package_snapshots

* Fix test_concurrent_snapshot_packaging

* Remove debug change.

* Fix snapshot_links path

* change to borrow for bank_snapshots_dir

* Reverted changes in create_and_verify_snapshot

* Fix param errors

* Fix rebase errors

* Remove NOTE 1

* Remove unwrap

* Remove the variables to make it apparent taht snapshot_links is the bank_snapshots_dir

* Use soft link instead of hard link for snapshot and status cache

* After switching to soft symlinking, the src path should be absolute
2023-04-26 11:48:48 -07:00
Xiang Zhu 16f3dcd5d2
Update fn create_and_verify_snapshot (#31245)
* only 1 snapshot per archive in create_and_verify_snapshot

* Update create_and_verify_snapshot with the newer funtion calls

* Fix test_package_snapshots

* Remove account path access change

* Rename slot to num_snapshots

* Remove unncessary purge_old_bank_snapshots in test

* Update non-deterministic format comment

* Cleanup unnecessary hash calls

* Use get_accounts_hash

* Remove extra declaration

* Remove rehash

* Remove clean_accounts

* Revert "Cleanup unnecessary hash calls"

This reverts commit 06b1457462cf6d7acf62e0e5531633caf5d9fc58.

Removing unncessary hash calls should be done for create_and_verify_snapshot,
not bank_to_full_snapshot_archive

* Fix typo appenedvecs

* Remove bank_snapshots_dir after rebasing
2023-04-24 18:52:50 -07:00
Xiang Zhu a5275f8839
Remove bank_snapshots_dir param (#31249)
* Remove bank_snapshots_dir param

* Remove outdated comment

* Revert "Remove outdated comment"

This reverts commit e4441432bec57edb0dc22c4bacf4d48ce26ed818.

* Handle parent() error

* Fix format error
2023-04-19 09:37:46 -07:00
Brooks ca1bde3591
Use Arc instead of &Arc in AccountsBackgroundService::new (#31268) 2023-04-19 11:10:41 -04:00
Brooks 80b27f3cd9
Use Arc instead of &Arc in AccountsHashVerifier::new (#31269) 2023-04-19 11:10:08 -04:00