Jeff Washington (jwash)
ce53b84cdc
refactor untar_snapshot_in to push parallelism deeper for further refactoring ( #18310 )
2021-06-29 18:26:15 -05:00
dependabot[bot]
52fd10ce03
chore: bump byteorder from 1.3.4 to 1.4.3 ( #18300 )
...
* chore: bump byteorder from 1.3.4 to 1.4.3
Bumps [byteorder](https://github.com/BurntSushi/byteorder ) from 1.3.4 to 1.4.3.
- [Release notes](https://github.com/BurntSushi/byteorder/releases )
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md )
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.4...1.4.3 )
---
updated-dependencies:
- dependency-name: byteorder
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Bump expected instruction count
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-29 15:08:04 -06:00
Brooks Prumo
c1b9d40a64
Add snapshot_runtime_info module ( #18199 )
...
The `snapshot_runtime_info` module adds the `SnapshotRuntimeInfo` and
`SyncSnapshotRuntimeInfo` types to handle sharing runtime snapshot information
between threads; in particular: AccountsDb, Bank, SnapshotPackagerService, and
snapshot_utils.
2021-06-29 15:13:34 -05:00
dependabot[bot]
9bd02c9570
chore: bump zstd from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0 ( #18299 )
...
* chore: bump zstd from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits/0.9.0 )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-29 11:12:57 -06:00
carllin
47cafb70da
test ( #18159 )
2021-06-28 18:27:26 -07:00
Jeff Washington (jwash)
afd64d27c9
bin accounts index ( #18111 )
2021-06-28 15:03:57 -05:00
Jeff Washington (jwash)
1f1e54b9d8
reduce pubkey copies on insert path ( #18240 )
2021-06-28 12:04:29 -05:00
Jack May
2fbedd834f
Don't update if already an executable
2021-06-26 19:19:26 -06:00
Jeff Washington (jwash)
e06376664b
replace account index bulk insert with iterator ( #18198 )
2021-06-25 14:31:55 -05:00
dependabot[bot]
91576fdd9b
chore: bump blake3 from 0.3.7 to 0.3.8 ( #18223 )
...
* chore: bump blake3 from 0.3.7 to 0.3.8
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.7...0.3.8 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-06-25 11:44:52 -06:00
Jeff Washington (jwash)
f2a2581259
metric for accounts index insertion time ( #18202 )
2021-06-25 08:36:55 -05:00
dependabot[bot]
9429d0463c
chore: bump ouroboros from 0.5.1 to 0.9.3 ( #18189 )
...
* chore: bump ouroboros from 0.5.1 to 0.9.3
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.5.1 to 0.9.3.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Api changes
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-25 00:53:54 -06:00
Jeff Washington (jwash)
f9fccdee85
remove unnecessary copies from accounts index code paths ( #18196 )
2021-06-24 17:56:25 -05:00
Jeff Washington (jwash)
31ec986ea1
untar in parallel ( #18184 )
...
* untar in parallel
* make enum for 'ignore' return value
2021-06-24 17:29:49 -05:00
Jeff Washington (jwash)
4b314be5bd
rework dirty_pubkeys from insert_new_if_missing_into_primary_index ( #18200 )
2021-06-24 14:52:11 -05:00
Jeff Washington (jwash)
77c3ffe4cf
add min/max bin size metrics to hash calculation ( #18155 )
2021-06-23 12:57:05 -05:00
sakridge
3b1738c000
Keep track of dirty stores on remove accounts to clean ( #17601 )
...
* Keep track of dirty stores on remove accounts to clean
and not zero_lamport key set
* Only dirty when count==0?
* Add another clean
2021-06-23 10:28:35 +02:00
Lijun Wang
ccf6b21bf8
Removed unused purged_slots in purge_older_root_entries ( #18102 )
...
Removed unused purged_slots in purge_older_root_entries
2021-06-22 15:27:37 -07:00
Jeff Washington (jwash)
d3ee73e151
capture sort time in hash calculation ( #18118 )
2021-06-22 16:02:21 -05:00
Jeff Washington (jwash)
fbc94d84c8
apply 'debug_do_not_add_builtins' to feature activations ( #18110 )
2021-06-22 14:21:49 -05:00
Jeff Washington (jwash)
d5f9f3b7ce
correctly calculate hash_total for hash/lamport calculation ( #18144 )
2021-06-22 11:36:43 -05:00
sakridge
e808f34b0b
Add batch stats ( #18096 )
2021-06-22 15:23:26 +02:00
Tyera Eulberg
0ab361b828
Use newly stabilized Duration methods/consts ( #18101 )
2021-06-22 00:11:16 +00:00
Tyera Eulberg
19fe1dd463
Move stake_weighted_timestamp module ( #18114 )
...
* Move timestamp module into runtime
* Less public
* Remove unused enum
2021-06-21 16:14:54 -06:00
Jeff Washington (jwash)
2087f5da94
refactor generate_index_for_slot ( #17984 )
2021-06-21 15:32:50 -05:00
Jeff Washington (jwash)
bf97627021
eliminate flatten and sort in hash calculation ( #17802 )
...
* eliminate flatten and sort in hash calculation
* reduce critical section time
* remove now no-longer necessary test code
* conflict with reset bins to 0 pr
2021-06-21 15:32:03 -05:00
Tyera Eulberg
4d46cb686a
Fix build: clippy lint
2021-06-21 10:18:21 -07:00
dependabot[bot]
d458fac2ff
chore: bump bincode from 1.3.1 to 1.3.3 ( #18087 )
...
* chore: bump bincode from 1.3.1 to 1.3.3
Bumps [bincode](https://github.com/servo/bincode ) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/servo/bincode/releases )
- [Commits](https://github.com/servo/bincode/compare/v1.3.1...v1.3.3 )
---
updated-dependencies:
- dependency-name: bincode
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-21 16:06:31 +00:00
Jeff Washington (jwash)
ae82e4e25b
refactor reconstruct_accountsdb_from_fields ( #17987 )
2021-06-21 10:47:58 -05:00
dependabot[bot]
491e4450cc
chore: bump libc from 0.2.96 to 0.2.97 ( #18060 )
...
* chore: bump libc from 0.2.96 to 0.2.97
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-18 11:51:06 -06:00
Alexander Meißner
789f33e8db
chore: cargo fmt
2021-06-18 10:42:46 -07:00
Alexander Meißner
6514096a67
chore: cargo +nightly clippy --fix -Z unstable-options
2021-06-18 10:42:46 -07:00
dependabot[bot]
abc9839d86
chore: bump flate2 from 1.0.14 to 1.0.20 ( #18062 )
...
* chore: bump flate2 from 1.0.14 to 1.0.20
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.14 to 1.0.20.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.14...1.0.20 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-18 15:55:48 +00:00
Jeff Washington (jwash)
0a81c37fce
introduce LockMapType for accounts_index ( #18021 )
2021-06-17 14:41:52 -05:00
Jeff Washington (jwash)
1feaaf009d
generate_index timings ( #17983 )
2021-06-17 13:35:25 -05:00
Jeff Washington (jwash)
20e714b3d0
remove unused code ( #17981 )
2021-06-17 13:30:46 -05:00
Jeff Washington (jwash)
71796f4951
break out generate index reporting ( #17980 )
2021-06-16 12:09:17 -05:00
Jeff Washington (jwash)
7ca04d6a86
refactor generate_index process_storage_slot ( #17982 )
2021-06-16 11:02:34 -05:00
Jeff Washington (jwash)
bb7413c9f9
refactor rebuild_bank_from_snapshots ( #17988 )
2021-06-16 11:01:52 -05:00
Jeff Washington (jwash)
eee5414c64
avoid unnecessary empty arrays when binning ( #17944 )
2021-06-16 09:24:59 -05:00
Jeff Washington (jwash)
adc683956f
do not use index for verify_bank_hash_and_lamports ( #17812 )
2021-06-16 09:10:38 -05:00
Jeff Washington (jwash)
55ee3b5f2f
set hash bins to 65k ( #17912 )
2021-06-16 09:09:55 -05:00
Jeff Washington (jwash)
7de79425ce
parallel storage -> accounts folder ( #17955 )
2021-06-16 08:48:24 -05:00
Jeff Washington (jwash)
dbd4dc04b0
ledger tool limit_load_slot_count_from_snapshot avoids assert failures ( #17974 )
2021-06-15 15:39:22 -05:00
Jeff Washington (jwash)
f558b9b6bf
verify bank hash on startup with ledger tool option ( #17939 )
2021-06-15 11:52:12 -05:00
Jon Cinque
1b1d34da59
Refactor stake program into solana_program ( #17906 )
...
* Move stake state / instructions into solana_program
* Update account-decoder
* Update cli and runtime
* Update all other parts
* Commit Cargo.lock changes in programs/bpf
* Update cli stake instruction import
* Allow integer arithmetic
* Update ABI digest
* Bump rust mem instruction count
* Remove useless structs
* Move stake::id() -> stake::program::id()
* Re-export from solana_sdk and mark deprecated
* Address feedback
* Run cargo fmt
2021-06-15 18:04:00 +02:00
Jeff Washington (jwash)
5ecb30ff58
nit: rename measure ( #17946 )
2021-06-15 09:01:11 -05:00
Jeff Washington (jwash)
4d8ffdcc11
remove unused parameters from CalculateHashIntermediate ( #17949 )
2021-06-15 09:00:43 -05:00
carllin
ccc013e134
Handle removing slots during account scans ( #17471 )
2021-06-14 21:04:01 -07:00
Jeff Washington (jwash)
471b34132e
add metrics for startup ( #17913 )
...
* add metrics for startup
* roll timings up higher
* fix test
* fix duplicate
2021-06-14 17:46:49 -05:00
Jeff Washington (jwash)
7fde9b6ff0
name arguments to help with confusion ( #17942 )
2021-06-14 16:21:48 -05:00
Jeff Washington (jwash)
62e1b20e56
refactor so hash verify can be done by more callers ( #17941 )
2021-06-14 16:20:03 -05:00
Jeff Washington (jwash)
e6bbd4b3f0
add metrics to handle_snapshot_requests ( #17937 )
2021-06-14 15:46:19 -05:00
dependabot[bot]
a0872232d3
chore: bump itertools from 0.9.0 to 0.10.1 ( #17929 )
...
* chore: bump itertools from 0.9.0 to 0.10.1
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.9.0...v0.10.1 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Fix versions
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-14 18:32:20 +00:00
sakridge
c2191d885d
Fix accounts index panic in purge_exact ( #17757 )
2021-06-14 16:10:26 +02:00
Jeff Washington (jwash)
d4cc975fe9
calculate_capitalization uses hash calculation ( #17443 )
...
* calculate_capitalization uses hash calculation
* feedback
* remove debugging code, clean up slot math
2021-06-14 08:53:07 -05:00
Jeff Washington (jwash)
2dc6969858
use trait to simplify and consolidate cumulative code ( #17852 )
2021-06-14 08:43:59 -05:00
dependabot[bot]
2e75023840
chore: bump regex from 1.4.5 to 1.5.4 ( #17885 )
...
* chore: bump regex from 1.4.5 to 1.5.4
Bumps [regex](https://github.com/rust-lang/regex ) from 1.4.5 to 1.5.4.
- [Release notes](https://github.com/rust-lang/regex/releases )
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/regex/compare/1.4.5...1.5.4 )
---
updated-dependencies:
- dependency-name: regex
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-06-11 09:50:54 -06:00
dependabot[bot]
f08ed1eb2d
chore: bump rayon from 1.5.0 to 1.5.1 ( #17869 )
...
* chore: bump rayon from 1.5.0 to 1.5.1
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases )
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.5.0...v1.5.1 )
---
updated-dependencies:
- dependency-name: rayon
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-06-10 13:18:47 -06:00
Lijun Wang
269d995832
Make account shrink configurable #17544 ( #17778 )
...
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true
Fixes #17544
2021-06-09 21:21:32 -07:00
Jack May
a1fab0c5ca
Log more info on runtime account verification errors ( #17861 )
2021-06-09 16:27:00 -07:00
Justin Starry
050bb5446d
Add local cluster tests that broadcast duplicate slots ( #13995 )
...
* Add duplicate node local cluster test
* fix clippy
* remove dupe test
2021-06-09 15:01:48 -07:00
dependabot[bot]
9a2ca8dd2f
chore: bump rustc_version from 0.2.3 to 0.4.0 ( #17854 )
...
* chore: bump rustc_version from 0.2.3 to 0.4.0
Bumps [rustc_version](https://github.com/Kimundi/rustc-version-rs ) from 0.2.3 to 0.4.0.
- [Release notes](https://github.com/Kimundi/rustc-version-rs/releases )
- [Commits](https://github.com/Kimundi/rustc-version-rs/compare/v0.2.3...v0.4.0 )
---
updated-dependencies:
- dependency-name: rustc_version
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Make versions consistent
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-09 16:57:39 +00:00
Jeff Washington (jwash)
576e3d95f7
refactor: create type ( #17818 )
2021-06-08 14:49:08 -06:00
Jeff Washington (jwash)
2a9b127029
Pass iterator for SortedStorages::new_with_slots ( #17811 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-06-08 13:59:50 -06:00
Jeff Washington (jwash)
01ef2a5c4a
test: simple CalculateHashIntermediate refactoring ( #17813 )
2021-06-08 13:54:10 -06:00
Jeff Washington (jwash)
58fe1d0764
add min to roots_tracker ( #17806 )
2021-06-08 08:32:16 -06:00
Jeff Washington (jwash)
817d48be21
binary search in slices for hashing ( #17755 )
2021-06-07 12:09:17 -05:00
Jeff Washington (jwash)
b5bb91b50f
rework hash calculation to not keep slot and write version ( #17685 )
...
* rework hash calculation to not keep slot and write version
* refactor functions and add tests
* always use multiple slot code path
2021-06-07 09:01:16 -05:00
Jon Cinque
8f5e773caf
system-program: Remove zero lamport check on transfers ( #17726 )
...
* system-program: Move lamports == 0 check on transfers
* Address feedback
* Update stake split to explicitly allocate + assign
* Update stake tests referring to split instruction
* Revert whitespace
* Update split instruction index in test
* Remove unnecessary `assign_with_seed` from `split_with_seed`
* Fix stake instruction parser
* Update test to allow splitting into account with lamports
2021-06-06 01:45:45 +02:00
Jeff Washington (jwash)
00ee84af37
break out hash time in metric ( #17721 )
2021-06-04 14:54:06 -05:00
Jeff Washington (jwash)
f6fb8906c7
add data point for cap mismatch ( #17746 )
2021-06-04 13:54:39 -05:00
Jeff Washington (jwash)
5197454fea
increase bin count for hash scan ( #17562 )
2021-06-04 10:53:13 -05:00
sakridge
f97ce2cd7e
Per-program id timings ( #17554 )
2021-06-04 16:04:31 +02:00
Jeff Washington (jwash)
81bafd9daf
add info for vm.max_map_count incorrectly set. ( #17727 )
2021-06-04 03:22:12 +00:00
dependabot[bot]
3dcc8e0046
chore: bump libloading from 0.6.2 to 0.7.0 ( #17704 )
...
* chore: bump libloading from 0.6.2 to 0.7.0
Bumps [libloading](https://github.com/nagisa/rust_libloading ) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/nagisa/rust_libloading/releases )
- [Commits](https://github.com/nagisa/rust_libloading/compare/0.6.2...0.7.0 )
---
updated-dependencies:
- dependency-name: libloading
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* New libloading methods are marked unsafe
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-06-03 18:08:31 -06:00
Jeff Washington (jwash)
9388aaca15
use slots returned from get_snapshot_storages to sort ( #17638 )
...
* use slots returned from get_snapshot_storages to sort
* add tests
2021-06-02 18:24:55 -05:00
Jeff Washington (jwash)
4bd32d891f
remove slot lookup ( #17691 )
2021-06-02 17:56:19 -05:00
Jeff Washington (jwash)
83ceedc091
simplify test construction ( #17686 )
2021-06-02 17:32:31 -05:00
Jeff Washington (jwash)
db8811eacd
support bin divisions up to 65536 ( #17563 )
...
* support bin divisions up to 65536
* add tests
2021-06-02 17:31:15 -05:00
dependabot[bot]
8e1e57a181
chore: bump tar from 0.4.28 to 0.4.35 ( #17666 )
...
* chore: bump tar from 0.4.28 to 0.4.35
Bumps [tar](https://github.com/alexcrichton/tar-rs ) from 0.4.28 to 0.4.35.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases )
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.28...0.4.35 )
---
updated-dependencies:
- dependency-name: tar
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-06-02 14:17:58 -06:00
dependabot[bot]
19c49e2e25
chore: bump crossbeam-channel from 0.4.4 to 0.5.0 ( #17665 )
...
* chore: bump crossbeam-channel from 0.4.4 to 0.5.0
Bumps [crossbeam-channel](https://github.com/crossbeam-rs/crossbeam ) from 0.4.4 to 0.5.0.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases )
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md )
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-channel-0.4.4...crossbeam-channel-0.5.0 )
---
updated-dependencies:
- dependency-name: crossbeam-channel
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-06-02 19:36:28 +00:00
Jeff Washington (jwash)
d802eb303c
reduce copy ( #17672 )
2021-06-02 14:03:02 -05:00
Jeff Washington (jwash)
07bac27ac7
remove antiquated assert ( #17643 )
2021-06-02 09:47:26 -05:00
Jeff Washington (jwash)
eec996ba41
implement ancestors as rolling bit field ( #17482 )
2021-06-02 09:32:32 -05:00
Jeff Washington (jwash)
107af52deb
minor test code cleanup ( #17645 )
2021-06-02 08:35:10 -05:00
carllin
bbcdf073ba
Support out of band dumping of unrooted slots in AccountsDb ( #17269 )
...
* Accounts dumping logic
* Add test for interaction between cache flush and remove_unrooted_slot()
* Update comments
* Rename
* renaming
* Add more comments
* Renaming
* Fixup test and bad check
2021-06-02 09:51:10 +00:00
Jack May
80e5b24b38
Fix CPI recursion depth ( #17659 )
...
* Fix CPI recursion depth
2021-06-02 11:15:19 +02:00
sakridge
14c52ab018
Skip shrink when it doesn't save anything ( #17405 )
2021-06-02 09:51:46 +02:00
Jeff Washington (jwash)
738cc9549f
parallel get_snapshot_storages ( #17589 )
2021-06-01 19:33:12 -05:00
Jeff Washington (jwash)
c8fdf0762c
add Ancestors to get_snapshot_storages to prepare for the future ( #17591 )
2021-06-01 15:14:35 -05:00
Jeff Washington (jwash)
ef5169ff24
sort storages by slot before scan ( #17411 )
...
* sort storages by slot before scan
* fix return value
2021-06-01 15:07:46 -05:00
Jeff Washington (jwash)
72bb271a94
add metric for collecting storages ( #17527 )
2021-06-01 13:17:49 -05:00
Jeff Washington (jwash)
886898eabf
rework test for flexibility ( #17592 )
...
* rework test for flexibility
* respond to pr feedback
2021-05-31 22:49:17 -05:00
dependabot[bot]
ab0f4ff835
Bump serde from 1.0.122 to 1.0.126 ( #17618 )
...
* Bump serde from 1.0.122 to 1.0.126
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.122 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.122...v1.0.126 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-05-31 22:41:25 +00:00
dependabot[bot]
ac452f9bb8
Bump tempfile from 3.1.0 to 3.2.0 ( #17587 )
...
* Bump tempfile from 3.1.0 to 3.2.0
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases )
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.1.0...v3.2.0 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-05-31 18:05:07 +00:00
dependabot[bot]
a35024123c
chore: bump memmap2 from 0.1.0 to 0.2.3 ( #17548 )
...
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.1.0 to 0.2.3.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.1.0...v0.2.3 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-05-28 13:11:58 -06:00
Jeff Washington (jwash)
654918ab27
refactor SnapshotStorage helpers to prepare for later changes ( #17560 )
2021-05-28 13:54:53 -05:00
Jack May
a3240aebde
Always bail if program modifies a ro account ( #17569 )
2021-05-28 09:50:25 -07:00
Jeff Washington (jwash)
8924fbf6a0
RollingBitField: bug fixes and add tests ( #17525 )
...
* RollingBitField: bug fixes and add tests
* respond to feedback
2021-05-28 16:33:07 +00:00
Jeff Washington (jwash)
1d02dba06f
remove unnecessary clone ( #17559 )
2021-05-28 10:30:01 -05:00
Jeff Washington (jwash)
55c22d3b76
add check_hash to non-index hash calculation ( #17558 )
2021-05-28 10:24:40 -05:00
Brooks Prumo
1953543274
Refactor bank_forks_utils::load() and some snapshot logic ( #17492 )
...
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.
Additionally, add some tests and doc comments.
2021-05-27 12:00:27 -05:00
Michael Vines
8eab0e8602
Bump version to v1.8.0 ( #17541 )
2021-05-27 08:51:53 -07:00
sakridge
49402b7d82
Only add keys with multiple index entries to the clean set ( #17473 )
2021-05-27 16:31:24 +02:00
Jeff Washington (jwash)
3973bf08f1
move local out of call ( #17519 )
2021-05-27 08:53:11 -05:00
Michael Vines
9541411c15
Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method
2021-05-27 03:05:05 +00:00
Michael Vines
cbce440af4
simulateTransaction can now return accounts modified by the simulation
2021-05-26 14:20:23 -07:00
sakridge
3f9e3c7375
Remove copied shrink code ( #17385 )
2021-05-26 19:27:18 +02:00
Jeff Washington (jwash)
6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool ( #17417 )
2021-05-26 10:36:12 -05:00
Trent Nelson
2a6c5ed0ac
add tests from `RentDebits`
2021-05-26 08:19:10 +00:00
Trent Nelson
97eab7edf9
runtime: add rent debit charges to block metadata
2021-05-26 08:19:10 +00:00
Tyera Eulberg
e9bc1c6b07
Add last valid block height to rpc Fees ( #17506 )
...
* Add last_valid_block_height to fees rpc
* Add getBlockHeight rpc
* Update docs
2021-05-26 07:26:19 +00:00
steviez
a5c2067273
fixup! Ancestors to HashSet: From(Vec) ( #17447 ) ( #17481 )
2021-05-25 12:24:34 -05:00
Jeff Washington (jwash)
98f1b11edd
Ancestors to HashSet: From(Vec) ( #17447 )
...
* ancestors.large_range_slots becomes HashSet
* Ancestors to HashSet: From(Vec)
2021-05-25 11:01:11 -05:00
Jeff Washington (jwash)
d39a327138
ancestors.large_range_slots becomes HashSet ( #17446 )
2021-05-25 11:00:50 -05:00
Jeff Washington (jwash)
40bb6577fc
initialize purge stats ( #17345 )
2021-05-25 09:24:38 -05:00
Jeff Washington (jwash)
d4fc9e7ed4
ad metric for collect ( #17418 )
2021-05-24 18:14:52 -05:00
Jeff Washington (jwash)
dd22ae047b
ancestors is set instead of map ( #17363 )
2021-05-24 17:01:02 -05:00
carllin
d8bc56fa51
Refactor purge_slots_from_cache_and_store() and handle_reclaims() ( #17319 )
2021-05-24 13:51:17 -07:00
jon-chuang
ad1f24d487
runtime: Executor usage counts retain only single-epoch memory ( #17162 )
2021-05-24 12:01:56 -07:00
sakridge
a8dca3976b
Refactor genesis download/load/check functions ( #17276 )
...
* Refactor genesis ingest functions
* Consolidate genesis.bin/genesis.tar.bz2 references
2021-05-24 16:45:36 +02:00
Alexander Meißner
855ae79598
Fix InvokeContext::push() account_deps ( #17350 )
...
* Reverts aliasing of account_deps with the previous invocation stack frame in InvokeContext::push().
* Adds explicit assert of programdata account content in test_program_bpf_upgrade_via_cpi.
2021-05-21 22:34:07 +02:00
Jeff Washington (jwash)
3f3324231d
Hold read lock during startup shrink ( #17309 )
...
* hold read lock during account scan of shrink
* rename and improve rusty
2021-05-21 13:59:32 -05:00
Jeff Washington (jwash)
44831c18d2
reuse work in is_non_loader_key ( #16521 )
2021-05-21 10:07:04 -05:00
Jeff Washington (jwash)
8143ee5502
add AccountsIndex AccountMapsReadLock type ( #17365 )
2021-05-21 10:02:14 -05:00
Jeff Washington (jwash)
2f55547d37
remove flatten in storage scan ( #17354 )
...
* remove flatten in storage scan
* fix tests
2021-05-21 09:26:46 -05:00
Michael Vines
2c99b23ad7
Add get_sysvar() helper to sdk
2021-05-21 01:30:12 +00:00
Jeff Washington (jwash)
45552d271a
take mutex once in process_dead_slots ( #17357 )
2021-05-20 19:50:58 -05:00
Jeff Washington (jwash)
0486df02ba
don't limit to thread pool when cleaning on startup ( #17317 )
2021-05-20 14:36:35 -05:00
Jeff Washington (jwash)
33ab9c4e8d
batch insert account_index items in generate_index ( #17290 )
2021-05-20 10:29:13 -05:00
Jeff Washington (jwash)
9b74988fc6
calculate_accounts_hash does not throttle threads on startup ( #17338 )
2021-05-20 10:25:54 -05:00
Jeff Washington (jwash)
a9aa533684
replace Ancestors HashMap for performance ( #16477 )
...
* replace Ancestors HashMap for performance
* add ancestors module
2021-05-20 10:11:56 -05:00
Jeff Washington (jwash)
a544010b03
don't log shrink metrics on first call ( #17328 )
...
* don't log shrink metrics on first call
* simplify logic
2021-05-20 00:15:35 +00:00
Jeff Washington (jwash)
32ec8341f9
generate_index inserts ideal initial data ( #17247 )
...
* improve insert into map initially
* rework towards single code path
* rename
* update test
2021-05-19 16:21:24 -05:00
Jeff Washington (jwash)
b5302e7619
add calc_stored_meta_us metric ( #17318 )
2021-05-19 12:29:41 -05:00
Jeff Washington (jwash)
c20b27bc8f
shink all in parallel on startup ( #17308 )
2021-05-19 12:15:24 -05:00
Jeff Washington (jwash)
ed9cbd50f0
move Ancestors to its own module ( #17316 )
2021-05-19 11:50:34 -05:00
Tao Zhu
0781fe1b4f
Upgrade Rust to 1.52.0 ( #17096 )
...
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script
* upgrade to Rust 1.52.1
* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
Jeff Washington (jwash)
4788976517
rework slot list update ( #17232 )
2021-05-19 08:48:46 -05:00
Jeff Washington (jwash)
7fe24c455c
simplify generate_index ( #17292 )
2021-05-19 08:11:41 -05:00
Tyera Eulberg
827355a6b1
Create solana-rpc crate and move subscriptions ( #17320 )
...
* Move non_circulating_supply to runtime
* Add solana-rpc crate and move max_slots
* Move subscriptions to solana-rpc
* Single use statements
2021-05-19 00:54:28 -06:00
Jeff Washington (jwash)
a3c0833a1c
tests: cleanup test ( #17305 )
...
* tests: cleanup test
* 2 more replacements
2021-05-18 18:08:37 -05:00
Jeff Washington (jwash)
75335b4f58
datapoint for verify_snapshot_bank ( #17306 )
2021-05-18 12:14:43 -05:00
Jeff Washington (jwash)
2175fc0625
calculate account infos before lock ( #17245 )
2021-05-18 08:06:19 -05:00
Jeff Washington (jwash)
3a56a56d69
StoredMetaWriteVersion ( #17293 )
2021-05-17 23:58:36 +00:00
Ryo Onodera
bcbe155575
Minor test cleanup and comments ( #17283 )
2021-05-18 07:22:43 +09:00
Jeff Washington (jwash)
d461aba6a4
don't try to build secondary indexes if we don't need them ( #17246 )
2021-05-17 17:21:15 -05:00
Jeff Washington (jwash)
9d6837c904
fix comment ( #17286 )
2021-05-17 14:06:25 -05:00
Jeff Washington (jwash)
d1db5448b9
hold lock to speed up insert ( #17194 )
...
* hold lock to speed up insert
* add tests
2021-05-17 11:58:33 -05:00
Jeff Washington (jwash)
3745e0babc
minor refactoring of AccountsIndex in preparation of bigger things ( #17237 )
2021-05-14 15:27:10 -05:00
Jeff Washington (jwash)
fd88db7339
ancestors::new -> default ( #17195 )
2021-05-14 15:20:59 -05:00
Jeff Washington (jwash)
c6f7867aa1
log mismatch ( #17209 )
2021-05-14 09:50:11 -05:00
Brooks Prumo
76dfacca61
Add documentation for BankForks::prune_non_rooted() ( #17213 )
2021-05-14 07:47:48 -05:00
Tyera Eulberg
27004f1b76
Return error for excluded secondary-index keys ( #17193 )
...
* Add runtime helpers to check secondary indexes for key
* Add custom rpc error
* Check secondary-index key inclusion in rpc
* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
2021-05-13 21:04:21 +00:00
Jeff Washington (jwash)
3dbc7744ab
metrics for generating index time ( #17192 )
...
* metrics for generating index time
* update metrics to include scan time
2021-05-13 14:32:19 -05:00
carllin
239ab8799c
Remove bloat from secondary indexes ( #17048 )
2021-05-12 15:29:30 -07:00
Lijun Wang
9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. ( #17158 )
...
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
Jeff Washington (jwash)
7d96f78821
include/exclude keys on account secondary index ( #17110 )
...
* AccountSecondaryIndexes.include/exclude
* use normal scan if key is not indexed
* add a test to ask for a scan for an excluded secondary index
* fix up cli args
2021-05-11 22:06:22 +00:00
Jeff Washington (jwash)
f39dda00e0
type AccountSecondaryIndexes = HashSet ( #17108 )
2021-05-10 14:22:48 +00:00
Jeff Washington (jwash)
a6a1355b80
fix secondary index test ( #17111 )
2021-05-10 08:54:30 -05:00
Ryo Onodera
c616c34825
Add comment for another unspoken eager collection subtlety ( #17137 )
2021-05-10 22:44:46 +09:00
Jeff Washington (jwash)
225ec00ec4
add comment ( #17059 )
2021-05-06 15:17:41 -05:00
Jeff Washington (jwash)
dc0429f5e6
add metric for assumption ( #17061 )
2021-05-06 15:04:13 -05:00
steviez
edc2ab3e48
Test account index and store alignment ( #17038 )
...
* Use ReclaimResult::Default() instead of building subtypes
* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
Ryo Onodera
d19526e6c2
Dump rent_collector/inflation with ledger-tool cap ( #17069 )
2021-05-06 19:29:46 +09:00
Jeff Washington (jwash)
ffbe8906ed
get root lock once ( #16829 )
2021-05-05 15:17:45 -05:00
Brooks Prumo
e6f49a3e79
Clean unrooted unfrozen banks ( #16580 ) ( #17003 )
...
Cleanup pubkeys when an unrooted, unfrozen bank is dropped. This is a
continuation of PR #16911 .
2021-05-05 15:02:02 -05:00
Jeff Washington (jwash)
ab7c96aa81
insert accounts in parallel when building initial index ( #17040 )
...
* insert accounts in parallel when building initial index
* rename nits from pr review
* rename nits from pr review
* rename nits from pr review
* rename nits from pr review
2021-05-05 17:08:45 +00:00
Stephen Akridge
3e0fed48e7
Don't recognize temp snapshots as possible snapshots to open
2021-05-05 08:45:03 -07:00
Jeff Washington (jwash)
5786be13a4
flatten_hash_intermediate sets capacity first ( #17013 )
...
* flatten_hash_intermediate sets capacity first
* use iterator instead of for
2021-05-05 09:07:05 -05:00
sakridge
d94e4ef7c9
More docs for bank.rs ( #17006 )
2021-05-03 17:37:18 +00:00
Jeff Washington (jwash)
541aa5ad85
tests: lamports -> lamports() ( #16982 )
2021-05-03 10:45:54 -05:00
Jeff Washington (jwash)
744ac1535f
tests: lamports -> ReadableAccount ( #16983 )
2021-05-03 10:45:15 -05:00
Jeff Washington (jwash)
1fe131a5bb
Readable account trait for stored account meta ( #16690 )
...
* ReadableAccount for StoredAccountMeta
* add test
2021-04-30 16:25:02 -05:00
Jeff Washington (jwash)
1a9954f85b
bank deposit checked_add error ( #16917 )
...
* bank deposit checked_add error
* add id
* rename variables
* rename error and metric
2021-04-30 16:22:17 -05:00
Jeff Washington (jwash)
01308cd890
distribute_rent_to_validators checked_add_lamports unwrap ( #16847 )
...
* distribute_rent_to_validators checked_add_lamports unwrap
* make rent disappear on add failure
* add pubkey to message
* update message text
* don't store account that we failed to transfer to
* format
2021-04-30 16:19:20 -05:00
Jeff Washington (jwash)
763c04adf3
lamports = -> .set_lamports() ( #16980 )
2021-04-30 21:17:05 +00:00
Brooks Prumo
17e6bd579f
Clean unrooted dropped banks ( #16580 ) ( #16911 )
...
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.
This work was originally based on PR #15106 .
2021-04-30 15:34:38 -05:00
Jeff Washington (jwash)
6f88aeac56
tests: .lamports= -> .set_lamports() ( #16977 )
...
* .lamports= -> .set_lamports()
* another
* fix
* another
2021-04-30 18:44:15 +00:00
Jeff Washington (jwash)
0af84bb91e
lamports= -> .set_lamports() ( #16978 )
2021-04-30 18:20:54 +00:00
Jeff Washington (jwash)
c4943f3cb0
checked_sub_lamports ( #16932 )
2021-04-30 09:56:34 -05:00
Jeff Washington (jwash)
63c56b57a9
checked_sub_lamports with unwrap ( #16922 )
...
* checked_sub_lamports with unwrap
* avoid unwrap
2021-04-29 15:04:28 -05:00
Jeff Washington (jwash)
c9c94a7ef2
checked_sub_lamports ( #16923 )
2021-04-29 12:38:21 -05:00
Jeff Washington (jwash)
ca7b36ad8f
lamports -> lamports() ( #16920 )
2021-04-29 10:44:46 -05:00
Jeff Washington (jwash)
23d67e4ac7
stretchy roots tracker ( #16830 )
...
* stretchy roots tracker
* rename hash to hash_set in tests
* update comment
* try 2 widths in test
* bool iter
* add assert
* helper function for bitfield insert/remove
* introduce RollingBitFieldTester
* another bool iter replacement
* map cleanup
* map to cloned
2021-04-29 09:11:28 -05:00
Jeff Washington (jwash)
a7070a5ca7
set_lamports() ( #16921 )
2021-04-28 21:30:06 +00:00
Jeff Washington (jwash)
21c75d9d29
lamports = -> set_lamports ( #16919 )
2021-04-28 15:39:54 -05:00
Jeff Washington (jwash)
f533d3be77
Write account path impl ReadableAccount ( #16779 )
2021-04-28 15:29:22 -05:00
Jeff Washington (jwash)
9070191b8b
stats on clean for removing roots ( #16849 )
...
* stats on clean for removing roots
* rename
* accumulate and swap metrics
2021-04-28 13:24:01 -05:00
Brooks Prumo
77f1ffd84b
Collect uncleaned pubkeys from all slots ( #16786 )
...
While working on another issue (#16580 ), the list of uncleaned pubkeys
returned from `remove_uncleaned_slots_and_collect_pubkeys_up_to_slot()`
did not include unrooted slots. This meant that during cleaning,
unrooted slots would not have their pubkeys cleaned up properly.
Now, return all uncleaned pubkeys, regardless if the slot is rooted or
not. Additionally, update the tests to have unrooted slots to ensure
this behavior.
This is part two of PR #16879 , and originally based on PR #15106 .
2021-04-28 10:57:42 -05:00
Jeff Washington (jwash)
a7a671b3aa
AccountSharedData.set_executable() ( #16881 )
2021-04-28 14:07:43 +00:00
Jeff Washington (jwash)
2021255f91
executable() ( #16880 )
2021-04-28 08:53:54 -05:00
Jeff Washington (jwash)
da3342759b
private AccountSharedData.rent_epoch ( #16877 )
2021-04-28 08:52:20 -05:00
Jeff Washington (jwash)
6381ee38eb
reclaims unref accounts from index ( #16838 )
2021-04-28 08:50:38 -05:00
Jeff Washington (jwash)
1864bc2080
write Option<AccountSharedData> ( #16874 )
...
* write Option<&AccountSharedData>
* add comment
2021-04-28 08:47:26 -05:00
Brooks Prumo
1eaff394da
Refactor `collect_uncleaned_pubkeys_to_slot()` ( #16879 )
...
* Refactor `collect_uncleaned_pubkeys_to_slot()`
While working on another issue (#16580 ), I came across
`collect_unclean_pubkeys_to_slot()` and had difficulty understanding it.
Since the function does a few logically separate things, I split the
function up. I also added documentation, removed an unused return value,
and renamed the functions to be more specific.
This commit is to split up an existing PR (#16786 ), where I had both this
aesthetic change _and_ a behavioral change.
2021-04-28 08:16:12 -05:00
Jeff Washington (jwash)
1bd623cd15
private AccountSharedData.rent_epoch ( #16844 )
2021-04-27 13:51:13 -05:00
Jeff Washington (jwash)
8f56c116d7
tests: lamports += to checked_add ( #16842 )
2021-04-27 10:23:07 -05:00
Jeff Washington (jwash)
d533f77301
bank.withdraw uses checked_sub_lamports ( #16848 )
...
* bank.withdraw uses checked_sub_lamports
* retain previous codepath for error
* map_err from clippy
2021-04-27 09:58:41 -05:00
Jeff Washington (jwash)
3887169db0
lamports += to checked_add ( #16841 )
2021-04-27 09:56:18 -05:00
Jeff Washington (jwash)
47ca7063f2
rootstracker.remove returns previous state ( #16831 )
2021-04-27 09:15:44 -05:00
Jeff Washington (jwash)
3fdbaefaa6
tests: lamports -= to checked_sub ( #16843 )
2021-04-27 09:12:48 -05:00
Jeff Washington (jwash)
998cba74b5
AccountSharedData.executable() ( #16835 )
2021-04-27 09:12:17 -05:00
Jeff Washington (jwash)
81402ee7f5
pass &Pubkey through account storage, slot clean code to reduce copies ( #16778 )
...
* &Pubkey
* use trait to pass &Hash or Hash
* use impl Borrow<Hash> instead of trait
* remove old code line commented out
2021-04-27 09:10:06 -05:00
Jeff Washington (jwash)
603872685d
private AccountSharedData.owner ( #16760 )
...
* private AccountSharedData.owner
* fix perf test
2021-04-26 14:59:17 -05:00
Jeff Washington (jwash)
f2ab0384e4
owner -> owner() ( #16783 )
2021-04-26 17:06:40 +00:00
Jeff Washington (jwash)
aeff911c93
owner -> owner() ( #16784 )
2021-04-26 10:40:11 -05:00
Michael Vines
d9dcd28d82
Ignore racy test_load_account_and_purge_race_without_retry
2021-04-23 23:17:56 +00:00
Jeff Washington (jwash)
ca14c18998
owner -> owner() ( #16782 )
2021-04-23 22:49:47 +00:00
Jeff Washington (jwash)
1a4a7059af
owner -> owner() ( #16785 )
2021-04-23 15:59:13 -05:00
Jeff Washington (jwash)
48c07d32f0
WritableAccount.add/subtract_lamports ( #16750 )
...
* add/sub lamports
* make add/sub return Result
* sample replacements
* cleanup
* fix up a few tests as examples
* move enum, cleanup, impl from
* fmt
* cleanup
* add lamports.rs
2021-04-23 20:20:48 +00:00
Trent Nelson
be29568318
runtime: checked math for Bank::withdraw
2021-04-23 20:13:35 +00:00
Jeff Washington (jwash)
dcf2d84b24
refactor complicated 'len' ( #16777 )
2021-04-23 18:36:49 +00:00
Jeff Washington (jwash)
04e6aebf35
.owner = X -> .set_owner(X) ( #16759 )
2021-04-23 18:26:33 +00:00
Jeff Washington (jwash)
fb0b76c1f3
account.owner = X -> account.set_owner(X) ( #16754 )
2021-04-23 12:34:22 -05:00
Jeff Washington (jwash)
7602911f9a
read_only_cache remove updates cache size ( #16742 )
2021-04-23 12:00:17 -05:00
Jeff Washington (jwash)
1cc9a1c0eb
log roots range metric ( #16636 )
...
* log roots range metric
* rename
2021-04-23 16:09:39 +00:00
Jeff Washington (jwash)
96d21335e1
.owner -> .owner() ( #16758 )
2021-04-23 09:35:55 -05:00
Jeff Washington (jwash)
5591cd6ef3
.owner -> .owner() ( #16757 )
...
* .owner -> .get_owner()
* use
2021-04-23 09:35:09 -05:00
Jeff Washington (jwash)
6c5b7ecf90
account.owner = X -> account.set_owner(X) ( #16756 )
2021-04-23 09:34:28 -05:00
Jeff Washington (jwash)
91be2903da
Reduce account index lookups during clean ( #16689 )
...
* reduce account index lookups in clean
* rename
* rename enum
* hold locks during removal from zero pubkey list
* merge with zero lamport fix
* tests
2021-04-23 09:33:14 -05:00
Jeff Washington (jwash)
fc12841d95
Readonlyaccounts ( #16743 )
...
* lamports -> lamports()
* format
2021-04-22 20:04:55 +00:00
Jeff Washington (jwash)
739f0698ab
clean read_only_cache in the bg ( #16722 )
...
* clean read_only_cache in the bg
* reset lru_index
* cleanup
* cleanup
* drop for read only cache
* join bg thread
* reuse existing lru vec
* simplify reserve logic
2021-04-22 13:59:58 -05:00
Jeff Washington (jwash)
333998d008
.lamports = <number> -> .set_lamports(<number>) ( #16746 )
2021-04-22 18:56:47 +00:00
Jeff Washington (jwash)
8a6b80095e
Set lamports ( #16747 )
...
* lamports = -> set_lamports()
* .lamports = X -> .set_lamports(X)
2021-04-22 13:53:06 -05:00
Jeff Washington (jwash)
8d9d6b62d9
use cheaper account get ( #16682 )
2021-04-22 12:54:08 -05:00
Jeff Washington (jwash)
123e0bdba7
don't iterate slot_list if zero-lamport ( #16678 )
2021-04-21 20:38:48 -05:00
Jeff Washington (jwash)
69cbad0869
some ReadableAccount changes ( #16688 )
...
* some ReadableAccount changes
* deref
2021-04-21 12:20:37 -05:00
Jeff Washington (jwash)
189d2121e6
simplify do_shrink_slot_stores, delay/reduce account clone ( #16691 )
2021-04-21 11:17:38 -05:00
Jeff Washington (jwash)
9852572eb9
increase rolling slot width to 4M ( #16583 )
2021-04-20 21:18:15 -05:00
Jeff Washington (jwash)
03f7b251b8
ReadableAccount.data returns slice ( #16686 )
...
* ReadAbleAccount.data returns slice
* more slice fixup
* more slice
* slice
2021-04-20 16:41:16 -05:00
Jeff Washington (jwash)
36e11998c7
read only accounts cache uses accurate size representation ( #16610 )
...
* read ony accounts cache uses accurate size representation
* add comment and test
2021-04-20 09:58:01 -05:00
Jack May
01786f684e
Remove unwrap ( #16652 )
2021-04-19 20:17:56 -07:00
Jeff Washington (jwash)
4aa753ff01
rename threads: 15 char limit ( #16625 )
2021-04-19 12:16:58 -05:00
Alexander Meißner
9dfcb921cf
Refactoring: Move KeyedAccounts to InvokeContext ( #15410 )
...
Collects all parametric occurrences and the construction of keyed_accounts and puts them into InvokeContext.
2021-04-19 18:48:48 +02:00
Jeff Washington (jwash)
015bc034a5
improve failing assert ( #16581 )
2021-04-19 08:55:01 -05:00
Michael Vines
a911ae00ba
clippy
2021-04-18 20:55:02 -07:00
Trent Nelson
285f3c9d56
Feature-gate hash-based duplicate transaction check
2021-04-16 18:51:18 +00:00
carllin
d747614b27
Account for possibility of cache flush in load() ( #15454 )
...
* Account for possibility of cache flush in load()
* More cleaning
* More cleaning
* Remove unused method and some comment cleaning
* Fix typo
* Make the detected impossible purge race panic()!
* Finally revert to original .expect()
* Fix typos...
* Add assertion for max_root for easier reasoning
* Reframe races with LoadHint as possible opt.
* Fix test
* Make race bug tests run longer for less flaky
* Delay the clone-in-lock slow path even for RPC
* Make get_account panic-free & add its onchain ver.
* Fix rebase conflicts...
* Clean up
* Clean up comment
* Revert fn name change
* Fix flaky test...
* fmt...
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-04-17 00:23:32 +09:00
Brooks Prumo
17aa45fad1
Remove old way of account hashing
...
Account data hashing used to use different ways of hashing on different
clusters. That is no longer the case, but the old code still existed.
This commit removes that old, now used code.
**NOTE** The golden hash values in bank.rs needed to be updated. Since
the original code that selected the hash algorithm used `if >` instead
of `if >=`, this meant that the genesis block's hash _always_ used the
old hashing method, which is no longer valid.
Validated by running `cargo test` successfully.
2021-04-13 14:42:21 -05:00
Justin Starry
f641429056
Remove blake3 from bpf program dependencies ( #16506 )
2021-04-13 10:06:05 +00:00
Justin Starry
85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
...
* Track transaction check time separately from account loads
* banking packet process metrics
* Remove signature clone in status cache lookup
* Reduce allocations when converting packets to transactions
* Add blake3 hash of transaction messages in status cache
* Bug fixes
* fix tests and run fmt
* Address feedback
* fix simd tx entry verification
* Fix rebase
* Feedback
* clean up
* Add tests
* Remove feature switch and fall back to signature check
* Bump programs/bpf Cargo.lock
* clippy
* nudge benches
* Bump `BankSlotDelta` frozen ABI hash`
* Add blake3 to sdk/programs/Cargo.lock
* nudge bpf tests
* short circuit status cache checks
Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
Jack May
fa83f3bd73
Return sysvars via syscalls ( #16422 )
2021-04-12 16:04:57 -07:00
Brooks Prumo
54ef065cc8
Use slot 1 for tests' sample store and accounts
...
Anticipating the removal of the old account hashing way, the Account DB
tests use slot 0 for the sample storages and accounts data. Slot 0 is a
special case, and so to make future changes not break any tests, update
just the tests here now to use any other slot.
The golden hash values were obtained by running `cargo test` and copying
in the "actual" values from the test output into the `raw_expected`
vector of the test code.
Validated by running `cargo test` successfully.
2021-04-12 17:09:07 -05:00
Jeff Washington (jwash)
105a6bfb46
Replace RootsTracker HashSet ( #16310 )
...
* Replace RootsTracker HashSet
* use bitvec
* cleanup, add brenchmark test
* test cleanup
* add lots of tests
* get rid of demo
* change warp test constant
* get rid of unused function
* pr feedback
* reorder use
* rework get_all to remove range checks
* add tests, fix bugs
2021-04-12 12:11:33 -05:00
Jeff Washington (jwash)
7ba3e710d3
latest_slot uses > current_max ( #16374 )
2021-04-12 10:52:24 -05:00
Jeff Washington (jwash)
6930a77a0f
prepare replace Ancestors HashMap for performance ( #16476 )
2021-04-12 10:51:57 -05:00
carllin
dc7030ffaa
Allow fork choice to support multiple versions of a slot ( #16266 )
2021-04-12 01:00:59 -07:00
Trent Nelson
ac5462e7a2
Merge pull request from GHSA-fmvj-vqp5-qqh9
...
* Sanitize permissions
* Forbid creating directories under ledger/rocksdb/
* hardened_unpack: Disallow dirs under rocksdb/ in genesis
* hardened_unpack: expand valid genesis entry test coverage
* hardened_unpack: rework old-style bsd directory entry rejection
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
2021-04-10 00:57:32 -06:00
François Garillot
b08cff9e77
Simplify some pattern-matches ( #16402 )
...
When those match an exact combinator on Option / Result.
Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust ).
2021-04-08 12:40:37 -06:00
steviez
e69f37a464
Remove old function: account_balance_for_capitalization ( #16383 )
...
This function currently returns one of its' parameters and is thus useless.
2021-04-07 23:57:40 -05:00
Jack May
476fd40948
Add program-test tests to bpf programs ( #16407 )
2021-04-07 01:47:15 -07:00
Jack May
92f4018b07
Cleanup unsupported sysvars ( #16390 )
...
* Cleanup unsupported sysvars
* fix ser description
2021-04-06 00:08:03 -07:00
Jeff Washington (jwash)
95dc7b5449
reduce lock contention on latest_root ( #16306 )
2021-04-05 11:35:14 -05:00
Jeff Washington (jwash)
7a997759fa
reduce locking on get_rooted_entries ( #16304 )
2021-04-05 11:26:53 -05:00
Justin Starry
0596cf5405
Remove unprocessed transactions from log notifications ( #16349 )
2021-04-04 23:44:59 +08:00
Jeff Washington (jwash)
3996b699dc
read only account cache for executable accounts - improve replay ( #16150 )
...
* read only account cache
* tests
* clippy
* cleanup
* new file, add tests
* remove copy/paste code from test
* remove dead code
* all loads use cache
* remove stale comments
* add metrics logging for read only cache size
* report read only cache hits and misses
* consistency
* formatting
* rename, add comment
* u64
* better interaction with existing cache
* lru list saved between cleans
2021-04-01 07:16:34 -05:00
behzad nouri
3f63ed9a72
removes OrderedIterator and transaction batch iteration order ( #16153 )
...
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817
However load_and_execute_transactions is iterating over
lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.
If iteratorion_order is [1, 2, 0], and i is 0, then:
lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).
This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
Jeff Washington (jwash)
416312b30b
update timings only on outer call ( #16275 )
...
* update timings only on outer call
* test & format
2021-03-31 18:02:59 -05:00
Jeff Washington (jwash)
f374b35944
Hash stored accounts in bg ( #16157 )
...
* lazy calculate account hash
* push to bg thread
* remove deadlock
* logs
* format
* some cleanup on aisle 9
* format, fix up some metrics
* fix test, remove legacy function only there for tests
* cleanup
* remove unused store_hasher
* Switch to crossbeam
* clippy
* format
* use iter()
* rework from feedback
* hash_slot -> slot
* hash(cluster_type)
Co-authored-by: Carl Lin <carl@solana.com>
2021-03-31 15:39:34 -05:00
Jeff Washington (jwash)
a5dcee254d
get rid of unnecessary data copy ( #16214 )
...
* get rid of unnecessary data copy
* preserve rent_epoch
2021-03-31 09:11:39 -05:00
sakridge
54c68ea83f
Drop write lock on sysvars ( #15497 )
...
* Drop write lock on sysvars
* adds env var for demoting sysvar write lock demotion
* moves demote logic to is_writable
* feature gates sysvar write lock demotion
* adds builtins to write lock demotion
* adds system program id to builtins
* adds Feature111...
* adds an abi-freeze test
* mvines set of builtin program keys
Co-authored-by: Michael Vines <mvines@gmail.com>
* update tests
* adds bpf loader keys
* Add test sysvar
* Plumb demote_sysvar to is_writable
* more plumbing of demote_sysvar_write_locks to is_writable
* patches test_program_bpf_instruction_introspection
* hard codes demote_sysvar_write_locks to false for serialization/encoding methods
* Revert "hard codes demote_sysvar_write_locks to false for serialization/encoding methods"
This reverts commit ae3e2d2e777437bddd753933097a210dcbc1b1fc.
* change the hardcoded ones to demote_sysvar_write_locks=true
* Use data_as_mut_slice
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-30 10:05:09 -07:00
sakridge
5e5b63712b
Status cache improvements ( #16174 )
2021-03-28 19:10:14 -07:00
Jeff Washington (jwash)
66c42f62d8
make AccountSharedData.data private to abstract storage ( #16091 )
...
* format
* 2 more tests
* use
2021-03-25 11:04:20 -05:00
Ryo Onodera
6d5c6c17c5
Simplify account.rent_epoch handling for sysvar rent ( #16049 )
...
* Add some code for special local testing
* Add comment to store_account_and_update_capitalization
* Simplify account.rent_epoch handling for sysvar rent
* Introduce *_for_test functions
* Add deprecation messages to existing api
2021-03-25 15:23:20 +09:00
sakridge
26af2d4c26
Up store limit for test_account_grow ( #16099 )
...
There can be more than 5 stores
2021-03-24 15:24:17 -07:00
Jeff Washington (jwash)
2d24d13046
AccountsSharedData: data copy on write ( #15800 )
...
* Arc<AccountData>
* try custom serializer
* adapt test from Behzad's change
* clippy
* simplify serialization
* remove abi example derive
* refactor 'take'
* remove serialization
* remove serialize calls
* remove account_data
* remove intos
* remove left over file
2021-03-23 15:19:31 -05:00
Jeff Washington (jwash)
fde43a906d
revert test to use Account for serialization ( #16063 )
2021-03-22 22:48:09 +00:00
Jeff Washington (jwash)
4beb39f7a1
multi-pass bin scanning ( #15377 )
...
* multi-pass bin scanning
* pr feedback
* format
* fix typo
* adjust metrics for code changes
* merge errors
2021-03-18 10:32:07 -05:00
carllin
f548a04fae
Allow unbounded wallclock processing time in tests ( #15961 )
2021-03-17 15:48:50 -07:00
DimAn
a5d144b00f
Revert to removing only tmp-
2021-03-17 10:31:01 -07:00
DimAn
0209d334bd
Add option for separate snapshot location
...
(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
2021-03-17 10:31:01 -07:00
Michael Vines
a2eb655322
=1.7.0
2021-03-16 07:51:07 +00:00
Jeff Washington (jwash)
efee8b62d7
a few missed set_data calls ( #15846 )
...
* a few missed set_data calls
* another set data call
2021-03-15 21:57:23 -05:00
Jeff Washington (jwash)
e35d738906
rework BankClient.get_account_data ( #15853 )
2021-03-15 19:27:53 -05:00
carllin
c1ba265dd9
Wallclock BankingStage Throttle ( #15731 )
2021-03-15 17:11:15 -07:00
Michael Vines
0c9ca5522c
Bump version to v1.7.0
2021-03-13 09:01:21 +00:00
Justin Starry
918d04e3f0
Add more slot update notifications ( #15734 )
...
* Add more slot update notifications
* fix merge
* Address feedback and add integration test
* switch to datapoint
* remove unused shred method
* fix clippy
* new thread for rpc completed slots
* remove extra constant
* fixes
* rely on channel closing
* fix check
2021-03-12 21:44:06 +08:00
Ryo Onodera
4bbeb9c033
Remove old feature: simple_capitalization ( #15763 )
...
* Remove old feature: simple_capitalization
* Fix another failing test in core
* Finish up test cleanup
* Further clean up a bit
2021-03-12 11:12:40 +09:00
Jeff Washington (jwash)
952c3bcbb7
AccountSharedData construction ( #15790 )
2021-03-11 18:09:04 -06:00
Jeff Washington (jwash)
3419a5446e
AccountSharedData.set_data ( #15781 )
...
* account.set_data and resize_data
* remove data_resize
2021-03-11 16:40:45 -06:00
Michael Vines
1061d021c9
Unpack snapshot AppendVecs directly into account paths
2021-03-11 08:25:24 -08:00
Trent Nelson
24d18b3cf2
docs: add docs links for crates published to crates.io
2021-03-11 08:38:18 +00:00
Jack May
478518308d
Share RO and Executable accounts within invocations ( #15799 )
2021-03-10 23:04:00 -08:00
Michael Vines
65dd177601
AppendVec cleanup: remove dead code, rename non-idiomatic "new" function
2021-03-10 14:50:59 -08:00
Jeff Washington (jwash)
1135ffd595
mut data refs as slice ( #15782 )
2021-03-10 15:28:03 -06:00
Tyera Eulberg
9c1198c0c7
Improve load_largest_accounts more ( #15785 )
...
* Add load_largest_accounts bench
* Check lamports before address filter
* Use BinaryHeap, add Accounts test
* Use pubkey reference in the min-heap
Also, flatten code with early returns
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2021-03-10 11:22:02 -07:00
Jack May
369e13b111
cleanup old runtime features ( #15787 )
2021-03-10 09:48:41 -08:00
Tyera Eulberg
3355efbb5a
Fix test to use AccountSharedData
2021-03-09 20:47:54 -07:00
Tyera Eulberg
5991cef5f5
Improve load_largest_accounts and add Bank test ( #15775 )
2021-03-10 00:23:41 +00:00
Jeff Washington (jwash)
52e54e1100
account.data -> data() ( #15778 )
2021-03-09 22:31:33 +00:00
Jeff Washington (jwash)
8a3135d17b
Account->AccountSharedData ( #15691 )
2021-03-09 15:06:07 -06:00
Tyera Eulberg
e3e8179f2d
Sdk: start to appease clippy's integer arithmetic check ( #15736 )
...
* Move to module-level clippy allowance
* Fix stake_weighted_timestamp math
* Fix genesis_config
* Fix shred_version
* Fix hard_forks
* Fix process_instruction
* Add ArithmeticOverflow ix error
* Fix nonce_keyed_account
* Update BankSlotDelta frozen abi due to new ix error
2021-03-08 18:37:57 -07:00
Ryo Onodera
beac2d6d3f
Remove stale comment
2021-03-08 10:18:32 +09:00
Ryo Onodera
8b0c6db871
Remove old feature: cumulative_rent_related_fixes ( #15754 )
2021-03-08 09:58:50 +09:00
sakridge
c5371fdc85
More debug on shrink and flush ( #15518 )
2021-03-05 14:10:34 -08:00
Jeff Washington (jwash)
13b9bb6790
refactor function to store into account after data is finished ( #15710 )
2021-03-05 11:02:35 -06:00
Greg Fitzgerald
edd159e7d7
Fix typos ( #15721 )
2021-03-05 07:16:53 +00:00
Tyera Eulberg
872f7117c3
Use OrderedIterator to produce TransactionLogInfo ( #15712 )
...
* Add failing test
* Fix iteration_order issue with stored logs
2021-03-04 15:26:19 -07:00