Commit Graph

251 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 1cc9a1c0eb
log roots range metric (#16636)
* log roots range metric

* rename
2021-04-23 16:09:39 +00:00
Jeff Washington (jwash) 96d21335e1
.owner -> .owner() (#16758) 2021-04-23 09:35:55 -05:00
Jeff Washington (jwash) 5591cd6ef3
.owner -> .owner() (#16757)
* .owner -> .get_owner()

* use
2021-04-23 09:35:09 -05:00
Jeff Washington (jwash) 91be2903da
Reduce account index lookups during clean (#16689)
* reduce account index lookups in clean

* rename

* rename enum

* hold locks during removal from zero pubkey list

* merge with zero lamport fix

* tests
2021-04-23 09:33:14 -05:00
Jeff Washington (jwash) 333998d008
.lamports = <number> -> .set_lamports(<number>) (#16746) 2021-04-22 18:56:47 +00:00
Jeff Washington (jwash) 8a6b80095e
Set lamports (#16747)
* lamports = -> set_lamports()

* .lamports = X -> .set_lamports(X)
2021-04-22 13:53:06 -05:00
Jeff Washington (jwash) 8d9d6b62d9
use cheaper account get (#16682) 2021-04-22 12:54:08 -05:00
Jeff Washington (jwash) 123e0bdba7
don't iterate slot_list if zero-lamport (#16678) 2021-04-21 20:38:48 -05:00
Jeff Washington (jwash) 69cbad0869
some ReadableAccount changes (#16688)
* some ReadableAccount changes

* deref
2021-04-21 12:20:37 -05:00
Jeff Washington (jwash) 189d2121e6
simplify do_shrink_slot_stores, delay/reduce account clone (#16691) 2021-04-21 11:17:38 -05:00
Jeff Washington (jwash) 4aa753ff01
rename threads: 15 char limit (#16625) 2021-04-19 12:16:58 -05:00
Michael Vines a911ae00ba clippy 2021-04-18 20:55:02 -07:00
carllin d747614b27
Account for possibility of cache flush in load() (#15454)
* Account for possibility of cache flush in load()

* More cleaning

* More cleaning

* Remove unused method and some comment cleaning

* Fix typo

* Make the detected impossible purge race panic()!

* Finally revert to original .expect()

* Fix typos...

* Add assertion for max_root for easier reasoning

* Reframe races with LoadHint as possible opt.

* Fix test

* Make race bug tests run longer for less flaky

* Delay the clone-in-lock slow path even for RPC

* Make get_account panic-free & add its onchain ver.

* Fix rebase conflicts...

* Clean up

* Clean up comment

* Revert fn name change

* Fix flaky test...

* fmt...

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-04-17 00:23:32 +09:00
Brooks Prumo 17aa45fad1 Remove old way of account hashing
Account data hashing used to use different ways of hashing on different
clusters.  That is no longer the case, but the old code still existed.
This commit removes that old, now used code.

**NOTE** The golden hash values in bank.rs needed to be updated.  Since
the original code that selected the hash algorithm used `if >` instead
of `if >=`, this meant that the genesis block's hash _always_ used the
old hashing method, which is no longer valid.

Validated by running `cargo test` successfully.
2021-04-13 14:42:21 -05:00
Justin Starry 85eb37fab0
Merge pull request from GHSA-8v47-8c53-wwrc
* Track transaction check time separately from account loads

* banking packet process metrics

* Remove signature clone in status cache lookup

* Reduce allocations when converting packets to transactions

* Add blake3 hash of transaction messages in status cache

* Bug fixes

* fix tests and run fmt

* Address feedback

* fix simd tx entry verification

* Fix rebase

* Feedback

* clean up

* Add tests

* Remove feature switch and fall back to signature check

* Bump programs/bpf Cargo.lock

* clippy

* nudge benches

* Bump `BankSlotDelta` frozen ABI hash`

* Add blake3 to sdk/programs/Cargo.lock

* nudge bpf tests

* short circuit status cache checks

Co-authored-by: Trent Nelson <trent@solana.com>
2021-04-13 00:28:08 -06:00
Brooks Prumo 54ef065cc8 Use slot 1 for tests' sample store and accounts
Anticipating the removal of the old account hashing way, the Account DB
tests use slot 0 for the sample storages and accounts data.  Slot 0 is a
special case, and so to make future changes not break any tests, update
just the tests here now to use any other slot.

The golden hash values were obtained by running `cargo test` and copying
in the "actual" values from the test output into the `raw_expected`
vector of the test code.

Validated by running `cargo test` successfully.
2021-04-12 17:09:07 -05:00
Jeff Washington (jwash) 6930a77a0f
prepare replace Ancestors HashMap for performance (#16476) 2021-04-12 10:51:57 -05:00
steviez e69f37a464
Remove old function: account_balance_for_capitalization (#16383)
This function currently returns one of its' parameters and is thus useless.
2021-04-07 23:57:40 -05:00
Jeff Washington (jwash) 3996b699dc
read only account cache for executable accounts - improve replay (#16150)
* read only account cache

* tests

* clippy

* cleanup

* new file, add tests

* remove copy/paste code from test

* remove dead code

* all loads use cache

* remove stale comments

* add metrics logging for read only cache size

* report read only cache hits and misses

* consistency

* formatting

* rename, add comment

* u64

* better interaction with existing cache

* lru list saved between cleans
2021-04-01 07:16:34 -05:00
Jeff Washington (jwash) f374b35944
Hash stored accounts in bg (#16157)
* lazy calculate account hash

* push to bg thread

* remove deadlock

* logs

* format

* some cleanup on aisle 9

* format, fix up some metrics

* fix test, remove legacy function only there for tests

* cleanup

* remove unused store_hasher

* Switch to crossbeam

* clippy

* format

* use iter()

* rework from feedback

* hash_slot -> slot

* hash(cluster_type)

Co-authored-by: Carl Lin <carl@solana.com>
2021-03-31 15:39:34 -05:00
Ryo Onodera 6d5c6c17c5
Simplify account.rent_epoch handling for sysvar rent (#16049)
* Add some code for special local testing

* Add comment to store_account_and_update_capitalization

* Simplify account.rent_epoch handling for sysvar rent

* Introduce *_for_test functions

* Add deprecation messages to existing api
2021-03-25 15:23:20 +09:00
sakridge 26af2d4c26
Up store limit for test_account_grow (#16099)
There can be more than 5 stores
2021-03-24 15:24:17 -07:00
Jeff Washington (jwash) 4beb39f7a1
multi-pass bin scanning (#15377)
* multi-pass bin scanning

* pr feedback

* format

* fix typo

* adjust metrics for code changes

* merge errors
2021-03-18 10:32:07 -05:00
Ryo Onodera 4bbeb9c033
Remove old feature: simple_capitalization (#15763)
* Remove old feature: simple_capitalization

* Fix another failing test in core

* Finish up test cleanup

* Further clean up a bit
2021-03-12 11:12:40 +09:00
Jeff Washington (jwash) 952c3bcbb7
AccountSharedData construction (#15790) 2021-03-11 18:09:04 -06:00
Jeff Washington (jwash) 3419a5446e
AccountSharedData.set_data (#15781)
* account.set_data and resize_data

* remove data_resize
2021-03-11 16:40:45 -06:00
Michael Vines 1061d021c9 Unpack snapshot AppendVecs directly into account paths 2021-03-11 08:25:24 -08:00
Michael Vines 65dd177601 AppendVec cleanup: remove dead code, rename non-idiomatic "new" function 2021-03-10 14:50:59 -08:00
Jeff Washington (jwash) 1135ffd595
mut data refs as slice (#15782) 2021-03-10 15:28:03 -06:00
Jeff Washington (jwash) 52e54e1100
account.data -> data() (#15778) 2021-03-09 22:31:33 +00:00
Jeff Washington (jwash) 8a3135d17b
Account->AccountSharedData (#15691) 2021-03-09 15:06:07 -06:00
sakridge c5371fdc85
More debug on shrink and flush (#15518) 2021-03-05 14:10:34 -08:00
Greg Fitzgerald edd159e7d7
Fix typos (#15721) 2021-03-05 07:16:53 +00:00
Jeff Washington (jwash) a2c47db920
extract expected cluster type (#15615)
* extract expected cluster type

* fix clippy warning

* more clippy
2021-03-02 13:55:34 -06:00
Jeff Washington (jwash) da47ab7e50
fix indentation format problem (#15618)
* fix indentation format problem

* fix indentation format problem
2021-03-02 10:46:00 -06:00
Greg Fitzgerald 2463cc1e6a
Fix typos (#15610) 2021-03-02 06:36:49 -08:00
carllin 97eaf3c334
Fix finalize_dead_slot_removal() of cached slots decrementing refcount (#15534) 2021-02-26 17:49:37 -08:00
Ryo Onodera 21b43009f6
Introduce ttl eviction for RecycleStore (#15513) 2021-02-25 17:27:27 +09:00
Michael Vines 5df36aec7d Pacify clippy 2021-02-19 20:08:41 -08:00
Jeff Washington (jwash) 7fee0bc69c
move hashing functionality to accounts_hash (#15353) 2021-02-17 00:29:50 +00:00
Jeff Washington (jwash) 8367740ff9
compute merkle root on chunks of fanout^3 (#15344)
* compute merkle root on chunks of fanout^3

* improve test_accountsdb_compute_merkle_root_large
2021-02-16 17:03:35 -06:00
Jeff Washington (jwash) d83aeebcca
change 2d cumulative calculation (#15346) 2021-02-16 14:24:45 -06:00
Jeff Washington (jwash) 708b8380b7
assert 2 items in stores don't have same slot and version with different values (#15250)
* assert 2 items in stores don't have same slot and version with different values

* add message to assert
2021-02-16 18:12:59 +00:00
Jeff Washington (jwash) 53959b4bbc
divide account scan into bins for more efficient downstream processing (#15303) 2021-02-15 21:27:39 -06:00
Jeff Washington (jwash) 78a176fc00
rename CumulativeOffset in preparation of 2d support (#15338) 2021-02-15 20:57:05 -06:00
Ryo Onodera 30f18319f2
Track RecycleStore basic stats with needed refactor (#15291)
* Track RecycleStore basic stats with needed refactor

* Fix another wrong metrics def
2021-02-15 16:09:17 +09:00
Jeff Washington (jwash) b8448f4189
merkle root code no longer adds lamports (#15298) 2021-02-12 17:30:14 -06:00
Jeff Washington (jwash) 8944efddf7
combine flatten and hash (#15269) 2021-02-12 11:35:54 -06:00
Jeff Washington (jwash) eced3776b0
add metrics to hash calculation and consolidate (#15165)
* add metrics to hash calculation

* add metrics to hash calculation

* create HashStats struct to consolidate hash timing

* formatting
2021-02-11 20:06:04 -06:00
Jeff Washington (jwash) eb54e544c2
add div_ceil (#15267) 2021-02-11 20:05:41 -06:00