6930a77a0f
prepare replace Ancestors HashMap for performance ( #16476 )
2021-04-12 10:51:57 -05:00
95dc7b5449
reduce lock contention on latest_root ( #16306 )
2021-04-05 11:35:14 -05:00
7a997759fa
reduce locking on get_rooted_entries ( #16304 )
2021-04-05 11:26:53 -05:00
8a3135d17b
Account->AccountSharedData ( #15691 )
2021-03-09 15:06:07 -06:00
edd159e7d7
Fix typos ( #15721 )
2021-03-05 07:16:53 +00:00
2463cc1e6a
Fix typos ( #15610 )
2021-03-02 06:36:49 -08:00
e4d0d4bfae
Fix pubkey refcount for shrink + clean ( #14987 )
2021-02-04 12:44:19 -08:00
e5225b7e68
Add ref count from storage ( #15078 )
2021-02-04 09:11:05 -08:00
2970b59853
Don't load all accounts into memory for capitalization check ( #14957 )
...
* Don't load all accounts into memory for capitalization check
2021-02-03 15:00:42 -08:00
72f10f5f29
Aggregate purge and shrink metrics ( #14763 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-27 01:39:47 -08:00
0d32a0e0f4
Partial clean ( #14800 )
...
* Revert "Revert "Partial accounts clean (#14652 )" (#14777 )"
This reverts commit ad2e10e17b
2021-01-24 09:50:19 -08:00
96b27ca0cb
Remove zero_lamport_pubkeys reference ( #14808 )
2021-01-23 18:31:55 +00:00
424bb797a6
Speed up generate_index ( #14792 )
2021-01-23 08:05:05 -08:00
c77461e428
Remove unnecesary flushes in previous roots ( #14596 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-23 04:02:44 -08:00
ad2e10e17b
Revert "Partial accounts clean ( #14652 )" ( #14777 )
...
This reverts commit ca7914aa3d
2021-01-22 18:05:49 -08:00
ca7914aa3d
Partial accounts clean ( #14652 )
...
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
2021-01-21 11:01:53 -08:00
5f14f45282
More generic accounts purge functions ( #14595 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-17 20:31:03 -08:00
6dfad0652f
Cache account stores, flush from AccountsBackgroundService ( #13140 )
2021-01-11 17:00:23 -08:00
c9df6134fa
Save 7G mem on mainnet fixing AccIndex overalloc. ( #14435 )
2021-01-06 09:48:59 +09:00
5affd8aa72
Add secondary indexes ( #14212 )
2020-12-31 18:06:03 -08:00
baa9602411
Add shrink paths ( #14238 )
2020-12-21 21:33:37 -08:00
7143aaa89b
Clippy
2020-12-14 08:03:29 -08:00
955b99cf69
Fix roots_and_ref_count ( #13827 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-26 13:09:20 -08:00
791fb17437
Prevent scans on unrooted slots from seeing partial clean ( #13628 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 13:01:04 -08:00
6276360468
Prevent scans from seeing root updates/clean ( #13464 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-16 17:23:11 -08:00
a8a77614fe
Fix potential undefined behavior ( #13555 )
...
* Switch to ouroboros 0.5.1
* Update other lock files
2020-11-13 18:12:41 +09:00
959880db60
Remove unused pubkey::Pubkey imports
2020-10-21 19:08:13 -07:00
e6b821c392
Finer grained AccountsIndex locking ( #12787 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-21 17:05:27 -07:00
64c4861e04
Fix error in max root calculation ( #12661 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-10-03 15:18:58 -07:00
06f84c65f1
Fix rooted accounts cleanup, simplify locking ( #12194 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-28 16:04:46 -07:00
00b36e6db2
Fix ref-count for multiple stores to the same pubkey in a slot, fixes zero lamport purge detection ( #12462 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-09-25 18:54:48 -07:00
832d47317e
Move clean accounts to background service ( #10898 )
2020-07-02 22:25:17 -07:00
50f7ed80c8
Dont skip eager rent collect across gapped epochs ( #10206 )
...
* Dont skip eager rent collect across gapped epochs
* Adjust style and comment
* Adjust ascii chart and comment a bit
* Moar assert
* Relax the partition_count assert for completeness
* Tweak comment...
* tweak a bit
* Add gating logic
* Address reviews
* small formatting
* Clarify the code by replacing auto_generated...
* small formatting
* small formatting
* small formatting
* small formatting
* Narrow down conditional compilation scope
2020-06-25 23:11:33 +09:00
dfe72d5242
Optimize stale slot shrinking for previously cleaned roots ( #10099 )
...
* Prioritize shrinking of previously cleaned roots
* measure time of stale slot shrinking
* Disable shrink for test
* shrink: budgeting, store count, force for snapshot
* Polish implementation and fix tests
* Fix ci..
* Clean up a bit
* Further polish implementation and fix/add tests
* Rebase fixes
* Remove unneeded Default for AccountStorageEntry
* Address review comments
* More cleanup
* More cleanup
2020-06-12 14:51:43 +09:00
e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) ( #10445 )
...
* address warnings from 'rustup run beta cargo clippy --workspace'
minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs
expect some performance improvement AccountsDB::clean_accounts()
* address warnings from 'rustup run beta cargo clippy --workspace --tests'
* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'
* rustfmt
* fix warning stragglers
* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned
* Remove lock around JsonRpcRequestProcessor (#10417 )
automerge
* make ancestors parameter optional to avoid forcing construction of empty hash maps
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
1eb40c3fe0
Introduce eager rent collection ( #9527 )
...
* Switch AccountsIndex.account_maps from HashMap to BTreeMap
* Introduce eager rent collection
* Start to add tests
* Avoid too short eager rent collection cycles
* Add more tests
* Add more tests...
* Refacotr!!!!!!
* Refactoring follow up
* More tiny cleanups
* Don't rewrite 0-lamport accounts to be deterministic
* Refactor a bit
* Do hard fork, restore tests, and perf. mitigation
* Fix build...
* Refactor and add switch over for testnet (TdS)
* Use to_be_bytes
* cleanup
* More tiny cleanup
* Rebase cleanup
* Set Bank::genesis_hash when resuming from snapshot
* Reorder fns and clean ups
* Better naming and commenting
* Yet more naming clarifications
* Make prefix width strictly uniform for 2-base partition_count
* Fix typo...
* Revert cluster-dependent gate
* kick ci?
* kick ci?
* kick ci?
2020-05-13 16:22:14 +09:00
3442f36f8a
Repair alternate versions of dead slots ( #9805 )
...
Co-authored-by: Carl <carl@solana.com>
2020-05-05 14:07:21 -07:00
9918539229
Introduce type alias Ancestors ( #9699 )
...
* Introduce type alias AncestorList
* Rename AncestorList => Ancestors
2020-04-27 11:07:03 +09:00
2623c71ed3
Use type aliases/resulting var names consistently ( #9060 )
2020-03-26 13:08:56 +09:00
4b397d15b3
Accounts cleanup service and perf improvements ( #8799 )
...
* Use atomic for ref count instead of taking rwlock
* Accounts cleanup service
* Review comments
2020-03-23 08:50:23 -07:00
952cd38b7b
Avoid early clean and bad snapshot by ref-counting ( #8724 )
...
* Avoid early clean and bad snapshot by ref-counting
* Add measure
* Clean ups
* clean ups
2020-03-13 14:14:37 +09:00
d86103383a
Do periodic inbound cleaning for rooted slots ( #8436 )
...
* Do periodic inbound compaction for rooted slots
* Add comment
* nits
* Consider not_compacted_roots in cleanup_dead_slot
* Renames in AccountsIndex
* Rename to reflect expansion of removed accounts
* Fix a comment
* rename
* Parallelize clean over AccountsIndex
* Some niceties
* Reduce locks and real chunked parallelism
* Measure each step for sampling opportunities
* Just noticed par iter is maybe lazy
* Replace storage scan with optimized index scan
* Various clean-ups
* Clear uncleared_roots even if no updates
2020-03-03 14:57:25 +09:00
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
58727463e1
Remove needless last_root for better reclaims ( #8148 )
...
* Restore last_root to fix unintended storage delete
* Remove last_root thing altogether
* Remove unneeded test...
2020-02-13 08:19:53 +09:00
f016c9a669
Maintenance : simplify a few patterns, remove unneeded dependencies ( #8137 )
...
* Simplify a few pattern matches
* Removing unneeded dependencies, upgrading internal version #s
net-shaper: Removing log, semver, serde_derive
bench-tps: Removing serde, serde_derive
banking-bench: Removing solana
ledger-tool: Removing bincode, serde, serde_derive
librapay: Removing solana, language_e2e_tests
log-analyzer: Removing log, semver, serde_derive
exchange: Removing solana
core: Removing crc, memmap, symlink, untrusted
perf: Removing serde_derive
genesis: Removing hex, serde_derive
sdk-c: Removing sha2
sys-tuner: Removing semver
bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
btc_spv_bin: Removing serde_json
btc_spv: Removing chrono
bpf_loader: Removing serde
ledger: Removing dlopen, dlopen_derive, serde_derive
move_loader: Removing byteorder, libc, language_e2e_tests
ownable: Removing serde, serde_derive
client: Removing rand
archiver-utils: Removing rand_chacha
validator: Removing serde_json, tempfile
param_passing_dep: Removing solana
failure: Removing log
vest: Removing log
vote-signer: Removing bs58, serde
local-cluster: Removing symlink
keygen: Removing rpassword
install: Removing bs58, log
upload-perf: Removing log
runtime: Removing serde_json
stake: Removing rand
* Add modified Cargo.lock
* fixup! Simplify a few pattern matches
* fixup! Simplify a few pattern matches
2020-02-06 10:02:38 -07:00
83718a3b3e
Cleanup runtime use syntax ( #8002 )
2020-01-28 17:03:20 -08:00
7c92bf15e2
Only return accounts that have changed since the bank's parent ( #7520 )
...
* Only return accounts that have changed since the bank's parent
* add test
2019-12-16 23:24:18 -08:00
0aa4dc904e
Accounts index updates, remove pubkey when dead account ( #7408 )
2019-12-11 11:11:31 -08:00
887bff572a
More conservative purge_zero_lamport_accounts purge logic ( #7157 )
2019-12-02 09:51:05 -08:00
b4119c454a
credit_only credits forwarding ( #6509 )
...
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
Jeff Washington (jwash)
Greg Fitzgerald
carllin
sakridge
Ryo Onodera
Michael Vines
joshua-maros
Kristofer Peterson
Ryo Onodera
Tyera Eulberg
François Garillot
Jack May
sakridge
Rob Walker