sakridge
b084c1d437
Remove accounts hack and correctly restore accounts store counts ( #8569 )
...
* Remove accounts hack and correctly restore append-vec counts
* Add test
2020-03-03 20:48:55 -08:00
sakridge
4f05f08f5d
Use fs::rename which is much faster than move_items ( #8579 )
2020-03-03 10:03:17 -08:00
Ryo Onodera
d86103383a
Do periodic inbound cleaning for rooted slots ( #8436 )
...
* Do periodic inbound compaction for rooted slots
* Add comment
* nits
* Consider not_compacted_roots in cleanup_dead_slot
* Renames in AccountsIndex
* Rename to reflect expansion of removed accounts
* Fix a comment
* rename
* Parallelize clean over AccountsIndex
* Some niceties
* Reduce locks and real chunked parallelism
* Measure each step for sampling opportunities
* Just noticed par iter is maybe lazy
* Replace storage scan with optimized index scan
* Various clean-ups
* Clear uncleared_roots even if no updates
2020-03-03 14:57:25 +09:00
Ryo Onodera
5d9130a3c4
Hack to skip cleanup_dead_slots upon snapshot load
2020-03-02 10:24:12 -07:00
Ryo Onodera
0b48c8eb35
Promote dangerous cond. from just warning to panic ( #8439 )
2020-02-26 05:09:57 +09:00
sakridge
947a339714
Add snapshot hash of full accounts state ( #8295 )
...
* Add snapshot hash of full accounts state
* Use normal hashing for the accounts delta state
* Add merkle
2020-02-22 13:46:40 -08:00
Ryo Onodera
d238371b0c
Correct missing entry handling to avoid bad warns ( #8339 )
...
* Correct missing entry handling to avoid bad warns
* Pass storage entries to AccountStorageSerialize
* Fix CI.....
* Add tests and reorder condition for cheapest first
* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Tyera Eulberg
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
Ryo Onodera
027ec71aa9
Remove AccountInfo's (De)Serialize ( #8313 )
...
automerge
2020-02-17 03:07:36 -08:00
Ryo Onodera
58727463e1
Remove needless last_root for better reclaims ( #8148 )
...
* Restore last_root to fix unintended storage delete
* Remove last_root thing altogether
* Remove unneeded test...
2020-02-13 08:19:53 +09:00
sakridge
a8028fbb93
Fix accounts_db store counts in purging accounts logic ( #8218 )
...
* Show insufficient purge_zero_lamport_account logic
* Add another pass to detect non-deleted values and increment the count
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-02-12 08:51:03 -08:00
Michael Vines
3ac0192d40
Better surface bank hash verification failures
2020-02-05 11:39:47 -07:00
sakridge
78f6ddc5b7
Fix spelling of verification in accounts_db ( #8117 )
...
automerge
2020-02-04 11:45:47 -08:00
Jack May
83718a3b3e
Cleanup runtime use syntax ( #8002 )
2020-01-28 17:03:20 -08:00
Ryo Onodera
2c7447b73e
Secure sysvars under hash by freezing all strictly ( #7892 )
...
* Secure sysvars under hash by freezing all strictly
* Fix hash's non-idempotnet and add new test
* Clean up
* More cleanups
2020-01-24 16:10:32 +09:00
Michael Vines
8b8033c72b
Set BankRc slot correctly when restoring a bank snapshot
2020-01-23 13:37:13 -07:00
Justin Starry
87598c7612
Consolidate tx error counters and update metrics dashboard ( #7724 )
...
automerge
2020-01-16 23:26:50 -08:00
Ryo Onodera
b16c30b4c6
Fix cluster collapse due to no proper shifted read ( #7797 )
...
* Fix cluster collapse due to no proper shifted read
* Add test for bank hash mismatch
Co-authored-by: sakridge <sakridge@gmail.com>
2020-01-15 11:45:19 +09:00
sakridge
3b78be83cf
Add hash stats information to check hashes between validators ( #7780 )
...
automerge
2020-01-14 11:57:29 -08:00
Ryo Onodera
865c42465a
Cap file size for snapshot data files ( #7182 )
...
* save limit deserialize
* save
* Save
* Clean up
* rustfmt
* rustfmt
* Just comment out to please CI
* Fix ci...
* Move code
* Rustfmt
* Crean up control flow
* Add another comment
* Introduce predetermined constant limit on snapshot data files (deserialize side)
* Introduce predetermined constant limit on snapshot data files (serialize side)
* rustfmt
* Tweak message
* Revert dynamic memory limit
* Limit size of snapshot data file (de)serialization
* Fix test breakage
* Clean up
* Fix uses formatting
* Rename: deserialize_{for,from}_snapshot
* Simplify comment
* Use Slot
* Provide slot for status cache
* Align variable name with snapshot_status_cache_file_path
* Define serialize_snapshot_data_file_with_metrics
* Fix build.......
* De-marco serialize_snapshot_data_file_with_metrics
* Revert u64 => Slot
2020-01-10 09:49:36 +09:00
Michael Vines
9720c894f1
Use commas to make a log message more readable
2020-01-06 22:31:01 -07:00
Ryo Onodera
c8fe4043b6
Rename slot_hash => bank_hash in AcoountsDB ( #7579 )
...
* Rename slot_hash => bank_hash in AcoountsDB
2019-12-23 10:50:31 +09:00
Ryo Onodera
3c361eb759
Check account hashes in snapshot ( #7559 )
...
* Check for incorrect hash value
* Finish up checking for incorrect hash value
* Fix comment a bit
Co-authored-by: sakridge <sakridge@gmail.com>
2019-12-20 09:39:30 +09:00
sakridge
9d5a07bac4
Move create_dir_all to AccountsDB::new ( #7465 )
...
AppendVec create doesn't need to try and create
paths every time and it can stall while snapshot
create is happening.
2019-12-13 16:46:16 -08:00
Ryo Onodera
a1ab81a896
Include rent_epoch and executable into account hash ( #7415 )
...
* Clean a bit
* Add rent_epoch and executable into account hashing
* Remove comment and instead create an issue
2019-12-12 13:03:33 +09:00
sakridge
0aa4dc904e
Accounts index updates, remove pubkey when dead account ( #7408 )
2019-12-11 11:11:31 -08:00
Justin Starry
b7d4330dd4
Fail fast if account paths cannot be canonicalized ( #7300 )
...
* Canonicalize account paths to avoid symlink issues
* fixes
2019-12-05 21:41:29 -05:00
sakridge
cfc21e1225
Only serialize rooted append vecs ( #7281 )
2019-12-05 14:27:46 -08:00
sakridge
887bff572a
More conservative purge_zero_lamport_accounts purge logic ( #7157 )
2019-12-02 09:51:05 -08:00
sakridge
6c89226ccf
Purge zero lamport accounts on snapshot ingestion ( #7010 )
...
Snapshots do not load the original index, so they must
purge zero lamport accounts again.
2019-11-22 18:22:28 -08:00
anatoly yakovenko
b150da837a
Use epoch as the gossip purge timeout for staked nodes. ( #7005 )
...
automerge
2019-11-20 11:25:18 -08:00
Jack May
d184d3a732
Merge native programs parts into one unit ( #7047 )
2019-11-20 10:12:43 -08:00
Sagar Dhawan
42da1ce4e2
Fix bank hash not changing when no internal state has changed ( #7052 )
...
* Fix bank hash not changing when no internal state has changed
* Fix unnecessary call to hash_internal_state
* Add blockhash into the bank_hash
* Add blockhash into the bank_hash and update tests
* Refactor accounts_db slot_hashes
* More clarity in comments
* Add clippy suggestion
* Grammar
* Fix compile after clippy made me break it
* Schooled by clippy
2019-11-19 20:19:43 -08:00
Michael Vines
50a17fc00b
Use Slot and Epoch type aliases instead of raw u64 ( #6693 )
...
automerge
2019-11-02 00:38:30 -07:00
Jack May
5264fded00
Avoid alloc due to vector pushes ( #6632 )
2019-10-30 21:55:17 -07:00
Rob Walker
b4119c454a
credit_only credits forwarding ( #6509 )
...
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
sakridge
f1172617cc
Purge accounts with lamports=0 on rooted forks ( #6315 )
2019-10-23 12:46:48 -07:00
Greg Fitzgerald
fb39bd45d7
Revert "Rename solana-runtime to sealevel ( #6239 )" ( #6247 )
...
This reverts commit 2e921437cd
.
2019-10-04 19:33:29 -06:00
Greg Fitzgerald
2e921437cd
Rename solana-runtime to sealevel ( #6239 )
...
automerge
2019-10-04 15:02:44 -07:00
Michael Vines
f9f5bc2eb5
More clippy
2019-10-02 21:21:07 -07:00
carllin
35365974bf
Remove serializing all ForkHashes ( #6110 )
2019-09-26 02:01:25 -07:00
carllin
701d90a41d
Remove some AccountStorage Serialization ( #6047 )
...
* Remove serialization of AccountStorageEntry fields
* Add metric for evaluating BankRc serialization time
* Serialize AppendVec current len
* Add dashboard metrics
* Move flush of AppendVecs to packaging thread
2019-09-25 18:07:41 -07:00
sakridge
093b5b5267
Prune fork_hashes with dead forks ( #6085 )
2019-09-25 11:16:14 -07:00
Rob Walker
72fb52ec60
rename balance ( #5984 )
2019-09-23 15:20:45 -07:00
sakridge
19ae556857
hash account state on store ( #5573 )
2019-09-20 13:21:12 -07:00
Sagar Dhawan
c1d788880d
Limit Rayon threadpool threads ( #5871 )
2019-09-12 11:39:39 -07:00
Michael Vines
4e827af392
Remove unnecessary trailing semicolons ( #5636 )
2019-08-23 22:47:54 -07:00
Rob Walker
0ffe7a9c8f
plumb some rent ( #5610 )
...
* plumb some rent
* nits
* fixups
* fixups
* fixups
2019-08-23 14:04:53 -07:00
Michael Vines
3fc5009ef2
Snapshot pipefitting through the validator cli ( #5617 )
...
* Handle 404 errors better
* Snapshot pipefitting through the validator cli
* Add download progress bar
* Log the current entrypoint slot
2019-08-23 13:02:07 -07:00
carllin
087c43b9ef
Add snapshotting integration test ( #5519 )
...
* Add snapshotting integration test
* Update ContactInfo on restart in local cluster nodes
2019-08-21 23:59:11 -07:00