Commit Graph

98 Commits

Author SHA1 Message Date
carllin 588c0464b8
Add sampling logic and DuplicateSlotRepairStatus module (#18721) 2021-07-21 11:15:08 -07:00
carllin 8a846b048e
Add AncestorHashesRepair type (#18681) 2021-07-15 19:29:53 -07:00
behzad nouri a0551b4054
persists repair-peers cache across repair service loops (#18400)
The repair-peers cache is reset each time repair service loop runs,
and so computed repeatedly for the same slots:
https://github.com/solana-labs/solana/blob/d2b07dca9/core/src/repair_service.rs#L275

This commit uses an LRU cache to persists repair-peers for each slot.
In addition to LRU eviction rules, in order to avoid re-using outdated
data, each entry also has 10 seconds TTL.
2021-07-07 14:12:09 +00:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
carllin ccc013e134
Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
carllin 96ba2edfeb
Switch EpochSlots to be frozen slots, not completed slots (#17168) 2021-06-03 00:20:00 +00:00
carllin bbcdf073ba
Support out of band dumping of unrooted slots in AccountsDb (#17269)
* Accounts dumping logic

* Add test for interaction between cache flush and remove_unrooted_slot()

* Update comments

* Rename

* renaming

* Add more comments

* Renaming

* Fixup test and bad check
2021-06-02 09:51:10 +00:00
Tyera Eulberg 9a5330b7eb
Move gossip modules into solana-gossip crate (#17352)
* Move gossip modules to solana-gossip

* Update Protocol abi digest due to move

* Move gossip benches and hook up CI

* Remove unneeded Result entries

* Single use statements
2021-05-26 09:15:46 -06:00
sakridge 8e69dd42c1
Add non-default repair nonce values (#16512)
* Track outstanding nonces in repair

* Rework outstanding requests to use lru cache and randomize nonces

Co-authored-by: Carl <carl@solana.com>
2021-04-20 09:37:33 -07:00
behzad nouri b6f231b60e
removes locked pubkey references (#15152) 2021-02-08 02:07:00 +00:00
carllin 5f14f45282
More generic accounts purge functions (#14595)
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-17 20:31:03 -08:00
Michael Vines 7143aaa89b Clippy 2020-12-14 08:03:29 -08:00
sakridge 0d663158d0
Reduce repair_stats debug (#13393) 2020-11-04 10:32:48 -08:00
carllin c8d67aa8eb
Add option for repairing only from trusted validators (#11752)
Co-authored-by: Carl <carl@solana.com>
2020-08-21 00:35:11 -07:00
carllin 7ef50a9352
Move cluster slots update to separate thread (#11523)
* Add cluster_slots_service

Co-authored-by: Carl <carl@solana.com>
2020-08-11 12:48:13 -07:00
carllin a7ea340f22
Track votes from gossip for optimistic confirmation (#11209)
* Add check in cluster_info_vote_listenere to see if optimstic conf was achieved
Add OptimisticConfirmationVerifier

* More fixes

* Fix merge conflicts

* Remove gossip notificatin

* Add dashboards

* Fix rebase

* Count switch votes as well toward optimistic conf

* rename

Co-authored-by: Carl <carl@solana.com>
2020-07-28 09:33:27 +00:00
carllin 6578ad7d08
Speed up local cluster partitioning tests (#11177)
* Fix long local cluster partition tests by skipping slot warmup

Co-authored-by: Carl <carl@solana.com>
2020-07-23 18:50:42 -07:00
carllin 73f3d04798
Add replay votes to gossip vote tracking (#11119)
* Plumb replay vote channel for notifying vote listener of replay votes

* Keep gossip only notification for debugging gossip in the future

Co-authored-by: Carl <carl@solana.com>
2020-07-20 17:29:07 -07:00
carllin 7a14e359d7
Switch to using weighted repair in RepairService (#10735)
* Plumb votes into repair service

* Remove refactoring

* Fix tests

* Switch to using RepairWeight for generating repairs

* Revert "Weight repair slots based on vote stake (#10741)"

This reverts commit cabd0a09c3.

* Update logging

Co-authored-by: Carl <carl@solana.com>
2020-07-10 05:52:54 +00:00
carllin 3f6042d8b3
Add RepairWeight to track votes seen in gossip for weighted repair (#10903)
* Add RepairWeight

Co-authored-by: Carl <carl@solana.com>
2020-07-06 22:49:40 -07:00
carllin 28e15a63e5
Repair timing (#10913)
Co-authored-by: Carl <carl@solana.com>
2020-07-03 18:37:01 -07:00
carllin f17ac70bb2
Add weighted traversal (#10877)
Co-authored-by: Carl <carl@solana.com>
2020-07-02 14:33:04 -07:00
sakridge d9b389f510
Reduce logging lines (#10835) 2020-06-29 15:57:28 -07:00
Greg Fitzgerald 50b3fa83a0
Move BankCommitmentCache to solana_runtime (#10816)
* Remove Blockstore member variable from BlockCommitmentCache

* Hoist is_confirmed_rooted() to its only caller

BlockCommitmentCache no longer depends on Blockstore

* Move BlockCommitmentCache to solana_runtime
2020-06-25 22:06:58 -06:00
carllin 764c7b0403
Disable entire codepath (#10767)
Co-authored-by: Carl <carl@solana.com>
2020-06-24 12:09:55 -07:00
sakridge cabd0a09c3
Weight repair slots based on vote stake (#10741)
* Weight repair slots based on vote stake

* Add test
2020-06-22 20:27:45 -07:00
carllin a33fef9af2
Add repair breakdown by slot and index (#10717)
* Slot full logging

* Repair stats logging

Co-authored-by: Carl <carl@solana.com>
2020-06-19 18:28:15 -07:00
Greg Fitzgerald 0550b893b0
Fix typos (#10675)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-06-17 20:54:52 -07:00
Greg Fitzgerald 6ee222363e
Move BankForks to solana_runtime (#10637)
* Move BankForks to solana_runtime

* Update imports
2020-06-17 15:27:03 +00:00
Greg Fitzgerald 8dd5384d6d
Split commitment module (#10541)
automerge
2020-06-12 17:16:10 -07:00
carllin 97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header (#10109)
* Add nonce to shreds/repairs

* Add data shred size to header

Co-authored-by: Carl <carl@solana.com>
2020-05-19 12:38:18 -07:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030)
automerge
2020-05-15 09:35:43 -07:00
Jack May eb1acaf927
Remove archiver and storage program (#9992)
automerge
2020-05-14 18:22:47 -07:00
carllin 3442f36f8a
Repair alternate versions of dead slots (#9805)
Co-authored-by: Carl <carl@solana.com>
2020-05-05 14:07:21 -07:00
sakridge fa20963b93
Revert shred fs (#9712)
* Revert "Untar is called for shred archives that do not exist. (#9565)"

This reverts commit 729cb5eec6.

* Revert "Dont insert shred payload into rocksdb (#9366)"

This reverts commit 5ed39de8c5.
2020-04-24 15:04:23 -07:00
carllin bab3502260
Push down cluster_info lock (#9594)
* Push down cluster_info lock

* Rework budget decrement

Co-authored-by: Carl <carl@solana.com>
2020-04-21 12:54:45 -07:00
carllin bcfd379f32
Simplify EpochSlots update (#9545)
Co-authored-by: Carl <carl@solana.com>
2020-04-16 19:32:19 -07:00
anatoly yakovenko 5ed39de8c5
Dont insert shred payload into rocksdb (#9366)
automerge
2020-04-16 18:20:55 -07:00
sakridge c67703e7a3
Make repair metrics less chatty (#9094) 2020-03-29 14:43:58 -07:00
carllin 076fef5e57
Update Cluster Slots to support multiple threads (#9071)
Co-authored-by: Carl <carl@solana.com>
2020-03-25 18:09:19 -07:00
anatoly yakovenko 9a79be5ca0
Use cluster information about slots to prioritize repair (#8820)
automerge
2020-03-12 17:34:46 -07:00
anatoly yakovenko f64ab49307
Cluster has no way to know which slots are available (#8732)
automerge
2020-03-11 21:31:50 -07:00
carllin f47a789b15
Add find_incomplete_slots (#8654)
* Add find_incomplete_slots

* Add live slots iterator
2020-03-05 10:58:00 -08:00
carllin dec3da8f9d
Add orphan iterator (#8636) 2020-03-04 18:10:30 -08:00
Justin Starry 021d0a46f8
Move docs from book/ to docs/ (#8469)
automerge
2020-02-26 07:11:38 -08:00
Pankaj Garg 0d5c1239c6
Update epoch slots to include all missing slots (#8276)
* Update epoch slots to include all missing slots

* new test for compress/decompress

* address review comments

* limit cache based on size, instead of comparing roots
2020-02-17 12:39:30 -08:00
carllin d3712dd26d Factor repair from gossip (#8044) 2020-02-11 13:11:48 -07:00
Pankaj Garg 1d06aa3b31
Remove repairman as its spamming cluster with unwanted repairs (#8193)
* Remove repairman as its spamming cluster with unwanted repairs

* remove obsolete test
2020-02-10 17:00:00 -08:00
carllin fe590da3b6
Revert "Factor repair from gossip (#8044)" (#8143)
This reverts commit e61257695f.
2020-02-06 11:44:20 -08:00
carllin e61257695f
Factor repair from gossip (#8044) 2020-01-31 14:23:50 -08:00