Commit Graph

66 Commits

Author SHA1 Message Date
Jack May eb1acaf927
Remove archiver and storage program (#9992)
automerge
2020-05-14 18:22:47 -07:00
carllin 3442f36f8a
Repair alternate versions of dead slots (#9805)
Co-authored-by: Carl <carl@solana.com>
2020-05-05 14:07:21 -07:00
sakridge fa20963b93
Revert shred fs (#9712)
* Revert "Untar is called for shred archives that do not exist. (#9565)"

This reverts commit 729cb5eec6.

* Revert "Dont insert shred payload into rocksdb (#9366)"

This reverts commit 5ed39de8c5.
2020-04-24 15:04:23 -07:00
carllin bab3502260
Push down cluster_info lock (#9594)
* Push down cluster_info lock

* Rework budget decrement

Co-authored-by: Carl <carl@solana.com>
2020-04-21 12:54:45 -07:00
carllin bcfd379f32
Simplify EpochSlots update (#9545)
Co-authored-by: Carl <carl@solana.com>
2020-04-16 19:32:19 -07:00
anatoly yakovenko 5ed39de8c5
Dont insert shred payload into rocksdb (#9366)
automerge
2020-04-16 18:20:55 -07:00
sakridge c67703e7a3
Make repair metrics less chatty (#9094) 2020-03-29 14:43:58 -07:00
carllin 076fef5e57
Update Cluster Slots to support multiple threads (#9071)
Co-authored-by: Carl <carl@solana.com>
2020-03-25 18:09:19 -07:00
anatoly yakovenko 9a79be5ca0
Use cluster information about slots to prioritize repair (#8820)
automerge
2020-03-12 17:34:46 -07:00
anatoly yakovenko f64ab49307
Cluster has no way to know which slots are available (#8732)
automerge
2020-03-11 21:31:50 -07:00
carllin f47a789b15
Add find_incomplete_slots (#8654)
* Add find_incomplete_slots

* Add live slots iterator
2020-03-05 10:58:00 -08:00
carllin dec3da8f9d
Add orphan iterator (#8636) 2020-03-04 18:10:30 -08:00
Justin Starry 021d0a46f8
Move docs from book/ to docs/ (#8469)
automerge
2020-02-26 07:11:38 -08:00
Pankaj Garg 0d5c1239c6
Update epoch slots to include all missing slots (#8276)
* Update epoch slots to include all missing slots

* new test for compress/decompress

* address review comments

* limit cache based on size, instead of comparing roots
2020-02-17 12:39:30 -08:00
carllin d3712dd26d Factor repair from gossip (#8044) 2020-02-11 13:11:48 -07:00
Pankaj Garg 1d06aa3b31
Remove repairman as its spamming cluster with unwanted repairs (#8193)
* Remove repairman as its spamming cluster with unwanted repairs

* remove obsolete test
2020-02-10 17:00:00 -08:00
carllin fe590da3b6
Revert "Factor repair from gossip (#8044)" (#8143)
This reverts commit e61257695f.
2020-02-06 11:44:20 -08:00
carllin e61257695f
Factor repair from gossip (#8044) 2020-01-31 14:23:50 -08:00
Greg Fitzgerald b5dba77056 Rename blocktree to blockstore (#7757)
automerge
2020-01-13 13:13:52 -08:00
Sagar Dhawan a95d37ea25
Fix repair slowness when most peers are unable to serve requests (#7287)
* Fix repair when most peers are incapable of serving requests

* Add a test for getting the lowest slot in blocktree

* Replace some more u64s with Slot
2019-12-05 11:25:13 -08:00
Jack May d2ed921bc6
Cleanup nightly warnings (#7055) 2019-11-19 20:15:37 -08:00
Sagar Dhawan 79d7090867
Remove obsolete references to Blob (#6957)
* Remove the name "blob" from archivers

* Remove the name "blob" from broadcast

* Remove the name "blob" from Cluset Info

* Remove the name "blob" from Repair

* Remove the name "blob" from a bunch more places

* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
carllin 43e2301e2c
Fix roots overrunning broadcast (#6884)
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
Greg Fitzgerald a3a830e1ab
Delete Service trait (#6921) 2019-11-13 11:12:09 -07:00
Greg Fitzgerald 30a08f4282 Cleanup ledger macros (#6916)
automerge
2019-11-13 07:14:09 -08:00
Sagar Dhawan 95d6586dd7
Remove debug datapoint that isn't being plotted (#6873) 2019-11-11 14:25:25 -08:00
Sagar Dhawan 816b2d7ff8
Tune repair to be less aggressive (#6868) 2019-11-11 13:12:22 -08:00
Sagar Dhawan 67d1e2903c
Upgrade Repair be more intelligent and agressive (#6789)
* Upgrade Repair be more intelligent and agressive

* Fix u64 casts

* Fix missing bracket

* Add 1 second delay to test to allow repair to kick in
2019-11-07 11:08:09 -08:00
Michael Vines 50a17fc00b Use Slot and Epoch type aliases instead of raw u64 (#6693)
automerge
2019-11-02 00:38:30 -07:00
Greg Fitzgerald 9232057e95
Rename replicator to archiver (#6464)
* Rename replicator to archiver

* cargo fmt

* Fix grammar
2019-10-21 11:29:37 -06:00
Greg Fitzgerald 2636418659
Move blocktree_processor to solana_ledger (#6460)
* Drop core::result dependency in bank_forks

* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
Greg Fitzgerald 5468be2ef9 Add solana-ledger crate (#6415)
automerge
2019-10-18 09:28:51 -07:00
carllin dd66d16fdb
Broadcast final shred for slots that are interrupted (#6269)
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
Rob Walker 7cf90766a3
add epoch_schedule sysvar (#6256)
* add epoch_schedule sysvar

* book sheesh!
2019-10-08 22:34:26 -07:00
carllin ac2374e9a1
Shred entries in parallel (#6180)
* Make shredding more parallel

* Fix erasure tests

* Fix replicator test

* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
sakridge 5ef012b2c1
Tweak debug to remove unreadable datapoints (#6060) 2019-10-04 16:25:22 -07:00
sakridge 211cae5811
Remove dead constants (#6207) 2019-10-01 18:22:57 -07:00
Michael Vines 3d44cffcda
Beautify metrics datapoint logging (#5998) 2019-09-20 12:00:43 -07:00
Rob Walker a2595b44c6
test randomize with error (#5916)
* test randomize with error

* update magic numbers

* fixup

* fixup

* fixup

* no more blobs

* fixup
2019-09-17 15:11:29 -07:00
Pankaj Garg 3d3b03a123
Verify signature of recovered shred before adding them to blocktree (#5811)
* Verify signature of recovered shred before adding them to blocktree

* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
Pankaj Garg 3b0d48e3b8
Remove blocktree blob references (#5691)
* Remove blocktree blob references

* fixes and cleanup

* replace uninitialized() call with MaybeUninit

* fix bench
2019-09-03 21:32:51 -07:00
carllin 8b9c3a2561
Blocktree last_root to enforce a slot floor (#5593)
* Add last_root to blocktree

* Don't repair earlier than last_root

* Add integration test to make sure blocktree floor is enforced
2019-08-27 15:09:41 -07:00
Pankaj Garg 4ac1213c9c
Integrate coding shreds and recovery (#5625)
* Integrate coding shreds and recovery

* More tests for shreds and some fixes

* address review comments

* fixes to code shred generation

* unignore tests

* fixes to recovery
2019-08-26 18:27:45 -07:00
Pankaj Garg 0dc0594aaa
Fixes to repair and orphan logic for data shreds (#5587) 2019-08-21 15:27:42 -07:00
Sagar Dhawan cc48773b03 Add "download from replicator" utility (#4709)
automerge
2019-06-17 18:12:13 -07:00
carllin aa3c00231a Fix should_update check to update EpochSlots in gossip (#4435)
automerge
2019-05-25 06:44:40 -07:00
carllin d772a27936 Plumb ClusterInfoRepairListener (#4428)
automerge
2019-05-24 19:20:09 -07:00
carllin 6a9e0bc593 Change EpochSlots to use BtreeSet so that serialization/deserialization returns the same order (#4404)
automerge
2019-05-23 03:50:41 -07:00
carllin f1e5edee14
Modify Roots Column To Support Multiple Roots (#4321)
* Fix 1) Roots column family to handle storing multiple slots, 2) Store all slots on the rooted path in the roots column family
2019-05-20 19:04:18 -07:00
Pankaj Garg 9476fe5ce3
Use log levels for datapoint metrics (#4335)
* Use log levels for datapoint metrics

* address review comments

* fix cyclomatic complexity
2019-05-17 17:34:05 -07:00