Commit Graph

26 Commits

Author SHA1 Message Date
carllin e20a8329d3
Add API to iterate over slot's blobs (#4276) 2019-05-13 22:04:54 -07:00
Jack May f567877d1d
Cleanup metrics (#4230) 2019-05-10 08:33:58 -07:00
Mark E. Sinclair ed48d8323c
Reduce locking in Blocktree (#4075)
* Reduce lock contention in blocktree

* Store root slot in separate column
2019-05-03 16:46:02 -05:00
Mark E. Sinclair 916458e132
Change erasure set size to 8:8 (#4129)
* Change erasure set size to 8:8

* Change tests to be agnostic to exact set size and ratio

* Add convenience methods for setting presence
2019-05-02 19:04:40 -05:00
Mark E. Sinclair 8b1724bb70
Serialize blocktree writes by locking the database (#4008)
Move several private methods to free functions
2019-04-26 10:52:10 -05:00
carllin c4d9dff590
Cherry-pick #3934 (#3982)
* Fix inserting bogus is_last blobs into blocktree

* Check for pre-existing blob before insert

* Ignore test that performs concurrent writes on blocktree as that is not supported
2019-04-25 00:04:49 -07:00
Mark E. Sinclair 9a40ad76bd
Fix race in erasure metadata tracking (#3962)
* Fix erasure metadata race condition

* make erasure return the underlying error without wrapping it in the `solana::Error` type

* Add metric for erasure failures

* add tests to `ErasureMeta` indexing logic

* Add test to ensure erasure recovery failures don't cause panics
2019-04-24 17:53:01 -05:00
Rob Walker 3ab492ccf8
save erasure set size, initialize coding blob erasures with that size (#3910)
* save erasure set size, initialize coding blob erasures with that size

* fixup

* fixup
2019-04-19 20:22:51 -07:00
Rob Walker 6eac5951ed
Revert "Revert "revert-revert-erasure and erasure fixes (#3833)" (#3855)" (#3889)
This reverts commit 596f611ede.
2019-04-18 21:56:43 -07:00
Pankaj Garg 596f611ede
Revert "revert-revert-erasure and erasure fixes (#3833)" (#3855)
This reverts commit 6bef16a6a1.
2019-04-17 18:04:30 -07:00
Rob Walker 6bef16a6a1
revert-revert-erasure and erasure fixes (#3833)
* fix erasure, more tests for full blobs, more metrics

* Revert "Revert "Use Rust erasure library and turn on erasure (#3768)" (#3827)"

This reverts commit 4b8cb72977.
2019-04-17 15:13:54 -07:00
sakridge 4b8cb72977
Revert "Use Rust erasure library and turn on erasure (#3768)" (#3827)
This reverts commit b9bb5af4a5.
2019-04-17 12:52:12 -07:00
Mark E. Sinclair b9bb5af4a5
Use Rust erasure library and turn on erasure (#3768)
* split out erasure into new crate; add implementation using rust reed-solomon-library

* Track erasures with a &[bool] instead of indexes

* fix bug that reported the number of erasures incorrectly

* Introduce erasure `Session` for consistent config

* Increase test coverage; fix bugs

* Add ability to remove blobs from erasure meta tracking. test added

* Track deletion of coding blobs in blocktree via ErasureMeta. Added to
test

* Remove unused functions in blocktree

* add randomness to recovery thread to exercise recovery due to either new
data or coding blobs

* Add unit test for ErasureMeta index handling

* Re-enable test in broadcast stage
2019-04-16 23:00:24 -05:00
Rob Walker 64c6f05da2
persist set_root() and use it in blocktree_processor to limit squashes (#3782)
* rename locktower's slot to epoch

* persist set_root() and use it in blocktree_processor to limit squashes
2019-04-15 13:12:28 -07:00
Rob Walker e57a0ab05d
test some bits (#3762) 2019-04-14 17:10:30 -07:00
Rob Walker efd19b07e7
implement erasure-based recovery inside blocktree (#3739)
* implement recover in blocktree

* erasures metric

* erasure metrics only

* fixup
2019-04-11 14:14:57 -07:00
carllin 20aa4434e2
Fix repair (#3581)
Add DetachedHeads repair protocol

Add DetachedHeads repair test

Repair starting from root
2019-04-06 19:41:22 -07:00
Mark E. Sinclair d90b8c331d
Refactor blocktree storage abstraction (#3588) 2019-04-02 16:58:07 -05:00
carllin 9369ea86ea
Track detached slots in blocktree (#3536)
* Add contains_all_parents flag to SlotMeta to prep for tracking detached heads

* Add new DetachedHeads column family

* Remove has_complete_parents

* Fix test
2019-03-29 16:07:24 -07:00
Mark 16ff4ac1a8 Simplify storage interface in blocktree (#3522) 2019-03-27 01:36:39 -05:00
carllin 60437a8dcb
Multiple entries per blob (#3337)
* Pack multiple entries into blob

* fix tests

* Add test for deserializing multi-entry blobs in blocktree

* more test fixes
2019-03-17 18:48:23 -07:00
Mark 656fb173f9
Extract kvstore into separate crate (#3327)
* extract kvstore into new crate

* add kvstore crate to CI publishing list
2019-03-15 18:42:47 -05:00
Mark a15927f8d0
make KvStore Send+Sync (#3317) 2019-03-15 13:01:34 -05:00
Mark be1511a7ff
delete accidental file (#3316) 2019-03-15 11:28:08 -05:00
Rob Walker 9f02a8d3d0
remove ticks_per_slot from blocktree (#3297) 2019-03-14 15:18:37 -07:00
Mark 56b0ba2601
KvStore - A data-store to support BlockTree (#2897)
* Mostly implement key-value store and add integration points

Essential key-value store functionality is implemented, needs more work to be integrated, tested, and activated.

Behind the `kvstore` feature.
2019-03-11 17:53:14 -05:00