carllin
534244b322
Fix set_roots to use cached bank parents instead of searching blocktree ( #4466 )
2019-05-29 09:43:22 -07:00
carllin
591fd72e0b
Implement listener for serving repairs through Repairman protocol ( #4306 )
...
* Make listener for serving repairs through Repairman protocol
2019-05-23 03:10:16 -07:00
carllin
1a77486f8e
Make RootedSlotsIterator for traversing slots on the root fork ( #4361 )
2019-05-20 23:09:00 -07:00
carllin
f1e5edee14
Modify Roots Column To Support Multiple Roots ( #4321 )
...
* Fix 1) Roots column family to handle storing multiple slots, 2) Store all slots on the rooted path in the roots column family
2019-05-20 19:04:18 -07:00
Pankaj Garg
9476fe5ce3
Use log levels for datapoint metrics ( #4335 )
...
* Use log levels for datapoint metrics
* address review comments
* fix cyclomatic complexity
2019-05-17 17:34:05 -07:00
Pankaj Garg
e920191de0
Rate limit metrics per log level ( #4313 )
...
* Rate limit metrics per log level
* fix tests
2019-05-16 22:27:05 -07:00
carllin
abd7f6b090
Change slot_meta_iterator() to return an iterator not a cursor ( #4303 )
...
automerge
2019-05-15 18:28:23 -07:00
carllin
a79fbbafc9
SlotMeta is incorrectly updated on insertion of erasure blobs ( #4289 )
...
* Fix put_coding_blob_bytes to properly update slotmetas and chaining
2019-05-15 00:28:31 -07:00
carllin
e20a8329d3
Add API to iterate over slot's blobs ( #4276 )
2019-05-13 22:04:54 -07:00
Mark E. Sinclair
796000e96f
Improve erasure metrics ( #4268 )
...
* Improve erasure metrics
* Simplify metrics submission
2019-05-13 16:04:43 -05:00
Jack May
f567877d1d
Cleanup metrics ( #4230 )
2019-05-10 08:33:58 -07:00
carllin
9881820444
RepairService saves db_iterator instead of reconstructing on every search ( #4242 )
2019-05-09 19:57:51 -07:00
Rob Walker
81fa69d347
Revert "Create bank snapshots ( #3671 )" ( #4243 )
...
This reverts commit abf2b300da
.
2019-05-09 19:27:27 -07:00
Sathish
abf2b300da
Create bank snapshots ( #3671 )
...
* Be able to create bank snapshots
* fix clippy
* load snapshot on start
* regenerate account index from the storage
* Remove rc feature dependency
* cleanup
* save snapshot for slot 0
2019-05-09 19:27:06 -07:00
carllin
575a0e318b
Add newly completed slots signal to Blocktree ( #4225 )
...
* Add channel to blocktree for communicating when slots are completed
* Refactor RepairService options into a RepairStrategy
2019-05-09 14:10:04 -07:00
Mark E. Sinclair
ed48d8323c
Reduce locking in Blocktree ( #4075 )
...
* Reduce lock contention in blocktree
* Store root slot in separate column
2019-05-03 16:46:02 -05:00
Mark E. Sinclair
916458e132
Change erasure set size to 8:8 ( #4129 )
...
* Change erasure set size to 8:8
* Change tests to be agnostic to exact set size and ratio
* Add convenience methods for setting presence
2019-05-02 19:04:40 -05:00
Pankaj Garg
87f1bd58b9
Reduce submit frequency of counters that are very frequent and killing metrics DB ( #4027 )
2019-04-26 20:37:40 -07:00
Mark E. Sinclair
8b1724bb70
Serialize blocktree writes by locking the database ( #4008 )
...
Move several private methods to free functions
2019-04-26 10:52:10 -05:00
carllin
c4d9dff590
Cherry-pick #3934 ( #3982 )
...
* Fix inserting bogus is_last blobs into blocktree
* Check for pre-existing blob before insert
* Ignore test that performs concurrent writes on blocktree as that is not supported
2019-04-25 00:04:49 -07:00
Mark E. Sinclair
9a40ad76bd
Fix race in erasure metadata tracking ( #3962 )
...
* Fix erasure metadata race condition
* make erasure return the underlying error without wrapping it in the `solana::Error` type
* Add metric for erasure failures
* add tests to `ErasureMeta` indexing logic
* Add test to ensure erasure recovery failures don't cause panics
2019-04-24 17:53:01 -05:00
Sagar Dhawan
349e8a9462
Ensure forwarded Blobs don't break Erasure ( #3907 )
2019-04-20 16:44:06 -07:00
Rob Walker
3ab492ccf8
save erasure set size, initialize coding blob erasures with that size ( #3910 )
...
* save erasure set size, initialize coding blob erasures with that size
* fixup
* fixup
2019-04-19 20:22:51 -07:00
Rob Walker
6eac5951ed
Revert "Revert "revert-revert-erasure and erasure fixes ( #3833 )" ( #3855 )" ( #3889 )
...
This reverts commit 596f611ede
.
2019-04-18 21:56:43 -07:00
Pankaj Garg
596f611ede
Revert "revert-revert-erasure and erasure fixes ( #3833 )" ( #3855 )
...
This reverts commit 6bef16a6a1
.
2019-04-17 18:04:30 -07:00
Rob Walker
6bef16a6a1
revert-revert-erasure and erasure fixes ( #3833 )
...
* fix erasure, more tests for full blobs, more metrics
* Revert "Revert "Use Rust erasure library and turn on erasure (#3768 )" (#3827 )"
This reverts commit 4b8cb72977
.
2019-04-17 15:13:54 -07:00
sakridge
4b8cb72977
Revert "Use Rust erasure library and turn on erasure ( #3768 )" ( #3827 )
...
This reverts commit b9bb5af4a5
.
2019-04-17 12:52:12 -07:00
Mark E. Sinclair
b9bb5af4a5
Use Rust erasure library and turn on erasure ( #3768 )
...
* split out erasure into new crate; add implementation using rust reed-solomon-library
* Track erasures with a &[bool] instead of indexes
* fix bug that reported the number of erasures incorrectly
* Introduce erasure `Session` for consistent config
* Increase test coverage; fix bugs
* Add ability to remove blobs from erasure meta tracking. test added
* Track deletion of coding blobs in blocktree via ErasureMeta. Added to
test
* Remove unused functions in blocktree
* add randomness to recovery thread to exercise recovery due to either new
data or coding blobs
* Add unit test for ErasureMeta index handling
* Re-enable test in broadcast stage
2019-04-16 23:00:24 -05:00
Rob Walker
64c6f05da2
persist set_root() and use it in blocktree_processor to limit squashes ( #3782 )
...
* rename locktower's slot to epoch
* persist set_root() and use it in blocktree_processor to limit squashes
2019-04-15 13:12:28 -07:00
Rob Walker
dd005fb50e
fix broadcast to *always* call erasure generation, simplify generator, test slot reset better ( #3764 )
2019-04-14 18:12:37 -07:00
Rob Walker
efd19b07e7
implement erasure-based recovery inside blocktree ( #3739 )
...
* implement recover in blocktree
* erasures metric
* erasure metrics only
* fixup
2019-04-11 14:14:57 -07:00
carllin
20aa4434e2
Fix repair ( #3581 )
...
Add DetachedHeads repair protocol
Add DetachedHeads repair test
Repair starting from root
2019-04-06 19:41:22 -07:00
Mark E. Sinclair
d90b8c331d
Refactor blocktree storage abstraction ( #3588 )
2019-04-02 16:58:07 -05:00
Greg Fitzgerald
fcef54d062
Add a constructor to generate random pubkeys
2019-03-31 16:23:18 -06:00
carllin
9369ea86ea
Track detached slots in blocktree ( #3536 )
...
* Add contains_all_parents flag to SlotMeta to prep for tracking detached heads
* Add new DetachedHeads column family
* Remove has_complete_parents
* Fix test
2019-03-29 16:07:24 -07:00
Mark E. Sinclair
50b0a5ae83
Blocktree+Erasure tests of basic erasure functionality ( #3535 )
...
* Remove WindowSlot; add Blocktree based tests to erasure
2019-03-28 01:55:51 -05:00
Mark
16ff4ac1a8
Simplify storage interface in blocktree ( #3522 )
2019-03-27 01:36:39 -05:00
Stephen Akridge
4fb0782892
Rename blocktree SlotMeta::is_rooted to is_connected
...
is_rooted is now is_connected and (still) indicates the set of connected
completed slots. 'rooted' slot terminology is used for a different
meaning in bank_forks and replay_stage.
2019-03-20 14:43:39 -07:00
Stephen Akridge
682b1b89b3
Adjust for vector of entries in blobs.
2019-03-19 13:49:48 -07:00
carllin
60437a8dcb
Multiple entries per blob ( #3337 )
...
* Pack multiple entries into blob
* fix tests
* Add test for deserializing multi-entry blobs in blocktree
* more test fixes
2019-03-17 18:48:23 -07:00
Mark
656fb173f9
Extract kvstore into separate crate ( #3327 )
...
* extract kvstore into new crate
* add kvstore crate to CI publishing list
2019-03-15 18:42:47 -05:00
Rob Walker
9f02a8d3d0
remove ticks_per_slot from blocktree ( #3297 )
2019-03-14 15:18:37 -07:00
Mark
56b0ba2601
KvStore - A data-store to support BlockTree ( #2897 )
...
* Mostly implement key-value store and add integration points
Essential key-value store functionality is implemented, needs more work to be integrated, tested, and activated.
Behind the `kvstore` feature.
2019-03-11 17:53:14 -05:00
anatoly yakovenko
0c592c52f6
Wake up replay stage when the poh bank is cleared. ( #3211 )
...
* wake up replay stage when the poh bank is cleared
* bump ticks per second
* Increase ticks per slot to match faster tick rate
* Remove check that working bank must be the bank for the greatest slot
* Make start_leader() skip starting TPU for slots we've already been leader for
2019-03-11 13:58:23 -07:00
Rob Walker
9e9c0785e7
groom broadcast ( #3170 )
2019-03-07 09:43:42 -08:00
Rob Walker
b9e878ee80
slot_height considered harmful ( #3135 )
...
* slot_height considered harmful
* fix test_tick_slot_epoch_indexes
2019-03-05 14:18:29 -08:00
Rob Walker
43bab23651
remove duplicate child creation ( #3100 )
...
* remove duplicate child creation
* resurrect test for partial slot
* simplify blocktree_processor some more (no tick_height, yay!)
* ensure frozen
2019-03-04 19:22:23 -08:00
Michael Vines
a94880574b
block_hash => blockhash
2019-03-02 12:13:30 -07:00
Michael Vines
258cf21416
Purge remaining last_id (now called block_hash)
2019-03-02 12:13:30 -07:00
Michael Vines
5f5d779ee1
Move src/ into core/src. Top-level crate is now called solana-workspace
2019-03-02 09:52:18 -08:00