87b2525e03
Limit maximum number of shreds in a slot to 32K ( #7584 )
...
* Limit maximum number of shreds in a slot to 32K
* mark dead slot replay as fatal error
2019-12-30 07:42:09 -08:00
e98132fd76
Move window insertion to separate thread ( #7397 )
...
* Move window insertion to separate thread
2019-12-19 00:15:49 -08:00
d94041e98d
Allow coding shred index to be different than data shred index ( #7438 )
...
* Allow coding shred index to be different than data shred index
* move fec_set_index to shred's common header
* fix bench
2019-12-12 16:50:29 -08:00
6bfe0fca1f
Add a version field to shreds ( #7023 )
...
* Add a version field to shreds
* Clippy
* Fix Chacha Golden
* Fix shredder bench compile
* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
79d7090867
Remove obsolete references to Blob ( #6957 )
...
* Remove the name "blob" from archivers
* Remove the name "blob" from broadcast
* Remove the name "blob" from Cluset Info
* Remove the name "blob" from Repair
* Remove the name "blob" from a bunch more places
* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
f108f483b7
Remove Blobs and switch to Packets ( #6937 )
...
* Remove Blobs and switch to Packets
* Fix some gossip messages not respecting MTU size
* Failure to serialize is not fatal
* Add log macros
* Remove unused extern
* Apparently macro use is required
* Explicitly scope macro
* Fix test compile
2019-11-14 10:24:53 -08:00
43e2301e2c
Fix roots overrunning broadcast ( #6884 )
...
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
a3a830e1ab
Delete Service trait ( #6921 )
2019-11-13 11:12:09 -07:00
30a08f4282
Cleanup ledger macros ( #6916 )
...
automerge
2019-11-13 07:14:09 -08:00
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
d1daeb44e6
Remove custom stack_size() ( #6844 )
2019-11-08 17:11:07 -07:00
8e81bc1b49
Fix pinning ( #6604 )
...
Remove Deref implementations and add more pass-throughs to the PinnedVec
wrapper.
Warm recyclers
set_pinnable
2019-11-07 19:48:33 -08:00
0ace79939b
Add reference tick to data shreds ( #6772 )
...
* Add reference tick to data shreds
* fix tests
2019-11-06 13:27:58 -08:00
50a17fc00b
Use Slot and Epoch type aliases instead of raw u64 ( #6693 )
...
automerge
2019-11-02 00:38:30 -07:00
385b4ce959
Get rid of verified packets and use the Meta::discard flag ( #6674 )
...
* get rid of verified packets and use the disabled meta field everywhere
2019-11-01 14:23:03 -07:00
e8e5ddc55d
Verify number of hashes for each block of entries ( #6262 )
...
* Verify number of hashes for each block of entries
* Fix blocktree processor tick check
* Rebase once more
2019-10-31 16:38:50 -04:00
34a9619806
SigVerify stage for shreds. ( #6563 )
2019-10-28 16:07:51 -07:00
6efaaa9d7a
Blocktree metrics ( #6527 )
...
* Add metrics for blocktree performance
* Plumb metrics through window service
2019-10-26 16:15:59 -07:00
364583ea5c
Fix copying packets in Window Service ( #6429 )
...
* Fix copying packets in Window Service
* Parallelize over batches instead of within batches
2019-10-21 12:04:52 -07:00
9232057e95
Rename replicator to archiver ( #6464 )
...
* Rename replicator to archiver
* cargo fmt
* Fix grammar
2019-10-21 11:29:37 -06:00
badeb4d31a
Rework shred headers to fix position of signature ( #6451 )
...
* Rework shred headers to fix position of signature
* fix clippy
2019-10-18 22:55:59 -07:00
5468be2ef9
Add solana-ledger crate ( #6415 )
...
automerge
2019-10-18 09:28:51 -07:00
298ba34c3c
Add flag to mark a packet as discarded ( #6427 )
2019-10-17 16:26:29 -07:00
7cf90766a3
add epoch_schedule sysvar ( #6256 )
...
* add epoch_schedule sysvar
* book sheesh!
2019-10-08 22:34:26 -07:00
ac2374e9a1
Shred entries in parallel ( #6180 )
...
* Make shredding more parallel
* Fix erasure tests
* Fix replicator test
* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
f9f5bc2eb5
More clippy
2019-10-02 21:21:07 -07:00
5763d63737
Additional tests for should_retransmit_and_persist ( #6062 )
...
automerge
2019-09-24 14:54:10 -07:00
3ed34b571c
Window service is filtering out coding shreds ( #6052 )
...
* Window service is filtering out coding shreds
* update erasure stats to indicate recovery count
* filter out outdated coding shreds
* address review comments
2019-09-24 12:25:25 -07:00
783e8672e7
Removed Shred enum ( #5963 )
...
* Remove shred enum and it's references
* rename ShredInfo to Shred
* clippy
2019-09-18 16:24:30 -07:00
ff608992ee
Replace Shred usage with ShredInfo ( #5939 )
...
* Replace Shred usage with ShredInfo
* Fix tests
* fix clippy
2019-09-17 18:22:46 -07:00
a2595b44c6
test randomize with error ( #5916 )
...
* test randomize with error
* update magic numbers
* fixup
* fixup
* fixup
* no more blobs
* fixup
2019-09-17 15:11:29 -07:00
d5ba90d375
Don't verify blobs that are less than root in window service ( #5901 )
2019-09-16 13:13:53 -07:00
c1d788880d
Limit Rayon threadpool threads ( #5871 )
2019-09-12 11:39:39 -07:00
385086359c
Reduce serializations/deserializations of shreds ( #5879 )
2019-09-12 10:10:25 -07:00
a452249bf3
Use retain on Packets instead of creating new ones ( #5804 )
...
* Use remove on Packets instead of creating a new one
* Fix compile after rebase
2019-09-05 19:16:18 -07:00
3d3b03a123
Verify signature of recovered shred before adding them to blocktree ( #5811 )
...
* Verify signature of recovered shred before adding them to blocktree
* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
f78b865cba
Cleanup shreds to remove FirstShred data structure ( #5789 )
...
* Cleanup shreds to remove FirstShred data structure
* Also reduce size used by parent slot information in shred header
* clippy
* fixes
* fix chacha test
2019-09-04 21:06:47 -07:00
62f6a78ccd
Make data plane shred filter parallel again ( #5740 )
2019-09-03 21:50:57 +00:00
2510f3d352
Remove extra call to serialize in shred verify ( #5698 )
2019-08-27 19:28:00 -07:00
47535b9ff1
Use serialize_into to fill in shreds instead of writing one byte at a time ( #5695 )
...
automerge
2019-08-27 17:11:24 -07:00
f1d58f980b
Ignore retransmit channel error ( #5680 )
...
automerge
2019-08-27 12:41:04 -07:00
4ac1213c9c
Integrate coding shreds and recovery ( #5625 )
...
* Integrate coding shreds and recovery
* More tests for shreds and some fixes
* address review comments
* fixes to code shred generation
* unignore tests
* fixes to recovery
2019-08-26 18:27:45 -07:00
4e827af392
Remove unnecessary trailing semicolons ( #5636 )
2019-08-23 22:47:54 -07:00
0dc0594aaa
Fixes to repair and orphan logic for data shreds ( #5587 )
2019-08-21 15:27:42 -07:00
4798e7fa73
Integrate data shreds ( #5541 )
...
* Insert data shreds in blocktree and database
* Integrate data shreds with rest of the code base
* address review comments, and some clippy fixes
* Fixes to some tests
* more test fixes
* ignore some local cluster tests
* ignore replicator local cluster tests
2019-08-20 17:16:06 -07:00
0fde19239b
Rate limit counter metrics points to one per second ( #5496 )
...
* Rate limit counter metrics points to one per second
* Remove old env var
* Test that metrics counter is incrementing
* Fix typo
2019-08-12 18:15:34 -04:00
4f86c0b74a
Rate limit transaction counters ( #5447 )
...
* Rate limit transaction counters
* @sakridge feedback
* Set default high metrics rate for multinode demo
* Fix tests
* Swap defaults and fix env var tests
* Only set metrics rate if not already set
2019-08-08 17:05:06 -04:00
937816e67b
Post warning if window service isn't receiving any data ( #5269 )
...
automerge
2019-07-24 12:46:10 -07:00
a383ea532f
Implement new Index Column ( #4827 )
...
* Implement new Index Column
* Correct slicing of blobs
* Mark coding blobs as coding when they're recovered
* Prevent broadcast stages from mixing coding and data blobs in blocktree
* Mark recovered blobs as present in the index
* Fix indexing error in recovery
* Fix broken tests, and some bug fixes
* increase min stack size for coverage runs
2019-07-10 11:08:17 -07:00
ef7022d638
Refactor replicators to not block on startup ( #4932 )
...
* Refactor replicators to not block on startup
* Ignore setup failure
2019-07-08 10:17:25 -07:00
d6737b8cc9
Set epoch schedule in set_root in leader schedule cache ( #4821 )
2019-06-26 00:19:48 -07:00
36c9e22e3d
Revert "Dynamic erasure ( #4653 )"
...
This reverts commit ada4d16c4c
2019-06-20 20:53:03 -07:00
ada4d16c4c
Dynamic erasure ( #4653 )
...
Remove erasure-related constants
Remove unneeded `Iterator::collect` call
Document erasure module
Randomize coding blobs used for repair
2019-06-20 20:27:41 -05:00
cc48773b03
Add "download from replicator" utility ( #4709 )
...
automerge
2019-06-17 18:12:13 -07:00
4c6a6d63bf
add MiningPools, fund validator MiningPools from inflation ( #4676 )
...
* add MiningPool fund validator MinigPools from inflation
* fixup
* finish rename of MINIMUM_SLOT_LENGTH to MINIMUM_SLOTS_PER_EPOCH
* deterministic miningpool location
* point_value, not credit_value... use f64
2019-06-14 11:38:37 -07:00
8c1b9a0b67
Data plane verification ( #4639 )
...
* Add signature to blob
* Change Signable trait to support returning references to signable data
* Add signing to broadcast
* Verify signatures in window_service
* Add testing for signatures to erasure
* Add RPC for getting current slot, consume RPC call in test_repairman_catchup for more deterministic results
2019-06-12 16:43:05 -07:00
348c2263ba
Remove genesis blockhash ( #4471 )
...
* Remove genesis blockhash
* Remove genesis blockhash from tests
* Fix golden
2019-05-29 17:29:02 -07:00
d772a27936
Plumb ClusterInfoRepairListener ( #4428 )
...
automerge
2019-05-24 19:20:09 -07:00
cfe5afd34c
_id => _pubkey variable renaming ( #4419 )
...
* wallet: rename *_account_id to *_account_pubkey
* s/from_id/from_pubkey/g
* s/node_id/node_pubkey/g
* s/stake_id/stake_pubkey/g
* s/voter_id/voter_pubkey/g
* s/vote_id/vote_pubkey/g
* s/delegate_id/delegate_pubkey/g
* s/account_id/account_pubkey/g
* s/to_id/to_pubkey/g
* s/my_id/my_pubkey/g
* cargo fmt
* s/staker_id/staker_pubkey/g
* s/mining_pool_id/mining_pool_pubkey/g
* s/leader_id/leader_pubkey/g
* cargo fmt
* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
2ed77b040a
create_genesis_block() now returns a struct ( #4403 )
2019-05-22 20:39:00 -07:00
b2ce5dc9f5
Adjust log level for counter metrics ( #4323 )
2019-05-17 07:00:06 -07:00
88c2d0fad4
Add genesis blockhash to replicators and blob filter for window ( #4275 )
...
* Add genesis blockhash to replicators and blob filter for window
* Fixes to mining submission and ledger download
* Add todo over sleep
* Update log
2019-05-13 21:19:51 -07:00
3bd921264a
Move EpochSchedule into own module ( #4272 )
2019-05-13 16:24:32 -07:00
7501ed65e5
Initialize and Update EpochSlots in RepairService ( #4255 )
...
* Initialize EpochSlots in RepairService
* Fix flaky test
2019-05-13 15:37:50 -07:00
f567877d1d
Cleanup metrics ( #4230 )
2019-05-10 08:33:58 -07:00
575a0e318b
Add newly completed slots signal to Blocktree ( #4225 )
...
* Add channel to blocktree for communicating when slots are completed
* Refactor RepairService options into a RepairStrategy
2019-05-09 14:10:04 -07:00
bba94c43b9
Add BankForks to RepairService ( #4223 )
...
automerge
2019-05-08 18:51:43 -07:00
8e400fc4bd
rework genesis (passive staking groundwork) ( #4187 )
...
* rework genesis
* fixup
2019-05-07 11:16:22 -07:00
1a259d4a3f
Prevent Requests/Caching of leader schedules for epochs beyond confirmed roots ( #4033 )
...
automerge
2019-04-30 13:23:21 -07:00
73f250f03a
Make minimum warmup period 32 slots long ( #4031 )
...
* Make minimum warmup period 32 slots long
* PR fixes
2019-04-29 15:26:52 -07:00
87f1bd58b9
Reduce submit frequency of counters that are very frequent and killing metrics DB ( #4027 )
2019-04-26 20:37:40 -07:00
0cbac26591
Add genesis blockhash to blobs ( #3953 )
2019-04-23 16:24:44 -07:00
43f7cd8149
Fix Retransmit slamming the leader with its own blobs ( #3938 )
2019-04-22 18:41:01 -07:00
4b04c37c36
verify that blobs match a known leader for the slot ( #3927 )
...
* validate that blobs match a known leader for the slot
* clippy
2019-04-22 15:21:10 -07:00
349e8a9462
Ensure forwarded Blobs don't break Erasure ( #3907 )
2019-04-20 16:44:06 -07:00
bd1db51e07
delete db_window.rs, move contents to window_service, clean up process_blobs ( #3746 )
2019-04-14 18:52:05 -07:00
483cc2fa4e
Support old repair strategy for reparing slots in a range for supporting replicators ( #3665 )
2019-04-08 12:46:23 -07:00
ec9e13d1f4
Add repair slot range
...
Use default impl RepairSlotRange
2019-03-13 10:15:03 -07:00
ac226c3e14
Remove superfluous set_leader() usage
2019-03-08 19:59:54 -08:00
a3cab470d3
Rename ClusterInfo::new_with_keypair() to ClusterInfo::new()
2019-03-07 13:05:42 -08:00
bb93504965
Rename ClusterInfo::new() to ClusterInfo::new_with_invalid_keypair()
2019-03-07 13:05:42 -08:00
6ab6e6cb9b
Clean up exit flag handing across TVU
2019-03-04 21:26:50 -08:00
5f5d779ee1
Move src/ into core/src. Top-level crate is now called solana-workspace
2019-03-02 09:52:18 -08:00
Pankaj Garg
carllin
Sagar Dhawan
Greg Fitzgerald
Justin Starry
Michael Vines
sakridge
anatoly yakovenko
Rob Walker
Mark E. Sinclair
Jack May