723f9a9b81
Remove unnecessary locking in retransmit stage ( #6276 )
...
* Add more detailed metrics to retransmit
* Remove unnecessary locking and add more metrics
2019-10-08 14:41:16 -07:00
c5e5342325
Rearrange broadcast stats ( #6274 )
2019-10-08 12:50:59 -07:00
788296047a
Increase batch size for recvmmsg() ( #6260 )
...
* Increase batch size for recvmmsg()
* fix tests
* new test
2019-10-08 09:54:49 -07:00
9dceb8ac74
Broadcast/Shredding Metrics ( #6270 )
...
automerge
2019-10-08 01:42:42 -07:00
ac2374e9a1
Shred entries in parallel ( #6180 )
...
* Make shredding more parallel
* Fix erasure tests
* Fix replicator test
* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
e12c577b16
remove verify_hash_internal_state ( #6261 )
2019-10-07 16:38:54 -07:00
ba7efbb136
Retransmit stage optimization, don't copy packets ( #6250 )
2019-10-07 15:33:22 -07:00
17f169f446
BlobFetchStage cleanup post shred work ( #6254 )
2019-10-07 11:08:01 -07:00
4870a2cbac
Panic when a snapshot fails to verify
2019-10-07 08:12:18 +09:00
da7d94d0f0
Retransmit stage benchmark ( #6249 )
2019-10-06 12:56:17 -07:00
cdef065cca
Broadcast Metrics ( #6166 )
...
* Add timing logigng to broadcast
* Shred metrics
* Fixes
2019-10-05 22:51:47 -07:00
5ef012b2c1
Tweak debug to remove unreadable datapoints ( #6060 )
2019-10-04 16:25:22 -07:00
0c3ff6b75c
Cli refactor: vote and storage program functionalities ( #6242 )
...
automerge
2019-10-04 14:18:19 -07:00
7f53737000
Periodically pull from the entrypoint if it's no longer in Crdt ( #6240 )
2019-10-04 14:18:07 -07:00
23ea8ae56b
Optimize retransmit stage ( #6231 )
...
* Optimize retransmit stage
* Remove comment
* Fix test
* Skip iteration to fixup 0 stakes
2019-10-04 11:52:02 -07:00
f9f5bc2eb5
More clippy
2019-10-02 21:21:07 -07:00
f331f1d1e9
Don't forward transaction to self ( #6218 )
2019-10-02 14:07:34 -07:00
211cae5811
Remove dead constants ( #6207 )
2019-10-01 18:22:57 -07:00
774e9df2e5
Finish unfininished slot before processing new slots ( #6197 )
2019-10-01 11:46:14 -07:00
ae7700296d
broadcast_shreds opt ( #6175 )
...
* Don't clone/copy/sort ContactInfo vec
2019-10-01 09:38:29 -07:00
520f7c3e18
Optimize BPF logs ( #6186 )
2019-09-30 14:21:29 -07:00
2c8c2029d8
cli: enforce rent-exemption balance for stake, vote and program accounts in cli ( #6118 )
...
* require minimum balance for stake, vote and program accounts
2019-10-01 01:14:49 +05:30
4f4618441c
split wallet staking commands ( #6168 )
...
* split wallet staking commands
* elide real home
* unit->UNIT for usage
* unit->UNIT, don't try to run SUBCOMMANDS: ;)
* more fixup
* fixups
* actually check
* shellcheck
* preserve #6158 after rebase
* fixup
* test
* too hard
* remove test
2019-09-29 21:18:15 -07:00
e5a7d08966
Add --expected-genesis-blockhash validator argument ( #6174 )
...
automerge
2019-09-29 19:09:24 -07:00
d50aef8404
Add get-epoch-info command ( #6161 )
...
automerge
2019-09-27 22:00:30 -07:00
5637f88aff
Don't try signging coding shred if fec rate is 0 ( #6171 )
...
automerge
2019-09-27 21:58:53 -07:00
c9e58743e7
Prevent subtract overflow panic when slot < MAX_LOCKOUT_HISTORY ( #6135 )
2019-09-26 19:40:18 -06:00
b4da83a3ab
Remove CUDA feature ( #6094 )
2019-09-26 13:36:51 -07:00
a964570b1a
add authorities to stake init ( #6104 )
...
* add authorities to stake init
* fixups
* code review
2019-09-26 13:29:29 -07:00
67d07254c2
Add rent estimation rpc ( #6109 )
...
* server side new rpc endpoint
* client side rpc
* take data_len as usize
Co-Authored-By: Tyera Eulberg <teulberg@gmail.com>
* add test and documentation
2019-09-26 23:27:13 +05:30
701d90a41d
Remove some AccountStorage Serialization ( #6047 )
...
* Remove serialization of AccountStorageEntry fields
* Add metric for evaluating BankRc serialization time
* Serialize AppendVec current len
* Add dashboard metrics
* Move flush of AppendVecs to packaging thread
2019-09-25 18:07:41 -07:00
43795193c4
add authorized parameters to vote api ( #6072 )
...
* add authorized parameters to vote api
* code review
2019-09-25 13:53:49 -07:00
e987d0094f
Move status cache serialization to the Snapshot Packager service ( #6081 )
...
* Move status cache serialization to the Snapshot Packager service
* Minor comment updates
* use ok_or_else instead of ok_or
* satus cache
* Remove assert when snapshot format is wrong
* Fix compile
* Remove slots_to_snapshot from bank forks
* Address review comment
* Remove unused imports
2019-09-25 13:42:19 -07:00
de3765ab70
Add erasure recovery stats to dashboard ( #6079 )
...
automerge
2019-09-24 19:59:42 -07:00
5f079137e5
Remove kvstore ( #6075 )
...
automerge
2019-09-24 19:59:32 -07:00
374b776a3e
Fix using temp file for archive ( #6058 )
...
* Fix using temp file for archive
* Rename the temp archive instead of hardlinking it
2019-09-24 15:24:54 -07:00
5763d63737
Additional tests for should_retransmit_and_persist ( #6062 )
...
automerge
2019-09-24 14:54:10 -07:00
3ed34b571c
Window service is filtering out coding shreds ( #6052 )
...
* Window service is filtering out coding shreds
* update erasure stats to indicate recovery count
* filter out outdated coding shreds
* address review comments
2019-09-24 12:25:25 -07:00
57e90948a8
Remove dead code from cluster_info ( #6051 )
2019-09-24 10:27:59 -07:00
26a20a7e62
nits ( #6032 )
2019-09-24 10:10:49 -07:00
7fa809c16d
Avoid hardlinking as that confuses tar ( #6042 )
2019-09-23 20:12:16 -07:00
84f74807d4
Skip considering banks older than the latest vote slot ( #6037 )
...
automerge
2019-09-23 19:40:03 -07:00
3a9c03cc89
Don't recover coding shreds ( #6034 )
...
* Don't recover coding shreds
* cleanup
2019-09-23 16:24:21 -07:00
62c22c6cb1
Fix really old banks triggering log spam ( #6025 )
2019-09-23 13:59:16 -07:00
dbd337c616
Upgrade to ReedSolomon 4.0 release ( #6026 )
2019-09-23 13:53:52 -07:00
eeda7338cc
Dump tar stdout/err on failure for better debug ( #6024 )
2019-09-23 13:05:09 -07:00
8240d1fe0a
Confidence implementation ( #5993 )
...
* Change confidence parameters
* Add status_cache_ancestors to get all relevant ancestors of a bank including roots from status cache
* Fix and add tests
* Clippy
2019-09-20 19:38:56 -07:00
fd6e7020eb
Fix bank overlapping another bank's broadcast ( #6012 )
2019-09-20 19:37:40 -07:00
a60a3efc1a
Revert "require stake, vote and executable accounts to be rent exempt ( #5928 )" ( #6005 )
...
This reverts commit 11e6197a83
2019-09-20 14:10:39 -07:00
19ae556857
hash account state on store ( #5573 )
2019-09-20 13:21:12 -07:00
3d44cffcda
✨ Beautify metrics datapoint logging ( #5998 )
2019-09-20 12:00:43 -07:00
c9be9acd14
log snapshot time ( #5996 )
2019-09-20 08:03:00 -07:00
11e6197a83
require stake, vote and executable accounts to be rent exempt ( #5928 )
...
* require vote account to be exempt
* make stake account rent exempt
* add rent exempted system instruction
* use rent exemption instruction in vote and stake api
* use rent exempted account while creating executable account
* updating chacha golden hash as instruction data has changed
* rent will be initialized for genesis bank too
2019-09-20 16:52:17 +05:30
d379786c90
Fix bind errors ( #5986 )
...
* Add ability to bind to a common tcp/udp port
* Extend port range for local-net sanity and fix validator executable
2019-09-19 17:16:22 -07:00
ca9d4e34df
Broadcast stage tuning ( #5989 )
2019-09-19 16:29:52 -07:00
5cbd1190b2
transaction batch ( #5962 )
...
* transaction batch
* fixup
2019-09-19 10:06:08 -07:00
0dbf7995b5
Remove unnecessary serialize of shred data ( #5967 )
...
* Remove unnecessary serialize of shred data
* remove obsolete code
* fix golden hash
2019-09-18 20:08:27 -07:00
e0858cfe06
Add parallel shred signing to shredder ( #5964 )
2019-09-18 18:00:07 -07:00
783e8672e7
Removed Shred enum ( #5963 )
...
* Remove shred enum and it's references
* rename ShredInfo to Shred
* clippy
2019-09-18 16:24:30 -07:00
6c4e656795
Remove obsoleted code from shred ( #5954 )
...
* Remove obsoleted code from shred
* fix broken test
2019-09-18 13:56:44 -07:00
b757294864
Add minor performance bump to shredding ( #5956 )
2019-09-18 12:35:52 -07:00
0d4a2c5eb0
simplify poh recorder => broadcast channel ( #5940 )
...
* simplify poh recorder broadcast channel
* fixup
* fixup
2019-09-18 12:16:22 -07:00
2f50d0e145
Refactor confidence from replay stage ( #5938 )
2019-09-17 19:43:40 -07:00
ff608992ee
Replace Shred usage with ShredInfo ( #5939 )
...
* Replace Shred usage with ShredInfo
* Fix tests
* fix clippy
2019-09-17 18:22:46 -07:00
a2595b44c6
test randomize with error ( #5916 )
...
* test randomize with error
* update magic numbers
* fixup
* fixup
* fixup
* no more blobs
* fixup
2019-09-17 15:11:29 -07:00
14cb6353c0
Change erasure ratio to 0.25 and increase data shreds to 16 ( #5931 )
...
* Change erasure ratio to 0.25 and increase data shreds to 16
* Fix case where no coding shreds are requested
2019-09-17 11:59:14 -07:00
e26f68fe62
Get transactions from LockedAccountsResults when possible ( #5923 )
2019-09-17 08:41:56 -07:00
f10438d530
Respect randomized transaction order when unlocking accounts ( #5918 )
2019-09-16 21:45:16 -07:00
7459eb15c3
A new data-structure in shreds for partial deserialization ( #5915 )
...
* A new datastructure in shreds for partial deserialization
* fix chacha golden hash
* fix clippy and address review comments
2019-09-16 20:28:54 -07:00
c44e7ce184
Leaders should not broadcast to replicators ( #5917 )
2019-09-16 17:56:34 -07:00
82615c703b
Switch erasure to solana-reed-solomon-erasure ( #5913 )
...
* Switch to solana-reed-solomon-erasure
* Disable Rayon for solana-reed-solomon-erasure
2019-09-16 16:14:55 -07:00
d5ba90d375
Don't verify blobs that are less than root in window service ( #5901 )
2019-09-16 13:13:53 -07:00
5f54573613
More shred related cleanup ( #5909 )
...
* More shred related cleanup
* fix uncle
2019-09-16 10:28:28 -07:00
c1880e3f3e
Reduce number of shreds per FEC block ( #5908 )
2019-09-15 10:37:12 -07:00
ee791e2e3e
Optimizations to shred writing and signing ( #5890 )
...
* Optimizations to shred writing and signing
* fix broken tests
* fixes
2019-09-14 21:05:54 -07:00
ceacc42126
Call gpu init earlier to force compilation. ( #5902 )
2019-09-14 12:32:57 -07:00
84c8a5bbec
Add replay-stage-mark_dead_slot datapoint ( #5907 )
2019-09-14 08:50:53 -07:00
8135279335
Reduce serialize/deserialize in shred recovery ( #5887 )
2019-09-12 21:52:13 -07:00
c1d788880d
Limit Rayon threadpool threads ( #5871 )
2019-09-12 11:39:39 -07:00
385086359c
Reduce serializations/deserializations of shreds ( #5879 )
2019-09-12 10:10:25 -07:00
1853771930
Add support for SDK sysvar types ( #5876 )
2019-09-10 18:53:02 -07:00
ee4266bc59
Remove banks in locktower not in bank_forks ( #5837 )
...
* Remove unnecessary calculations from collect_vote_lockouts
* Add test for locktower startup from snapshot
2019-09-10 13:58:27 -07:00
b426dfb2c0
Change tx batching in banking process and record ( #5832 )
...
* Change tx batching in banking process and record
* Change batching to reduce impact on replay stage
2019-09-10 11:04:03 -07:00
fd33b27af1
Fix coding shred generator ( #5865 )
2019-09-10 09:35:07 -07:00
7682db4826
Generate coding shreds on the fly based on erasure limits ( #5852 )
...
* Generate coding shreds on the fly based on erasure limits
* fix uncle
2019-09-09 17:26:51 -07:00
7607800d47
Refactor restart function in local cluster to support separate exit and restart functions ( #5845 )
2019-09-08 17:53:34 -07:00
df205f8752
Use ureq instead of influx_db_client ( #5839 )
2019-09-07 12:48:45 -07:00
e8d88f3237
Split SDK's timing.rs ( #5823 )
2019-09-06 14:30:56 -07:00
1f9fde5f7b
ThinClient internal name grooming ( #5800 )
2019-09-06 09:07:40 -07:00
a452249bf3
Use retain on Packets instead of creating new ones ( #5804 )
...
* Use remove on Packets instead of creating a new one
* Fix compile after rebase
2019-09-05 19:16:18 -07:00
3d3b03a123
Verify signature of recovered shred before adding them to blocktree ( #5811 )
...
* Verify signature of recovered shred before adding them to blocktree
* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
05d2eec45c
Remove unnecessary erasure config references ( #5809 )
2019-09-05 14:46:41 -07:00
3237e897d7
Adjust packet batching post-decoupling from blobs ( #5783 )
2019-09-05 11:22:39 -07:00
bd74e63702
Offload remaining confidence cache computation to separate thread ( #5792 )
...
* Move remaining confidence cache computation to separate thread
* Move confidence cache out of bank forks
2019-09-04 23:10:25 -07:00
f78b865cba
Cleanup shreds to remove FirstShred data structure ( #5789 )
...
* Cleanup shreds to remove FirstShred data structure
* Also reduce size used by parent slot information in shred header
* clippy
* fixes
* fix chacha test
2019-09-04 21:06:47 -07:00
7062fe4b47
Refactor Blocktree for clarity and correctness ( #5700 )
...
* Refactor shreds to prevent insertion of any metadata on bad shreds
* Refactor fetching Index in blocktree
* Refactor get_slot_meta_entry
* Re-enable local cluster test
* cleanup
* Add tests for success/fail insertion of coding/data shreds
* Remove assert
* Fix and add tests for should_insert coding and data blobs
2019-09-04 17:14:42 -07:00
02ee2a601c
Further cleanup of blocktree after Blob deprecation ( #5780 )
2019-09-04 12:47:09 -07:00
dfa6238342
Remove unnecessary construction of descendants ( #5742 )
2019-09-04 01:49:42 -07:00
3b0d48e3b8
Remove blocktree blob references ( #5691 )
...
* Remove blocktree blob references
* fixes and cleanup
* replace uninitialized() call with MaybeUninit
* fix bench
2019-09-03 21:32:51 -07:00
62f6a78ccd
Make data plane shred filter parallel again ( #5740 )
2019-09-03 21:50:57 +00:00
Sagar Dhawan
sakridge
Pankaj Garg
carllin
Michael Vines
Tyera Eulberg
sakridge
Jack May
Parth
Rob Walker