b66c03667c
Log for threshold failure ( #7008 )
2019-11-17 17:10:16 -08:00
c3926e6af0
|solana-gossip spy| no longer requires an entrypoint ( #6999 )
2019-11-16 14:16:28 -07:00
70322d1ff8
Add error logging to dead slots ( #7000 )
2019-11-16 02:54:51 -08:00
0ca943f49b
RecyclerCache for shred_sigverify ( #6986 )
...
automerge
2019-11-15 12:16:56 -08:00
59413b3124
Fix rules for fork selection ( #6906 )
...
automerge
2019-11-15 08:36:33 -08:00
006c39380a
Display 'none' instead of 0.0.0.0 ( #6973 )
2019-11-14 20:24:35 -07:00
852a2146ab
Add Blocktree api to get transactions by slot ( #6966 )
...
* Add blocktree method to get confirmed-block txs
* Clean up use statements
* Add test, and fmt
* Plumb new blocktree method into getConfirmedBlock
2019-11-14 16:34:39 -07:00
03f7f0d18c
Rename getBlock to getConfirmedBlock; remove getBlocksSince ( #6961 )
...
automerge
2019-11-14 13:14:42 -08:00
79d7090867
Remove obsolete references to Blob ( #6957 )
...
* Remove the name "blob" from archivers
* Remove the name "blob" from broadcast
* Remove the name "blob" from Cluset Info
* Remove the name "blob" from Repair
* Remove the name "blob" from a bunch more places
* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
e7f63cd336
Upgrade to rust 1.39.0 ( #6939 )
...
* Upgrade to rust 1.39.0
* 1.39.0 clippy
2019-11-14 12:27:01 -07:00
f108f483b7
Remove Blobs and switch to Packets ( #6937 )
...
* Remove Blobs and switch to Packets
* Fix some gossip messages not respecting MTU size
* Failure to serialize is not fatal
* Add log macros
* Remove unused extern
* Apparently macro use is required
* Explicitly scope macro
* Fix test compile
2019-11-14 10:24:53 -08:00
43e2301e2c
Fix roots overrunning broadcast ( #6884 )
...
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
7b05b3dbb3
rent collector improvments ( #6888 )
...
* avoid account copying + pre-empt rent
* adding support for base rent
2019-11-14 10:56:49 +05:30
4fc767b3f6
Move version! from core:: to clap_utils:: ( #6944 )
...
* Move version! from core to clap-utils
* Completely move version! from core:: to clap_utils::
* rustfmt
* Do remaining transition after rebase
2019-11-14 13:10:38 +09:00
a3a830e1ab
Delete Service trait ( #6921 )
2019-11-13 11:12:09 -07:00
fcc2874591
Remove/address some TODOs ( #6923 )
2019-11-13 09:43:15 -07:00
30a08f4282
Cleanup ledger macros ( #6916 )
...
automerge
2019-11-13 07:14:09 -08:00
86faa3f995
Properly type RpcClient::get_version() ( #6919 )
2019-11-12 22:01:04 -07:00
148a58865e
Make creating new snapshot.tar.bz2 truly-atomic ( #6902 )
2019-11-12 14:21:56 -07:00
3faeb7fa79
Rename solana-netutil to solana-net-utils for consistency ( #6895 )
...
* sed -i -e 's/netutil/net_utils/g' $(git grep --files-with-matches netutil :**.rs)
* sed -i -e 's/netutil/net-utils/g' $(git grep --files-with-matches netutil)
* git mv netutil/ net-utils
* Tweak a bit
* Fix rustfmt & clippy
2019-11-12 13:37:13 -07:00
bb00904fc8
add rent reserve for bootstrap stakes ( #6876 )
...
* genesis investor stakes
* assert rent is sufficient for these bootstrappers
2019-11-12 12:33:40 -08:00
5903339c17
feat: return bank/block info with block-related results ( #6716 )
2019-11-12 14:49:41 -05:00
c6b108ef4f
Don't panic in sdk/ when genesis fails to load ( #6892 )
2019-11-12 10:24:49 -07:00
7e82450d7b
Serialize transaction in proper wire format instead of json ( #6889 )
2019-11-12 10:45:10 -05:00
95d6586dd7
Remove debug datapoint that isn't being plotted ( #6873 )
2019-11-11 14:25:25 -08:00
816b2d7ff8
Tune repair to be less aggressive ( #6868 )
2019-11-11 13:12:22 -08:00
4be646c695
discover() by gossip sockaddr instead of just by gossip ip address ( #6865 )
2019-11-11 12:42:58 -07:00
cc6e1ea200
Stub out `getBlocksSince` and `getBlock` methods ( #6853 )
...
* Add getBlocksSince rpc method, and initial stub of getBlock method
* Return test transactions from getBlock method
* clippy
* Add comment on get_block method
2019-11-11 13:18:34 -05:00
cfab36cb1d
Include channel and commit info in the version of pre-release builds ( #6819 )
2019-11-10 22:39:13 -07:00
5835b3b8eb
Increase timeout when confirming airdrop for max commitment ( #6858 )
...
* Increase timeout when confirming airdrop for max commitment
* Add commitment to airdrop rpc trace
* Flip commitment check
2019-11-10 12:20:52 -05:00
1ddf90ed08
Compress contact_info_trace() output to improve CI log rendering ( #6852 )
2019-11-09 01:12:18 -07:00
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
02058ea699
Reject blocks with invalid last ticks in replay stage ( #6833 )
...
* Reject blocks with invalid last ticks in replay stage
* slot_full
2019-11-08 20:21:54 -05:00
d1daeb44e6
Remove custom stack_size() ( #6844 )
2019-11-08 17:11:07 -07:00
efdfc5c327
Remove TODOs ( #6843 )
2019-11-08 16:43:18 -07:00
9c00ad9ff2
Remove some low-hanging TODOs ( #6839 )
2019-11-08 16:41:36 -07:00
da425cc225
Don't insert coding shreds into blocktree on leader ( #6831 )
2019-11-08 13:54:23 -08:00
b800642fa4
Add new fork log message for when the node is leader for consistency ( #6808 )
2019-11-08 12:30:25 -07:00
5bd05fba09
require `to` account signature ( #6658 )
...
* require to signature
* fixing invocation to create_account
* fix create_account references
* address review comment
* whacking bugs in tests
* fixing stake program tests
2019-11-08 15:57:35 +05:30
8e81bc1b49
Fix pinning ( #6604 )
...
Remove Deref implementations and add more pass-throughs to the PinnedVec
wrapper.
Warm recyclers
set_pinnable
2019-11-07 19:48:33 -08:00
b64b54f48f
unfork dalek ed25519 ( #6776 )
2019-11-07 17:08:10 -08:00
67d1e2903c
Upgrade Repair be more intelligent and agressive ( #6789 )
...
* Upgrade Repair be more intelligent and agressive
* Fix u64 casts
* Fix missing bracket
* Add 1 second delay to test to allow repair to kick in
2019-11-07 11:08:09 -08:00
65de227520
Don't panic on packet data ( #6769 )
2019-11-06 14:32:37 -08:00
0ace79939b
Add reference tick to data shreds ( #6772 )
...
* Add reference tick to data shreds
* fix tests
2019-11-06 13:27:58 -08:00
b3a75a60a4
Use rooted bank by default in rpc bank selection ( #6759 )
...
* Name anonymous parameters for clarity
* Add CommitmentConfig to select bank for rpc
* Add commitment information to jsonrpc docs
* Update send_and_confirm retries as per commitment defaults
* Pass CommitmentConfig into client requests; also various 'use' cleanup
* Use _with_commitment methods to speed local_cluster tests
* Pass CommitmentConfig into Archiver in order to enable quick confirmations in local_cluster tests
* Restore solana ping speed
* Increase wallet-sanity timeout to account for longer confirmation time
2019-11-06 14:15:00 -07:00
5e8668799c
Fewer recyclers. ( #6770 )
...
automerge
2019-11-06 12:35:51 -08:00
67f636545a
Refactor sigverify to stage for signing shreds on the GPU ( #6635 )
...
automerge
2019-11-06 10:52:30 -08:00
7ff2a44a63
Make last shred for an interrupted slot signed + typed ( #6760 )
2019-11-06 08:25:17 -07:00
b5074d8577
Enable JSON RPC request/response logging by default ( #6758 )
2019-11-06 08:23:13 -07:00
24102a7435
Allow voting on empty banks ( #6719 )
...
* Allow votes on empty banks
* Remove making first bank is_delta true, no longer necessary for idling
* Remove votable from ledger tool
2019-11-06 01:02:26 -08:00
9614d17024
Limit deserialization of data coming off the wire ( #6751 )
...
* Limit deserialization of data coming off the wire
* Feedback and cleanup
2019-11-06 00:07:57 -08:00
09e648f957
ledger-tool/: Include full validator voting history in fork-graph ( #6756 )
2019-11-05 19:40:00 -07:00
8f91b5aab3
Add threshold to repairman for same slot ( #6728 )
2019-11-05 12:48:45 -08:00
c312d4fba0
Calculate proofs collected and don't encrypt if there are none ( #6698 )
2019-11-05 11:38:50 -08:00
7203036e3e
Adjust nofiles within Blocktree::open() for all ledger/ users ( #6737 )
...
automerge
2019-11-05 11:18:49 -08:00
c6931dcb07
Remove credit-only account handling ( #6726 )
...
* Renaming
- credit-only/credit-debit to read-only/read-write
- debitable to writable
* Remove credit handling, making credit-only accounts read-only
* Update programs to remove deprecated credit-only account designation
* Use readonly and writable instead of underscored types
2019-11-05 09:38:35 -07:00
fba1af6ea9
ledger-tool can now load a ledger snapshot ( #6729 )
2019-11-04 22:14:55 -07:00
b825d04597
Pull perf into a separate module. ( #6718 )
...
automerge
2019-11-04 20:13:43 -08:00
4d52f47f87
Move get_bank_forks() into ledger/ so its available for use by ledger-tool/ ( #6720 )
2019-11-04 19:10:06 -07:00
f54cfcdb8f
Store and persists full stack of tower votes in gossip ( #6695 )
...
* vote array
wip
wip
wip
update
gossip index should match tower index
tests build
clippy
test index after expired vote
test
bank specific last vote sync time
* verify
* we are likely to see many more warnings about old votes now
2019-11-04 16:19:54 -08:00
57983980a7
Lower verify-batch-size to debug ( #6722 )
...
automerge
2019-11-04 16:00:59 -08:00
33f4aaf3fd
Rename confidence to commitment ( #6714 )
2019-11-04 16:44:27 -07:00
c138d692b1
Show all ports for nodes in gossip table ( #6717 )
...
* Show all ports for nodes in gossip table
* review comments
2019-11-04 15:05:08 -08:00
5e3697807c
Fail gracefully if AVX support is missing ( #6705 )
2019-11-04 11:03:39 -07:00
568475e2db
Fix incorrectly signed CrdsValues ( #6696 )
2019-11-03 10:07:51 -08:00
9ea398416e
Sign shreds on the GPU ( #6595 )
...
* sign gpu shreds
* wip
* checks
* tests build
* test
* tests
* test
* nits
* sign cpu test
* write out the sigs in parallel
* clippy
* cpu test
* prepare secret for gpu
* woot!
* update
* bump perf libs
2019-11-02 06:23:14 -07:00
50a17fc00b
Use Slot and Epoch type aliases instead of raw u64 ( #6693 )
...
automerge
2019-11-02 00:38:30 -07:00
a57f6b70da
Fix swapped repair and forwards addrs ( #6691 )
...
automerge
2019-11-01 16:01:42 -07:00
385b4ce959
Get rid of verified packets and use the Meta::discard flag ( #6674 )
...
* get rid of verified packets and use the disabled meta field everywhere
2019-11-01 14:23:03 -07:00
2d67962c2f
Send repairman shreds to the repair socket ( #6671 )
2019-10-31 18:23:50 -07:00
e8e5ddc55d
Verify number of hashes for each block of entries ( #6262 )
...
* Verify number of hashes for each block of entries
* Fix blocktree processor tick check
* Rebase once more
2019-10-31 16:38:50 -04:00
59f3dc3b6b
Fix PohRecorder Metrics ( #6644 )
...
* Update Poh Recorder Dashboard
* Update PohRecorder logging
2019-10-30 18:55:29 -07:00
fa12a5f70b
kill rent calculator ( #6625 )
2019-10-30 16:25:12 -07:00
38491c8c4b
Reduce verify-batch-size log ( #6623 )
2019-10-30 13:41:11 -07:00
801337a422
Refactor Weighted Shuffle ( #6614 )
...
automerge
2019-10-29 21:02:11 -07:00
ef3aa2731c
Fix Weighted Best calculation ( #6606 )
...
automerge
2019-10-29 17:04:11 -07:00
34a9619806
SigVerify stage for shreds. ( #6563 )
2019-10-28 16:07:51 -07:00
579a02529d
Fix unnecessarily copying shreds in broadcast stage ( #6588 )
...
* Optimize coalesce_shreds to not explictly clone
* Remove Coalesce Shreds altogether
* fn no longer needs clippy exception
2019-10-28 14:58:27 -07:00
243fa6cf63
Shred gpu sigverify ( #6520 )
...
Implement APIs for verifying shred signatures on the GPU.
2019-10-28 10:29:38 -07:00
6efaaa9d7a
Blocktree metrics ( #6527 )
...
* Add metrics for blocktree performance
* Plumb metrics through window service
2019-10-26 16:15:59 -07:00
53c7be32b6
Add more retransmit and streamer stats ( #6534 )
2019-10-24 19:27:19 -07:00
397ea05aa7
spy nodes are now gossip entrypoints ( #6532 )
2019-10-24 15:35:33 -07:00
b4119c454a
credit_only credits forwarding ( #6509 )
...
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
03d29a8311
Async poh verify ( #6353 )
...
* Async poh verify
* Up ticks_per_s to 160
GPU poh verify needs shorter poh sequences or it takes forever to
verify. Keep slot time the same at 400ms.
* Fix stats
* Don't halt on ticks
* Increase retries for local_cluster tests and make repairman test serial
2019-10-23 12:11:04 -07:00
35cc74ef25
Add GenesisBlock::OperatingMode to control how cluster features are activated ( #6430 )
2019-10-23 11:50:10 -07:00
0b2d4f32fa
feat: get epoch schedule rpc, update cli ( #6500 )
2019-10-22 16:41:18 -04:00
e6438098e1
Increase archiver polling timeout ( #6501 )
...
automerge
2019-10-22 12:15:55 -07:00
b38bf90de7
Deshred blocks in parallel ( #6461 )
...
* Deshred in parallel
* Add tests for corrupt slots and parallel deshred
* Rename load_blocktree_entries to load_blocktree_entries_with_shred_count
2019-10-21 16:15:10 -07:00
84e911361a
Use constants instead of lazy_static for shred header sizes ( #6472 )
2019-10-21 12:46:16 -07:00
364583ea5c
Fix copying packets in Window Service ( #6429 )
...
* Fix copying packets in Window Service
* Parallelize over batches instead of within batches
2019-10-21 12:04:52 -07:00
9232057e95
Rename replicator to archiver ( #6464 )
...
* Rename replicator to archiver
* cargo fmt
* Fix grammar
2019-10-21 11:29:37 -06:00
2636418659
Move blocktree_processor to solana_ledger ( #6460 )
...
* Drop core::result dependency in bank_forks
* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
31e9074ae5
Rename leader_after_slots to leader_after_n_slots ( #6459 )
2019-10-19 23:28:33 -04:00
e2c316d2d0
system_instruction_processor updates ( #6448 )
...
* zero lamport account creation
* whack create_user_account, take 2
* target->to
* ..
* ..
* update chacha golden
* update chacha golden
* ..
* ..
2019-10-19 18:23:27 -07:00
74ee88d9bc
Add storage stage and bank_forks tests to integration ( #6458 )
2019-10-19 12:09:45 -07:00
badeb4d31a
Rework shred headers to fix position of signature ( #6451 )
...
* Rework shred headers to fix position of signature
* fix clippy
2019-10-18 22:55:59 -07:00
785c2574cd
Check that transaction fee-payer is a debitable account ( #6454 )
...
automerge
2019-10-18 20:39:05 -07:00
1a77f7ce3b
Change to 0x7f which is a valid short_vec len ( #6455 )
...
automerge
2019-10-18 19:56:48 -07:00
6e7dccbbfb
Add an error enum to snapshot_utils ( #6453 )
2019-10-18 19:16:06 -06:00
32bfced6a4
Add offset checks for sigverify ( #6452 )
...
* Add offset checks for sigverify
* decode_len returning error instead of unwrap
2019-10-18 17:52:59 -07:00
e81ba8e79f
Split snapshot_package module ( #6447 )
...
automerge
2019-10-18 14:58:16 -07:00
a19ffb353d
Don't hide serialization errors ( #6443 )
...
automerge
2019-10-18 13:35:05 -07:00
193c9a08e0
Reject TXs when there is a mismatch ( #6236 )
...
automerge
2019-10-18 09:48:35 -07:00
5468be2ef9
Add solana-ledger crate ( #6415 )
...
automerge
2019-10-18 09:28:51 -07:00
9cf9de6044
Remove the Cursor struct ( #6426 )
2019-10-18 09:18:36 -06:00
27c0d30a07
Fix logging ( #6417 )
2019-10-18 02:06:41 -07:00
6c33c3a5ba
Update shred tests to use specific error codes ( #6428 )
...
automerge
2019-10-17 22:50:38 -07:00
298ba34c3c
Add flag to mark a packet as discarded ( #6427 )
2019-10-17 16:26:29 -07:00
24846b7b61
Don't use BlocktreeError from Shredder ( #6423 )
...
automerge
2019-10-17 15:44:15 -07:00
9e52d11ad0
Remove Backend trait ( #6407 )
2019-10-17 15:19:27 -06:00
d3068c3918
Remove circular dependencies in core ( #6408 )
...
* Remove core::result dependency from blocktree
* Remove core::result dependency from shred
* Move Packet from core::packet to sdk::packet
This way we don't need to split perf_libs yet.
* Disable packet when compiling BPF programs
2019-10-17 11:37:30 -06:00
a931ad40c8
Remove unused code in entry ( #6414 )
...
automerge
2019-10-17 09:59:40 -07:00
83c1831a01
Fix replay stage test ( #6406 )
2019-10-16 15:41:43 -07:00
ccb4e32ee0
ReplayStage metrics ( #6358 )
...
* ReplayStage metrics
* Add more metrics
* Refactor get_slot_entries_with_shred_count() to detect wasted work
* Update dashboard
* Update broadcast slots to micros
* Add broadcast dashboard
2019-10-16 14:32:18 -07:00
7e6e7e8406
Remove special handling of first ledger tick ( #6263 )
...
* Remove special handling of first ledger tick
* Fix subtraction overflow
* @garious feedback
* Back to height
* More tick_height name changes
* Fix off-by-one
* Fix leader tick error
* Fix merge conflict
* Fix recently added test
2019-10-16 15:53:11 -04:00
e267dfacdd
Stabilize some banking stage tests ( #6251 )
...
* Stabilize some banking stage tests
Fixes #5660
* Fix CI...
* clean up
* Fix ci
* Address review nits
* Use bank.max_tick_height due to off-by-one for no PohRecord's clearing bank
* Fix CI...
* Use bank.max_tick_height() instead for clarity
2019-10-16 12:37:27 -07:00
f4c5da3c72
Fix unaligned read of short_vec pubkey_size in sigverify ( #6388 )
...
automerge
2019-10-16 11:09:17 -07:00
a258e1e0b3
Fix flaky test_recv_mmsg_batch_size ( #6399 )
...
automerge
2019-10-16 11:01:41 -07:00
33052c1dd2
Cleanup shred header structures ( #6378 )
...
automerge
2019-10-15 20:48:45 -07:00
20e800230f
Don't deserialize coding header for data shreds ( #6367 )
...
* Don't deserialize coding hdr for data shreds
* review comments
* fix tests
2019-10-15 15:18:23 -07:00
1afc527919
Lower cluster_info-num_nodes datapoint ( #6368 )
2019-10-15 14:42:19 -07:00
dfca2b510b
Lower shred/receiver stats ( #6365 )
...
too many messages
2019-10-15 11:43:52 -07:00
f8ccd90eeb
Add ForkConfidenceCache methods ( #6359 )
...
automerge
2019-10-14 22:14:20 -07:00
88bb55ffd2
Add get_vote_accounts() to RPC client
2019-10-14 20:38:51 -07:00
5508ac6272
Add root slot to getVoteAccounts
2019-10-14 20:38:51 -07:00
5d99853502
Add getBlockConfidence rpc endpoint ( #6350 )
...
automerge
2019-10-14 15:24:10 -07:00
3313b2ff58
Fetch stage batching of forwarded txs ( #6349 )
...
automerge
2019-10-14 13:32:29 -07:00
322fcea6e5
More fullnode to validator renaming ( #6337 )
2019-10-11 13:30:52 -06:00
5650231df3
Increase buffer size for erasure meta DB column ( #6335 )
2019-10-11 12:18:11 -07:00
bf9c815b9e
Increase Index column buffers ( #6268 )
2019-10-10 23:17:39 -07:00
364781366a
Use sendmmsg for broadcasting shreds ( #6325 )
...
* Replace packet with slice of data in sendmmsg
* fixes
* fix bench
2019-10-10 19:38:48 -07:00
c6e4641781
Remove many uses of legacy term 'fullnode' ( #6324 )
2019-10-10 17:33:00 -06:00
753bd77b41
Use multicast to send retransmit packets ( #6319 )
2019-10-10 15:02:36 -07:00
1960ea8ed7
Add benches for shredding and poh ( #6307 )
...
* Add benches for shredding and poh
* ignore poh bench
* Factor out Poh bench as separate function
2019-10-10 14:00:24 -07:00
1b775044f7
Use multiple retransmit stage threads/sockets ( #6279 )
2019-10-10 13:24:03 -07:00
570b98c7bc
Multicast same packet to multiple destinations via sendmmg ( #6316 )
...
* Implement multicast same packet to multiple destinations using sendmmsg()
2019-10-10 13:09:15 -07:00
1f345ce2d9
Don't grab keypair from cluster info on every iteration of broadccast ( #6303 )
2019-10-09 17:36:45 -07:00
ed85aa43a4
Implement sendmmsg() API ( #6304 )
...
* Implement sendmmsg()
* fixes
2019-10-09 17:06:56 -07:00
4b0250192a
Remove remnants of the cuda feature flag ( #6298 )
...
automerge
2019-10-09 16:09:36 -07:00
dd66d16fdb
Broadcast final shred for slots that are interrupted ( #6269 )
...
* Broadcast final shred for slots that are interrupted
2019-10-09 16:07:18 -07:00
32312f3c16
Do not retransmit Repair responses ( #6284 )
...
* Do not retransmit Repair responses
* Add a test
* Refactor neighboring functionality
2019-10-09 13:11:19 -07:00
2db83e1a21
Remove greedy fetch in shred_fetch stage ( #6278 )
...
* Remove greedy fetch in shred_fetch stage
* cleanup
2019-10-09 10:36:05 -07:00
7cf90766a3
add epoch_schedule sysvar ( #6256 )
...
* add epoch_schedule sysvar
* book sheesh!
2019-10-08 22:34:26 -07:00
f2ee01ace3
Fix blocktree processor entry callback test ( #6285 )
2019-10-08 20:38:05 -04:00
5e31565574
Expand blocktree processor options ( #6248 )
...
* Refactor blocktree processor args and support full leader cache
* Add entry callback option
* Rename num_threads to override_num_threads
* Add test for entry callback
* Refactor cached leader schedule changes
* Add tests for blocktree process options
* Refactor test
* @mvines feedback
2019-10-08 17:58:49 -04:00
723f9a9b81
Remove unnecessary locking in retransmit stage ( #6276 )
...
* Add more detailed metrics to retransmit
* Remove unnecessary locking and add more metrics
2019-10-08 14:41:16 -07:00
c5e5342325
Rearrange broadcast stats ( #6274 )
2019-10-08 12:50:59 -07:00
788296047a
Increase batch size for recvmmsg() ( #6260 )
...
* Increase batch size for recvmmsg()
* fix tests
* new test
2019-10-08 09:54:49 -07:00
9dceb8ac74
Broadcast/Shredding Metrics ( #6270 )
...
automerge
2019-10-08 01:42:42 -07:00
ac2374e9a1
Shred entries in parallel ( #6180 )
...
* Make shredding more parallel
* Fix erasure tests
* Fix replicator test
* Remove UnfinishedSlotInfo
2019-10-08 00:42:51 -07:00
e12c577b16
remove verify_hash_internal_state ( #6261 )
2019-10-07 16:38:54 -07:00
ba7efbb136
Retransmit stage optimization, don't copy packets ( #6250 )
2019-10-07 15:33:22 -07:00
17f169f446
BlobFetchStage cleanup post shred work ( #6254 )
2019-10-07 11:08:01 -07:00
4870a2cbac
Panic when a snapshot fails to verify
2019-10-07 08:12:18 +09:00
da7d94d0f0
Retransmit stage benchmark ( #6249 )
2019-10-06 12:56:17 -07:00
cdef065cca
Broadcast Metrics ( #6166 )
...
* Add timing logigng to broadcast
* Shred metrics
* Fixes
2019-10-05 22:51:47 -07:00
5ef012b2c1
Tweak debug to remove unreadable datapoints ( #6060 )
2019-10-04 16:25:22 -07:00
0c3ff6b75c
Cli refactor: vote and storage program functionalities ( #6242 )
...
automerge
2019-10-04 14:18:19 -07:00
7f53737000
Periodically pull from the entrypoint if it's no longer in Crdt ( #6240 )
2019-10-04 14:18:07 -07:00
23ea8ae56b
Optimize retransmit stage ( #6231 )
...
* Optimize retransmit stage
* Remove comment
* Fix test
* Skip iteration to fixup 0 stakes
2019-10-04 11:52:02 -07:00
f9f5bc2eb5
More clippy
2019-10-02 21:21:07 -07:00
f331f1d1e9
Don't forward transaction to self ( #6218 )
2019-10-02 14:07:34 -07:00
211cae5811
Remove dead constants ( #6207 )
2019-10-01 18:22:57 -07:00
774e9df2e5
Finish unfininished slot before processing new slots ( #6197 )
2019-10-01 11:46:14 -07:00
ae7700296d
broadcast_shreds opt ( #6175 )
...
* Don't clone/copy/sort ContactInfo vec
2019-10-01 09:38:29 -07:00
520f7c3e18
Optimize BPF logs ( #6186 )
2019-09-30 14:21:29 -07:00
2c8c2029d8
cli: enforce rent-exemption balance for stake, vote and program accounts in cli ( #6118 )
...
* require minimum balance for stake, vote and program accounts
2019-10-01 01:14:49 +05:30
4f4618441c
split wallet staking commands ( #6168 )
...
* split wallet staking commands
* elide real home
* unit->UNIT for usage
* unit->UNIT, don't try to run SUBCOMMANDS: ;)
* more fixup
* fixups
* actually check
* shellcheck
* preserve #6158 after rebase
* fixup
* test
* too hard
* remove test
2019-09-29 21:18:15 -07:00
e5a7d08966
Add --expected-genesis-blockhash validator argument ( #6174 )
...
automerge
2019-09-29 19:09:24 -07:00
d50aef8404
Add get-epoch-info command ( #6161 )
...
automerge
2019-09-27 22:00:30 -07:00
5637f88aff
Don't try signging coding shred if fec rate is 0 ( #6171 )
...
automerge
2019-09-27 21:58:53 -07:00
c9e58743e7
Prevent subtract overflow panic when slot < MAX_LOCKOUT_HISTORY ( #6135 )
2019-09-26 19:40:18 -06:00
b4da83a3ab
Remove CUDA feature ( #6094 )
2019-09-26 13:36:51 -07:00
a964570b1a
add authorities to stake init ( #6104 )
...
* add authorities to stake init
* fixups
* code review
2019-09-26 13:29:29 -07:00
67d07254c2
Add rent estimation rpc ( #6109 )
...
* server side new rpc endpoint
* client side rpc
* take data_len as usize
Co-Authored-By: Tyera Eulberg <teulberg@gmail.com>
* add test and documentation
2019-09-26 23:27:13 +05:30
701d90a41d
Remove some AccountStorage Serialization ( #6047 )
...
* Remove serialization of AccountStorageEntry fields
* Add metric for evaluating BankRc serialization time
* Serialize AppendVec current len
* Add dashboard metrics
* Move flush of AppendVecs to packaging thread
2019-09-25 18:07:41 -07:00
43795193c4
add authorized parameters to vote api ( #6072 )
...
* add authorized parameters to vote api
* code review
2019-09-25 13:53:49 -07:00
e987d0094f
Move status cache serialization to the Snapshot Packager service ( #6081 )
...
* Move status cache serialization to the Snapshot Packager service
* Minor comment updates
* use ok_or_else instead of ok_or
* satus cache
* Remove assert when snapshot format is wrong
* Fix compile
* Remove slots_to_snapshot from bank forks
* Address review comment
* Remove unused imports
2019-09-25 13:42:19 -07:00
de3765ab70
Add erasure recovery stats to dashboard ( #6079 )
...
automerge
2019-09-24 19:59:42 -07:00
5f079137e5
Remove kvstore ( #6075 )
...
automerge
2019-09-24 19:59:32 -07:00
374b776a3e
Fix using temp file for archive ( #6058 )
...
* Fix using temp file for archive
* Rename the temp archive instead of hardlinking it
2019-09-24 15:24:54 -07:00
5763d63737
Additional tests for should_retransmit_and_persist ( #6062 )
...
automerge
2019-09-24 14:54:10 -07:00
3ed34b571c
Window service is filtering out coding shreds ( #6052 )
...
* Window service is filtering out coding shreds
* update erasure stats to indicate recovery count
* filter out outdated coding shreds
* address review comments
2019-09-24 12:25:25 -07:00
57e90948a8
Remove dead code from cluster_info ( #6051 )
2019-09-24 10:27:59 -07:00
26a20a7e62
nits ( #6032 )
2019-09-24 10:10:49 -07:00
7fa809c16d
Avoid hardlinking as that confuses tar ( #6042 )
2019-09-23 20:12:16 -07:00
84f74807d4
Skip considering banks older than the latest vote slot ( #6037 )
...
automerge
2019-09-23 19:40:03 -07:00
3a9c03cc89
Don't recover coding shreds ( #6034 )
...
* Don't recover coding shreds
* cleanup
2019-09-23 16:24:21 -07:00
62c22c6cb1
Fix really old banks triggering log spam ( #6025 )
2019-09-23 13:59:16 -07:00
dbd337c616
Upgrade to ReedSolomon 4.0 release ( #6026 )
2019-09-23 13:53:52 -07:00
eeda7338cc
Dump tar stdout/err on failure for better debug ( #6024 )
2019-09-23 13:05:09 -07:00
8240d1fe0a
Confidence implementation ( #5993 )
...
* Change confidence parameters
* Add status_cache_ancestors to get all relevant ancestors of a bank including roots from status cache
* Fix and add tests
* Clippy
2019-09-20 19:38:56 -07:00
fd6e7020eb
Fix bank overlapping another bank's broadcast ( #6012 )
2019-09-20 19:37:40 -07:00
a60a3efc1a
Revert "require stake, vote and executable accounts to be rent exempt ( #5928 )" ( #6005 )
...
This reverts commit 11e6197a83
2019-09-20 14:10:39 -07:00
19ae556857
hash account state on store ( #5573 )
2019-09-20 13:21:12 -07:00
3d44cffcda
✨ Beautify metrics datapoint logging ( #5998 )
2019-09-20 12:00:43 -07:00
c9be9acd14
log snapshot time ( #5996 )
2019-09-20 08:03:00 -07:00
11e6197a83
require stake, vote and executable accounts to be rent exempt ( #5928 )
...
* require vote account to be exempt
* make stake account rent exempt
* add rent exempted system instruction
* use rent exemption instruction in vote and stake api
* use rent exempted account while creating executable account
* updating chacha golden hash as instruction data has changed
* rent will be initialized for genesis bank too
2019-09-20 16:52:17 +05:30
d379786c90
Fix bind errors ( #5986 )
...
* Add ability to bind to a common tcp/udp port
* Extend port range for local-net sanity and fix validator executable
2019-09-19 17:16:22 -07:00
ca9d4e34df
Broadcast stage tuning ( #5989 )
2019-09-19 16:29:52 -07:00
5cbd1190b2
transaction batch ( #5962 )
...
* transaction batch
* fixup
2019-09-19 10:06:08 -07:00
0dbf7995b5
Remove unnecessary serialize of shred data ( #5967 )
...
* Remove unnecessary serialize of shred data
* remove obsolete code
* fix golden hash
2019-09-18 20:08:27 -07:00
e0858cfe06
Add parallel shred signing to shredder ( #5964 )
2019-09-18 18:00:07 -07:00
783e8672e7
Removed Shred enum ( #5963 )
...
* Remove shred enum and it's references
* rename ShredInfo to Shred
* clippy
2019-09-18 16:24:30 -07:00
6c4e656795
Remove obsoleted code from shred ( #5954 )
...
* Remove obsoleted code from shred
* fix broken test
2019-09-18 13:56:44 -07:00
b757294864
Add minor performance bump to shredding ( #5956 )
2019-09-18 12:35:52 -07:00
0d4a2c5eb0
simplify poh recorder => broadcast channel ( #5940 )
...
* simplify poh recorder broadcast channel
* fixup
* fixup
2019-09-18 12:16:22 -07:00
2f50d0e145
Refactor confidence from replay stage ( #5938 )
2019-09-17 19:43:40 -07:00
ff608992ee
Replace Shred usage with ShredInfo ( #5939 )
...
* Replace Shred usage with ShredInfo
* Fix tests
* fix clippy
2019-09-17 18:22:46 -07:00
a2595b44c6
test randomize with error ( #5916 )
...
* test randomize with error
* update magic numbers
* fixup
* fixup
* fixup
* no more blobs
* fixup
2019-09-17 15:11:29 -07:00
14cb6353c0
Change erasure ratio to 0.25 and increase data shreds to 16 ( #5931 )
...
* Change erasure ratio to 0.25 and increase data shreds to 16
* Fix case where no coding shreds are requested
2019-09-17 11:59:14 -07:00
e26f68fe62
Get transactions from LockedAccountsResults when possible ( #5923 )
2019-09-17 08:41:56 -07:00
f10438d530
Respect randomized transaction order when unlocking accounts ( #5918 )
2019-09-16 21:45:16 -07:00
7459eb15c3
A new data-structure in shreds for partial deserialization ( #5915 )
...
* A new datastructure in shreds for partial deserialization
* fix chacha golden hash
* fix clippy and address review comments
2019-09-16 20:28:54 -07:00
c44e7ce184
Leaders should not broadcast to replicators ( #5917 )
2019-09-16 17:56:34 -07:00
82615c703b
Switch erasure to solana-reed-solomon-erasure ( #5913 )
...
* Switch to solana-reed-solomon-erasure
* Disable Rayon for solana-reed-solomon-erasure
2019-09-16 16:14:55 -07:00
d5ba90d375
Don't verify blobs that are less than root in window service ( #5901 )
2019-09-16 13:13:53 -07:00
5f54573613
More shred related cleanup ( #5909 )
...
* More shred related cleanup
* fix uncle
2019-09-16 10:28:28 -07:00
c1880e3f3e
Reduce number of shreds per FEC block ( #5908 )
2019-09-15 10:37:12 -07:00
ee791e2e3e
Optimizations to shred writing and signing ( #5890 )
...
* Optimizations to shred writing and signing
* fix broken tests
* fixes
2019-09-14 21:05:54 -07:00
ceacc42126
Call gpu init earlier to force compilation. ( #5902 )
2019-09-14 12:32:57 -07:00
84c8a5bbec
Add replay-stage-mark_dead_slot datapoint ( #5907 )
2019-09-14 08:50:53 -07:00
8135279335
Reduce serialize/deserialize in shred recovery ( #5887 )
2019-09-12 21:52:13 -07:00
c1d788880d
Limit Rayon threadpool threads ( #5871 )
2019-09-12 11:39:39 -07:00
385086359c
Reduce serializations/deserializations of shreds ( #5879 )
2019-09-12 10:10:25 -07:00
1853771930
Add support for SDK sysvar types ( #5876 )
2019-09-10 18:53:02 -07:00
ee4266bc59
Remove banks in locktower not in bank_forks ( #5837 )
...
* Remove unnecessary calculations from collect_vote_lockouts
* Add test for locktower startup from snapshot
2019-09-10 13:58:27 -07:00
b426dfb2c0
Change tx batching in banking process and record ( #5832 )
...
* Change tx batching in banking process and record
* Change batching to reduce impact on replay stage
2019-09-10 11:04:03 -07:00
fd33b27af1
Fix coding shred generator ( #5865 )
2019-09-10 09:35:07 -07:00
7682db4826
Generate coding shreds on the fly based on erasure limits ( #5852 )
...
* Generate coding shreds on the fly based on erasure limits
* fix uncle
2019-09-09 17:26:51 -07:00
7607800d47
Refactor restart function in local cluster to support separate exit and restart functions ( #5845 )
2019-09-08 17:53:34 -07:00
df205f8752
Use ureq instead of influx_db_client ( #5839 )
2019-09-07 12:48:45 -07:00
e8d88f3237
Split SDK's timing.rs ( #5823 )
2019-09-06 14:30:56 -07:00
1f9fde5f7b
ThinClient internal name grooming ( #5800 )
2019-09-06 09:07:40 -07:00
a452249bf3
Use retain on Packets instead of creating new ones ( #5804 )
...
* Use remove on Packets instead of creating a new one
* Fix compile after rebase
2019-09-05 19:16:18 -07:00
3d3b03a123
Verify signature of recovered shred before adding them to blocktree ( #5811 )
...
* Verify signature of recovered shred before adding them to blocktree
* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
05d2eec45c
Remove unnecessary erasure config references ( #5809 )
2019-09-05 14:46:41 -07:00
3237e897d7
Adjust packet batching post-decoupling from blobs ( #5783 )
2019-09-05 11:22:39 -07:00
bd74e63702
Offload remaining confidence cache computation to separate thread ( #5792 )
...
* Move remaining confidence cache computation to separate thread
* Move confidence cache out of bank forks
2019-09-04 23:10:25 -07:00
f78b865cba
Cleanup shreds to remove FirstShred data structure ( #5789 )
...
* Cleanup shreds to remove FirstShred data structure
* Also reduce size used by parent slot information in shred header
* clippy
* fixes
* fix chacha test
2019-09-04 21:06:47 -07:00
7062fe4b47
Refactor Blocktree for clarity and correctness ( #5700 )
...
* Refactor shreds to prevent insertion of any metadata on bad shreds
* Refactor fetching Index in blocktree
* Refactor get_slot_meta_entry
* Re-enable local cluster test
* cleanup
* Add tests for success/fail insertion of coding/data shreds
* Remove assert
* Fix and add tests for should_insert coding and data blobs
2019-09-04 17:14:42 -07:00
02ee2a601c
Further cleanup of blocktree after Blob deprecation ( #5780 )
2019-09-04 12:47:09 -07:00
dfa6238342
Remove unnecessary construction of descendants ( #5742 )
2019-09-04 01:49:42 -07:00
3b0d48e3b8
Remove blocktree blob references ( #5691 )
...
* Remove blocktree blob references
* fixes and cleanup
* replace uninitialized() call with MaybeUninit
* fix bench
2019-09-03 21:32:51 -07:00
62f6a78ccd
Make data plane shred filter parallel again ( #5740 )
2019-09-03 21:50:57 +00:00
85e5fbeb35
Add absoluteSlot to getEpochInfo ( #5765 )
2019-09-02 12:21:06 -07:00
475f6fe666
votes only need slots and the last bank hash ( #5499 )
...
churn
cleanup
reverse test slot hashes
test check_slots_are_valid
updates
only send the minimum bank vote difference
fixup! only send the minimum bank vote difference
some banks may not have a voting account setup
fixup! votes only need slots and the last bank hash
fixup! fixup! votes only need slots and the last bank hash
fmt
fixed compare
fixed vote
fixup! fixed vote
poke ci
filter the local votes via the last bank vote
2019-09-02 12:01:09 -07:00
0c2a49391a
Disable pinned gpu memory ( #5753 )
2019-08-31 16:44:07 -07:00
e3a6c9234a
Entrypoint RPC service discovery now blocks until the entrypoint is actually found ( #5756 )
...
automerge
2019-08-30 16:12:58 -07:00
4786143524
Add a more helpful error on genesis block mismatch ( #5744 )
...
automerge
2019-08-30 09:10:22 -07:00
carllin
Michael Vines
anatoly yakovenko
Tyera Eulberg
Sagar Dhawan
Parth
Ryo Onodera
Greg Fitzgerald
Ryo Onodera
Rob Walker
Sunny Gleason
Justin Starry
Pankaj Garg
sakridge
Jack May
sakridge
TristanDebrunner
TristanDebrunner