445e6668c2
Fix ( #9896 )
...
Co-authored-by: Carl <carl@solana.com>
2020-05-06 11:44:49 -07:00
72312ad615
Avoid holding the entire rooted path while loading bank forks ( #9885 )
2020-05-05 19:45:41 -07:00
3442f36f8a
Repair alternate versions of dead slots ( #9805 )
...
Co-authored-by: Carl <carl@solana.com>
2020-05-05 14:07:21 -07:00
a7f33b5014
Cache banks in BankForks until optional largest_confirmed_root ( #9678 )
...
automerge
2020-04-24 15:49:57 -07:00
76b1c2baf0
One less alloc per transaction ( #9705 )
...
* One less alloc per transaction
* Fix benches
* Fix compiler warnings in bench build
* Fix move build
* Fix bench
2020-04-24 13:03:46 -06:00
d5abff82e0
Add largest_confirmed_root to BlockCommitmentCache ( #9640 )
...
* Add largest_confirmed_root to BlockCommitmentCache
* clippy
* Add blockstore to BlockCommitmentCache to check root
* Add rooted_stake helper fn and test
* Nodes that are behind should correctly id confirmed roots
* Simplify rooted_stake collector
2020-04-22 12:22:09 -06:00
bab3502260
Push down cluster_info lock ( #9594 )
...
* Push down cluster_info lock
* Rework budget decrement
Co-authored-by: Carl <carl@solana.com>
2020-04-21 12:54:45 -07:00
1607891b29
log proper slot ( #9576 )
...
Co-authored-by: Carl <carl@solana.com>
2020-04-19 14:24:45 -07:00
66abe45ea1
Decouple accounts hash calculation from snapshot hash ( #9507 )
2020-04-16 15:12:20 -07:00
3037eb8d4f
Remove slot field, add test ( #9444 )
...
Co-authored-by: Carl <carl@solana.com>
2020-04-10 23:52:37 -07:00
aa8dfac313
Simplify vote simulation ( #9435 )
...
Co-authored-by: Carl <carl@solana.com>
2020-04-10 15:16:12 -07:00
4522e85ac4
Add Metrics/Dashboards tracking block production ( #9342 )
...
* Add metric tracking blocks/dropped blocks
Co-authored-by: Carl <carl@solana.com>
2020-04-08 14:35:24 -07:00
ad0997e15f
RPC: add `err` field to TransactionStatus, alongside the now deprecated `status` field ( #9296 )
...
automerge
2020-04-04 16:13:26 -07:00
0139236464
ReplayStage fixes ( #9271 ) ( #9279 )
...
automerge
2020-04-02 21:05:33 -07:00
268e04cb4a
Rename CustomError to Custom ( #9207 )
2020-04-01 09:01:11 -07:00
0e2722c638
solana-validator now supports multiple --authorized-voter arguments ( #9174 )
...
* Use Epoch type
* Vote account's authorized voter is now supported without a validator restart
2020-03-31 08:23:42 -07:00
66946a4680
Check ClusterSlots for confirmation of block propagation ( #9115 )
2020-03-30 19:57:11 -07:00
50fa577af8
Use cluster confirmations in rpc and pubsub ( #9138 )
...
* Add runtime methods to simply get status and slot
* Add helper function to get slot confirmation_count from BlockCommitmentCache
* Return cluster confirmations in getSignatureStatus
* Remove use of invalid get_signature_confirmation_status
* Remove unused methods
* Update pubsub to use cluster confirmations
* Fix test_check_signature_subscribe failure
* Refactor confirmations to read commitment cache only once
* Review comments
* Use bank, root from BlockCommitmentCache
* Update docs
* Add metric for block-commitment aggregations
Co-authored-by: Justin Starry <justin@solana.com>
2020-03-30 17:53:25 -06:00
62040cef56
Store BlockCommitmentCache slot and root metadata ( #9154 )
...
automerge
2020-03-30 10:29:30 -07:00
c1a3b6ecc2
Add RPC subscription api for rooted slots ( #9118 )
...
automerge
2020-03-27 09:33:40 -07:00
d47262d233
Reduce transmit frequency ( #9113 )
...
Co-authored-by: Carl <carl@solana.com>
2020-03-26 23:33:28 -07:00
5a8658283a
Add check for propagation of leader block before generating further blocks ( #8758 )
...
Co-authored-by: Carl <carl@solana.com>
2020-03-26 19:57:27 -07:00
f3d556e3f9
Refactor VoteTracker ( #9084 )
...
* Refactor VoteTracker
Co-authored-by: Carl <carl@solana.com>
2020-03-26 17:55:17 -07:00
b7b4aa5d4d
move rpc types from client to client-types crate ( #9039 )
...
* Separate client types into own crate, so ledger does not need it
Removes about 50 crates of dependency from ledger
* Drop Rpc name from transaction-status types
2020-03-26 13:29:30 -07:00
aa24181a53
Remove blockstream unix socket support. RPC or bust ( #9004 )
...
automerge
2020-03-21 20:17:11 -07:00
18c1f0dfe9
Remove stub core/src/genesis_utils.rs ( #8999 )
2020-03-21 10:54:40 -07:00
dc1db33ec9
Add Capabilities to Signal BroadcastStage to Retransmit ( #8899 )
2020-03-19 23:35:01 -07:00
dc347dd3d7
Add Accounts hash consistency halting ( #8772 )
...
* Accounts hash consistency halting
* Add option to inject account hash faults for testing.
Enable option in local cluster test to see that node halts.
2020-03-16 08:37:31 -07:00
ead6dc553a
If let
2020-03-16 07:57:07 -07:00
009c124fac
Remove generic
2020-03-16 07:57:07 -07:00
9411fc00b8
Lower error level
2020-03-16 07:57:07 -07:00
53b8d0d528
Remove holding Poh lock ( #8838 )
...
automerge
2020-03-13 15:15:13 -07:00
9872430bd2
Add VoteTracker for tracking cluster's votes in gossip ( #8327 )
...
Track votes by slot in cluster_vote_listener
2020-03-09 22:03:09 -07:00
f23dc11a86
compute_bank_stats needs to return newly computed ForkStats ( #8608 )
...
* Fix broken confirmation, add test
2020-03-04 11:49:56 -08:00
8ef8c9094a
Add ReplayStage changes for checking switch threshold ( #8504 )
...
* Refactor for supporting switch threshold check
2020-03-02 12:43:43 -08:00
5f766cd20b
Remove loop ( #8493 )
2020-02-26 19:59:28 -08:00
d47a47924a
Update voting simulation ( #8460 )
2020-02-26 14:09:07 -08:00
7a2bf7e7eb
Limit leader schedule search space ( #8468 )
...
* Limit leader schedule search space
* Fix and add test
* Rename
2020-02-26 13:35:50 -08:00
d821fd29d6
Add versioning ( #8348 )
...
automerge
2020-02-25 17:12:01 -08:00
004f1d5aed
Combine replay stage memory reporting ( #8455 )
...
automerge
2020-02-25 16:04:27 -08:00
1caeea8bc2
Refactor new bank paths into common function ( #8454 )
2020-02-25 15:49:59 -08:00
aa80f69171
Promote some datapoints to `info` to fix dashboard ( #8381 )
...
automerge
2020-02-21 13:41:49 -08:00
ab361a8073
Rename KeypairUtil to Signer ( #8360 )
...
automerge
2020-02-20 13:28:55 -08:00
ccad5d5aaf
change warnings to infos ( #8322 )
2020-02-19 14:25:49 -08:00
73a278dc64
Factor out creating genesis with vote accounts into a utility function ( #8315 )
...
automerge
2020-02-18 02:39:47 -08:00
17fb8258e5
Datapoints overwhelm the metrics queue and blow up ram usage. ( #8272 )
...
automerge
2020-02-14 11:11:55 -08:00
c4fd81fc1c
The getConfirmedBlock RPC API is now disabled by default
...
The --enable-rpc-get-confirmed-block flag allows validators to opt-in to
the higher disk usage and IOPS.
2020-02-11 22:24:08 -07:00
72b11081a4
Report validator rewards in getConfirmedBlock JSON RPC
2020-02-11 17:25:45 -07:00
0c8cee8c4a
Refactor select_fork() to avoid clones and for clarity ( #8081 )
...
* Refactor select_fork() to avoid clones and for clarity
* Add test that fork weights are increasing
2020-02-03 16:48:24 -08:00
4197cce8c9
Tower tests ( #7974 )
...
* Add testing framework for voting
2020-01-28 16:02:28 -08:00
4ffd7693d6
Add lock to make sure slot-based locktree calls are safe ( #7993 )
2020-01-28 13:45:41 -08:00
5cf090c896
feat: implement RPC notification queue ( #7863 )
2020-01-20 16:08:29 -05:00
6d3b8b6d7d
Remove tuples from JSON RPC responses ( #7806 )
...
* Remove RpcConfirmedBlock tuple
* Remove getRecentBlockhash tuple
* Remove getProgramAccounts tuple
* Remove tuple from get_signature_confirmation_status
* Collect Rpc response types
* Camel-case epoch schedule for rpc response
* Remove getBlockCommitment tuple
* Remove getStorageTurn tuple
* Update json-rpc docs
2020-01-15 00:25:45 -07:00
ff1ca1e0d3
Consolidate entry tick verification into one function ( #7740 )
...
* Consolidate entry tick verification into one function
* Mark bad slots as dead in blocktree processor
* more feedback
* Add bank.is_complete
* feedback
2020-01-15 09:15:26 +08:00
b5dba77056
Rename blocktree to blockstore ( #7757 )
...
automerge
2020-01-13 13:13:52 -08:00
a17d5795fb
getConfirmedBlock: add encoding optional parameter ( #7756 )
...
automerge
2020-01-12 21:34:30 -08:00
73c93cc345
Print bank hash and hash inputs. ( #7733 )
2020-01-09 16:33:10 -08:00
4fe0b116ae
Measure heap usage while processing the ledger
2020-01-03 13:25:37 -07:00
a0fb9de515
Move thread_mem_usage module into measure/
2020-01-03 13:25:37 -07:00
87b2525e03
Limit maximum number of shreds in a slot to 32K ( #7584 )
...
* Limit maximum number of shreds in a slot to 32K
* mark dead slot replay as fatal error
2019-12-30 07:42:09 -08:00
727be309b2
fix entryverification state ( #7169 )
...
automerge
2019-12-23 23:26:27 -08:00
6a9005645a
Update "limit-ledger-size" to use DeleteRange for much faster deletes ( #7515 )
...
* Update "limit-ledger-size" to use DeleteRange for much faster deletes
* Update core/src/ledger_cleanup_service.rs
Co-Authored-By: Michael Vines <mvines@gmail.com>
* Rewrite more idiomatically
* Move max_ledger_slots to a fn for clippy
* Remove unused import
* Detect when all columns have been purged and fix a bug in deletion
* Check that more than 1 column is actually deleted
* Add helper to test that ledger meets minimum slot bounds
* Remove manual batching of deletes
* Refactor to keep some N slots older than the highest root
* Define MAX_LEDGER_SLOTS that ledger_cleanup_service will try to keep around
* Refactor compact range
2019-12-18 11:50:09 -08:00
98b80288ed
Optimize bank_forks critical section ( #7477 )
2019-12-13 17:20:31 -08:00
dd54fff978
Use pinned memory for entry verify ( #7440 )
2019-12-12 10:36:27 -08:00
6d2861f358
add unit test for minority fork overcommit attack ( #7292 )
...
* add unit test for minority fork overcommit attack
* add generic function to simulate fork selection
2019-12-10 22:06:16 +05:30
9ecb844de7
Split up ReplayStageConfig to make it derive Default ( #7334 )
...
automerge
2019-12-06 14:39:35 -08:00
3ab8185777
Add intermittent Timestamping to Votes ( #7233 )
...
* Add intermittent timestamp to Vote
* Add timestamp to VoteState, add timestamp processing to program
* Print recent timestamp with solana show-vote-account
* Add offset of 1 to timestamp Vote interval to initialize at node boot (slot 1)
* Review comments
* Cache last_timestamp in Tower and use for interval check
* Move work into Tower method
* Clarify timestamping interval
* Replace tuple with struct
2019-12-06 14:38:49 -07:00
fae9c08815
Add ReplayStageConfig ( #7195 )
2019-12-04 11:17:17 -07:00
f0a40862d6
Support local cluster edge case testing ( #7135 )
...
* Refactor local cluster to support killing a partition
* Rework run_network_partition
* Introduce fixed leader schedule
* Plumb fixed schedule into test
2019-12-03 16:31:59 -08:00
0a0f15baca
RPC subscriptions for new slot notifications ( #7114 )
...
* feat: slot notifications via pubsub rpc w/ tests
2019-11-26 03:42:54 -05:00
0f66e5e49b
Add getConfirmedBlock test to rpc ( #7120 )
...
automerge
2019-11-25 11:08:03 -08:00
0bd41f98ed
Avoid jemalloc in windows build ( #7089 )
...
automerge
2019-11-21 18:39:29 -08:00
d9e7a5fcbe
Use fork weight instead of individual bank weight for fork selection. ( #7079 )
...
* Fix weight calculation
* Fix tests
* fork weight
* wait until nodes are in the leader schedule
* enable sanity
* fewer long tests
2019-11-21 15:47:08 -08:00
c965a110f2
Use unbounded channel ( #7081 )
2019-11-21 14:23:40 -07:00
e66b29943b
datapoint for best fork weight and slot in replay ( #7066 )
2019-11-20 17:26:52 -08:00
97ca6858b7
Write transaction status and fee into persistent store ( #7030 )
...
* Pass blocktree into execute_batch, if persist_transaction_status
* Add validator arg to enable persistent transaction status store
* Pass blocktree into banking_stage, if persist_transaction_status
* Add validator params to bash scripts
* Expose actual transaction statuses outside Bank; add tests
* Fix benches
* Offload transaction status writes to a separate thread
* Enable persistent transaction status along with rpc service
* nudge
* Review comments
2019-11-20 16:43:10 -07:00
b150da837a
Use epoch as the gossip purge timeout for staked nodes. ( #7005 )
...
automerge
2019-11-20 11:25:18 -08:00
d184d3a732
Merge native programs parts into one unit ( #7047 )
2019-11-20 10:12:43 -08:00
d32a072190
Use ticks_per_slot to calculate maximum grace ticks ( #7024 )
...
* Use ticks_per_slot to calculate maximum grace ticks
* fix test
* fix votable candidate ordering
* fixes to pick_best_fork() and a unit test
* fixes
2019-11-19 17:55:42 -08:00
b958bf9086
Fix confirmation metrics ( #7035 )
2019-11-19 09:51:50 -05:00
43144cfe8b
Make banks that fail threshhold check resettable ( #7027 )
2019-11-19 02:36:30 -08:00
11d2d2eccd
Fix progress map losing banks and recomputing stats ( #7026 )
...
* Fix progress map missing banks
* Fix confirmations
* Fix test
* Initialiize progress with frozen banks atartup
2019-11-19 02:36:00 -08:00
6bfe0fca1f
Add a version field to shreds ( #7023 )
...
* Add a version field to shreds
* Clippy
* Fix Chacha Golden
* Fix shredder bench compile
* Fix blocktree bench compile
2019-11-18 18:05:02 -08:00
b66c03667c
Log for threshold failure ( #7008 )
2019-11-17 17:10:16 -08:00
70322d1ff8
Add error logging to dead slots ( #7000 )
2019-11-16 02:54:51 -08:00
59413b3124
Fix rules for fork selection ( #6906 )
...
automerge
2019-11-15 08:36:33 -08:00
79d7090867
Remove obsolete references to Blob ( #6957 )
...
* Remove the name "blob" from archivers
* Remove the name "blob" from broadcast
* Remove the name "blob" from Cluset Info
* Remove the name "blob" from Repair
* Remove the name "blob" from a bunch more places
* Remove the name "blob" from tests and book
2019-11-14 11:49:31 -08:00
43e2301e2c
Fix roots overrunning broadcast ( #6884 )
...
* Add trusted pathway for insert_shreds to avoid checks
2019-11-14 00:32:07 -08:00
a3a830e1ab
Delete Service trait ( #6921 )
2019-11-13 11:12:09 -07:00
30a08f4282
Cleanup ledger macros ( #6916 )
...
automerge
2019-11-13 07:14:09 -08:00
9807f47d4e
Rename genesis block to genesis config ( #6816 )
2019-11-08 23:56:57 -05:00
02058ea699
Reject blocks with invalid last ticks in replay stage ( #6833 )
...
* Reject blocks with invalid last ticks in replay stage
* slot_full
2019-11-08 20:21:54 -05:00
b800642fa4
Add new fork log message for when the node is leader for consistency ( #6808 )
2019-11-08 12:30:25 -07:00
0ace79939b
Add reference tick to data shreds ( #6772 )
...
* Add reference tick to data shreds
* fix tests
2019-11-06 13:27:58 -08:00
24102a7435
Allow voting on empty banks ( #6719 )
...
* Allow votes on empty banks
* Remove making first bank is_delta true, no longer necessary for idling
* Remove votable from ledger tool
2019-11-06 01:02:26 -08:00
09e648f957
ledger-tool/: Include full validator voting history in fork-graph ( #6756 )
2019-11-05 19:40:00 -07:00
f54cfcdb8f
Store and persists full stack of tower votes in gossip ( #6695 )
...
* vote array
wip
wip
wip
update
gossip index should match tower index
tests build
clippy
test index after expired vote
test
bank specific last vote sync time
* verify
* we are likely to see many more warnings about old votes now
2019-11-04 16:19:54 -08:00
57983980a7
Lower verify-batch-size to debug ( #6722 )
...
automerge
2019-11-04 16:00:59 -08:00
33f4aaf3fd
Rename confidence to commitment ( #6714 )
2019-11-04 16:44:27 -07:00
50a17fc00b
Use Slot and Epoch type aliases instead of raw u64 ( #6693 )
...
automerge
2019-11-02 00:38:30 -07:00
e8e5ddc55d
Verify number of hashes for each block of entries ( #6262 )
...
* Verify number of hashes for each block of entries
* Fix blocktree processor tick check
* Rebase once more
2019-10-31 16:38:50 -04:00
38491c8c4b
Reduce verify-batch-size log ( #6623 )
2019-10-30 13:41:11 -07:00
b4119c454a
credit_only credits forwarding ( #6509 )
...
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
2019-10-23 22:01:22 -07:00
03d29a8311
Async poh verify ( #6353 )
...
* Async poh verify
* Up ticks_per_s to 160
GPU poh verify needs shorter poh sequences or it takes forever to
verify. Keep slot time the same at 400ms.
* Fix stats
* Don't halt on ticks
* Increase retries for local_cluster tests and make repairman test serial
2019-10-23 12:11:04 -07:00
b38bf90de7
Deshred blocks in parallel ( #6461 )
...
* Deshred in parallel
* Add tests for corrupt slots and parallel deshred
* Rename load_blocktree_entries to load_blocktree_entries_with_shred_count
2019-10-21 16:15:10 -07:00
84e911361a
Use constants instead of lazy_static for shred header sizes ( #6472 )
2019-10-21 12:46:16 -07:00
2636418659
Move blocktree_processor to solana_ledger ( #6460 )
...
* Drop core::result dependency in bank_forks
* Move blocktree_processor into solana_ledger
2019-10-20 09:54:38 -06:00
e2c316d2d0
system_instruction_processor updates ( #6448 )
...
* zero lamport account creation
* whack create_user_account, take 2
* target->to
* ..
* ..
* update chacha golden
* update chacha golden
* ..
* ..
2019-10-19 18:23:27 -07:00
badeb4d31a
Rework shred headers to fix position of signature ( #6451 )
...
* Rework shred headers to fix position of signature
* fix clippy
2019-10-18 22:55:59 -07:00
5468be2ef9
Add solana-ledger crate ( #6415 )
...
automerge
2019-10-18 09:28:51 -07:00
27c0d30a07
Fix logging ( #6417 )
2019-10-18 02:06:41 -07:00
d3068c3918
Remove circular dependencies in core ( #6408 )
...
* Remove core::result dependency from blocktree
* Remove core::result dependency from shred
* Move Packet from core::packet to sdk::packet
This way we don't need to split perf_libs yet.
* Disable packet when compiling BPF programs
2019-10-17 11:37:30 -06:00
83c1831a01
Fix replay stage test ( #6406 )
2019-10-16 15:41:43 -07:00
ccb4e32ee0
ReplayStage metrics ( #6358 )
...
* ReplayStage metrics
* Add more metrics
* Refactor get_slot_entries_with_shred_count() to detect wasted work
* Update dashboard
* Update broadcast slots to micros
* Add broadcast dashboard
2019-10-16 14:32:18 -07:00
7e6e7e8406
Remove special handling of first ledger tick ( #6263 )
...
* Remove special handling of first ledger tick
* Fix subtraction overflow
* @garious feedback
* Back to height
* More tick_height name changes
* Fix off-by-one
* Fix leader tick error
* Fix merge conflict
* Fix recently added test
2019-10-16 15:53:11 -04:00
dfca2b510b
Lower shred/receiver stats ( #6365 )
...
too many messages
2019-10-15 11:43:52 -07:00
7cf90766a3
add epoch_schedule sysvar ( #6256 )
...
* add epoch_schedule sysvar
* book sheesh!
2019-10-08 22:34:26 -07:00
5e31565574
Expand blocktree processor options ( #6248 )
...
* Refactor blocktree processor args and support full leader cache
* Add entry callback option
* Rename num_threads to override_num_threads
* Add test for entry callback
* Refactor cached leader schedule changes
* Add tests for blocktree process options
* Refactor test
* @mvines feedback
2019-10-08 17:58:49 -04:00
5ef012b2c1
Tweak debug to remove unreadable datapoints ( #6060 )
2019-10-04 16:25:22 -07:00
f9f5bc2eb5
More clippy
2019-10-02 21:21:07 -07:00
84f74807d4
Skip considering banks older than the latest vote slot ( #6037 )
...
automerge
2019-09-23 19:40:03 -07:00
62c22c6cb1
Fix really old banks triggering log spam ( #6025 )
2019-09-23 13:59:16 -07:00
8240d1fe0a
Confidence implementation ( #5993 )
...
* Change confidence parameters
* Add status_cache_ancestors to get all relevant ancestors of a bank including roots from status cache
* Fix and add tests
* Clippy
2019-09-20 19:38:56 -07:00
ca9d4e34df
Broadcast stage tuning ( #5989 )
2019-09-19 16:29:52 -07:00
783e8672e7
Removed Shred enum ( #5963 )
...
* Remove shred enum and it's references
* rename ShredInfo to Shred
* clippy
2019-09-18 16:24:30 -07:00
2f50d0e145
Refactor confidence from replay stage ( #5938 )
2019-09-17 19:43:40 -07:00
ff608992ee
Replace Shred usage with ShredInfo ( #5939 )
...
* Replace Shred usage with ShredInfo
* Fix tests
* fix clippy
2019-09-17 18:22:46 -07:00
84c8a5bbec
Add replay-stage-mark_dead_slot datapoint ( #5907 )
2019-09-14 08:50:53 -07:00
3d3b03a123
Verify signature of recovered shred before adding them to blocktree ( #5811 )
...
* Verify signature of recovered shred before adding them to blocktree
* fix failing tests, and review comments
2019-09-05 18:20:30 -07:00
bd74e63702
Offload remaining confidence cache computation to separate thread ( #5792 )
...
* Move remaining confidence cache computation to separate thread
* Move confidence cache out of bank forks
2019-09-04 23:10:25 -07:00
02ee2a601c
Further cleanup of blocktree after Blob deprecation ( #5780 )
2019-09-04 12:47:09 -07:00
dfa6238342
Remove unnecessary construction of descendants ( #5742 )
2019-09-04 01:49:42 -07:00
3b0d48e3b8
Remove blocktree blob references ( #5691 )
...
* Remove blocktree blob references
* fixes and cleanup
* replace uninitialized() call with MaybeUninit
* fix bench
2019-09-03 21:32:51 -07:00
475f6fe666
votes only need slots and the last bank hash ( #5499 )
...
churn
cleanup
reverse test slot hashes
test check_slots_are_valid
updates
only send the minimum bank vote difference
fixup! only send the minimum bank vote difference
some banks may not have a voting account setup
fixup! votes only need slots and the last bank hash
fixup! fixup! votes only need slots and the last bank hash
fmt
fixed compare
fixed vote
fixup! fixed vote
poke ci
filter the local votes via the last bank vote
2019-09-02 12:01:09 -07:00
7dfb735db9
randomize tx ordering ( #4978 )
...
Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
2019-08-28 21:08:32 +05:30
8b9c3a2561
Blocktree last_root to enforce a slot floor ( #5593 )
...
* Add last_root to blocktree
* Don't repair earlier than last_root
* Add integration test to make sure blocktree floor is enforced
2019-08-27 15:09:41 -07:00
4ac1213c9c
Integrate coding shreds and recovery ( #5625 )
...
* Integrate coding shreds and recovery
* More tests for shreds and some fixes
* address review comments
* fixes to code shred generation
* unignore tests
* fixes to recovery
2019-08-26 18:27:45 -07:00
faafee6b42
to to/the the ( #5590 )
2019-08-21 17:46:59 -07:00
4798e7fa73
Integrate data shreds ( #5541 )
...
* Insert data shreds in blocktree and database
* Integrate data shreds with rest of the code base
* address review comments, and some clippy fixes
* Fixes to some tests
* more test fixes
* ignore some local cluster tests
* ignore replicator local cluster tests
2019-08-20 17:16:06 -07:00
ccee6241a6
Revert "publish fixes"
...
This reverts commit 4d13d3871d
2019-08-16 17:28:07 -07:00
4d13d3871d
publish fixes
2019-08-16 17:03:57 -07:00
94f1132fb6
fix single node testnet, remove bootstrap vote ( #5534 )
2019-08-15 18:58:46 -07:00
d22848f9b1
use live stakes for consensus ( #5426 )
...
* use live stakes for consensus
* lint
* re-enable leader_failure_4
* fixups
* re-ignore leader_failure_4
2019-08-14 13:30:21 -07:00
cd14a940d8
Allow process_blocktree() to start processing from any root ( #5484 )
...
* Remove unnecessary entry_height from BankInfo
* Refactor process_blocktree to support process_blocktree_from_root
* Refactor to process blocktree after loading from snapshot
* On restart make sure bank_forks contains all the banks between the root and the tip of each fork, not just the head of each fork
* Account for 1 tick_per_slot in bank 0 so that blockhash of bank0 matches the tick
2019-08-13 17:20:14 -07:00
07d11be6ab
add global stake warmup ( #5483 )
...
* add global stake warmup
* integrate stake history into runtime
* fixup core tests
* fixup
* remove existing cooldown tests for now
2019-08-12 20:59:57 -07:00
0fde19239b
Rate limit counter metrics points to one per second ( #5496 )
...
* Rate limit counter metrics points to one per second
* Remove old env var
* Test that metrics counter is incrementing
* Fix typo
2019-08-12 18:15:34 -04:00
4f86c0b74a
Rate limit transaction counters ( #5447 )
...
* Rate limit transaction counters
* @sakridge feedback
* Set default high metrics rate for multinode demo
* Fix tests
* Swap defaults and fix env var tests
* Only set metrics rate if not already set
2019-08-08 17:05:06 -04:00
efd09ecd37
Revert fork metrics ( #5427 )
...
* Revert "Remove duplicate row (#5419 )"
This reverts commit a81dd80d60#5406 )"
This reverts commit 92e419f1c7
2019-08-05 15:53:36 -07:00
92e419f1c7
Log fork stake-percentage in metrics, and display ( #5406 )
...
* Log fork stake percentage data
* Add fork stake percentage to dashboard
* Call out parent slot
2019-08-02 19:16:23 -06:00
eb3991b9ba
Replay stage log message nits ( #5408 )
2019-08-02 10:08:42 -07:00
6cb2040a1b
Snapshot Packaging Service ( #5262 )
...
* Snapshot serialization and packaging
2019-07-31 17:58:10 -07:00
84368697af
Fix metrics when leader does not report metrics ( #5291 )
2019-07-30 16:18:33 -04:00
aef7bae60d
Let grace ticks to roll over into multiple leader slots ( #5268 )
...
* Let grace ticks to roll over into multiple leader slots
* address review comments
2019-07-26 11:33:51 -07:00
dab7de7496
Add confidence cache to BankForks ( #5066 )
...
* Add confidence cache to BankForks
* Include stake-weighted lockouts in cache
* Add cache test
* Move confidence cache updates to handle_votable_bank
* Prune confidence cache on prune_non_root()
* Spin thread to process aggregate_stake_lockouts
* Add long-running thread for stake_weighted_lockouts computation
2019-07-26 11:27:57 -06:00
547a7a345f
Add logs to indicate when the leader changes ( #5253 )
2019-07-23 22:19:20 -04:00
0d8f5379a0
Add time units
2019-07-22 08:22:21 -07:00
a07b17b9b5
Drop older slots in the ledger ( #5188 )
...
* Add facility to delete blocktree columns in range
* Add ledger cleanup service
* Add local_cluster test
2019-07-20 13:13:55 -07:00
afa05acb32
more replay_stage grooming ( #5163 )
2019-07-18 14:54:27 -07:00
a3a91ba222
Fix misleading variable name ( #5176 )
...
automerge
2019-07-18 14:07:32 -07:00
751b54b60b
Skip sleeping in replay stage if a bank was recently processed ( #5161 )
...
* Skip sleeping in replay stage if a bank was recently processed
* Remove return
2019-07-18 12:04:53 -07:00
0ffd91df27
groom poh_recorder ( #5127 )
...
* groom poh_recorder
* fixup
* nits
* slot() from the outside means "the slot the recorder is working on"
* remove redundant check
* review comments, put next_tick back in the "is reset" check
* remove redundant check
2019-07-17 14:10:15 -07:00
40faaef9da
Revert "Logging ( #5017 )" ( #5096 )
...
This reverts commit b50a3bae72
2019-07-14 18:48:15 -07:00
4c90898f0b
Dynamic erasure set configuration ( #5018 )
...
* Use local erasure session to create/broadcast coding blobs
* Individual session for each recovery (as the config might be different)
* address review comments
* new constructors for session and coding generator
* unit test for dynamic erasure config
2019-07-11 13:58:33 -07:00
b50a3bae72
Logging ( #5017 )
...
* Add logging to replay_stage
* locktower logging
2019-07-10 15:52:31 -07:00
f537482c86
remove set_leader from cluster_info ( #4998 )
2019-07-09 22:06:47 -07:00
f777a1a74c
groom replay_stage and poh_recorder ( #4961 )
...
* groom replay_stage and poh_recorder
* fixup
* fixup
* don't freeze() parent, need to review bank_forks and maybe vote...
2019-07-09 15:36:30 -07:00
eb4edd75e6
make bank commit_credits non public ( #4944 )
...
* make bank commit_credits non pub
* track down create() failure
* move bank_client to process_transaction(), which commits credits
2019-07-08 15:37:54 -07:00
2aac094f63
Ensure blobs are deserializable without unwrapping ( #4948 )
...
* Return result from deserializing blobs in blocktree instead of assuming deserialization will succeed
* Mark bad deserialization as dead fork
* Add test for corrupted blobs in blocktree and replay_stage
2019-07-07 14:37:12 -07:00
66552d7047
Credit-Only Accounts: Cache account balance for thread-safe load/store ( #4691 )
...
* Implement CreditOnlyLocks
* Update credit-only atomic on account load
* Update credit-only atomic after bank.freeze_lock; store credits if all credit-only lock references are dropped
* Commit credit-only credits on bank freeze
* Update core to CreditAccountLocks
* Impl credit-only in System Transfer
* Rework CreditAccountLocks, test, and fix bugs
* Review comments: Pass CreditAccountLocks by reference; Tighten up insert block
* Only store credits on completed slot
* Check balance in bench_exchange funding to ensure commit_credits has completed
* Add is_debitable info to KeyedAccount meta to pass into programs
* Reinstate CreditOnlyLocks check on lock_account
* Rework CreditAccountLocks to remove strong_count usage
* Add multi-threaded credit-only locks test
* Improve RwLocks usage
* Review comments: panic if bad things happen; tighter code
* Assert lock_accounts race does not happen
* Revert panic if bad things happen; not a bad thing
2019-06-27 17:25:10 -04:00
d6737b8cc9
Set epoch schedule in set_root in leader schedule cache ( #4821 )
2019-06-26 00:19:48 -07:00
29611fb61d
tower consensus naming ( #4598 )
...
s/locktower/tower/g
2019-06-24 13:41:23 -07:00
aacb38864c
Mark dead forks in replay stage ( #4715 )
...
* Add DeadSlots column family
* Filter dead forks from get_slots_since
* Mark erroring slots as dead in replay stage, add test
* Mark dead forks in progress instead of removing them
* Fix logging process_entries failures in replay_stage
* Unignore test_fail_entry_verification_leader
2019-06-20 15:50:41 -07:00
8019bff391
Fixes for storage program and rework storage stage ( #4654 )
...
automerge
2019-06-11 18:27:47 -07:00
3217a1d70c
use highest staked node as bootstrap leader, remove bootstrap_leader from genesis_block ( #4635 )
...
* use highest staked node as bootstrap leader, remove bootstrap_leader from genesis_block
* clippy
* fixup
* fixup
2019-06-11 11:44:58 -07:00
028e111fbc
remove payer from vote instructions ( #4475 )
2019-05-31 11:45:17 -07:00
182096dc1a
Create bank snapshots ( #4244 )
...
* Revert "Revert "Create bank snapshots (#3671 )" (#4243 )"
This reverts commit 81fa69d347
2019-05-30 21:31:35 -07:00
534244b322
Fix set_roots to use cached bank parents instead of searching blocktree ( #4466 )
2019-05-29 09:43:22 -07:00
d772a27936
Plumb ClusterInfoRepairListener ( #4428 )
...
automerge
2019-05-24 19:20:09 -07:00
cfe5afd34c
_id => _pubkey variable renaming ( #4419 )
...
* wallet: rename *_account_id to *_account_pubkey
* s/from_id/from_pubkey/g
* s/node_id/node_pubkey/g
* s/stake_id/stake_pubkey/g
* s/voter_id/voter_pubkey/g
* s/vote_id/vote_pubkey/g
* s/delegate_id/delegate_pubkey/g
* s/account_id/account_pubkey/g
* s/to_id/to_pubkey/g
* s/my_id/my_pubkey/g
* cargo fmt
* s/staker_id/staker_pubkey/g
* s/mining_pool_id/mining_pool_pubkey/g
* s/leader_id/leader_pubkey/g
* cargo fmt
* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
2ed77b040a
create_genesis_block() now returns a struct ( #4403 )
2019-05-22 20:39:00 -07:00
578c2ad3ea
add bank hash to votes ( #4381 )
2019-05-21 21:45:38 -07:00
f1e5edee14
Modify Roots Column To Support Multiple Roots ( #4321 )
...
* Fix 1) Roots column family to handle storing multiple slots, 2) Store all slots on the rooted path in the roots column family
2019-05-20 19:04:18 -07:00
af82b0dce9
Fix sending root slots instead of current slot ( #4354 )
...
automerge
2019-05-20 15:01:55 -07:00
86e03a6d1b
support issuing vote instructions from system account ( #4338 )
...
* issue vote instructions from system account
* fixup
* bring back KeypairUtil
2019-05-20 13:32:32 -07:00
06eb2364f2
Handle missed slots in storage stage ( #4337 )
...
* Handle missed slots in storage stage
* Fix test compile
* Make test use the new missed slot support
2019-05-18 15:24:50 -07:00
9476fe5ce3
Use log levels for datapoint metrics ( #4335 )
...
* Use log levels for datapoint metrics
* address review comments
* fix cyclomatic complexity
2019-05-17 17:34:05 -07:00
b2ce5dc9f5
Adjust log level for counter metrics ( #4323 )
2019-05-17 07:00:06 -07:00
fd3f2cb910
Add Storage accounts for all nodes ( #4298 )
...
* Setup storage keypairs for all nodes
* Clean up naming
* clippy
* Update arg value_names
2019-05-15 15:19:29 -07:00
f567877d1d
Cleanup metrics ( #4230 )
2019-05-10 08:33:58 -07:00
81fa69d347
Revert "Create bank snapshots ( #3671 )" ( #4243 )
...
This reverts commit abf2b300da
2019-05-09 19:27:27 -07:00
abf2b300da
Create bank snapshots ( #3671 )
...
* Be able to create bank snapshots
* fix clippy
* load snapshot on start
* regenerate account index from the storage
* Remove rc feature dependency
* cleanup
* save snapshot for slot 0
2019-05-09 19:27:06 -07:00
8e400fc4bd
rework genesis (passive staking groundwork) ( #4187 )
...
* rework genesis
* fixup
2019-05-07 11:16:22 -07:00
71f9b44687
Add Confirmations parameter to RPC Subscriptions ( #4154 )
...
* Add optional depth parameter to pubsub, and store in subscriptions
* Pass bank_forks into rpc_subscription; add method to check depth before notify and impl for account subscriptions
* Impl check-depth for signature subscriptions
* Impl check-depth for program subscriptions
* Plumb fork id through accounts
* Use fork id and root to prevent repeated account notifications; also s/Depth/Confirmations
* Write tests in terms of bank_forks
* Fixup accounts tests
* Add pubsub-confirmations tests
* Update pubsub documentation
2019-05-06 08:31:50 -06:00
a7b695c27a
Change replicators to slot-based ( #4118 )
2019-05-03 16:27:53 -07:00
1a259d4a3f
Prevent Requests/Caching of leader schedules for epochs beyond confirmed roots ( #4033 )
...
automerge
2019-04-30 13:23:21 -07:00
512bfc93cb
Add a cache for leader schedules ( #3841 )
...
* Add a cache for leader schedules
2019-04-19 02:39:44 -07:00
64c6f05da2
persist set_root() and use it in blocktree_processor to limit squashes ( #3782 )
...
* rename locktower's slot to epoch
* persist set_root() and use it in blocktree_processor to limit squashes
2019-04-15 13:12:28 -07:00
1db80d79fc
Update get recent blockhashes to return confirmed blockhashes only
2019-04-12 14:58:22 -07:00
877ec08280
Send recent votes in Vote Transactions ( #3734 )
2019-04-11 14:48:36 -07:00
f1e7237c09
vote_api cleanup ( #3710 )
...
* vote_api cleanup
* fixups
* fixup
* remove unused code
* revert removal of serialize and deserialize
* ...
* increase coverage, bootstrap staking
* Sagar's STAKE to my VOTE
2019-04-10 17:52:47 -07:00
d4eebcc2aa
Check for frozen in confirm_forks ( #3678 )
2019-04-09 11:45:38 -07:00
20aa4434e2
Fix repair ( #3581 )
...
Add DetachedHeads repair protocol
Add DetachedHeads repair test
Repair starting from root
2019-04-06 19:41:22 -07:00
a5b5248a09
move vote_accounts up ( #3647 )
2019-04-05 14:23:00 -07:00
35298e01a8
Remove Instruction wrapper structs and name functions after enum fields
2019-04-03 13:34:27 -07:00
fcef54d062
Add a constructor to generate random pubkeys
2019-03-31 16:23:18 -06:00
f886b3b12b
Fix resetting PohRecorder to wrong bank ( #3553 )
...
* Check whether future slot already has transmission
2019-03-29 20:00:36 -07:00
5646daa820
Delete lots of fee parameters
...
So many zeros!
2019-03-29 19:21:51 -06:00
dee5ede16d
Get rid of unnecessary frozen banks ( #3572 )
2019-03-29 16:06:48 -07:00
e45f7afd85
use the right id for delegate id
2019-03-27 15:04:09 -07:00
36ea088387
Fix Storage Stage not receiving entries when node is leader ( #3528 )
2019-03-27 13:10:33 -07:00
0346b9cb5c
hang out on progress until fork is confirmed
2019-03-27 08:41:41 -07:00
6956bf635e
validator confirmaiton
2019-03-27 08:41:41 -07:00
e27d6d0988
validator confirmation
2019-03-27 08:41:41 -07:00
4bc3f70150
Boot VoteTransaction
2019-03-25 17:11:57 -07:00
51004881f8
filter out banks that have an older epoch ( #3472 )
2019-03-25 11:09:39 -07:00
46ecac3310
Update leader slot in poh recorder if we skipped it ( #3452 )
...
* reset poh recorder with the original start slot
2019-03-23 13:07:09 -07:00
f09b8d3921
Demote log level
2019-03-22 17:00:09 -07:00
06b0c98c75
Remove accounts when the fork is removed ( #3384 )
...
* Fix test
* Cleanup accounts when the fork is removed
* Update test to check for deleted accounts
2019-03-21 17:36:10 -07:00
412ebfcaf2
ReplayStage::new is too long
...
break into more functions
2019-03-21 14:08:11 -07:00
07d55d0092
Downgrade 'No next leader found' to warning
2019-03-21 11:18:49 -06:00
148e08a8a5
Enable cluster tests ( #3372 )
...
* Cluster tests
* stable!
* fixup! stable!
* fixup! fixup! stable!
* fixup! fixup! fixup! stable!
* fixup! fixup! fixup! fixup! stable!
* fixed space
* add getNumBlocksSinceSignatureConfirmation entry for the json rpc docs
* Check in upcoming epochs for potential leadership slots in next_leader_slot()
2019-03-21 07:43:21 -07:00
fe9e771b9b
Clear progress map on squash ( #3377 )
2019-03-19 17:30:36 -07:00
7246d72f03
fix is_locked_out logic
2019-03-19 16:21:46 -07:00
5e21268ca0
PR comments
2019-03-18 20:46:11 -07:00
b38e3bef01
Modify bank_forks to support squashing/filtering new root and also don't remove parents from bank_forks when inserting, otherwise we lose potential fork points when querying blocktree for child slots
2019-03-18 20:46:11 -07:00
ef111dcbe1
Decendent is not a word
2019-03-18 15:58:27 -07:00
efc39ffdde
Report how many grace ticks were afforded to previous leader ( #3350 )
2019-03-18 13:24:07 -07:00
61a4b998fa
Implement locktower voting ( #3251 )
...
* locktower components and tests
* integrate locktower into replay stage
* track locktower duration
* make sure threshold is checked after simulating the vote
* check vote lockouts using the VoteState program
* duplicate vote test
* epoch stakes
* disable impossible to verify tests
2019-03-18 12:12:33 -07:00
00f005af25
Fix leader rotation counter
2019-03-15 17:01:18 -07:00
971da7325d
Reduce log level for periodic debug messages
2019-03-15 15:41:26 -07:00
66fb1bbb2e
Give last leader some grace ticks to catch up ( #3299 )
...
* Wait for last leader for some ticks
* New tests and fixed existing tests
2019-03-15 13:22:16 -07:00
9f02a8d3d0
remove ticks_per_slot from blocktree ( #3297 )
2019-03-14 15:18:37 -07:00
af03df38b9
Don't vote for empty leader transmissions ( #3248 )
...
* Don't vote for empty leader transmissions
* Add is_delta flag to bank to detect empty leader transmissions
* Plumb new is_votable flag through replay stage
* Fix PohRecorder tests
* Change is_delta to AtomicBool to avoid making Bank references mutable
* Reset start slot in poh_recorder when working bank is cleared, so that connsecutive TPU's will start from the correct place
* Use proper max tick height calculation
* Test for not voting on empty transmission
* tests for is_votable
2019-03-13 14:06:12 -07:00
cb3eeace56
Replay Stage start_leader() can use wrong parent fork() ( #3238 )
...
* Make sure start_leader starts on the last voted block, not necessarily the biggest indexed bank in frozen_slots()
* Fix tvu test
2019-03-12 17:42:53 -07:00
b418525464
Update current leader information in metrics and dashboard
2019-03-11 17:43:59 -07:00
0c592c52f6
Wake up replay stage when the poh bank is cleared. ( #3211 )
...
* wake up replay stage when the poh bank is cleared
* bump ticks per second
* Increase ticks per slot to match faster tick rate
* Remove check that working bank must be the bank for the greatest slot
* Make start_leader() skip starting TPU for slots we've already been leader for
2019-03-11 13:58:23 -07:00
195a880576
pass Pubkeys as refs, copy only where values needed ( #3213 )
...
* pass Pubkeys as refs, copy only where values needed
* Pubkey is pervasive
* fixup
2019-03-09 19:28:43 -08:00
0acdbc0d03
plumb staking_account and voting_keypair from multinode-demo to Vote ( #3199 )
...
* plumb staking_account and voting_keypair from bash to Vote
2019-03-08 19:29:08 -07:00
02eb234399
Fix TVU and PoH Recorder going out of sync ( #3164 )
...
* Fix broadcast_stage error
* Account for very fast ticks in tick verification
2019-03-07 15:49:07 -08:00
a3cab470d3
Rename ClusterInfo::new_with_keypair() to ClusterInfo::new()
2019-03-07 13:05:42 -08:00
bb93504965
Rename ClusterInfo::new() to ClusterInfo::new_with_invalid_keypair()
2019-03-07 13:05:42 -08:00
1c0cfb17a3
Start leader based on Poh tick height. ( #3084 )
...
* Start leader based on poh and test
* Equalize validator and leader stakes in LocalCluster
* Clear WorkingBank on poh_recorder reset
2019-03-05 17:56:51 -08:00
6ab6e6cb9b
Clean up exit flag handing across TVU
2019-03-04 21:26:50 -08:00
794e961328
use Bank's notion of leader_id where possible ( #3119 )
2019-03-04 18:40:47 -08:00
6d82123125
rename bank_id to bank_slot
2019-03-04 17:10:27 -08:00
1654199b23
Use PohRecorder to synchronize instead of rotate. ( #3080 )
2019-03-03 16:44:06 -08:00
1edf6c361e
Move Vote program out of the SDK
2019-03-02 16:44:36 -08:00
a94880574b
block_hash => blockhash
2019-03-02 12:13:30 -07:00
0f1582c196
cargo fmt
2019-03-02 12:13:30 -07:00
258cf21416
Purge remaining last_id (now called block_hash)
2019-03-02 12:13:30 -07:00
carllin
Michael Vines
Tyera Eulberg
Greg Fitzgerald
sakridge
Jack May
Justin Starry
Carl
Pankaj Garg
anatoly yakovenko
Sunny Gleason
sakridge
Parth
Sagar Dhawan
TristanDebrunner
Dan Albert
Rob Walker
Justin Starry
Tyera Eulberg
Sathish
Anatoly Yakovenko