Commit Graph

83 Commits

Author SHA1 Message Date
Sagar Dhawan a07b17b9b5
Drop older slots in the ledger (#5188)
* Add facility to delete blocktree columns in range

* Add ledger cleanup service

* Add local_cluster test
2019-07-20 13:13:55 -07:00
Justin Starry bf5bce50a4
Fix stake pruning test (#5124) 2019-07-16 13:20:03 -04:00
Pankaj Garg 1c966aac25 Facility to generate a blocktree prune list using ledger tool (#5041)
automerge
2019-07-12 16:58:13 -07:00
Sagar Dhawan 35ec7a5156
Decouple turns from segments in PoRep (#5004)
* Decouple Segments from Turns in Storage

* Get replicator local cluster tests running in a reasonable amount of time

* Fix unused imports

* Document new RPC APIs

* Check for exit while polling
2019-07-10 13:33:29 -07:00
Sagar Dhawan b8e7736af2 Move SLOTS_PER_SEGMENT to genesis (#4992)
automerge
2019-07-09 16:48:40 -07:00
Rob Walker f777a1a74c
groom replay_stage and poh_recorder (#4961)
* groom replay_stage and poh_recorder

* fixup

* fixup

* don't freeze() parent, need to review bank_forks and maybe vote...
2019-07-09 15:36:30 -07:00
carllin 22ef3c7c54
Blob verify (#4951)
* Ensure signable data is not out of range

* Add a broadcast stage that puts bad sizes in blobs

* Resign blob after modifyign size

* Remove assertions that fail when size != meta.size
2019-07-08 18:21:52 -07:00
carllin fc180f4cbf
Halve stake of malicious validator (#4937) 2019-07-05 15:45:39 -07:00
Sagar Dhawan 55b8ff72d0
Enable parallel tests to reduce test time (#4919)
* Add crate to serialize some tests

* Ignore unused attribute warning

* Enable parallel run in CI

* Try to fix lograte tests

* Fix interdependent counter tests
2019-07-02 17:35:03 -07:00
Pankaj Garg 3615445a12
Broadcast run for injecting fake blobs in turbine (#4889)
* Broadcast run for injecting fake blobs in turbine

* address review comments

* new local cluster test that uses fake blob broadcast

* added a test to make sure tvu_peers ordering is guaranteed
2019-07-01 17:54:03 -07:00
carllin 6cec61dcfc
Re-enable tests (#4848) 2019-06-27 12:09:14 -07:00
Justin Starry 861d6468ca Stake weighted pruning for the gossip network (#4769)
* Stake weighted pruning

* Fix compile error

* Fix clippy errors

* Add helper for creating a connected staked network

* Bug fixes and test groundwork

* Small refactor

* Anatoly's feedback and tests

* Doc updates

* @rob-solana's feedback

* Fix test bug and add log trace

* @rob-solana's feedback
2019-06-26 00:30:16 -07:00
Michael Vines 206e62271b Ignore flaky test_two_unbalanced_stakes (#4794)
automerge
2019-06-23 20:55:43 -07:00
carllin aacb38864c
Mark dead forks in replay stage (#4715)
* Add DeadSlots column family

* Filter dead forks from get_slots_since

* Mark erroring slots as dead in replay stage, add test

* Mark dead forks in progress instead of removing them

* Fix logging process_entries failures in replay_stage

* Unignore test_fail_entry_verification_leader
2019-06-20 15:50:41 -07:00
Michael Vines 4a32bc48d2 ignore unstable test_repairman_catchup 2019-06-20 09:24:11 -07:00
carllin 46bb79df29
Support for custom BroadcastStage in local cluster tests (#4716)
* Refactor BroadcastStage to support custom implementations, add FailEntryVerificationBroadcastRun implementation

* Plumb switch on broadcast type through validator

* Add test for validator generating non-verifiable entries to local_cluster

* Fix bad initializers

* Refactor broadcast run code into utils
2019-06-19 00:13:19 -07:00
Sagar Dhawan cc48773b03 Add "download from replicator" utility (#4709)
automerge
2019-06-17 18:12:13 -07:00
Rob Walker 4c6a6d63bf
add MiningPools, fund validator MiningPools from inflation (#4676)
* add MiningPool fund validator MinigPools from inflation

* fixup

* finish rename of MINIMUM_SLOT_LENGTH to MINIMUM_SLOTS_PER_EPOCH

* deterministic miningpool location

* point_value, not credit_value... use f64
2019-06-14 11:38:37 -07:00
carllin 8c1b9a0b67
Data plane verification (#4639)
* Add signature to blob

* Change Signable trait to support returning references to signable data

* Add signing to broadcast

* Verify signatures in window_service

* Add testing for signatures to erasure

* Add RPC for getting current slot, consume RPC call in test_repairman_catchup for more deterministic results
2019-06-12 16:43:05 -07:00
anatoly yakovenko 82df267ec9
s/avalanche/turbine (#4561)
* s/avalanche/turbine/g
2019-06-06 12:48:40 -07:00
Pankaj Garg 70a16e91a5
Randomize avalanche broadcast peer table for each blob (#4529)
* fix clippy warnings
2019-06-03 20:38:05 -07:00
Pankaj Garg 3574469052
Add random distribution for avalanche peers (#4493)
* Add random distribution for avalanche peers

* fix clippy warnings

* bug fixes

* nits
2019-06-01 07:55:43 -07:00
Sathish 182096dc1a
Create bank snapshots (#4244)
* Revert "Revert "Create bank snapshots (#3671)" (#4243)"

This reverts commit 81fa69d347.

* keep saved and unsaved copies of status cache

* fix format check

* bench for status cache serialize

* misc cleanup

* remove appendvec storage on purge

* fix accounts restore

* cleanup

* Pass snapshot path as args

* Fix clippy
2019-05-30 21:31:35 -07:00
carllin 045d4d5294
Unignore test test_repairman_catchup (#4484) 2019-05-30 13:21:12 -07:00
carllin c82f4a1b6d
Unignore test_repairman_catchup 2019-05-29 21:59:41 -07:00
carllin 348c2263ba
Remove genesis blockhash (#4471)
* Remove genesis blockhash

* Remove genesis blockhash from tests

* Fix golden
2019-05-29 17:29:02 -07:00
Sagar Dhawan 335dfdc4d5
Fix Gossip skipping push for some values (#4463)
* Make gossip skip over values from Pruned nodes

* Add test and init blooms to contain the origin
2019-05-28 18:39:40 -07:00
Michael Vines 2adb98a4a0 Ignore flaky test_repairman_catchup (#4439)
automerge
2019-05-26 12:24:20 -07:00
carllin d772a27936 Plumb ClusterInfoRepairListener (#4428)
automerge
2019-05-24 19:20:09 -07:00
Michael Vines cfe5afd34c
_id => _pubkey variable renaming (#4419)
* wallet: rename *_account_id to *_account_pubkey

* s/from_id/from_pubkey/g

* s/node_id/node_pubkey/g

* s/stake_id/stake_pubkey/g

* s/voter_id/voter_pubkey/g

* s/vote_id/vote_pubkey/g

* s/delegate_id/delegate_pubkey/g

* s/account_id/account_pubkey/g

* s/to_id/to_pubkey/g

* s/my_id/my_pubkey/g

* cargo fmt

* s/staker_id/staker_pubkey/g

* s/mining_pool_id/mining_pool_pubkey/g

* s/leader_id/leader_pubkey/g

* cargo fmt

* s/funding_id/funding_pubkey/g
2019-05-23 23:20:04 -07:00
Michael Vines 94beb4b8c2
More fullnode -> validator renaming (#4414)
* s/fullnode_config/validator_config/g

* s/FullnodeConfig/ValidatorConfig/g

* mv core/lib/fullnode.rs core/lib/validator.rs

* s/Fullnode/Validator/g

* Add replicator-x.sh

* Rename fullnode.md to validator.md

* cargo fmt
2019-05-23 22:05:16 -07:00
Michael Vines 2ed77b040a
create_genesis_block() now returns a struct (#4403) 2019-05-22 20:39:00 -07:00
Michael Vines f079a78c5e Remove fee arg from system_transaction::* (#4346)
automerge
2019-05-20 10:03:19 -07:00
Michael Vines 392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet (#4292)
* Remove unused PohServiceConfig::Step

* Clarify variable name

* Poh::hash() now takes an iteration counter

* man -> max

* Inline functions with single call site

* Move PohServiceConfig into GenesisBlock

* Add plumbing to enable real PoH on testnets

* Batch hashes to improve PoH hash rate

* Ensure a constant hashes_per_tick

* Remove PohEntry mixin field

* Poh/PohEntry no longer maintains tick_height

* Ensure a constant hashes_per_tick

* ci/localnet-sanity.sh: Use real PoH

* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
2019-05-18 14:01:36 -07:00
Rob Walker 39e85a3e53
kill some bs58 (#4316)
* kill some bs58

* fixup
2019-05-16 21:43:18 -07:00
Sagar Dhawan a0ffbf50a5
Correctly remove replicator from data plane after its done repairing (#4301)
* Correctly remove replicator from data plane after its done repairing

* Update discover to report nodes and replicators separately

* Fix print and condition to be spy
2019-05-16 07:14:58 -07:00
Sagar Dhawan fd3f2cb910
Add Storage accounts for all nodes (#4298)
* Setup storage keypairs for all nodes

* Clean up naming

* clippy

* Update arg value_names
2019-05-15 15:19:29 -07:00
Rob Walker 0c1191c3ee
rework staking_utils (#4283) 2019-05-14 16:15:51 -07:00
carllin 3bd921264a
Move EpochSchedule into own module (#4272) 2019-05-13 16:24:32 -07:00
Michael Vines a2e3a92b01
Extend GetBlockHash RPC API to include the fee scehdule for using the returned blockhash (#4222) 2019-05-13 12:49:37 -07:00
Rob Walker 81fa69d347
Revert "Create bank snapshots (#3671)" (#4243)
This reverts commit abf2b300da.
2019-05-09 19:27:27 -07:00
Sathish abf2b300da Create bank snapshots (#3671)
* Be able to create bank snapshots

* fix clippy

* load snapshot on start

* regenerate account index from the storage

* Remove rc feature dependency

* cleanup

* save snapshot for slot 0
2019-05-09 19:27:06 -07:00
carllin 575a0e318b
Add newly completed slots signal to Blocktree (#4225)
* Add channel to blocktree for communicating when slots are completed

* Refactor RepairService options into a RepairStrategy
2019-05-09 14:10:04 -07:00
carllin b8fd51e97d
Add new gossip structure for supporting repairs (#4205)
* Add Epoch Slots to gossip

* Add new gossip structure to support Repair

* remove unnecessary clones

* Setup dummy fast repair in repair_service

* PR comments
2019-05-08 13:50:32 -07:00
Michael Vines 55e3b7d380
Storage transactions are now paid for by a system account (#4193)
* Pay program loading fees from a system account

* Pay transaction fees from a system account
2019-05-07 15:01:10 -07:00
Sagar Dhawan 2107e15bd3
Reduce Avalanche redundancy and implement traditional fanout (#4174)
* Reduce Avalanche redundancy and implement traditional fanout

* Revert tiny fanout

* Update diagrams and docs based on review comments
2019-05-07 13:24:58 -07:00
Michael Vines 4f3b22d04e
Minor code restyling, no functional changes 2019-05-07 12:35:29 -07:00
Rob Walker 8e400fc4bd
rework genesis (passive staking groundwork) (#4187)
* rework genesis

* fixup
2019-05-07 11:16:22 -07:00
carllin 297328ff9a Fix improper tick sleeping time in test (#4155)
automerge
2019-05-03 20:15:10 -07:00
Sagar Dhawan a7b695c27a
Change replicators to slot-based (#4118) 2019-05-03 16:27:53 -07:00