blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
6,232 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

726 Commits

Author SHA1 Message Date
Pankaj Garg b5324063f1
Use thread pools for rayon par_iter (#4473) 
* Use thread pools for rayon par_iter

* address review comments

* cleanup
 2019-05-29 17:16:36 -07:00
Sagar Dhawan 6ed071c4dd
Fix storage stage operating on empty slots (#4474) 
* Fix storage stage operating on empty slots

* Reduce fn argument count

* Fix tests
 2019-05-29 15:01:20 -07:00
sakridge 4404634b14
Coalesce packets better (#4456) 2019-05-29 12:17:50 -07:00
Rob Walker 6a1de33138
tighten up packets_to_blobs (#4464) 
* tighten up packets_to_blobs

* missed a test
 2019-05-29 10:08:35 -07:00
carllin 534244b322
Fix set_roots to use cached bank parents instead of searching blocktree (#4466) 2019-05-29 09:43:22 -07:00
Sagar Dhawan 335dfdc4d5
Fix Gossip skipping push for some values (#4463) 
* Make gossip skip over values from Pruned nodes

* Add test and init blooms to contain the origin
 2019-05-28 18:39:40 -07:00
Pankaj Garg c8a03c7b3d
Save RNG for generating random storage sampling offsets (#4450) 
* Save RNG for generating random storage sampling offsets

* fix clippy

* fix stable-perf

* fix chacha
 2019-05-28 14:14:46 -07:00
sakridge 5340800cea
Add some optimizing to ThinClient (#4112) 
Can create a multi-socketed ThinClient which will use request time
from get_recent_blockhash to tune for the best node to talk to.
 2019-05-27 20:54:44 -07:00
dependabot-preview[bot] 13c2e50b38 Bump sys-info from 0.5.6 to 0.5.7 (#4445) 
automerge
 2019-05-27 20:31:12 -07:00
carllin 65f89d6729
Bump logging level of validator procsesing errors (#4442) 2019-05-27 16:19:38 -07:00
Michael Vines ed6a438c51
v0.16.0 2019-05-26 19:42:15 -07:00
Michael Vines 2adb98a4a0 Ignore flaky test_repairman_catchup (#4439) 
automerge
 2019-05-26 12:24:20 -07:00
carllin aa3c00231a Fix should_update check to update EpochSlots in gossip (#4435) 
automerge
 2019-05-25 06:44:40 -07:00
carllin d772a27936 Plumb ClusterInfoRepairListener (#4428) 
automerge
 2019-05-24 19:20:09 -07:00
Rob Walker 0302f13b97
add datapoint for corrupt vote_account (#4424) 2019-05-24 18:34:56 -07:00
Pankaj Garg 1f71d05299 remove copying of forwarded packets (#4425) 
automerge
 2019-05-24 17:35:09 -07:00
Sagar Dhawan 4c9b7c9d2b
Submit all incoming proofs as valid (#4377) 2019-05-24 14:49:10 -07:00
carllin cf4bb70d80 Rename id to pubkey in cluster_info_repair_listener (#4421) 2019-05-24 04:31:32 -07:00
carllin 57f8a15b96
Fix issues in ClusterInfoRepairListener (#4418) 
* Sort repairmen before shuffling so order is the same across all validators

* Reduce repair redundancy to 1 for now

* Fix local cache of roots so that 1) Timestamps are only updated to acknowledge a repair was sent 2) Roots are updated even when timestamps aren't updated to keep in sync with network

* Refactor code, add test
 2019-05-24 00:47:51 -07:00
Michael Vines cfe5afd34c
_id => _pubkey variable renaming (#4419) 
* wallet: rename *_account_id to *_account_pubkey

* s/from_id/from_pubkey/g

* s/node_id/node_pubkey/g

* s/stake_id/stake_pubkey/g

* s/voter_id/voter_pubkey/g

* s/vote_id/vote_pubkey/g

* s/delegate_id/delegate_pubkey/g

* s/account_id/account_pubkey/g

* s/to_id/to_pubkey/g

* s/my_id/my_pubkey/g

* cargo fmt

* s/staker_id/staker_pubkey/g

* s/mining_pool_id/mining_pool_pubkey/g

* s/leader_id/leader_pubkey/g

* cargo fmt

* s/funding_id/funding_pubkey/g
 2019-05-23 23:20:04 -07:00
Michael Vines 94beb4b8c2
More fullnode -> validator renaming (#4414) 
* s/fullnode_config/validator_config/g

* s/FullnodeConfig/ValidatorConfig/g

* mv core/lib/fullnode.rs core/lib/validator.rs

* s/Fullnode/Validator/g

* Add replicator-x.sh

* Rename fullnode.md to validator.md

* cargo fmt
 2019-05-23 22:05:16 -07:00
Rob Walker 35e8f966e3
add freeze_lock() and fix par_process_entries() failure to detect self conflict (#4415) 
* add freeze_lock and fix par_process_entries failure to detect self conflict

* fixup

* fixup
 2019-05-23 17:35:15 -07:00
Michael Vines b37d2fde3d
Add storage mining pool (#4364) 
* Add storage mining pool

* Set gossip port

* Add create-storage-mining-pool-account wallet command

* Add claim-storage-reward wallet command

* Create storage account upfront

* Add storage program to genesis

* Use STORAGE_ACCOUNT_SPACE

* Fix tests

* Add wallet commands to create validator/replicator storage accounts

* Add create_validator_storage_account()

* Storage stage no longer implicitly creates a storage account
 2019-05-23 14:50:23 -07:00
carllin 6a9e0bc593 Change EpochSlots to use BtreeSet so that serialization/deserialization returns the same order (#4404) 
automerge
 2019-05-23 03:50:41 -07:00
carllin 591fd72e0b
Implement listener for serving repairs through Repairman protocol (#4306) 
* Make listener for serving repairs through Repairman protocol
 2019-05-23 03:10:16 -07:00
Michael Vines 2ed77b040a
create_genesis_block() now returns a struct (#4403) 2019-05-22 20:39:00 -07:00
Pankaj Garg b8f6c17dee Don't filter transactions if we are buffering it locally (#4395) 
automerge
 2019-05-22 17:54:28 -07:00
Pankaj Garg 36019cb1e3
Tweaks to real PoH based on perf testing (#4396) 
* Some counters for real poh perf analysis

* more metrics

* Comment on CPU affinity change, and reduce hash batch size based on TPS perf

* review comments
 2019-05-22 15:54:24 -07:00
Tyera Eulberg 99d2428041
Transaction format changes toward Credit-Only accounts (#4386) 
* Add num_readonly_accounts slice

* Impl programs in account_keys

* Emulate current account-loading functionality using program-account_keys (breaks exchange_program_api tests)

* Fix test

* Add temporary exchange faucet id

* Update chacha golden

* Split num_credit_only_accounts into separate fields

* Improve readability

* Move message field constants into Message

* Add MessageHeader struct and fixup comments
 2019-05-22 18:23:16 -04:00
Pankaj Garg eef2bdf690 Add CPU affinity for PoH service thread (#4394) 
automerge
 2019-05-22 14:21:43 -07:00
dependabot[bot] 190656967d Bump nix from 0.13.0 to 0.14.0 (#4382) 
Bumps [nix](https://github.com/nix-rust/nix) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.13.0...v0.14.0)

Signed-off-by: dependabot[bot] <support@dependabot.com>
 2019-05-22 14:07:35 -07:00
sakridge 1d7a758c97
Bump perf libs version to fix out buffer sizing (#4385) 2019-05-22 13:00:03 -07:00
Rob Walker 578c2ad3ea
add bank hash to votes (#4381) 2019-05-21 21:45:38 -07:00
sakridge 52d453d06f
Reduce broadcast prints (#4368) 2019-05-21 17:02:19 -07:00
Sagar Dhawan c9ba9e4eb7 Add storage space constant (#4366) 
automerge
 2019-05-21 11:07:13 -07:00
sakridge f877fb8c8f
Don't print leader update message unless leader actually updates (#4365) 2019-05-21 11:06:56 -07:00
Michael Vines 6374e69a69 Add mining pool wallet commands (#4360) 
automerge
 2019-05-21 07:32:38 -07:00
carllin 1a77486f8e
Make RootedSlotsIterator for traversing slots on the root fork (#4361) 2019-05-20 23:09:00 -07:00
Rob Walker ead15d294e
add get_epoch_vote_accounts rpc (#4317) 
* add get_epoch_vote_accounts rpc

* fixups

* documentation and type updates
 2019-05-20 22:21:13 -07:00
carllin f1e5edee14
Modify Roots Column To Support Multiple Roots (#4321) 
* Fix 1) Roots column family to handle storing multiple slots, 2) Store all slots on the rooted path in the roots column family
 2019-05-20 19:04:18 -07:00
sakridge 7153abd483
Revert "Performance tweaks (#4340)" (#4350) 
* Revert "Performance tweaks (#4340)"

This reverts commit 55cee5742f.

* Revert Rc change
 2019-05-20 17:48:42 -07:00
Sagar Dhawan af82b0dce9 Fix sending root slots instead of current slot (#4354) 
automerge
 2019-05-20 15:01:55 -07:00
Rob Walker d4da2fbacd
fix bench warnings (#4356) 2019-05-20 14:32:23 -07:00
Rob Walker 86e03a6d1b
support issuing vote instructions from system account (#4338) 
* issue vote instructions from system account

* fixup

* bring back KeypairUtil
 2019-05-20 13:32:32 -07:00
Michael Vines 114e2989fa Improve PoH unit test asserts (#4351) 
automerge
 2019-05-20 13:02:44 -07:00
sakridge 7024c73e9b
Revert banking threads increase which seems to be slower in testing. (#4349) 2019-05-20 11:42:37 -07:00
Michael Vines f079a78c5e Remove fee arg from system_transaction::* (#4346) 
automerge
 2019-05-20 10:03:19 -07:00
Michael Vines 6365c4c061
Use cleanup (#4347) 2019-05-20 09:58:27 -07:00
sakridge 55cee5742f
Performance tweaks (#4340) 
* Use Rc to prevent clone of Packets

* Fix min => max in banking_stage threads.

Coalesce packet buffers better since a larger batch will
be faster through banking and sigverify.

Deconstruct batches into banking_stage from sigverify since
sigverify likes to accumulate batches but then a single banking_stage
thread will be stuck with a large batch. Maximize parallelism by
creating more chunks of work for banking_stage.
 2019-05-20 09:15:00 -07:00
Michael Vines 034eda4546 Fix a couple replicator nits (#4345) 
automerge
 2019-05-20 08:55:45 -07:00
Sagar Dhawan 06eb2364f2
Handle missed slots in storage stage (#4337) 
* Handle missed slots in storage stage

* Fix test compile

* Make test use the new missed slot support
 2019-05-18 15:24:50 -07:00
Michael Vines 392a39dd54
Poh subsystem cleanup, genesis plumbing, enable real PoH on edge testnet (#4292) 
* Remove unused PohServiceConfig::Step

* Clarify variable name

* Poh::hash() now takes an iteration counter

* man -> max

* Inline functions with single call site

* Move PohServiceConfig into GenesisBlock

* Add plumbing to enable real PoH on testnets

* Batch hashes to improve PoH hash rate

* Ensure a constant hashes_per_tick

* Remove PohEntry mixin field

* Poh/PohEntry no longer maintains tick_height

* Ensure a constant hashes_per_tick

* ci/localnet-sanity.sh: Use real PoH

* Rework Poh/PohService to keep PohRecorder unlocked as much as possible while hashing
 2019-05-18 14:01:36 -07:00
Pankaj Garg 9476fe5ce3
Use log levels for datapoint metrics (#4335) 
* Use log levels for datapoint metrics

* address review comments

* fix cyclomatic complexity
 2019-05-17 17:34:05 -07:00
Sagar Dhawan 788290ad82 Rework Storage Program to accept multiple proofs per segment (#4319) 
automerge
 2019-05-17 14:52:54 -07:00
dependabot[bot] 6b5bcfaa58 Bump libc from 0.2.54 to 0.2.55 (#4324) 
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.54 to 0.2.55.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.54...0.2.55)

Signed-off-by: dependabot[bot] <support@dependabot.com>
 2019-05-17 13:24:12 -07:00
sakridge b114bc3674
Add benchmark for sigverify stage (#4320) 2019-05-17 11:09:42 -07:00
Pankaj Garg b2ce5dc9f5
Adjust log level for counter metrics (#4323) 2019-05-17 07:00:06 -07:00
Pankaj Garg e920191de0
Rate limit metrics per log level (#4313) 
* Rate limit metrics per log level

* fix tests
 2019-05-16 22:27:05 -07:00
Rob Walker 39e85a3e53
kill some bs58 (#4316) 
* kill some bs58

* fixup
 2019-05-16 21:43:18 -07:00
carllin b3e45fd6b7
Add erroring tx to unexpected validator error logging (#4314) 
* Add tx logging to error

* Add tx logging to unexpected validator errors
 2019-05-16 14:59:22 -07:00
Rob Walker 87414de3e2
switch over to passive stakes (#4295) 
* add failing test

* switch over to passive stakes

* test multiple stakers
 2019-05-16 08:23:31 -07:00
Sagar Dhawan a0ffbf50a5
Correctly remove replicator from data plane after its done repairing (#4301) 
* Correctly remove replicator from data plane after its done repairing

* Update discover to report nodes and replicators separately

* Fix print and condition to be spy
 2019-05-16 07:14:58 -07:00
carllin abd7f6b090 Change slot_meta_iterator() to return an iterator not a cursor (#4303) 
automerge
 2019-05-15 18:28:23 -07:00
Sagar Dhawan d8735df1de
Update replicator to use cluster_info instead of cached client (#4302) 2019-05-15 18:14:04 -07:00
dependabot[bot] 481853e1b1 Bump reqwest from 0.9.16 to 0.9.17 (#4296) 
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.9.16 to 0.9.17.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.9.16...v0.9.17)

Signed-off-by: dependabot[bot] <support@dependabot.com>
 2019-05-15 16:02:17 -07:00
Sagar Dhawan fd3f2cb910
Add Storage accounts for all nodes (#4298) 
* Setup storage keypairs for all nodes

* Clean up naming

* clippy

* Update arg value_names
 2019-05-15 15:19:29 -07:00
Sagar Dhawan 4576250342
Fix bug in storage processor and remove duplicate Constant (#4294) 
* Fix bug in storage processor and remove duplicate Constant

* Add test

* Bump replicator timeout
 2019-05-15 13:28:56 -07:00
Rob Walker 628128b376
add passive staking to local_cluster (#4285) 
* add passive staking to local_cluster

* add stake_program to genesis

* use equal stakes in local_cluster tests
 2019-05-15 12:15:31 -07:00
Sagar Dhawan 916017ca2c
Fix repair for a range of slots (#4286) 
* Fix repair for a range of slots

* Delete RepairInfo
 2019-05-15 11:37:20 -07:00
dependabot[bot] 1d327a5167 Bump bincode from 1.1.3 to 1.1.4 (#4290) 
Bumps [bincode](https://github.com/TyOverby/bincode) from 1.1.3 to 1.1.4.
- [Release notes](https://github.com/TyOverby/bincode/releases)
- [Commits](https://github.com/TyOverby/bincode/compare/v1.1.3...v1.1.4)

Signed-off-by: dependabot[bot] <support@dependabot.com>
 2019-05-15 09:47:55 -06:00
carllin a79fbbafc9
SlotMeta is incorrectly updated on insertion of erasure blobs (#4289) 
* Fix put_coding_blob_bytes to properly update slotmetas and chaining
 2019-05-15 00:28:31 -07:00
Sagar Dhawan 1d54d29076
Fix reading ledger for chacha encrypt (#4288) 2019-05-14 16:59:17 -07:00
Sagar Dhawan 10b9a4806b
Fix incorrect genesis blockhashes on restart (#4287) 2019-05-14 16:32:44 -07:00
Rob Walker 0c1191c3ee
rework staking_utils (#4283) 2019-05-14 16:15:51 -07:00
Rob Walker 18b386cd10
remove unused make_active_set_entries (#4284) 2019-05-14 15:08:49 -07:00
Rob Walker 216e9a61a0
expose stake directly from bank (#4281) 2019-05-14 13:35:14 -07:00
Rob Walker 0f498e6265
remove unused filter_zero_balance (#4279) 2019-05-14 10:44:29 -07:00
Rob Walker e8ad822111
get program names from programs (#4273) 
* get program names from programs

* fixup
 2019-05-14 10:44:16 -07:00
carllin 65a82ebf50 Fix broken tip (#4278) 
automerge
 2019-05-14 02:35:32 -07:00
carllin e20a8329d3
Add API to iterate over slot's blobs (#4276) 2019-05-13 22:04:54 -07:00
Sagar Dhawan 88c2d0fad4
Add genesis blockhash to replicators and blob filter for window (#4275) 
* Add genesis blockhash to replicators and blob filter for window

* Fixes to mining submission and ledger download

* Add todo over sleep

* Update log
 2019-05-13 21:19:51 -07:00
carllin 3bd921264a
Move EpochSchedule into own module (#4272) 2019-05-13 16:24:32 -07:00
carllin 7501ed65e5
Initialize and Update EpochSlots in RepairService (#4255) 
* Initialize EpochSlots in RepairService

* Fix flaky test
 2019-05-13 15:37:50 -07:00
Michael Vines 2eaa64c4e8
valhashators -> validators 2019-05-13 15:29:23 -07:00
Pankaj Garg c9b86018c6
Filter out all unprocessed transactions before forwarding them (#4266) 
* Filter out all unprocessed transactions before forwarding them

* fix clippy
 2019-05-13 14:40:05 -07:00
Mark E. Sinclair 796000e96f
Improve erasure metrics (#4268) 
* Improve erasure metrics

* Simplify metrics submission
 2019-05-13 16:04:43 -05:00
Sagar Dhawan 5e91f8f59d
Update reported tx count to exclude errors (#4201) 2019-05-13 13:23:52 -07:00
Michael Vines a2e3a92b01
Extend GetBlockHash RPC API to include the fee scehdule for using the returned blockhash (#4222) 2019-05-13 12:49:37 -07:00
Tyera Eulberg 1b68da7572 Use solana-ed25519-dalek v0.2.0 (#4264) 
automerge
 2019-05-13 09:51:59 -07:00
Pankaj Garg 133be2df51
Check for transaction forwarding delay to detect an expired transaction before forwarding it (#4249) 
Also refactored code for forwarding packets, and added test for it
 2019-05-10 14:28:38 -07:00
Tyera Eulberg 06a93dcb43
Move to solana-ed25519-dalek (#4248) 2019-05-10 15:26:46 -06:00
carllin ad7f04a245
Add genesis_blockhash to erasure blobs so they don't get filtered out by window_service (#4247) 2019-05-10 14:06:31 -07:00
Jack May f567877d1d
Cleanup metrics (#4230) 2019-05-10 08:33:58 -07:00
carllin 9881820444
RepairService saves db_iterator instead of reconstructing on every search (#4242) 2019-05-09 19:57:51 -07:00
Rob Walker 81fa69d347
Revert "Create bank snapshots (#3671)" (#4243) 
This reverts commit abf2b300da.
 2019-05-09 19:27:27 -07:00
Sathish abf2b300da Create bank snapshots (#3671) 
* Be able to create bank snapshots

* fix clippy

* load snapshot on start

* regenerate account index from the storage

* Remove rc feature dependency

* cleanup

* save snapshot for slot 0
 2019-05-09 19:27:06 -07:00
carllin 575a0e318b
Add newly completed slots signal to Blocktree (#4225) 
* Add channel to blocktree for communicating when slots are completed

* Refactor RepairService options into a RepairStrategy
 2019-05-09 14:10:04 -07:00
Sagar Dhawan a031b09190 Add replicator support to multinode demo (#4221) 
automerge
 2019-05-09 13:43:39 -07:00
sakridge 5d0d467287
fix banking_stage benches (#4231) 2019-05-09 11:20:26 -07:00
carllin bba94c43b9 Add BankForks to RepairService (#4223) 
automerge
 2019-05-08 18:51:43 -07:00
sakridge 9cdffc7d64 Don't push empty vecs into the unprocessed buffers (#4214) 2019-05-08 17:58:07 -07:00
carllin b8fd51e97d
Add new gossip structure for supporting repairs (#4205) 
* Add Epoch Slots to gossip

* Add new gossip structure to support Repair

* remove unnecessary clones

* Setup dummy fast repair in repair_service

* PR comments
 2019-05-08 13:50:32 -07:00
Rob Walker a80176496d
add /target/ to .gitignore files for all crates (#4217) 
* add /target/ to .gitignore files for all crates

* shellcheck
 2019-05-08 12:15:05 -07:00
Pankaj Garg 1a2b131ceb
Don't forward transactions that are expired or failed signature check (#4199) 2019-05-08 10:32:25 -07:00
carllin 791ee411a5
Add root to bank forks (#4206) 2019-05-07 23:34:10 -07:00
Jack May 427963f554
Core depends on vote and budget program directly (#4204) 2019-05-07 22:41:47 -07:00
Rob Walker b49f8c0984
reduce replicode, introduce passive staking support (#4207) 2019-05-07 22:22:43 -07:00
dependabot[bot] 674a49f8d7 Bump serde_derive from 1.0.90 to 1.0.91 (#4172) 
Bumps [serde_derive](https://github.com/serde-rs/serde) from 1.0.90 to 1.0.91.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.90...v1.0.91)

Signed-off-by: dependabot[bot] <support@dependabot.com>
 2019-05-07 20:05:41 -07:00
carllin 69eeb7cf08
Fix parent record locks usage in child banks (#4159) 
* Introduce record locks on txs that will be recorded

* Add tests for LockedAccountsResults

* Fix broken bench

* Exit process_entries on detecting conflicting locks within same entry
 2019-05-07 15:51:35 -07:00
Michael Vines 55e3b7d380
Storage transactions are now paid for by a system account (#4193) 
* Pay program loading fees from a system account

* Pay transaction fees from a system account
 2019-05-07 15:01:10 -07:00
Sagar Dhawan 2107e15bd3
Reduce Avalanche redundancy and implement traditional fanout (#4174) 
* Reduce Avalanche redundancy and implement traditional fanout

* Revert tiny fanout

* Update diagrams and docs based on review comments
 2019-05-07 13:24:58 -07:00
Michael Vines 4f3b22d04e
Minor code restyling, no functional changes 2019-05-07 12:35:29 -07:00
Michael Vines 2c78a93001
GenesisBlock::new(X) => create_genesis_block(X) 2019-05-07 12:34:17 -07:00
Rob Walker 8e400fc4bd
rework genesis (passive staking groundwork) (#4187) 
* rework genesis

* fixup
 2019-05-07 11:16:22 -07:00
Pankaj Garg 29c2a63c8b
Retry transactions that failed due to account lock (#4184) 
* added test
 2019-05-07 10:23:02 -07:00
dependabot[bot] 57038529e0 Bump hex-literal from 0.1.4 to 0.2.0 (#4185) 
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.1.4 to 0.2.0.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.1.4...hex-literal-v0.2.0)

Signed-off-by: dependabot[bot] <support@dependabot.com>
 2019-05-07 07:14:16 -07:00
Pankaj Garg e9f80e5542
Don't use default ticks per slot in calculating next slot leader (#4169) 2019-05-06 11:23:06 -07:00
Greg Fitzgerald a19df7a36c
Add type annotations for external crates (#4125) 2019-05-06 10:11:50 -06:00
Tyera Eulberg 71f9b44687
Add Confirmations parameter to RPC Subscriptions (#4154) 
* Add optional depth parameter to pubsub, and store in subscriptions

* Pass bank_forks into rpc_subscription; add method to check depth before notify and impl for account subscriptions

* Impl check-depth for signature subscriptions

* Impl check-depth for program subscriptions

* Plumb fork id through accounts

* Use fork id and root to prevent repeated account notifications; also s/Depth/Confirmations

* Write tests in terms of bank_forks

* Fixup accounts tests

* Add pubsub-confirmations tests

* Update pubsub documentation
 2019-05-06 08:31:50 -06:00
carllin 297328ff9a Fix improper tick sleeping time in test (#4155) 
automerge
 2019-05-03 20:15:10 -07:00
Pankaj Garg 3ef961fe37
Record poh ticks even when the node is not in leader schedule (#4148) 
* remove obsolete test
 2019-05-03 16:41:19 -07:00
Sagar Dhawan a7b695c27a
Change replicators to slot-based (#4118) 2019-05-03 16:27:53 -07:00
Mark E. Sinclair ed48d8323c
Reduce locking in Blocktree (#4075) 
* Reduce lock contention in blocktree

* Store root slot in separate column
 2019-05-03 16:46:02 -05:00
Michael Vines 7fe3c75c6b
Add a node-specific ip echo service to remove dependency on ifconfig.co (#4137) 2019-05-03 11:01:35 -07:00
Pankaj Garg 441e76ebeb
Index buffered transactions at the correct offset (#4126) 
* tests
 2019-05-02 19:05:53 -07:00
carllin c2dfb9900e Revert "Change forwarded metrics to be in line with fetch stage metrics (#4068)" (#4135) 
automerge
 2019-05-02 17:36:19 -07:00
Mark E. Sinclair 916458e132
Change erasure set size to 8:8 (#4129) 
* Change erasure set size to 8:8

* Change tests to be agnostic to exact set size and ratio

* Add convenience methods for setting presence
 2019-05-02 19:04:40 -05:00
Rob Walker 7f75cc8906
update nightly to 2019-05-01 (#4111) 
* update nightly to 2019-05-01

* cargo fmt

* cargo fmt

* increase bench timeout
 2019-05-01 20:08:42 -07:00
sakridge 1ab5098576
Move get_clients into gossip_service (#4109) 2019-05-01 17:14:01 -07:00
Pankaj Garg 3eec3cfac2
Cleanup banking stage in lieu of recent transaction forwarding changes (#4101) 2019-05-01 15:13:10 -07:00
Pankaj Garg 4f18fc836f
Forward transactions to the next slot leader (#4092) 
- this ensures that transactions will reach in time for the next node to process them
 2019-05-01 11:37:29 -07:00
Rob Walker cb528af4e2
fix accounts_db storage.reset() (#4094) 
* fix accounts_db storage.reset()

* fix compilation errors, remove unused, fix test_accounts_grow() failure
 2019-05-01 09:27:13 -07:00
Sagar Dhawan 9add8d0afc Add alternative to Spy Nodes that can fully participate in Gossip (#4087) 
automerge
 2019-04-30 16:42:56 -07:00
sakridge 408bdbce7a
Add non_default_stream parameter to cuda_verify (#4079) 2019-04-30 13:34:46 -07:00
carllin 1a259d4a3f Prevent Requests/Caching of leader schedules for epochs beyond confirmed roots (#4033) 
automerge
 2019-04-30 13:23:21 -07:00
carllin 73f250f03a
Make minimum warmup period 32 slots long (#4031) 
* Make minimum warmup period 32 slots long

* PR fixes
 2019-04-29 15:26:52 -07:00
carllin 3b2adbc9df Change forwarded metrics to be in line with fetch stage metrics (#4068) 
automerge
 2019-04-29 13:50:14 -07:00
Sagar Dhawan 4e41c81bc7 Fix the output from Gossip Discovery (#4067) 
automerge
 2019-04-29 13:19:24 -07:00
carllin fabba82173
ignore non-descendants of roots in blocktree (#4032) 2019-04-29 12:29:14 -07:00
anatoly yakovenko 3e14af5033
bast bank ancestor check (#4050) 2019-04-28 10:27:09 -07:00
Michael Vines 656e2649a7
v0.15.0 2019-04-27 07:38:46 -07:00
Pankaj Garg 87f1bd58b9
Reduce submit frequency of counters that are very frequent and killing metrics DB (#4027) 2019-04-26 20:37:40 -07:00
sakridge a056c1f18f
Don't send packets when msgs len is 0 (#4030) 
And don't send metrics every iteration.
 2019-04-26 17:27:31 -07:00
Jack May b912ee7fdf
Nit: Spelling (#4025) 2019-04-26 16:02:37 -07:00
Mark E. Sinclair 8b1724bb70
Serialize blocktree writes by locking the database (#4008) 
Move several private methods to free functions
 2019-04-26 10:52:10 -05:00
Michael Vines e71ab55288
Rename in-tree program_ids to be base-58 human readable (#4001) 2019-04-25 17:58:49 -07:00
Pankaj Garg 15aa07f2a0
Fix node count metrics (#3997) 
- the count was truncated to avalanche neighborhood size
 2019-04-25 13:47:38 -07:00
carllin e4536621df
Log all uncommittable errors on validator (#3999) 2019-04-25 13:37:30 -07:00
Sagar Dhawan a3c302c36a Add signature polling to SyncClient (#3996) 
automerge
 2019-04-25 12:46:40 -07:00
carllin c4d9dff590
Cherry-pick #3934 (#3982) 
* Fix inserting bogus is_last blobs into blocktree

* Check for pre-existing blob before insert

* Ignore test that performs concurrent writes on blocktree as that is not supported
 2019-04-25 00:04:49 -07:00
Sagar Dhawan cf91ff8694 Remove Thin Client from storage stage (#3976) 
automerge
 2019-04-24 22:34:10 -07:00
Mark E. Sinclair 9a40ad76bd
Fix race in erasure metadata tracking (#3962) 
* Fix erasure metadata race condition

* make erasure return the underlying error without wrapping it in the `solana::Error` type

* Add metric for erasure failures

* add tests to `ErasureMeta` indexing logic

* Add test to ensure erasure recovery failures don't cause panics
 2019-04-24 17:53:01 -05:00
Pankaj Garg 9c7ccc0e2b More metrics to track memory usage (#3966) 
automerge
 2019-04-24 11:11:30 -07:00
Sagar Dhawan 0cbac26591
Add genesis blockhash to blobs (#3953) 2019-04-23 16:24:44 -07:00
carllin ab11327e34
Fix mismatch between leader/validator bank votability (#3942) 
* Fix mismatch between leader/validator bank votability
 2019-04-23 15:32:19 -07:00
Michael Vines c309cd80aa
Add getClusterNodes/getSlotLeader JSON RPC API (#3940) 
* Minor cleanup

* Include _this_ node in the contact info trace

* Add getClusterNodes/getSlotLeader RPC API
 2019-04-23 14:46:41 -07:00
Pankaj Garg d22a1c9b1f
Use smaller batch size in sigverify stage if CUDA is not available (#3951) 2019-04-23 12:41:50 -07:00
Pankaj Garg 7372ec9e1a
Use poh would_be_leader check in banking stage to hold or forward txs (#3948) 2019-04-23 11:56:30 -07:00
Pankaj Garg de6af95061
Process forwarded packets only when the node is about to be the leader (#3935) 
* Tests and metrics
 2019-04-22 19:49:32 -07:00
Sagar Dhawan 43f7cd8149
Fix Retransmit slamming the leader with its own blobs (#3938) 2019-04-22 18:41:01 -07:00
Rob Walker 4b04c37c36
verify that blobs match a known leader for the slot (#3927) 
* validate that blobs match a known leader for the slot

* clippy
 2019-04-22 15:21:10 -07:00
sakridge 95a16426f3
Cleanup bench-exchange (#3919) 
* bench-exchange changes

Generate new keypair for each bench-exchange
Add metrics
Tweak network sleep parameters for better reliability.

* Bench-exchange contract metrics
 2019-04-22 13:16:28 -07:00
Michael Vines af9ebf1d1a
Add status logging while processing a ledger (#3916) 2019-04-20 20:17:57 -07:00
Sagar Dhawan 349e8a9462
Ensure forwarded Blobs don't break Erasure (#3907) 2019-04-20 16:44:06 -07:00
Sagar Dhawan c0bffb56df
Update Avalanche to use Epoch Stakes (#3900) 2019-04-19 22:31:40 -07:00
Sagar Dhawan 970cc32e65
Fix Data Plane computation when stakes are equal (#3913) 2019-04-19 21:07:21 -07:00
Rob Walker 3ab492ccf8
save erasure set size, initialize coding blob erasures with that size (#3910) 
* save erasure set size, initialize coding blob erasures with that size

* fixup

* fixup
 2019-04-19 20:22:51 -07:00
Rob Walker efbb573316
add an env::var driven localcluster test (#3906) 2019-04-19 15:47:03 -07:00
Rob Walker c3155a6e39
silence sigverify disabled, make disabling sigverify easier (#3901) 
* silence sigverify disabled, make disabling sigverify easier

* fixup
 2019-04-19 14:18:19 -07:00
Rob Walker 320bd66c84
handle zero num_will_fit (#3892) 
* handle zero num_will_fit

* clippy
 2019-04-19 12:02:33 -07:00
carllin 512bfc93cb
Add a cache for leader schedules (#3841) 
* Add a cache for leader schedules
 2019-04-19 02:39:44 -07:00
Rob Walker 6eac5951ed
Revert "Revert "revert-revert-erasure and erasure fixes (#3833)" (#3855)" (#3889) 
This reverts commit 596f611ede.
 2019-04-18 21:56:43 -07:00
Sagar Dhawan 475a74d37f
Reduce packet recv batches (#3894) 
* Reduce packet recv batches

* Fix type
 2019-04-18 19:24:37 -07:00
Sagar Dhawan 7e409a13cd
Ack on empty Gossip Pull Responses and keep Entrypoint around (#3881) 
* Ack on empty Gossip Pull Responses and keep Entrypoint around

* Address comments and fix test

* Update core/src/cluster_info.rs

Co-Authored-By: sagar-solana <sagar@solana.com>

* Update core/src/cluster_info.rs

Co-Authored-By: sagar-solana <sagar@solana.com>
 2019-04-18 15:12:17 -07:00
Rob Walker 67b8ad6a0f
fix entries.to_blobs() (#3882) 
* * rename Entry::serialized_size() to Entry::to_blob_size() to better
    reduce confusion with bincode, et al. and to better reflect its
    real meaning

* fix implementation of to_blob_size() to actually return what happens
    when we do entries.to_blobs() (i.e. we serialize Vec<Entry>, not Entry)

* update tests to be more rigorous

* clippy
 2019-04-18 14:45:41 -07:00
Tyera Eulberg a9cfae486c
Revert-revert migrate to ed25519-dalek crate (#3877) 
* Revert "Revert "Migrate from ring to ed25519-dalek, take 2 (#3844)" (#3868)"

This reverts commit 6a878602f2.

* Fix Signature::verify method
 2019-04-18 14:37:20 -06:00
Pankaj Garg f2187780d2
Do not forward vote transactions (#3871) 2019-04-18 11:18:49 -07:00
Tyera Eulberg 6a878602f2
Revert "Migrate from ring to ed25519-dalek, take 2 (#3844)" (#3868) 
This reverts commit e9b82bacda.
 2019-04-18 11:47:34 -06:00
Michael Vines f8543a268f
solana-gossip now displays other spy nodes and contact info age (#3867) 2019-04-18 09:48:21 -07:00
Tyera Eulberg e9b82bacda
Migrate from ring to ed25519-dalek, take 2 (#3844) 
* Migrate from ring to ed25519-dalek

* Move gen_keypair_file test to a more appropriate location

* Fixup bench-exchange and add helper fn for single deterministic keypair

* Update golden
 2019-04-18 10:38:32 -06:00
Pankaj Garg 2f4a3ed190
Use a separate channel to process votes in banking stage (#3861) 
- This will help expedite the vote processing on peer nodes
 2019-04-17 21:07:45 -07:00
Mark E. Sinclair aa21f5343a Remove broken read().unwrap() call to a non-shared packet 2019-04-17 21:24:40 -05:00
Sagar Dhawan 9c2809db21
Delete SharedPackets (#3843) 
* Delete SharedPackets

* Fix bench and sigverify
 2019-04-17 18:15:50 -07:00
Pankaj Garg 596f611ede
Revert "revert-revert-erasure and erasure fixes (#3833)" (#3855) 
This reverts commit 6bef16a6a1.
 2019-04-17 18:04:30 -07:00
Sagar Dhawan 2b3218b5f2
Fix flaky tests by waiting for test node to boot (#3845) 2019-04-17 17:50:34 -07:00
Pankaj Garg a2c8e3952f
Fixes to TPS calculation and reporting (#3836) 
Fixes to TPS calculations and reporting
 2019-04-17 15:37:01 -07:00
Rob Walker 6bef16a6a1
revert-revert-erasure and erasure fixes (#3833) 
* fix erasure, more tests for full blobs, more metrics

* Revert "Revert "Use Rust erasure library and turn on erasure (#3768)" (#3827)"

This reverts commit 4b8cb72977.
 2019-04-17 15:13:54 -07:00
Rob Walker 01657ddfe7
packet.rs optimizations (#3818) 
* packet.rs optimizations

* remove redundant and aggressive metric submission

* remove metrics submit(), get compiling again, honor log level in inc()
 2019-04-17 14:14:57 -07:00
Greg Fitzgerald 51a2988bb2
Revert "Rename programs to instruction_processors (#3789)" (#3824) 
This reverts commit 34344982a9.
 2019-04-17 15:05:49 -06:00
sakridge 4b8cb72977
Revert "Use Rust erasure library and turn on erasure (#3768)" (#3827) 
This reverts commit b9bb5af4a5.
 2019-04-17 12:52:12 -07:00
Mark E. Sinclair b9bb5af4a5
Use Rust erasure library and turn on erasure (#3768) 
* split out erasure into new crate; add implementation using rust reed-solomon-library

* Track erasures with a &[bool] instead of indexes

* fix bug that reported the number of erasures incorrectly

* Introduce erasure `Session` for consistent config

* Increase test coverage; fix bugs

* Add ability to remove blobs from erasure meta tracking. test added

* Track deletion of coding blobs in blocktree via ErasureMeta. Added to
test

* Remove unused functions in blocktree

* add randomness to recovery thread to exercise recovery due to either new
data or coding blobs

* Add unit test for ErasureMeta index handling

* Re-enable test in broadcast stage
 2019-04-16 23:00:24 -05:00
Rob Walker e94f268346
reduce sigverify verbosity (#3813) 2019-04-16 18:25:53 -07:00
Amr Ali 34344982a9
Rename programs to instruction_processors (#3789) 
* Rename programs to instruction_processors

* Updates around the code base to support instruction_processors rename

* Kabab instruction_processors

* Update Cargo.toml files and scripts to use instruction-processors

* Update Cargo.toml to use instruction-processors

* Update CI scripts to use instruction-processors
 2019-04-16 22:39:00 +02:00
Michael Vines 63d66ece57
net/ testnet nodes now stake more lamports (#3812) 
* Add --bootstrap-leader-lamports

* Generalize --no-stake into --stake NUM

* Use a large stake for net/ fullnodes

* Setup vote account before starting fullnode to avoid mixed log output
 2019-04-16 13:03:01 -07:00
Michael Vines d77359914f
Log the spy node id by default for better debug (#3796) 2019-04-15 20:58:37 -07:00
Dan Albert 8638b3bb19 Update Cargo.toml files for version = 0.14.0 2019-04-16 02:50:20 +00:00
anatoly yakovenko 68fc303b9b
Rework Accounts for fast squash, hashing state and checkpoint recovery. (#3613) 
* accounts rewrite

* ignore grow tests

* skip duplicate roots

* allow for a root race

* logger

* accounts_index tests

* tests

* tests
 2019-04-15 17:15:50 -07:00
Sagar Dhawan 3fcf03ff3e
Refactor LocalCluster and add support for listener nodes (#3790) 2019-04-15 15:27:45 -07:00
Greg Fitzgerald 80f3568062
Upgrade to Rust 1.34.0 (#3781) 
* Upgrade to Rust 1.34.0

* Remove redundant closures

Thanks Clippy!
 2019-04-15 15:56:08 -06:00
Michael Vines d026ebb83a
Use tvu_peers() since validators no longer run an RPC port by default (#3784) 2019-04-15 13:25:09 -07:00
Rob Walker 64c6f05da2
persist set_root() and use it in blocktree_processor to limit squashes (#3782) 
* rename locktower's slot to epoch

* persist set_root() and use it in blocktree_processor to limit squashes
 2019-04-15 13:12:28 -07:00
Rob Walker 5c4689a326
rename locktower's slot to epoch (#3776) 2019-04-15 10:46:14 -07:00
Rob Walker 6859907df9
more rigorous erasure constants, comments (#3766) 
* more rigorous erasure constants, comments

* new header size means new golden
 2019-04-14 21:10:09 -07:00
Rob Walker de52747950
remove max_tick_height replicode (#3765) 2019-04-14 19:15:31 -07:00
Rob Walker bd1db51e07
delete db_window.rs, move contents to window_service, clean up process_blobs (#3746) 2019-04-14 18:52:05 -07:00
Rob Walker dd005fb50e
fix broadcast to *always* call erasure generation, simplify generator, test slot reset better (#3764) 2019-04-14 18:12:37 -07:00
Rob Walker 542bafeb71
groom packet.rs, add blob.data alignment (#3763) 2019-04-14 17:30:08 -07:00
Rob Walker e57a0ab05d
test some bits (#3762) 2019-04-14 17:10:30 -07:00
Pankaj Garg 2c745ce108
Shorten recv wait when there are buffered packets in banking stage (#3757) 
- packets are buffered on leader rotation, when the next leader is
  unknown
- shortening the wait allows the banking stage to poll for next
  leader more frequently
 2019-04-14 12:34:07 -07:00
Michael Vines f6aa90e193 Add fullnode --dynamic-port-range option 2019-04-14 07:08:29 -07:00
Michael Vines 2277a39dd2 Default solana-gossip log-level to 'info' 2019-04-14 07:07:15 -07:00
Pankaj Garg ee35ed5250
Refactored buffered packet forwarding code (#3750) 
- Added unit tests
- Don't consume packets if bank is not known
 2019-04-13 23:19:54 -07:00
Stephen Akridge 92b5e131fe Name sigverify threads 2019-04-13 11:24:36 -07:00
Sagar Dhawan 1e8f83a74a Use a better name for new api 2019-04-12 14:58:22 -07:00
Sagar Dhawan 1db80d79fc Update get recent blockhashes to return confirmed blockhashes only 2019-04-12 14:58:22 -07:00
Stephen Akridge 1dac4c33b8 Change sigverify counter from entries to packets 
batch or entries kind of useless since it can have some
variable number of packets
 2019-04-12 13:19:46 -07:00
Pankaj Garg 44ebfa736a
Don't forward buffered packet to the same node (#3712) 
- instead, process the packets
 2019-04-11 17:23:45 -07:00
Sagar Dhawan 877ec08280
Send recent votes in Vote Transactions (#3734) 2019-04-11 14:48:36 -07:00
Rob Walker efd19b07e7
implement erasure-based recovery inside blocktree (#3739) 
* implement recover in blocktree

* erasures metric

* erasure metrics only

* fixup
 2019-04-11 14:14:57 -07:00
carllin 787dc5748a
Fixed DuplicateSigs (#3727) 
* Fixed DuplicateSigs by not recording errors in signature cache of bank
 2019-04-11 11:51:34 -07:00
Rob Walker e1d5bb1a26
add redundant broadcast (#3724) 
* add redundant broadcast

* crank up to full redundancy

* Update broadcast_stage.rs

* Update broadcast_stage.rs

* Update broadcast_stage.rs

* Update broadcast_stage.rs
 2019-04-11 09:15:17 -07:00
Jack May d0f46d6a8a
Cleanup client traits and create super trait (#3728) 2019-04-11 00:25:14 -07:00
Rob Walker 4b6c0198ad
reset coding generator on slot boundaries (#3726) 2019-04-10 18:18:55 -07:00
Rob Walker f1e7237c09
vote_api cleanup (#3710) 
* vote_api cleanup

* fixups

* fixup

* remove unused code

* revert removal of serialize and deserialize

* ...

* increase coverage, bootstrap staking

* Sagar's STAKE to my VOTE
 2019-04-10 17:52:47 -07:00
Sagar Dhawan 1b5845ac3e
Fix getting votes from gossip (#3723) 2019-04-10 17:16:08 -07:00
Sagar Dhawan d2ea782372 Always use bootstrap vote account for leader 2019-04-10 15:51:00 -07:00
Greg Fitzgerald 894135a084 Less pub in PohRecorder 2019-04-10 12:50:45 -07:00
Pankaj Garg 7da4142d33
Process votes from gossip only in leader node (#3707) 2019-04-09 22:06:32 -07:00
carllin 003fd6545c
Logging for unexpected validator errors (#3697) 2019-04-09 15:05:43 -07:00
Stephen Akridge 2c93062f54 Improve banking_stage performance messages 
Use transaction count instead of batch count,
and set the recv_start from when we finished processing
the previous batch to get a more accurate number.
 2019-04-09 14:54:12 -07:00
Pankaj Garg dc6c34da5d
Fast-track vote signature verification and processing (#3695) 2019-04-09 12:57:12 -07:00
carllin d4eebcc2aa
Check for frozen in confirm_forks (#3678) 2019-04-09 11:45:38 -07:00
carllin 483cc2fa4e
Support old repair strategy for reparing slots in a range for supporting replicators (#3665) 2019-04-08 12:46:23 -07:00
Jack May 44b391096d
Configurable local cluster native processors (#3676) 2019-04-08 11:15:58 -07:00
carllin 444e87f888
Fix metric (#3664) 2019-04-06 21:57:01 -07:00
carllin 20aa4434e2
Fix repair (#3581) 
Add DetachedHeads repair protocol

Add DetachedHeads repair test

Repair starting from root
 2019-04-06 19:41:22 -07:00
Greg Fitzgerald 878a842611
Move append_vec bench to the crate with append_vec (#3650) 
* Move append_vec bench to the crate with append_vec

* Use black_box to tell the compiler not to optimize away test data

```
pub fn black_box<T>(dummy: T) -> T {
    unsafe {
        let ret = std::ptr::read_volatile(&dummy);
        std::mem::forget(dummy);
        ret
    }
}
```

* Revert "Use black_box to tell the compiler not to optimize away test data"

This reverts commit 5610b8ee957f1d8bf6e270e392859e8b23b1e472.

* Use black_box to tell the compiler not to optimize away test data

* Create bench directories
 2019-04-06 07:18:56 -06:00
Tyera Eulberg 68e21911eb Remove redundant transfer_signed 2019-04-05 22:04:32 -07:00
Tyera Eulberg 95cc36af96 Impl SyncClient and AsyncClient for ThinClient 2019-04-05 22:04:32 -07:00
Tyera Eulberg f349c1f0dc Get everything off RpcSignatureStatus 2019-04-05 22:09:29 -06:00
Tyera Eulberg 90c1300bb6 Plumb TransactionError through Rpc 2019-04-05 22:09:29 -06:00
Sagar Dhawan f5f4434e0a Remove unnecessary lock in sigverify 2019-04-05 16:57:45 -07:00
Sagar Dhawan adcda3c715 Remove airdrop dependency from replicators 2019-04-05 16:11:39 -07:00
Rob Walker a5b5248a09
move vote_accounts up (#3647) 2019-04-05 14:23:00 -07:00
Tyera Eulberg ef5df6f3fe Add server specification 2019-04-05 11:44:57 -07:00
Tyera Eulberg 2f90f9fbd4 Version all jsonrpc crates, in core too 2019-04-05 11:44:57 -07:00
Greg Fitzgerald 46e6911ec1 Add get_signature_status() to SyncClient 
And move bank::Result to transaction module.
 2019-04-05 10:22:05 -07:00
carllin 4ea19b90a4
Fix update_ancestor_stakes in locktower (#3631) 
* Fix update_ancestor_stakes in locktower

* Add test for vote threshold
 2019-04-05 03:05:31 -07:00
Greg Fitzgerald 35298e01a8 Remove Instruction wrapper structs and name functions after enum fields 2019-04-03 13:34:27 -07:00
Greg Fitzgerald 867f6f107b Rename SystemInstruction::Move to SystemInstruction::Transfer 2019-04-03 08:35:57 -06:00