blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
12,927 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

2099 Commits

Author SHA1 Message Date
Michael Vines 1d40685b8e Remove support for 1.1.0 version snapshots 2020-08-04 01:01:50 +00:00
Michael Vines eefcf484cb clippy 2020-08-03 18:35:15 +00:00
Tyera Eulberg 0bc9bcc8b9
Fix bad rebase 2020-08-02 12:09:06 -06:00
Tyera Eulberg d1b2e6cdf2
Add getTokenLargestAccounts endpoint (#11322) 2020-08-02 10:23:44 -06:00
Michael Vines ffbd859c42 Update spl-token and spl-memo 2020-08-02 07:53:42 -07:00
Tyera Eulberg 9bcfc51df1
Add token rpc endpoints to rpc-client (#11315) 2020-07-31 20:53:49 +00:00
carllin d7e961dac4
Enable new fork choice on mainnet, 400_000 slots into epoch 61 (#11312) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-31 20:37:58 +00:00
Greg Fitzgerald 61d9d219f9
Withdraw authority no longer implies a custodian (#11302) 
* Withdraw authority no longer implies a custodian

Before this change, if the withdraw authority and custodian had
the same public key, then a withdraw authority signature would
imply a custodian signature and lockup would be not be enforced.

After this change, the client's withdraw instruction must
explictly reference a custodian account in its optional sixth
account argument.

Likewise, the fee-payer no longer implies either a withdraw
authority or custodian.

* Fix test

The test was configuring the stake account with the fee-payer as
the withdraw authority, but then passing in a different key to
the withdraw instruction's withdraw authority parameter. It only
worked because the second transaction was signed by the fee-payer.
 2020-07-31 13:37:53 -06:00
Michael Vines c78e12a262 Avoid spl-sdk dependency, which inhibits crate publishing 2020-07-30 23:47:26 +00:00
Greg Fitzgerald 1d2426ddbc
Move simulate_transaction from rpc to bank (#11294) 2020-07-30 19:15:45 +00:00
carllin a23e1129a9
Switch to optimistic conf for notifying gossip subscribers (#11289) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-30 02:52:27 -07:00
carllin bf18524368
Add hook for getting vote transactions on replay (#11264) 
* Add hook for getting vote transactions on replay

Co-authored-by: Carl <carl@solana.com>
 2020-07-29 23:17:40 -07:00
Trent Nelson a888f2f516 Reject TXs sent via RPC with no signatures 2020-07-30 00:07:33 -06:00
Trent Nelson 9778fedd7a Add new RPC error for TXs with no signatures 2020-07-30 00:07:33 -06:00
Trent Nelson a7079e4dde Dereplicode send_transaction and request_airdrop RPC handlers 2020-07-30 00:07:33 -06:00
Trent Nelson b962b2ce2d Add failing test for TX sent via RPC with no signatures 2020-07-30 00:07:33 -06:00
carllin ebe367f722
Add utility function (#11262) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-28 23:44:10 -07:00
Tyera Eulberg b45ac5d4db
Add SPL Token-specific rpc endpoints (#11231) 
* Simplify account-decoder program ids + spl_token helper

* Spl program namespace version

* Add getTokenAccountBalance endpoint

* Remove token program id from getTokenAccountBalance request

* Add getTokenSupply endpoint

* Remove token program id from getTokenSupply request

* Add getTokenAccountsByOwner/Delegate endpoints

* Remove token program id from getTokenAccountsByOwner/Delegate requests

* Named parameter
 2020-07-28 23:00:48 -06:00
Trent Nelson 678f94903b Fill out missing RPC request received debug logging 2020-07-29 01:51:20 +00:00
carllin a7ea340f22
Track votes from gossip for optimistic confirmation (#11209) 
* Add check in cluster_info_vote_listenere to see if optimstic conf was achieved
Add OptimisticConfirmationVerifier

* More fixes

* Fix merge conflicts

* Remove gossip notificatin

* Add dashboards

* Fix rebase

* Count switch votes as well toward optimistic conf

* rename

Co-authored-by: Carl <carl@solana.com>
 2020-07-28 09:33:27 +00:00
carllin c0dc21620b
Test cleanup (#11192) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-24 09:55:25 +00:00
carllin 6578ad7d08
Speed up local cluster partitioning tests (#11177) 
* Fix long local cluster partition tests by skipping slot warmup

Co-authored-by: Carl <carl@solana.com>
 2020-07-23 18:50:42 -07:00
Greg Fitzgerald 7484202282
Return root when bank not found (#11188) 
* Return root when bank not found

* Apply suggestions from code review

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-07-24 00:57:15 +00:00
Greg Fitzgerald 8ccce1e543
Upgrade ed25519-dalek (#11183) 2020-07-23 17:23:51 -06:00
Tyera Eulberg c90de8978d
Add encoding and filters parameters to rpc Subscriptions (#11065) 
* Plumb account configs and enable encoding

* Enable filters for pubsub program accounts

* Update docs
 2020-07-23 13:38:28 -06:00
Tyera Eulberg ca6480a8ac
Use OrderedIterator in collect_balances (#11166) 2020-07-23 13:35:10 -06:00
Tyera Eulberg b5a6a2f461
Initialize BlockCommitmentCache slot and root on node boot (#11178) 
* Initialize commitment-cache slot and root on node boot

* Ignore long tests
 2020-07-23 11:44:57 -06:00
carllin e556f85178
Add test for making sure switch doesn't happen past failure threshold (#11138) 
Fix switch threshold

Co-authored-by: Carl <carl@solana.com>
 2020-07-21 23:04:24 -07:00
Tyera Eulberg 6c38369042
Use OrderedIterator in TransactionStatusService (#11149) 
* Split out get-first-err for unit testing

* Add failing test

* Add missing ordering
 2020-07-21 20:06:49 +00:00
carllin 73f3d04798
Add replay votes to gossip vote tracking (#11119) 
* Plumb replay vote channel for notifying vote listener of replay votes

* Keep gossip only notification for debugging gossip in the future

Co-authored-by: Carl <carl@solana.com>
 2020-07-20 17:29:07 -07:00
Greg Fitzgerald 23c2e55cbf
Use BlockCommitmentCache for RPC slots, take 2 (#11137) 
* Use BlockCommitmentCache for RPC slots (#11103)

* Add BankForks::highest_descendant(slot)

* Update debug messages

* Update core/src/rpc.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* cargo fmt

* Remove highest_descendant

* Fix test

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* Fix crossed-in-flight compilation error

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-07-20 23:03:40 +00:00
Michael Vines d19cd3a18a
Revert "Use BlockCommitmentCache for RPC slots (#11103)" (#11122) 
This reverts commit 815b0f31b4.
 2020-07-18 16:39:12 -07:00
Greg Fitzgerald 815b0f31b4
Use BlockCommitmentCache for RPC slots (#11103) 
* Add BankForks::highest_descendant(slot)

* Update debug messages

* Update core/src/rpc.rs

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>

* cargo fmt

* Remove highest_descendant

* Fix test

Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
 2020-07-17 16:22:27 -06:00
Tyera Eulberg a255b0fc86
Rename CacheSlotInfo to CommitmentSlots (#11111) 2020-07-17 17:54:49 +00:00
Tyera Eulberg fdff681bcc
Simplify BlockCommitmentCache slot info (#11106) 
* Refactor BlockCommitmentCache, store a CacheSlotInfo

* Comma
 2020-07-17 09:24:51 -06:00
Dan Albert 7958dd88c2
Update non_circulating_supply.rs 2020-07-16 12:57:13 -06:00
sakridge 7fe870ba48
Make accounts file clean faster (#11071) 2020-07-15 09:37:40 -07:00
Tyera Eulberg 9a80e31bae
Expose tss to the other blockstore_processor path (#11070) 2020-07-14 21:14:48 -06:00
dependabot-preview[bot] 5055028231 Bump serde_json from 1.0.54 to 1.0.56 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.54 to 1.0.56.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.54...v1.0.56)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-07-14 11:13:22 -07:00
carllin e9cbdf711b
Add TreeDiff trait to reuse tree functions (#11046) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-14 07:38:48 +00:00
Kristofer Peterson ed5a2f2a90
Fold bank serialisation into serde snapshot (#10581) 
* Move bank (de)serialisation logic from bank and snapshot_utils to serde_snapshot.
Add sanity assertions between genesis config and bank fields on deserialisation.
Atomically update atomic bool in quote_for_specialization_detection().
Use same genesis config when restoring snapshots in test cases.

* Tidy up namings and duplicate structs to version

* Apply struct renames to tests

* Update abi hashes

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
 2020-07-13 23:00:59 +09:00
Greg Fitzgerald 145906123a
Simplify use of SendTransactionService (#10999) 
* Send transaction upon recv

This will allow us to move the channel to the public interface

* Use a channel, not a method, to communicate

* Pipeline the services

* Ignore unused return values

* Fix clippy warning
 2020-07-11 01:14:41 +00:00
Michael Vines 25ca99cdcf largest -> highest 2020-07-10 12:57:32 -07:00
Michael Vines aef6bf272e getConfirmedBlocks now has an upper limit on slot range 2020-07-10 12:18:31 -07:00
sakridge 631f051c68
Add jitter to cleanup to prevent all nodes cleaning at the same time (#10936) 2020-07-10 11:54:45 -07:00
carllin 7a14e359d7
Switch to using weighted repair in RepairService (#10735) 
* Plumb votes into repair service

* Remove refactoring

* Fix tests

* Switch to using RepairWeight for generating repairs

* Revert "Weight repair slots based on vote stake (#10741)"

This reverts commit cabd0a09c3.

* Update logging

Co-authored-by: Carl <carl@solana.com>
 2020-07-10 05:52:54 +00:00
Greg Fitzgerald 16eeea4f82
Move SendTransactionService to solana_runtime (#10972) 2020-07-09 18:28:26 +00:00
Greg Fitzgerald 17a8cc862b
Remove core dependencies from SendTransactionService (#10965) 
* Remove core dependencies from SendTransactionService

* Fix clippy warnings
 2020-07-08 19:13:42 -06:00
Greg Fitzgerald ebadbce920
Drop bank from BlockCommitmentCache (#10959) 
* Remove bank reference from BlockCommitmentCache

* Don't use a Bank to create BlockCommitmentCache

* Rename recent_slot to slot
 2020-07-09 00:50:13 +00:00
dependabot-preview[bot] 841ecfd927
chore(deps): bump bincode from 1.2.1 to 1.3.1 (#10867) 
* chore(deps): bump bincode from 1.2.1 to 1.3.1

Bumps [bincode](https://github.com/servo/bincode) from 1.2.1 to 1.3.1.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

* Switch from deprecated method

* Add options to maintain behavior with bincode::options()

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2020-07-09 00:08:05 +00:00
Tyera Eulberg 1a6bbd2867
Limited Deserialize isn't limiting anything (#10952) 
* Add failing test

* Use deserialize_from to enable limit
 2020-07-08 15:54:42 -06:00
Greg Fitzgerald 930162a079
Drop dependency on BlockCommitmentCache bank (#10946) 
* Drop dependency on BlockCommitmentCache bank

* cargo fmt
 2020-07-07 20:13:30 -06:00
Greg Fitzgerald 2fdbb97244
Rename largest_confirmed_root to highest_confirmed_root (#10947) 2020-07-07 23:59:46 +00:00
sakridge d492f4f15b
Clean accounts path on validator startup (#10929) 2020-07-07 09:41:45 -07:00
carllin 3f6042d8b3
Add RepairWeight to track votes seen in gossip for weighted repair (#10903) 
* Add RepairWeight

Co-authored-by: Carl <carl@solana.com>
 2020-07-06 22:49:40 -07:00
sakridge 58a475b789
Add db recovery methods (#10838) 2020-07-06 12:43:45 -07:00
Tyera Eulberg 4de0713aa3
Rpc: Add getStakeActivation endpoint (#10902) 
* Add getStakeActivation endpoint

* Add docs

* Update docs/src/apps/jsonrpc-api.md

Co-authored-by: Michael Vines <mvines@gmail.com>

* Rework return type

* Update docs

* Rebase

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-07-06 13:28:40 -06:00
Ryo Onodera 39b3ac6a8d
Introduce automatic ABI maintenance mechanism (2/2; rollout) (#8012) 
* Introduce automatic ABI maintenance mechanism (2/2; rollout)

* Fix stable clippy

* Change to symlink

* Freeze abi of Tower

* fmt...

* Improve dev-experience!

* Update BankSlotDelta

$ diff -u /tmp/abi8/*7dg6BreYxTuxiVz6aLvk3p2Z7GQk2cJqfGvC9h4FAoSj* /tmp/abi8/*9chBcbXVJ4fK7uGgydQzam5aHipaAKFw6V4LDFpjbE4w*
--- /tmp/abi8/bank__BankSlotDelta_frozen_abi__test_abi_digest_7dg6BreYxTuxiVz6aLvk3p2Z7GQk2cJqfGvC9h4FAoSj      2020-06-18 18:01:22.831228087 +0900
+++ /tmp/abi8/bank__BankSlotDelta_frozen_abi__test_abi_digest_9chBcbXVJ4fK7uGgydQzam5aHipaAKFw6V4LDFpjbE4w      2020-07-03 15:59:58.430695244 +0900
@@ -140,7 +140,7 @@
                                                         field u8
                                                             primitive u8
                                                         field solana_sdk::instruction::InstructionError
-                                                            enum InstructionError (variants = 34)
+                                                            enum InstructionError (variants = 35)
                                                                 variant(0) GenericError (unit)
                                                                 variant(1) InvalidArgument (unit)
                                                                 variant(2) InvalidInstructionData (unit)
@@ -176,6 +176,7 @@
                                                                 variant(31) CallDepth (unit)
                                                                 variant(32) MissingAccount (unit)
                                                                 variant(33) ReentrancyNotAllowed (unit)
+                                                                variant(34) MaxSeedLengthExceeded (unit)
                                                     variant(9) CallChainTooDeep (unit)
                                                     variant(10) MissingSignatureForFee (unit)
                                                     variant(11) InvalidAccountIndex (unit)

* Fix some merge conflicts...
 2020-07-06 20:22:23 +09:00
Ryo Onodera 92697973d0
Document lockout_intervals and tiny niceties (#10925) 2020-07-06 08:59:17 +00:00
carllin 28e15a63e5
Repair timing (#10913) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-03 18:37:01 -07:00
Greg Fitzgerald 2922494b4f
Cleanup rpc (#10910) 
* Remove unreachable error case

* Inline Ok

* Hoist Ok
 2020-07-03 17:39:14 -06:00
Tyera Eulberg 8d951776ab
Rpc: add filter to getProgramAccounts (#10888) 
* Add RpcFilterType, and implement CompareBytes for getProgramAccounts

* Accept bytes in bs58

* Rename to memcmp

* Add Memcmp optional encoding field

* Add dataSize filter

* Update docs

* Clippy

* Simplify tests that don't need to test account contents; add multiple-filter tests
 2020-07-03 07:46:29 +00:00
sakridge 832d47317e
Move clean accounts to background service (#10898) 2020-07-02 22:25:17 -07:00
carllin f1699721ef
Bench RaptorQ (#10886) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-02 18:31:32 -07:00
carllin f17ac70bb2
Add weighted traversal (#10877) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-02 14:33:04 -07:00
Michael Vines d53228e69d getInflationRate now only supports the current epoch 2020-07-02 02:11:31 +00:00
Tyera Eulberg 2669ccb864
Update struct prefixes to Ui (#10874) 
* Update comments

* Use Ui prefix
 2020-07-01 20:06:40 +00:00
dependabot-preview[bot] 4deec51703
chore(deps): bump rayon from 1.3.0 to 1.3.1 (#10605) 
* chore(deps): bump rayon from 1.3.0 to 1.3.1

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.3.0...v1.3.1)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-07-01 08:09:28 -07:00
carllin 7a71580d53
Add heaviest subtree utility functions (#10863) 
Co-authored-by: Carl <carl@solana.com>
 2020-07-01 08:45:32 +00:00
Ryo Onodera ecf6fb2cd2
Rename new_banks_from_blockstore => new_banks_from_ledger (#10864) 2020-07-01 06:19:40 +00:00
Tyera Eulberg d97850f1d9
Add account-decoder utilities (#10846) 
* Fix comment and make less pub

* Add account-decoder crate and use to decode vote and system (nonce) accounts

* Update docs

* Rename RpcAccount struct

* s/Rpc/Display

* Call it jsonParsed and update docs

* Revert "s/Rpc/Display"

This reverts commit 6e7149f503f560f1e9237981058ff05642bb7db5.

* s/Rpc/Ui

* Add tests

* Ui more things

* Comments
 2020-07-01 04:55:11 +00:00
sakridge 1ffad2d051
More guard rails for restart with ledger procedure (#10853) 
* Add expected_bank_hash required for supermajority
* Print snapshot hash in ledger-tool create-snapshot.
 2020-06-30 12:43:48 -07:00
carllin 4b93a7c1f6
Fix fork detection (#10839) 
* Fix fork detection

Co-authored-by: Carl <carl@solana.com>
 2020-06-29 18:49:57 -07:00
sakridge d9b389f510
Reduce logging lines (#10835) 2020-06-29 15:57:28 -07:00
sakridge 583cec922b
Remove ledger purge batching (#10830) 2020-06-29 14:44:35 -07:00
sakridge 17a2128a8f
More replay stage timing metrics (#10828) 2020-06-28 10:04:15 -07:00
carllin fb8e2d8cdb
Add debugging (#10822) 
Co-authored-by: Carl <carl@solana.com>
 2020-06-26 10:23:11 +00:00
Greg Fitzgerald 50b3fa83a0
Move BankCommitmentCache to solana_runtime (#10816) 
* Remove Blockstore member variable from BlockCommitmentCache

* Hoist is_confirmed_rooted() to its only caller

BlockCommitmentCache no longer depends on Blockstore

* Move BlockCommitmentCache to solana_runtime
 2020-06-25 22:06:58 -06:00
Greg Fitzgerald 62b873b054
Clean up rpc module (#10812) 
* Clean up rpc module

* Simplify getting bank
 2020-06-25 17:08:55 -06:00
carllin 84954ad0a0
Fix leaf propagation in case of no votes in HeaviestForkChoice (#10803) 
* Fix leaf propagation logic

Co-authored-by: Carl <carl@solana.com>
 2020-06-25 04:08:18 -07:00
Ryo Onodera 4164c69683
Tiny cleanups in preparation for persistent tower (#10805) 2020-06-25 18:24:16 +09:00
Dan Albert b3e382ab3f
Add non-circulating withdraw authority (#10798) 
Co-authored-by: publish-docs.sh <maintainers@solana.com>
 2020-06-25 04:17:49 +00:00
Greg Fitzgerald 1c498369b5
Remove fee-payer guesswork from Message and Transaction (#10776) 
* Make Message::new_with_payer the default constructor

* Remove Transaction::new_[un]signed_instructions

These guess the fee-payer instead of stating it explicitly
 2020-06-24 14:52:38 -06:00
carllin 764c7b0403
Disable entire codepath (#10767) 
Co-authored-by: Carl <carl@solana.com>
 2020-06-24 12:09:55 -07:00
sakridge a1ef921b88
Rework backup and clear function (#10751) 2020-06-23 21:29:07 +00:00
Ryo Onodera 685becad13
Further expand last_voted_slot terminology (#10747) 2020-06-23 21:52:45 +09:00
sakridge cabd0a09c3
Weight repair slots based on vote stake (#10741) 
* Weight repair slots based on vote stake

* Add test
 2020-06-22 20:27:45 -07:00
sakridge 2ba8fc5243
Remove slots past wait-for-supermajority slot. (#10720) 2020-06-22 20:27:25 -07:00
Ryo Onodera 0952b76f02
Add {Vote, Tower}::last_voted_slot() (#10734) 2020-06-23 11:59:54 +09:00
Ryo Onodera 44f5452013
Remove unused StakeLockout::lockout (#10719) 
* Remove unused StakeLockout::lockout

* Revert...

* Really revert to the original behavior...

* Use consistent naming after StakeLockout removal

* Furhter clean up

* Missed type aliases...

* More...

* Even more...
 2020-06-23 10:30:09 +09:00
Kristofer Peterson a87f490b5e
nit removal (#10721) 2020-06-21 16:12:21 +09:00
carllin a33fef9af2
Add repair breakdown by slot and index (#10717) 
* Slot full logging

* Repair stats logging

Co-authored-by: Carl <carl@solana.com>
 2020-06-19 18:28:15 -07:00
Tyera Eulberg fd2508b09e
Add jsonParsed option for EncodedTransactions; add memo parser (#10711) 
* Add jsonParsed option for EncodedTransactions; add memo parser

* Use kebab case for program names

* Add account-key parsing

* Add parse test
 2020-06-19 16:15:13 -06:00
Kristofer Peterson 6d81eede93
Add CLI options and runtime support for selection of output snapshot version. (#10536) 2020-06-19 14:38:37 +09:00
Greg Fitzgerald ed4c71fe2d
Merge heaviest bank modules (#10672) 
* Merge heaviest bank modules

* Merge heaviest fork choice modules

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2020-06-18 03:59:08 +00:00
Greg Fitzgerald 0550b893b0
Fix typos (#10675) 
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2020-06-17 20:54:52 -07:00
Justin Starry 39984cdcc3
Wait until bank is frozen before sending RPC notifications (#10654) 2020-06-18 00:44:51 +08:00
Greg Fitzgerald 6ee222363e
Move BankForks to solana_runtime (#10637) 
* Move BankForks to solana_runtime

* Update imports
 2020-06-17 15:27:03 +00:00
sakridge 0de6c444d6
Simd poh (#10604) 
* Simd poh

* Fix poh verify bench
 2020-06-16 23:03:26 -07:00
Greg Fitzgerald 891aa8c13e
Add RequestProcessor constructor to simplify testing (#10631) 
* Add JsonRpcRequestProcessor constructor

* Avoid start_rpc_handler_with_tx monolith in rpc tests

* pub

* Pass an Arc, so that caller can poke the bank
 2020-06-16 23:30:59 -06:00
publish-docs.sh 5673343f49 Add address to non-circulating supply 2020-06-16 21:20:33 -07:00
sakridge 1eca9b19ab
Entry verify cleanup and gossip counters (#10632) 
* Add prune message counter

* Switch to us verification time to match other counters

* Add separate transaction/poh verify timing
 2020-06-16 14:00:29 -07:00
carllin f8b88d717e
Enable fork choice and switch votes, devnet => now, testnet => epoch 63 (#10615) 
* Enable fork choice, devnet => now, testnet => epoch 63

* Set development to 0

* Enable switch vote slot

Co-authored-by: Carl <carl@solana.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2020-06-16 09:55:36 +00:00
dependabot-preview[bot] 480f69cc5e Bump serde from 1.0.111 to 1.0.112 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.111 to 1.0.112.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.111...v1.0.112)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-15 07:58:16 -07:00
Ryo Onodera a0692c9b4c
Further fix the ci (#10559) 2020-06-14 14:53:49 +09:00
Ryo Onodera d159bc71e7
Fix trailing whitespace (#10558) 2020-06-14 14:43:43 +09:00
anatoly yakovenko ba83e4ca50
Fix fannout gossip bench (#10509) 
* Gossip benchmark

* Rayon tweaking

* push pulls

* fanout to max nodes

* fixup! fanout to max nodes

* fixup! fixup! fanout to max nodes

* update

* multi vote test

* fixup prune

* fast propagation

* fixups

* compute up to 95%

* test for specific tx

* stats

* stats

* fixed tests

* rename

* track a lagging view of which nodes have the local node in their active set in the local received_cache

* test fixups

* dups are old now

* dont prune your own origin

* send vote to tpu

* tests

* fixed tests

* fixed test

* update

* ignore scale

* lint

* fixup

* fixup

* fixup

* cleanup

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
 2020-06-13 22:03:38 -07:00
R. M. Shea 48febf153b
Change documentation introduction to indicate Foundation IP transfer (#10440) 2020-06-13 14:41:05 -07:00
Jack May b6a9573748
Route all loader messages to log collector (#10528) 2020-06-13 13:20:08 -07:00
Greg Fitzgerald 8dd5384d6d
Split commitment module (#10541) 
automerge
 2020-06-12 17:16:10 -07:00
Michael Vines f54c049b43
Add FdGYQ... to non-circulation withdrawer authority list (#10542) 
automerge
 2020-06-12 16:45:20 -07:00
Greg Fitzgerald 2eb6f498a8
Remove redundant BankForks parameter (#10537) 2020-06-12 11:04:17 -06:00
Ryo Onodera dfe72d5242
Optimize stale slot shrinking for previously cleaned roots (#10099) 
* Prioritize shrinking of previously cleaned roots

* measure time of stale slot shrinking

* Disable shrink for test

* shrink: budgeting, store count, force for snapshot

* Polish implementation and fix tests

* Fix ci..

* Clean up a bit

* Further polish implementation and fix/add tests

* Rebase fixes

* Remove unneeded Default for AccountStorageEntry

* Address review comments

* More cleanup

* More cleanup
 2020-06-12 14:51:43 +09:00
Greg Fitzgerald fb8612be49
Update non-circulating pubkeys (#10524) 
automerge
 2020-06-11 16:30:25 -07:00
Greg Fitzgerald 9077c4a776
Enable jsonrpc client (#10522) 2020-06-11 16:51:25 -06:00
carllin 526eefac97
Fix clippy (#10521) 
Co-authored-by: Carl <carl@solana.com>
 2020-06-11 15:26:00 -06:00
carllin 2e1d59ff85
Adopt heaviest subtree fork choice rule (#10441) 
* Add HeaviestSubtreeForkChoice

* Make replay stage switch between two fork choice rules

Co-authored-by: Carl <carl@solana.com>
 2020-06-11 12:16:04 -07:00
sakridge 4c140acb3b
ClusterInfo cleanup (#10504) 
automerge
 2020-06-10 17:00:17 -07:00
sakridge 6eb5ef6ac7
Add back missing pull_response success counter (#10491) 2020-06-10 09:17:57 -07:00
sakridge ecb6959720
Optimize process pull responses (#10460) 
* Batch process pull responses

* Generate pull requests at 1/2 rate

* Do filtering work of process_pull_response in read lock

Only take write lock to insert if needed.
 2020-06-09 17:08:13 -07:00
dependabot-preview[bot] a264a9dd0b Bump serde_json from 1.0.53 to 1.0.54 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.53 to 1.0.54.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.53...v1.0.54)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-09 16:57:46 -07:00
Michael Vines 8e2745c2a2 Clean up delinquency slot distance computation 2020-06-09 11:27:30 -07:00
Michael Vines ed351400b2 Add SendTransactionService 2020-06-09 07:46:40 -07:00
anatoly yakovenko 832d324a23
Revert "Gossip PullRequests tend to return a lot of duplicates. (#10326)" (#10455) 
This reverts commit 31e20eff82.
 2020-06-09 07:27:00 -07:00
Kristofer Peterson e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445) 
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
 2020-06-09 09:38:14 +09:00
Greg Fitzgerald af8c21c559
Remove lock around JsonRpcRequestProcessor (#10417) 
automerge
 2020-06-07 20:54:03 -07:00
sakridge 0645a0c96d
Gossip cleanup remove duplicate gossip metrics and name worker threads (#10435) 
Refactor into functions
 2020-06-06 15:05:45 -07:00
sakridge ebb612ab4e
Enable and add tick rate to metrics (#10430) 2020-06-06 11:47:11 -07:00
Michael Vines 7e2651ca51
RPC simulateTransaction endpoint now returns program log output (#10432) 2020-06-06 10:18:28 -07:00
sakridge 2725acebea
Lower counter level (#10428) 2020-06-05 15:53:09 -07:00
dependabot-preview[bot] 2e37eccfc4
Bump jsonrpc-core-client from 14.1.0 to 14.2.0 (#10425) 
Bumps [jsonrpc-core-client](https://github.com/paritytech/jsonrpc) from 14.1.0 to 14.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.1.0...v14.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-06-05 15:51:01 -06:00
sakridge 3d2230f1a9
Add pull request count metrics (#10421) 2020-06-05 09:36:31 -07:00
anatoly yakovenko 31e20eff82
Gossip PullRequests tend to return a lot of duplicates. (#10326) 
* filter messages that are likely to be pushed from the response

* tests

* tests

* wait to start filtering responses, and push stats to influx

* wait to start filtering responses, and push stats to influx

* reduce the timers to match the publish self timeout

* fmt

* fmt
 2020-06-05 08:01:45 -07:00
Michael Vines aa6832964c
ledger_cleanup_service: compact at a slower rate than purging (#10414) 2020-06-04 21:06:06 -07:00
dependabot-preview[bot] 318835e3a0
Merge pull request #10410 from solana-labs/dependabot/cargo/jsonrpc-pubsub-14.2.0 
Bump jsonrpc-pubsub from 14.1.0 to 14.2.0
 2020-06-04 19:06:14 -07:00
Justin Starry 754f25ae99
Avoid AccountInUse errors when simulating transactions (#10391) 
automerge
 2020-06-04 19:06:01 -07:00
dependabot-preview[bot] 68f95c791a Bump jsonrpc-ws-server from 14.1.0 to 14.2.0 
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.1.0 to 14.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.1.0...v14.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-04 19:02:30 -07:00
dependabot-preview[bot] 8736247554 Bump jsonrpc-http-server from 14.1.0 to 14.2.0 
Bumps [jsonrpc-http-server](https://github.com/paritytech/jsonrpc) from 14.1.0 to 14.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/v14.1.0...v14.2.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-04 19:00:28 -07:00
dependabot-preview[bot] ea9b958dff Bump jsonrpc-derive from 14.1.0 to 14.2.1 
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.1.0 to 14.2.1.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-03 12:16:54 -07:00
dependabot-preview[bot] f58b6b3431 Bump jsonrpc-core from 14.1.0 to 14.2.0 
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.1.0 to 14.2.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-03 09:02:07 -07:00
dependabot-preview[bot] 6cf30c38a4 Bump reqwest from 0.10.5 to 0.10.6 
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.10.5 to 0.10.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-03 09:01:36 -07:00
sakridge 2cf719ac2c
Cache tvu peers for broadcast (#10373) 2020-06-03 08:24:05 -07:00
dependabot-preview[bot] f901571026 Bump serde from 1.0.110 to 1.0.111 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.110 to 1.0.111.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.110...v1.0.111)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-02 23:29:14 -07:00
dependabot-preview[bot] 8498b75d6c Bump indexmap from 1.3.2 to 1.4.0 
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.3.2 to 1.4.0.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.3.2...1.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-06-02 23:28:57 -07:00
Tyera Eulberg eee9a08376
Purge TransactionStatus and AddressSignatures exactly from ledger-tool (#10358) 
* Add failing test

* Add execution path to purge primary-index columns exactly

* Fail gracefully if older TransactionStatus rocksdb keys are present

* Remove columns_empty check for special columns

* Move blockstore purge methods to submodule

* Remove unused column empty check
 2020-06-02 19:49:31 -06:00
Michael Vines ebf8a65526 Add test_rpc_send_transaction_preflight 2020-06-01 17:51:04 -07:00
Michael Vines 9158479dc4 Add node health check to transaction preflight 2020-06-01 17:51:04 -07:00
Michael Vines 9dbf3d5430 Factor out RpcHealth module 2020-06-01 17:51:04 -07:00
Michael Vines 189aa7962e Add preflight checks to sendTransaction RPC method 2020-06-01 17:51:04 -07:00
sakridge ef37b82ffa
More cluster stats and add epoch stakes cache in retransmit stage (#10345) 
* More cluster info metrics for push request/response counts

* Cache staked peers for the epoch
 2020-06-01 08:37:54 -07:00
carllin d9366776b2
Add operating mode gating (#10332) 
Co-authored-by: Carl <carl@solana.com>
 2020-05-30 00:03:19 -07:00
Greg Fitzgerald 19d11800bf
Remove WithSigner (#10325) 
automerge
 2020-05-29 23:17:44 -07:00
Michael Vines eb45aaa6c3
Added --health-check-slot-distance (#10324) 2020-05-29 15:31:52 -07:00
carllin 778078e1dc
Distinguish switch/non-switching votes in ReplayStage (#10218) 
* Add SwitchForkDecision, change vote instruction based on decision

* Factor out SelectVoteAndResetForkResult

Co-authored-by: Carl <carl@solana.com>
 2020-05-29 14:40:36 -07:00
Tyera Eulberg b563b49ed5
Improve Rpc inflation tooling (#10309) 
automerge
 2020-05-29 11:50:25 -07:00
Kristofer Peterson fb4d8e1f62
cleanup clippy tests (#10172) 
automerge
 2020-05-29 00:26:06 -07:00
dependabot-preview[bot] a71833c17b
Bump regex from 1.3.8 to 1.3.9 (#10308) 
* Bump regex from 1.3.8 to 1.3.9

Bumps [regex](https://github.com/rust-lang/regex) from 1.3.8 to 1.3.9.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.8...1.3.9)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-05-28 21:48:39 -07:00
Michael Vines 9eefb2c929 Bump version to 1.3.0 2020-05-28 15:01:13 -07:00
carllin e68621b8bb
Fix repair dos (#10299) 
Co-authored-by: Carl <carl@solana.com>
 2020-05-28 14:16:24 -07:00
Michael Vines 0442c45d5b
Feign RPC health while in a `--wait-for-supermajority` holding pattern (#10295) 2020-05-28 12:22:19 -07:00
sakridge 3f508b37fd
Skip gossip requests with different shred version and split lock (#10240) 2020-05-28 11:38:13 -07:00
dependabot-preview[bot] 9227874ada
Bump regex from 1.3.7 to 1.3.8 (#10293) 
* Bump regex from 1.3.7 to 1.3.8

Bumps [regex](https://github.com/rust-lang/regex) from 1.3.7 to 1.3.8.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/regex-1.3.7...1.3.8)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-05-28 10:02:23 -07:00
Tyera Eulberg 3f0995d3b4
Add commitment parameter to getFeeCalculatorForBlockhash (#10255) 
* Accept commitment parameter on getFeeCalculatorForBlockhash

* Update docs

* Add get_fee_calculator_for_blockhash_with_commitment to rpc client
 2020-05-26 17:23:58 -06:00
Tyera Eulberg 4e431bc818
Add mechanism to get blockhash's last valid slot (#10239) 
automerge
 2020-05-26 12:06:21 -07:00
sakridge 7ebd8ee531
Cluster info metrics (#10215) 2020-05-25 15:03:34 -07:00
Michael Vines 156387aba4
LedgerCleanupService no longer causes an OOM and actually purges (#10199) 
* cleanup_ledger() now services new_root_receiver while purging
* purge_slots() now fully deletes before compacting
* Add ledger pruning grafana graph
 2020-05-24 21:41:54 -07:00
carllin 3aae98c8be
Add switching vote instruction (#10197) 
* Add switching vote

* Make sure vote size stays under gossip limit

Co-authored-by: Carl <carl@solana.com>
 2020-05-24 15:38:35 -07:00
carllin 42aaacf520
Factor out LockedPubkeyReferences (#10198) 
Co-authored-by: Carl <carl@solana.com>
 2020-05-22 23:23:17 -07:00
Michael Vines 48c0845359 Update another non-circulating account 2020-05-22 15:11:33 -07:00
sakridge 10b1895357
Optimize banking processing of AccountInUse (#10154) 
* Optimize banking processing of AccountInUse and thread count

* Add more options to banking-bench
 2020-05-22 15:01:01 -07:00
sakridge f1e932c90a
Trigger notifications on supermajority votes confirmation (#10137) 
automerge
 2020-05-22 14:53:47 -07:00
Michael Vines e2b5cd6d47
Add another non-circulating account (#10186) 
automerge
 2020-05-22 13:06:03 -07:00
Tyera Eulberg 2928c5d103
Add SingleGossip commitment level to use for subscriptions (#10147) 
automerge
 2020-05-22 12:55:17 -07:00
Kristofer Peterson b7a32f01c0
Multi-version snapshot support (#9980) 
* Multi-version snapshot support

* rustfmt

* Remove CLI options and runtime support for selection output snapshot version.
Address some clippy complaints.

* Muzzle clippy type complexity warning.

Despite clippy's suggestion, it is not currently possible to create type aliases
for traits and so everything within the 'Box<...>' cannot be type aliased.

This then leaves creating full blown traits, and either implementing
said traits by closure (somehow) or moving the closures into new structs
implementing said traits which seems a bit of a palaver.

Alternatively it is possible to define and use the type alias 'type ResultBox<T> = Result<Box<T>>'
which does seems rather pointless and not a great reduction in complexity but is enough to keep clippy happy.

In the end I simply went with squelching the clippy warning.

* Remove now unused Serialize/Deserialize trait implementations for AccountStorageEntry and AppendVec

* refactor versioned de/serialisers

* rename serde_utils to serde_snapshot

* move call to accounts_db.generate_index() back down to context_accountsdb_from_stream()

* update version 1.1.1 to 1.2.0
remove nested use of serialize_bytes

* cleanups

* Add back measurement of account storage entry serialization.
Remove construction of Vec and HashMap temporaries during serialization.

* consolidate serialisation test cases into serde_snapshot.
clean up leakage of implementation details in serde_snapshot.

* move short term / legacy snapshot code into child module

* add serialize_iter_as_tuple

* preliminary integration of following commit

commit 6d58b73c47294bfb93465d5a83cd2175660b6e6d
Author: Ryo Onodera <ryoqun@gmail.com>
Date:   Wed May 20 14:02:02 2020 +0900

    Confine snapshot 1.1 relic to versioned codepath

* refactored serde_snapshot, rustfmt
legacy accounts_db format now "owns" both leading u64s, legacy bank_rc format has none

* reduce type complexity (clippy)
 2020-05-23 02:54:24 +09:00
Tyera Eulberg 12a3b1ba6a
Fixup deserialize_bs58_transaction, and make a few error types more targeted (#10171) 
automerge
 2020-05-21 17:30:02 -07:00
Michael Vines 18be7a7966
REST API now returns supply in SOL rather than lamports (#10170) 
automerge
 2020-05-21 15:50:06 -07:00
Michael Vines 486168b796
Revert "Add AVX2 runtime checks (#10033)" (#10167) 
This reverts commit cf8eb7700b.
 2020-05-21 13:13:52 -07:00
Justin Starry 10d60288e8
Update default subscription commitment level from recent to single (#10161) 
* Update default subscription commitment level from recent to single

* Update jsonrpc docs

* Fix failing tests
 2020-05-22 00:33:24 +08:00
Michael Vines 324cfd40f0
Add v0 REST APIs for circulating and total supply (#10102) 2020-05-20 20:04:07 -07:00
sakridge ce17de7d25
Add option to wait for a specific epoch length to bench-tps (#10083) 2020-05-20 16:42:46 -07:00
Michael Vines 417f0e41fa
Add stake-o-matic (#10044) 
automerge
 2020-05-20 16:15:03 -07:00
Michael Vines ad7b113944
Ignore test_tvu_exit (#10134) 
automerge
 2020-05-19 23:40:27 -07:00
Greg Fitzgerald d9919b99d2
Remove folds (#10128) 
automerge
 2020-05-19 18:13:41 -07:00
carllin 439fd30840
Fix erasure (#10095) 
* Fix bad FEC blocks

* Add test

Co-authored-by: Carl <carl@solana.com>
 2020-05-19 16:13:12 -07:00
carllin 97f2bcff69
master: Add nonce to shreds repairs, add shred data size to header (#10109) 
* Add nonce to shreds/repairs

* Add data shred size to header

Co-authored-by: Carl <carl@solana.com>
 2020-05-19 12:38:18 -07:00
Michael Vines 427c78d891
Add SimulateTransaction RPC endpoint (#10106) 
automerge
 2020-05-19 12:08:19 -07:00
Tyera Eulberg 759c0e0b03
Update accounts whitelist (#10100) 2020-05-18 14:17:56 -06:00
Tyera Eulberg bac4aec16f
Trigger RPC notifications after block commitment cache update (#10077) 
* Fixup commitment-aggregation metric

* Trigger notifications after commitment-cache update

* Fixup fn name

* Add single-confirmation commitment level

* Rename to highest_confirmed_slot

* Pass commitment-cache info directly to notifications

* Use match

* Update commitment docs

* Update out of date pubsub docs
 2020-05-18 12:49:01 -06:00
Reisen bfcfbab818
Add Vote PubSub endpoint for live gossip votes. (#10045) 
* Add Vote PubSub endpoint for live gossip votes.

* Updated tests for Vote RPC and Vote Listener

* Add JSON RPC documentation for Vote RPC.

* Base58 encode hash in Vote RPC response.
 2020-05-17 15:01:08 -06:00
sakridge f562ed4cc8 Distinguish between shred type in shred fetch stage duplicate filter (#10068) 
* Shred type check

* Test
 2020-05-15 13:23:56 -07:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030) 
automerge
 2020-05-15 09:35:43 -07:00
Michael Vines 1da1667920
Forge a confirmed root before halting for RPC inspection (#10061) 2020-05-15 09:02:48 -07:00
Jack May eb1acaf927
Remove archiver and storage program (#9992) 
automerge
 2020-05-14 18:22:47 -07:00
Michael Vines 4e4a21f9b7
`solana-gossip spy` can now specify a shred version (#10040) 2020-05-13 19:37:40 -07:00
Trent Nelson cf8eb7700b
Add AVX2 runtime checks (#10033) 
automerge
 2020-05-13 12:19:22 -07:00
Ryo Onodera 1eb40c3fe0
Introduce eager rent collection (#9527) 
* Switch AccountsIndex.account_maps from HashMap to BTreeMap

* Introduce eager rent collection

* Start to add tests

* Avoid too short eager rent collection cycles

* Add more tests

* Add more tests...

* Refacotr!!!!!!

* Refactoring follow up

* More tiny cleanups

* Don't rewrite 0-lamport accounts to be deterministic

* Refactor a bit

* Do hard fork, restore tests, and perf. mitigation

* Fix build...

* Refactor and add switch over for testnet (TdS)

* Use to_be_bytes

* cleanup

* More tiny cleanup

* Rebase cleanup

* Set Bank::genesis_hash when resuming from snapshot

* Reorder fns and clean ups

* Better naming and commenting

* Yet more naming clarifications

* Make prefix width strictly uniform for 2-base partition_count

* Fix typo...

* Revert cluster-dependent gate

* kick ci?

* kick ci?

* kick ci?
 2020-05-13 16:22:14 +09:00
Tyera Eulberg ee7f15eff1
Rpc: optionally filter getLargestAccounts by circulating/nonCirculating (#10007) 
* Add circ/non-circ filter to getLargestAccounts

* Plumb largest accounts into client and cli

* Bump timeout toward CI flakiness

* Update docs
 2020-05-12 21:05:05 -06:00
sakridge a9b82cf95b
Enable disk metrics (#10009) 2020-05-12 15:24:39 -07:00
carllin 59de1b3b62
Compute Switch Threshold (#9218) 
* Add switching threshold check

Co-authored-by: Carl <carl@solana.com>
 2020-05-11 22:20:11 -07:00
Michael Vines 84b6120983
getClusterNodes RPC API now includes the node software version (#9990) 2020-05-11 21:30:01 -07:00
Michael Vines 2521f75c18
Advertise node software version in gossip (#9981) 
* Advertise node version in gossip

* Remove solana_clap_utils::version! macro
 2020-05-11 15:02:01 -07:00
Tyera Eulberg 965204b8e0
Check slot cleaned up for RPC blockstore/slot queries (#9982) 
automerge
 2020-05-11 14:47:40 -07:00
sakridge 903a8a3196
Add retransmit packets_by_slot metrics (#9975) 2020-05-11 13:49:10 -07:00
sakridge 1a47b1cd86
Retransmit and shred fetch metrics (#9965) 
* Retransmit stats

* Shred fetch stats
 2020-05-10 21:37:05 -07:00
dependabot-preview[bot] 288c9751c1
Bump serde from 1.0.106 to 1.0.110 (#9964) 
* Bump serde from 1.0.106 to 1.0.110

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.106 to 1.0.110.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.106...v1.0.110)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-05-10 16:24:25 -07:00
dependabot-preview[bot] 0541431ea8
Bump serde_json from 1.0.52 to 1.0.53 (#9957) 
* Bump serde_json from 1.0.52 to 1.0.53

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.52 to 1.0.53.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.52...v1.0.53)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-05-10 11:46:29 -07:00
Tyera Eulberg 3ee702a922
Rpc: Add getCirculatingSupply endpoint, redux (#9953) 
* Add Bank.clock() helper

* Add non-circulating calculations

* Plumb getSupply rpc endpoint

* Add docs for getSupply, and remove getTotalSupply from docs

* Add pubkeys! procedural macro

* Use procedural macro in non_circulating_supply
 2020-05-09 12:05:29 -06:00
sakridge f98bfda6f9
Security changes (#9923) 
* Move test-only functions to test modules

* Remove sigverify disable

* Remove chacha CTR code
 2020-05-08 10:00:23 -07:00
carllin 01ab1d1369
Add metrics for logging time taken in replaystage steps (#9933) 
automerge
 2020-05-08 03:46:29 -07:00
carllin e970c58330
Properly handle ancestor/descendant maps (#9932) 
* Account for descendants < root not existing in BankForks, purge ancestors/descendants map for consistency with BankForks and progress map


Co-authored-by: Carl <carl@solana.com>
 2020-05-07 23:39:57 -07:00
Tyera Eulberg 754c65c066
Refactor RPC subscriptions account handling (#9888) 
* Switch subscriptions to use commitment instead of confirmations

* Add bank method to return account and last-modified slot

* Add last_modified_slot to subscription data and use to filter account subscriptions

* Update tests to non-zero last_notified_slot

* Add accounts subscriptions to test; fails at higher tx load

* Pass BankForks to RpcSubscriptions

* Use BankForks on add_account_subscription to properly initialize last_notified_slot

* Bundle subscriptions

* Check for non-equality

* Use commitment to initialize last_notified_slot; revert context.slot chage
 2020-05-07 00:23:06 -06:00
carllin 445e6668c2
Fix (#9896) 
Co-authored-by: Carl <carl@solana.com>
 2020-05-06 11:44:49 -07:00
Michael Vines 09ae61651a
Eliminate BankForksInfo (#9887) 2020-05-06 08:24:59 -07:00
Michael Vines 16ddd001f6
Gossip no longer pushes/pulls from nodes with a different shred version (#9868) 2020-05-05 20:15:19 -07:00
Michael Vines 72312ad615
Avoid holding the entire rooted path while loading bank forks (#9885) 2020-05-05 19:45:41 -07:00
carllin 3442f36f8a
Repair alternate versions of dead slots (#9805) 
Co-authored-by: Carl <carl@solana.com>
 2020-05-05 14:07:21 -07:00
Tyera Eulberg 627bc7e3a9
Rpc: Filter blockstore data by cluster-confirmed root (#9873) 
automerge
 2020-05-04 18:39:27 -07:00
Tyera Eulberg f5b0d13f08
Rpc: add getLargestAccounts endpoint (#9869) 
automerge
 2020-05-04 16:46:10 -07:00
Tyera Eulberg 6deaf649ef
Add commitment Root variant, and add fleshed out --commitment arg to Cli (#9806) 
automerge
 2020-04-29 21:55:33 -07:00
Ryo Onodera a91236012d
Pass around --max-genesis-archive-unpacked-size (#9161) 
automerge
 2020-04-29 18:53:34 -07:00
anatoly yakovenko a0514eb2ae
thiserror, docs, remove general Failure case (#9741) 
automerge
 2020-04-29 18:12:51 -07:00
Michael Vines 230df0ec0c
Upgrade to Rust 1.43.0 (#9754) 2020-04-29 18:02:05 -07:00
Tyera Eulberg efb4988d10
Rpc: remove unwraps (#9793) 
automerge
 2020-04-29 12:06:09 -07:00
Stephen Akridge eb24f3df84 Update dalek version 2020-04-28 21:02:47 -06:00
Tyera Eulberg 9d91cca73c
Remove commented code 2020-04-28 15:47:41 -06:00
Tyera Eulberg 655e3bc418
Rpc: Use cluster largest_confirmed_root as commitment max (#9750) 
automerge
 2020-04-28 10:20:43 -07:00
Tyera Eulberg a12428a5b8
Use Blockstore lowest_slot to start root iterator (#9738) 2020-04-28 10:22:10 -06:00
anatoly yakovenko 193dbb1794
sanitize lowest slots (#9747) 2020-04-27 20:22:30 -07:00
Michael Vines c11abf88b7
Clean up `use` to keep rust 1.43.0 from complaining (#9740) 2020-04-27 16:54:11 -07:00
anatoly yakovenko 8ef097bf6f
Input values are not sanitized after they are deserialized, making it far too easy for Leo to earn SOL (#9706) 
* sanitize gossip protocol messages
* sanitize transactions
* crds protocol sanitize
 2020-04-27 11:06:00 -07:00
Tyera Eulberg a7f33b5014
Cache banks in BankForks until optional largest_confirmed_root (#9678) 
automerge
 2020-04-24 15:49:57 -07:00
dependabot-preview[bot] d7f37a703e
Bump jsonrpc-derive from 14.0.5 to 14.1.0 (#9702) 
Bumps [jsonrpc-derive](https://github.com/paritytech/jsonrpc) from 14.0.5 to 14.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/jsonrpc-derive-v14.0.5...v14.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-04-24 15:24:18 -07:00
dependabot-preview[bot] c92f95e0b8
Bump jsonrpc-ws-server from 14.0.6 to 14.1.0 (#9701) 
Bumps [jsonrpc-ws-server](https://github.com/paritytech/jsonrpc) from 14.0.6 to 14.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/ipc-14.0.6...v14.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-04-24 15:24:07 -07:00
sakridge fa20963b93
Revert shred fs (#9712) 
* Revert "Untar is called for shred archives that do not exist. (#9565)"

This reverts commit 729cb5eec6.

* Revert "Dont insert shred payload into rocksdb (#9366)"

This reverts commit 5ed39de8c5.
 2020-04-24 15:04:23 -07:00
Greg Fitzgerald 76b1c2baf0
One less alloc per transaction (#9705) 
* One less alloc per transaction

* Fix benches

* Fix compiler warnings in bench build

* Fix move build

* Fix bench
 2020-04-24 13:03:46 -06:00
Leopold Schabel 767a0f9384
Fix comment in serve_repair (#9703) 
automerge
 2020-04-24 10:00:23 -07:00
sakridge 79829c98db
Fix vote listener passing bad transactions (#9694) 2020-04-23 17:04:09 -07:00
dependabot-preview[bot] d2cef8ed9b
Bump num_cpus from 1.12.0 to 1.13.0 (#9682) 
* Bump num_cpus from 1.12.0 to 1.13.0

Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.12.0...v1.13.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2020-04-23 13:08:06 -07:00
dependabot-preview[bot] 17a8b0f783
Bump jsonrpc-core-client from 14.0.5 to 14.1.0 (#9683) 
Bumps [jsonrpc-core-client](https://github.com/paritytech/jsonrpc) from 14.0.5 to 14.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/jsonrpc-core-client-v14.0.5...v14.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-04-23 13:07:50 -07:00
Michael Vines d1cbccd9ba
solana-dos can now DoS gossip nodes (#9652) 
automerge
 2020-04-23 11:46:12 -07:00
Tyera Eulberg 5d2f488004
Add getLowestNonpurgedBlock rpc; use blockstore api in getConfirmedBlocks (#9656) 
automerge
 2020-04-22 13:33:06 -07:00
Tyera Eulberg d5abff82e0
Add largest_confirmed_root to BlockCommitmentCache (#9640) 
* Add largest_confirmed_root to BlockCommitmentCache

* clippy

* Add blockstore to BlockCommitmentCache to check root

* Add rooted_stake helper fn and test

* Nodes that are behind should correctly id confirmed roots

* Simplify rooted_stake collector
 2020-04-22 12:22:09 -06:00
dependabot-preview[bot] 08e73e5366
Bump jsonrpc-http-server from 14.0.6 to 14.1.0 (#9628) 
automerge
 2020-04-22 09:21:05 -07:00
sakridge 2e8349196e
Custom epochslots debug (#9647) 2020-04-22 08:55:08 -07:00
dependabot-preview[bot] 2a935ec15f
Bump jsonrpc-pubsub from 14.0.6 to 14.1.0 (#9649) 
Bumps [jsonrpc-pubsub](https://github.com/paritytech/jsonrpc) from 14.0.6 to 14.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/compare/ipc-14.0.6...v14.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-04-22 07:49:35 -07:00
carllin bab3502260
Push down cluster_info lock (#9594) 
* Push down cluster_info lock

* Rework budget decrement

Co-authored-by: Carl <carl@solana.com>
 2020-04-21 12:54:45 -07:00
Michael Vines ad186b8652
Flag test_tvu_exit as serial to hopefully reduce CI flakiness (#9509) 2020-04-21 12:54:16 -07:00
Justin Starry 3023691487
RPC: Allow single slot address history queries (#9630) 2020-04-22 01:08:06 +08:00
dependabot-preview[bot] 64e5684d45 Bump bs58 from 0.3.0 to 0.3.1 
Bumps [bs58](https://github.com/mycorrhiza/bs58-rs) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/mycorrhiza/bs58-rs/releases)
- [Commits](https://github.com/mycorrhiza/bs58-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-21 08:07:00 -07:00
Tyera Eulberg 18cba86f77
Wait for supermajority of cluster to have rooted a transaction to consider it finalized (#9618) 
* Add rooted stake to BlockCommitment

* Use rooted stake to include cluster check
 2020-04-20 23:25:49 -06:00
Michael Vines 914b022663
cli: Add transaction-history (#9614) 
automerge
 2020-04-20 22:01:09 -07:00
Anatoly Yakovenko 8a69ea971f test 2020-04-20 08:44:39 -07:00
dependabot-preview[bot] b1ca74ed30 Bump regex from 1.3.6 to 1.3.7 
Bumps [regex](https://github.com/rust-lang/regex) from 1.3.6 to 1.3.7.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.6...regex-1.3.7)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-20 08:02:22 -07:00
anatoly yakovenko 77fb4230d6
Calculate distance between u64 without overflow (#9592) 
* fix overflow

* fixed num_live_peers overflow
 2020-04-19 23:05:26 -07:00
carllin 1607891b29
log proper slot (#9576) 
Co-authored-by: Carl <carl@solana.com>
 2020-04-19 14:24:45 -07:00
sakridge 65a9658b13
Budget for gossip traffic (#9550) 2020-04-18 22:11:17 -07:00
Michael Vines a7aa7e172b
validator: Consider the activated stake of this node to be online again (#9573) 
automerge
 2020-04-18 10:16:19 -07:00
dependabot-preview[bot] cb0a1a94a7 Bump jsonrpc-core from 14.0.5 to 14.1.0 
Bumps [jsonrpc-core](https://github.com/paritytech/jsonrpc) from 14.0.5 to 14.1.0.
- [Release notes](https://github.com/paritytech/jsonrpc/releases)
- [Commits](https://github.com/paritytech/jsonrpc/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-17 20:06:15 -07:00
Michael Vines 8509dcb8a0 Report offline/wrong-shred nodes while waiting for a super majority in gossip 2020-04-17 13:32:19 -07:00
Michael Vines 7b5cdf6adf Reduce ReceiveUpdates log spam 2020-04-17 13:32:19 -07:00
Michael Vines 55ed52a71d
Increase the number of JSON RPC service threads (#9551) 
automerge
 2020-04-17 12:37:33 -07:00
Michael Vines b58338b066
Make rpc_subscriptions.rs tests serial (#9556) 
automerge
 2020-04-17 10:48:39 -07:00
sakridge e655cba5bd
Make rpc tests serial (#9537) 2020-04-16 22:02:55 -07:00
carllin bcfd379f32
Simplify EpochSlots update (#9545) 
Co-authored-by: Carl <carl@solana.com>
 2020-04-16 19:32:19 -07:00
Michael Vines 47ae57610a Only build x86_64-unknown-linux-gnu on docs.rs 2020-04-16 19:06:17 -07:00
anatoly yakovenko 5ed39de8c5
Dont insert shred payload into rocksdb (#9366) 
automerge
 2020-04-16 18:20:55 -07:00
sakridge 66abe45ea1
Decouple accounts hash calculation from snapshot hash (#9507) 2020-04-16 15:12:20 -07:00
carllin 7aa4d401f7
Fix broadcast metrics (#9461) 
* Rework broadcast metrics to support multiple threads

* Update dashboards

Co-authored-by: Carl <carl@solana.com>
 2020-04-15 15:22:16 -07:00
sakridge a8b8c2f438
Move slow compaction test to integration (#9506) 2020-04-15 11:54:03 -07:00
Jack May 241a05fc52
Add native loader entry points (#9486) 2020-04-15 09:41:29 -07:00
Michael Vines 40737e9efa Add health check URI 2020-04-15 09:06:24 -07:00
sakridge 69f1e487b3
Reduce cluster-info metrics. (#9465) 2020-04-14 21:21:58 -07:00
Michael Vines 83a96c557d Ensure --dynamic-port-range is wide enough 2020-04-14 12:21:05 -07:00
Michael Vines 8cda974552
Fix flaky new_archiver_external_ip_test (#9457) 
automerge
 2020-04-12 11:49:34 -07:00
carllin 3037eb8d4f
Remove slot field, add test (#9444) 
Co-authored-by: Carl <carl@solana.com>
 2020-04-10 23:52:37 -07:00
carllin aa8dfac313
Simplify vote simulation (#9435) 
Co-authored-by: Carl <carl@solana.com>
 2020-04-10 15:16:12 -07:00
sakridge ae5a6419d4
ReceiveUpdates spams the log, adjust the threshold higher (#9429) 2020-04-10 10:21:46 -07:00
Michael Vines 85feca305b Avoid port conflict in new_with_external_ip_test_gossip 2020-04-09 20:27:54 -07:00
Tyera Eulberg 91159ea8e3
Rpc: Add getConfirmedSignaturesForAddress (#9407) 
automerge
 2020-04-09 20:21:31 -07:00
Michael Vines d5a9ee97f2
Add --allow-dead-slots argument to `slot`/`print`/`json` commands (#9408) 
automerge
 2020-04-09 20:10:51 -07:00
sakridge 900933bbcc
Reduce rpc test code (#9413) 
automerge
 2020-04-09 18:05:56 -07:00
sakridge be77bdef12
Allow lower shred count (#9410) 2020-04-09 16:36:44 -07:00
Michael Vines f3afe5c99c
Remove dead code (#9404) 
automerge
 2020-04-09 13:09:59 -07:00
Tyera Eulberg e1aa247548
Rpc: Add getConfirmedTransaction (#9381) 
* Add blockstore method to return a complete transaction by signature

* Plumb getConfirmedTransaction rpc

* Add doc
 2020-04-09 00:57:30 -06:00
Jack May ad0482be73
Revert "Add native loader entry points (#9275)" Breaks genesis_config abi (#9377) 
This reverts commit ed86d8d1fc.
 2020-04-08 14:36:18 -07:00
carllin 4522e85ac4
Add Metrics/Dashboards tracking block production (#9342) 
* Add metric tracking blocks/dropped blocks

Co-authored-by: Carl <carl@solana.com>
 2020-04-08 14:35:24 -07:00
Tyera Eulberg 36e73cada4
Add blockstore address-to-signature index (#9367) 
automerge
 2020-04-08 12:50:39 -07:00
sakridge 4677cdb4c2
Optimize broadcast cluster_info critical section (#9327) 2020-04-06 17:36:22 -07:00
Tyera Eulberg 96c23110ae
Make TestValidator mint_lamports configurable (#9337) 
automerge
 2020-04-06 16:20:55 -07:00
dependabot-preview[bot] 62cb2cd13c Bump serde_json from 1.0.49 to 1.0.51 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.49 to 1.0.51.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.49...v1.0.51)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-06 08:40:55 -07:00
Tyera Eulberg acf64f8476
Update getSignatureStatuses to return historical statuses (#9314) 
automerge
 2020-04-06 03:04:54 -07:00
Ryo Onodera b28ec430e4
Introduce background stale AppendVec shrink mechanism (#9219) 
* Introduce background AppendVec shrink mechanism

* Support ledger tool

* Clean up

* save

* save

* Fix CI

* More clean up

* Add tests

* Clean up yet more

* Use account.hash...

* Fix typo....

* Add comment

* Rename accounts_cleanup_service
 2020-04-06 17:30:23 +09:00
carllin 7b68628e6c
Remove write lock (#9311) 
* Remove write lock

Co-authored-by: Carl <carl@solana.com>
 2020-04-05 15:18:45 -07:00
Justin Starry b584174d67
Deprecate `confirmTransaction`, `getSignatureStatus`, and `getSignatureConfirmation` (#9298) 
* Deprecate `confirmTransaction`, `getSignatureStatus`, etc

* Rename get_signature_statuses to get_signature_statuses_with_commitment

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-04-05 14:31:24 +08:00
Tyera Eulberg 49e2cc6593
Rework TransactionStatus index in blockstore (#9281) 
automerge
 2020-04-04 20:24:06 -07:00
Michael Vines ad0997e15f
RPC: add `err` field to TransactionStatus, alongside the now deprecated `status` field (#9296) 
automerge
 2020-04-04 16:13:26 -07:00
dependabot-preview[bot] 8cdf406dd3 Bump serde from 1.0.105 to 1.0.106 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.105 to 1.0.106.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.105...v1.0.106)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-04-04 15:13:52 -07:00
Jack May ed86d8d1fc
Add native loader entry points (#9275) 2020-04-03 17:40:59 -07:00
sakridge 9493de4443
Add snapshot compression option (#9276) 2020-04-03 13:13:49 -07:00
Jack May d61191db40
fix bench warnings (#9277) 2020-04-02 21:56:38 -07:00
carllin 0139236464
ReplayStage fixes (#9271) (#9279) 
automerge
 2020-04-02 21:05:33 -07:00
Michael Vines 8b14eb9020
Place AccountsHashes in same enum ordinal position as the v1.0 version (#9251) 
automerge
 2020-04-01 18:47:50 -07:00
Tyera Eulberg 25ee36bbba
Undo breaking rpc removal of getSignatureConfirmation (#9245) 2020-04-01 17:56:18 -06:00
Justin Starry c7ba1994ac
Undo getSignatureStatus breaking change, add getSignatureStatuses (#9228) 
automerge
 2020-04-01 11:30:58 -07:00
Jack May 268e04cb4a
Rename CustomError to Custom (#9207) 2020-04-01 09:01:11 -07:00
sakridge 7605f1f540
Fix repair dos (#9056) 2020-04-01 06:48:35 -07:00
sakridge a74a64084d
Fix error with account hash list getting too big for gossip (#9197) 
Nodes will stop publishing hashes on gossip and also
slow memory leak.
 2020-03-31 21:39:48 -07:00
sakridge a61ddb6f61
max_ledger_slots -> max_ledger_shreds (#9198) 
automerge
 2020-03-31 17:21:19 -07:00
Michael Vines 0e2722c638
solana-validator now supports multiple --authorized-voter arguments (#9174) 
* Use Epoch type

* Vote account's authorized voter is now supported without a validator restart
 2020-03-31 08:23:42 -07:00
carllin 66946a4680
Check ClusterSlots for confirmation of block propagation (#9115) 2020-03-30 19:57:11 -07:00
sakridge 73e99cc513
Ledger cleanup fixes (#9131) 
* Fix purging happening every slot when cleanup service is not started at slot 0
* Purge by shred count instead of slots since slots can have variable
number of shreds
 2020-03-30 19:02:12 -07:00
Tyera Eulberg 50fa577af8
Use cluster confirmations in rpc and pubsub (#9138) 
* Add runtime methods to simply get status and slot

* Add helper function to get slot confirmation_count from BlockCommitmentCache

* Return cluster confirmations in getSignatureStatus

* Remove use of invalid get_signature_confirmation_status

* Remove unused methods

* Update pubsub to use cluster confirmations

* Fix test_check_signature_subscribe failure

* Refactor confirmations to read commitment cache only once

* Review comments

* Use bank, root from BlockCommitmentCache

* Update docs

* Add metric for block-commitment aggregations

Co-authored-by: Justin Starry <justin@solana.com>
 2020-03-30 17:53:25 -06:00
Tyera Eulberg 62040cef56
Store BlockCommitmentCache slot and root metadata (#9154) 
automerge
 2020-03-30 10:29:30 -07:00
sakridge ae66c0e497
Add repair message support to dos tool (#9090) 2020-03-29 14:44:25 -07:00
sakridge c67703e7a3
Make repair metrics less chatty (#9094) 2020-03-29 14:43:58 -07:00
dependabot-preview[bot] 2ed3e2160d
Bump serde_json from 1.0.48 to 1.0.49 (#9139) 
* Bump serde_json from 1.0.48 to 1.0.49

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.48 to 1.0.49.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.48...v1.0.49)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Update serde_json in programs

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Co-authored-by: Justin Starry <justin@solana.com>
 2020-03-28 12:19:40 -07:00
Justin Starry 8bbf6e3f54
Fix race in RPC subscriptions test (#9142) 2020-03-29 01:58:51 +08:00
Michael Vines d7fa40087c
Bump version to 1.2.0 2020-03-28 09:44:13 -07:00
Michael Vines 4a8b1d9b2c
RpcClient now returns Signatures instead of Strings (#9129) 2020-03-27 15:46:00 -07:00
Justin Starry c1a3b6ecc2
Add RPC subscription api for rooted slots (#9118) 
automerge
 2020-03-27 09:33:40 -07:00
carllin d47262d233
Reduce transmit frequency (#9113) 
Co-authored-by: Carl <carl@solana.com>
 2020-03-26 23:33:28 -07:00
carllin 5a8658283a
Add check for propagation of leader block before generating further blocks (#8758) 
Co-authored-by: Carl <carl@solana.com>
 2020-03-26 19:57:27 -07:00
Tyera Eulberg 4b97e58cba
Consolidate signature-status rpcs (#9069) 
* getSignatureStatus: return confirmations for non-rooted transactions

* Remove getNumConfirmations.. rpc

* Remove getSignatureConfirmation

* Review comments

* More review comments
 2020-03-26 19:21:01 -06:00
carllin f3d556e3f9
Refactor VoteTracker (#9084) 
* Refactor VoteTracker

Co-authored-by: Carl <carl@solana.com>
 2020-03-26 17:55:17 -07:00
sakridge b7b4aa5d4d
move rpc types from client to client-types crate (#9039) 
* Separate client types into own crate, so ledger does not need it

Removes about 50 crates of dependency from ledger

* Drop Rpc name from transaction-status types
 2020-03-26 13:29:30 -07:00
sakridge ed036b978d
Accumulate blockstore metrics and submit every 2s (#9075) 2020-03-26 12:51:41 -07:00
Justin Starry e4472db33f
Unflake rpc subscriptions test by reducing sub count (#9078) 
automerge
 2020-03-25 20:43:38 -07:00
carllin 076fef5e57
Update Cluster Slots to support multiple threads (#9071) 
Co-authored-by: Carl <carl@solana.com>
 2020-03-25 18:09:19 -07:00
dependabot-preview[bot] 095c79e863
Bump regex from 1.3.5 to 1.3.6 (#9055) 
Bumps [regex](https://github.com/rust-lang/regex) from 1.3.5 to 1.3.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.5...1.3.6)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-25 17:24:14 -07:00
Ryo Onodera f987c18a7e
Strictly validate the contents of snapshot/genesis (#8959) 
automerge
 2020-03-25 02:46:41 -07:00
Justin Starry 8f38bc7dc0
Refactor how pubsub subscriptions are added (#9042) 2020-03-25 00:53:32 +08:00
Justin Starry 19dfb87b1f
Fix timeout for subscriptions test (#9043) 
automerge
 2020-03-24 01:57:28 -07:00
Justin Starry eed8087d87
Respect confirmations param for signature subscription notifications (#9019) 
automerge
 2020-03-23 17:00:34 -07:00
Tyera Eulberg 1b8f9e75dd
Update getSignatureStatus: support multiple signatures, include slot in each response item (#9022) 
* Rename enable-rpc-get-confirmed-block

* Rename RpcTransactionStatus -> RpcTransactionStatusMeta

* Return simplified RpcTransactionStatus; Add support for multiple transactions

* Update docs

* typo
 2020-03-23 11:25:39 -06:00
sakridge 4b397d15b3
Accounts cleanup service and perf improvements (#8799) 
* Use atomic for ref count instead of taking rwlock

* Accounts cleanup service

* Review comments
 2020-03-23 08:50:23 -07:00
sakridge 4d2b83d01f
Add option to disable rocks compaction (#9011) 2020-03-23 08:42:32 -07:00
Justin Starry a0ffcc61ae
Add slot info to Bank::get_signature_confirmation_status (#9018) 2020-03-23 21:55:15 +08:00
Justin Starry 4b4819cd07
Add slot context to rpc pubsub notifications (#9001) 
automerge
 2020-03-23 05:34:42 -07:00
Michael Vines 88ba8439fc
Add frozen account support (#8989) 
automerge
 2020-03-22 11:10:04 -07:00
Michael Vines aa24181a53
Remove blockstream unix socket support. RPC or bust (#9004) 
automerge
 2020-03-21 20:17:11 -07:00
Michael Vines 18c1f0dfe9
Remove stub core/src/genesis_utils.rs (#8999) 2020-03-21 10:54:40 -07:00
sakridge 909321928c
Shred fetch comment and debug message tweak (#8980) 
automerge
 2020-03-20 11:00:48 -07:00
sakridge 453f5ce8f2
Shred filter (#8975) 
Thread bank_forks into shred fetch
 2020-03-20 07:49:48 -07:00
carllin dc1db33ec9
Add Capabilities to Signal BroadcastStage to Retransmit (#8899) 2020-03-19 23:35:01 -07:00
dependabot-preview[bot] 24d871b529
Bump serde from 1.0.104 to 1.0.105 (#8954) 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.104 to 1.0.105.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.104...v1.0.105)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-18 22:35:30 -07:00
dependabot-preview[bot] 2dc50cff5b
Bump bv from 0.11.0 to 0.11.1 (#8952) 
automerge
 2020-03-18 21:37:21 -07:00
sakridge c51049a59b
Add counter for accounts hash verification. (#8928) 2020-03-18 08:39:09 -07:00
anatoly yakovenko 9cedeb0a8d
Pull streamer out into its own module. (#8917) 
automerge
 2020-03-17 23:30:23 -07:00
sakridge 7079559c2d
Fix windows build by removing sys-info (#8860) 
Doesn't build for windows.
 2020-03-16 12:53:13 -07:00
sakridge dc347dd3d7
Add Accounts hash consistency halting (#8772) 
* Accounts hash consistency halting

* Add option to inject account hash faults for testing.

Enable option in local cluster test to see that node halts.
 2020-03-16 08:37:31 -07:00
Carl ead6dc553a If let 2020-03-16 07:57:07 -07:00
Carl 009c124fac Remove generic 2020-03-16 07:57:07 -07:00
Carl 9411fc00b8 Lower error level 2020-03-16 07:57:07 -07:00
carllin 9afc5da2e1
Fix vote polling (#8829) 
Co-authored-by: Carl <carl@solana.com>
 2020-03-15 20:31:05 -07:00
Michael Vines 5481d1a039 Validators now run a full gossip node while looking for a snapshot 2020-03-15 09:31:55 -07:00
Michael Vines 29fb79382c Rework validator vote account defaults to half voting fees 2020-03-13 20:13:33 -07:00
carllin 53b8d0d528
Remove holding Poh lock (#8838) 
automerge
 2020-03-13 15:15:13 -07:00
dependabot-preview[bot] 52b254071c
Bump regex from 1.3.4 to 1.3.5 (#8830) 
Bumps [regex](https://github.com/rust-lang/regex) from 1.3.4 to 1.3.5.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.3.4...1.3.5)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-13 07:43:07 -06:00
anatoly yakovenko 9a79be5ca0
Use cluster information about slots to prioritize repair (#8820) 
automerge
 2020-03-12 17:34:46 -07:00
dependabot-preview[bot] f93c8290f4
Bump sys-info from 0.5.9 to 0.5.10 (#8810) 
Bumps [sys-info](https://github.com/FillZpp/sys-info-rs) from 0.5.9 to 0.5.10.
- [Release notes](https://github.com/FillZpp/sys-info-rs/releases)
- [Changelog](https://github.com/FillZpp/sys-info-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/FillZpp/sys-info-rs/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-12 13:42:31 -06:00
dependabot-preview[bot] 48ac038f7a
Bump serial_test from 0.3.2 to 0.4.0 (#8808) 
Bumps [serial_test](https://github.com/palfrey/serial_test) from 0.3.2 to 0.4.0.
- [Release notes](https://github.com/palfrey/serial_test/releases)
- [Commits](https://github.com/palfrey/serial_test/compare/v0.3.2...v0.4.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-12 11:14:37 -06:00
anatoly yakovenko f64ab49307
Cluster has no way to know which slots are available (#8732) 
automerge
 2020-03-11 21:31:50 -07:00
Jack May 6eb4973780
Don't use move semantics if not needed (#8793) 2020-03-11 14:37:23 -07:00
dependabot-preview[bot] 75a84ecdae
Bump reqwest from 0.10.1 to 0.10.4 (#8787) 
Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.10.1 to 0.10.4.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.10.1...v0.10.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-11 09:02:40 -06:00
dependabot-preview[bot] e41ff2df66
Bump chrono from 0.4.10 to 0.4.11 (#8755) 
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.10 to 0.4.11.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/master/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.10...v0.4.11)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-10 08:43:18 -06:00
dependabot-preview[bot] f88b79d42b
Bump itertools from 0.8.2 to 0.9.0 (#8756) 
Bumps [itertools](https://github.com/bluss/rust-itertools) from 0.8.2 to 0.9.0.
- [Release notes](https://github.com/bluss/rust-itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bluss/rust-itertools/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-10 08:42:11 -06:00
dependabot-preview[bot] 1a0dd53450
Bump rayon from 1.2.0 to 1.3.0 (#8757) 
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.2.0...rayon-core-v1.3.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-10 08:40:02 -06:00
carllin 9872430bd2
Add VoteTracker for tracking cluster's votes in gossip (#8327) 
Track votes by slot in cluster_vote_listener
 2020-03-09 22:03:09 -07:00
Michael Vines 36fa3a1a0a
Wait for 80% of the active stake instead of 75% 2020-03-09 20:31:09 -07:00
dependabot-preview[bot] fad08a19cc
Bump serde_json from 1.0.46 to 1.0.48 (#8260) 
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.46 to 1.0.48.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.46...v1.0.48)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-09 20:20:19 -06:00
dependabot-preview[bot] ff74452ef3
Bump libc from 0.2.66 to 0.2.67 (#8680) 
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.66 to 0.2.67.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.66...0.2.67)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
 2020-03-08 21:38:58 -07:00
Michael Vines f992ee3140
Remove unnecessary snapshot hash verification (#8711) 2020-03-07 09:04:52 -07:00
sakridge 97986a5241
Move download code to download-utils crate (#8704) 2020-03-07 07:08:01 -08:00
Michael Vines cea8067219
Disable setLogFilter RPC API by default (#8693) 
automerge
 2020-03-06 16:03:10 -08:00
Trent Nelson 4db074a5aa
RPC: Add `getFeeCalculatorForBlockhash` method call (#8687) 
Returns the `FeeCalculator` associated with the given blockhash, or
`null` if said blockhash has expired
 2020-03-06 17:01:31 -07:00
Michael Vines cb6848aa80
Publish initial snapshot hash in gossip on validator startup (#8679) 
automerge
 2020-03-05 22:52:31 -08:00
Greg Fitzgerald 2242b1b4a5
Bump byteorder from 1.3.2 to 1.3.4 (#8159) 
Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.3.2 to 1.3.4.
- [Release notes](https://github.com/BurntSushi/byteorder/releases)
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.2...1.3.4)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
 2020-03-05 17:07:18 -07:00
carllin f47a789b15
Add find_incomplete_slots (#8654) 
* Add find_incomplete_slots

* Add live slots iterator
 2020-03-05 10:58:00 -08:00
Trent Nelson fd00e5cb35
Store FeeCalculator with blockhash in nonce accounts (#8650) 
* Copy current state version to v0

* Add `FeeCalculator` to nonce state

* fixup compile

* Dump v0 handling...

Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...

* Create more representative test `RecentBlockhashes`

* Improve CLI nonce account display

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix that last bank test...

* clippy/fmt

Co-authored-by: Michael Vines <mvines@gmail.com>
 2020-03-05 07:40:26 -07:00
Michael Vines 448b957a13
Add --bind-address and --rpc-bind-address validator arguments (#8628) 2020-03-04 22:46:43 -07:00
carllin dec3da8f9d
Add orphan iterator (#8636) 2020-03-04 18:10:30 -08:00
Grimes f89c22b5ee
`solana catchup` now detects when you try to catchup to yourself (#8635) 
automerge
 2020-03-04 14:44:21 -08:00
carllin f23dc11a86
compute_bank_stats needs to return newly computed ForkStats (#8608) 
* Fix broken confirmation, add test
 2020-03-04 11:49:56 -08:00
Michael Vines 306783c661
Don't advertise the snapshot that the node was loaded from 
snapshot_packager_service will remove this snapshot hash from gossip
when it starts
 2020-03-02 18:58:53 -07:00
Justin Starry 8ec8204a30
Run pubsub test poller in tokio runtime (#8494) 2020-03-03 09:44:39 +08:00
Michael Vines 42c5c59800 Only gossip packaged snapshots 2020-03-02 14:17:17 -07:00
carllin 8ef8c9094a
Add ReplayStage changes for checking switch threshold (#8504) 
* Refactor for supporting switch threshold check
 2020-03-02 12:43:43 -08:00
Michael Vines 13551885c2 --wait-for-supermajority now requires a SLOT 2020-03-02 12:59:35 -07:00
Michael Vines 1ca4913328 Avoid is_x86_feature_detected when not building for x86 2020-03-01 18:10:43 -07:00
Michael Vines 862a4a243f
Demote gossip responder error log messages to info! 2020-03-01 10:43:20 -07:00
Michael Vines 2a5605db24 Reduce max snapshot hashes to stay under MTU 2020-02-29 09:21:52 -07:00
carllin 6a5a6387e2
Fix skipping own leader slots (#8533) 
automerge
 2020-02-29 00:05:35 -08:00
Michael Vines 0f31adeafb GET for /snapshot.tar.bz2 now redirects to the latest snapshot 2020-02-28 23:23:59 -07:00
Michael Vines ae817722d8 Include validator version in log 2020-02-28 23:23:59 -07:00
Trent Nelson 90bedd7e06
Split signature throughput tracking out of `FeeCalculator` (#8447) 
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`

Leaving `FeeCalculator` to *only* calculate transaction fees

* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate

* Expose recent `FeeRateGovernor` to clients

* Move `burn()` back into `FeeCalculator`

Appease BPF tests

* Revert "Move `burn()` back into `FeeCalculator`"

This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.

* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`

* Make `FeeRateGovernor`'s `lamports_per_signature` private

* rebase artifacts

* fmt

* Drop 'Recent'

* Drop _with_commitment variant

* Use a more portable integer for `target_signatures_per_slot`

* Add docs for `getReeRateCalculator` JSON RPC method

* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
 2020-02-28 13:27:01 -07:00
Michael Vines 3acf956f6f Fix test_concurrent_snapshot_packaging 2020-02-26 23:32:53 -07:00
Michael Vines 0d4cb252c4 Adapt local-cluster/ 2020-02-26 23:32:53 -07:00
Michael Vines fcabc6f799 Rename snapshot.tar.bz2 to snapshot-<slot>-<hash>.tar.bz2 2020-02-26 23:32:53 -07:00
carllin 5f766cd20b
Remove loop (#8493) 2020-02-26 19:59:28 -08:00
Michael Vines 27c5ec0149 Use the same reqwest features across the repo 2020-02-26 20:47:43 -07:00