Commit Graph

184 Commits

Author SHA1 Message Date
Greg Fitzgerald 0550b893b0
Fix typos (#10675)
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-06-17 20:54:52 -07:00
Kristofer Peterson e23340d89e
Clippy cleanup for all targets and nighly rust (also support 1.44.0) (#10445)
* address warnings from 'rustup run beta cargo clippy --workspace'

minor refactoring in:
- cli/src/cli.rs
- cli/src/offline/blockhash_query.rs
- logger/src/lib.rs
- runtime/src/accounts_db.rs

expect some performance improvement AccountsDB::clean_accounts()

* address warnings from 'rustup run beta cargo clippy --workspace --tests'

* address warnings from 'rustup run nightly cargo clippy --workspace --all-targets'

* rustfmt

* fix warning stragglers

* properly fix clippy warnings test_vote_subscribe()
replace ref-to-arc with ref parameters where arc not cloned

* Remove lock around JsonRpcRequestProcessor (#10417)

automerge

* make ancestors parameter optional to avoid forcing construction of empty hash maps

Co-authored-by: Greg Fitzgerald <greg@solana.com>
2020-06-09 09:38:14 +09:00
Greg Fitzgerald 5d248fe5f8
Permit paying oneself (#10337)
* Allow paying to oneself

* cargo fmt

* Permit pay-to-self in CLI

No test here because we're just removing an [untested] special case.

Fixes #10339
2020-05-30 20:12:48 -06:00
Kristofer Peterson b7a32f01c0
Multi-version snapshot support (#9980)
* Multi-version snapshot support

* rustfmt

* Remove CLI options and runtime support for selection output snapshot version.
Address some clippy complaints.

* Muzzle clippy type complexity warning.

Despite clippy's suggestion, it is not currently possible to create type aliases
for traits and so everything within the 'Box<...>' cannot be type aliased.

This then leaves creating full blown traits, and either implementing
said traits by closure (somehow) or moving the closures into new structs
implementing said traits which seems a bit of a palaver.

Alternatively it is possible to define and use the type alias 'type ResultBox<T> = Result<Box<T>>'
which does seems rather pointless and not a great reduction in complexity but is enough to keep clippy happy.

In the end I simply went with squelching the clippy warning.

* Remove now unused Serialize/Deserialize trait implementations for AccountStorageEntry and AppendVec

* refactor versioned de/serialisers

* rename serde_utils to serde_snapshot

* move call to accounts_db.generate_index() back down to context_accountsdb_from_stream()

* update version 1.1.1 to 1.2.0
remove nested use of serialize_bytes

* cleanups

* Add back measurement of account storage entry serialization.
Remove construction of Vec and HashMap temporaries during serialization.

* consolidate serialisation test cases into serde_snapshot.
clean up leakage of implementation details in serde_snapshot.

* move short term / legacy snapshot code into child module

* add serialize_iter_as_tuple

* preliminary integration of following commit

commit 6d58b73c47294bfb93465d5a83cd2175660b6e6d
Author: Ryo Onodera <ryoqun@gmail.com>
Date:   Wed May 20 14:02:02 2020 +0900

    Confine snapshot 1.1 relic to versioned codepath

* refactored serde_snapshot, rustfmt
legacy accounts_db format now "owns" both leading u64s, legacy bank_rc format has none

* reduce type complexity (clippy)
2020-05-23 02:54:24 +09:00
Kristofer Peterson 58ef02f02b
9951 clippy errors in the test suite (#10030)
automerge
2020-05-15 09:35:43 -07:00
Jack May eb1acaf927
Remove archiver and storage program (#9992)
automerge
2020-05-14 18:22:47 -07:00
Ryo Onodera 1eb40c3fe0
Introduce eager rent collection (#9527)
* Switch AccountsIndex.account_maps from HashMap to BTreeMap

* Introduce eager rent collection

* Start to add tests

* Avoid too short eager rent collection cycles

* Add more tests

* Add more tests...

* Refacotr!!!!!!

* Refactoring follow up

* More tiny cleanups

* Don't rewrite 0-lamport accounts to be deterministic

* Refactor a bit

* Do hard fork, restore tests, and perf. mitigation

* Fix build...

* Refactor and add switch over for testnet (TdS)

* Use to_be_bytes

* cleanup

* More tiny cleanup

* Rebase cleanup

* Set Bank::genesis_hash when resuming from snapshot

* Reorder fns and clean ups

* Better naming and commenting

* Yet more naming clarifications

* Make prefix width strictly uniform for 2-base partition_count

* Fix typo...

* Revert cluster-dependent gate

* kick ci?

* kick ci?

* kick ci?
2020-05-13 16:22:14 +09:00
Tyera Eulberg f5b0d13f08
Rpc: add getLargestAccounts endpoint (#9869)
automerge
2020-05-04 16:46:10 -07:00
sakridge f37f83fd12
Fuzzer test and fixes (#9853) 2020-05-02 08:07:52 -07:00
sakridge 894549f002
Put empty accounts in the accounts list on load (#9840)
Indexing into accounts array does not match account_keys otherwise.
Also enforce program accounts not at index 0
Enforce at least 1 Read-write signing fee-payer account.
2020-05-01 17:23:33 -07:00
Michael Vines 8dfe0affd4
Add incinerator sysvar (#9815) 2020-04-30 22:04:08 -07:00
Ryo Onodera 9918539229
Introduce type alias Ancestors (#9699)
* Introduce type alias AncestorList

* Rename AncestorList => Ancestors
2020-04-27 11:07:03 +09:00
sakridge e907c0e650
Filter program ids to store (#9721)
automerge
2020-04-26 00:11:37 -07:00
Ryo Onodera 2623c71ed3
Use type aliases/resulting var names consistently (#9060) 2020-03-26 13:08:56 +09:00
Justin Starry ef3af104ae
Return appropriate error for invalid program account (#9047)
automerge
2020-03-25 13:23:05 -07:00
Michael Vines 88ba8439fc
Add frozen account support (#8989)
automerge
2020-03-22 11:10:04 -07:00
sakridge 4dd0367136
Rwlock storage opt (#9006)
* Remove unecessary account paths rwlock

* Remove path rwlock in accounts_db and optimize storage critical section
2020-03-22 10:04:03 -07:00
sakridge 2435c3ce0c
Add accounts-bench, a benchmark to test the accounts store speed (#8866) 2020-03-17 11:02:07 -07:00
Ryo Onodera 4bbf09f582
Enable conservative out-of-bound snapshot cleaning (#8811)
* Enable conservative out-of-bound snapshot cleaning

* Add tests
2020-03-13 14:44:00 +09:00
Jack May 97c5fb8141
Allow passing of program_ids to programs (#8639) 2020-03-05 10:57:12 -08:00
Trent Nelson fd00e5cb35
Store FeeCalculator with blockhash in nonce accounts (#8650)
* Copy current state version to v0

* Add `FeeCalculator` to nonce state

* fixup compile

* Dump v0 handling...

Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...

* Create more representative test `RecentBlockhashes`

* Improve CLI nonce account display

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix that last bank test...

* clippy/fmt

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-03-05 07:40:26 -07:00
Jack May b85d7c1f70
Fix account tests (#8615) 2020-03-04 10:40:41 -08:00
Trent Nelson 1cc7131bb7
Consolidate Nonce state under one struct (#8624)
automerge
2020-03-04 08:51:48 -08:00
Trent Nelson be0cc0273f
SDK: Re-org nonce state module to facilitate versioning (#8603)
automerge
2020-03-03 17:00:39 -08:00
Trent Nelson 90bedd7e06
Split signature throughput tracking out of `FeeCalculator` (#8447)
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`

Leaving `FeeCalculator` to *only* calculate transaction fees

* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate

* Expose recent `FeeRateGovernor` to clients

* Move `burn()` back into `FeeCalculator`

Appease BPF tests

* Revert "Move `burn()` back into `FeeCalculator`"

This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.

* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`

* Make `FeeRateGovernor`'s `lamports_per_signature` private

* rebase artifacts

* fmt

* Drop 'Recent'

* Drop _with_commitment variant

* Use a more portable integer for `target_signatures_per_slot`

* Add docs for `getReeRateCalculator` JSON RPC method

* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
2020-02-28 13:27:01 -07:00
sakridge 947a339714
Add snapshot hash of full accounts state (#8295)
* Add snapshot hash of full accounts state

* Use normal hashing for the accounts delta state

* Add merkle
2020-02-22 13:46:40 -08:00
Ryo Onodera d238371b0c
Correct missing entry handling to avoid bad warns (#8339)
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360)
automerge
2020-02-20 13:28:55 -08:00
Greg Fitzgerald e8124324ff
Support transaction signing by heterogenous lists of keypairs (#8342)
automerge
2020-02-20 12:13:23 -08:00
Michael Vines 1bf2285fa2 ledger-tool: Add print-accounts command 2020-02-14 19:59:48 -07:00
Ryo Onodera 7614af2a45
Verify frozen bank from snapshot by hashing (#8184) 2020-02-11 16:46:33 +09:00
Michael Vines 3ac0192d40 Better surface bank hash verification failures 2020-02-05 11:39:47 -07:00
Jack May 83718a3b3e
Cleanup runtime use syntax (#8002) 2020-01-28 17:03:20 -08:00
Ryo Onodera 2c7447b73e
Secure sysvars under hash by freezing all strictly (#7892)
* Secure sysvars under hash by freezing all strictly

* Fix hash's non-idempotnet and add new test

* Clean up

* More cleanups
2020-01-24 16:10:32 +09:00
Jack May 023074650f
Allow the same account to be passed multiple times to a single instruction (#7795) 2020-01-22 09:11:56 -08:00
Justin Starry 87598c7612 Consolidate tx error counters and update metrics dashboard (#7724)
automerge
2020-01-16 23:26:50 -08:00
sakridge 3b78be83cf Add hash stats information to check hashes between validators (#7780)
automerge
2020-01-14 11:57:29 -08:00
Trent Nelson 9754fc789e
Manage durable nonce stored value in runtime (#7684)
* Bank: Return nonce pubkey/account from `check_tx_durable_nonce`

* Forward account with HashAgeKind::DurableNonce

* Add durable nonce helper for HashAgeKind

* Add nonce util for advancing stored nonce in runtime

* Advance nonce in runtime

* Store rolled back nonce account on TX InstructionError

* nonce: Add test for replayed InstErr fee theft
2020-01-10 16:57:31 -07:00
Trent Nelson 7e94cc2cc3 Move nonce into system program (#7645)
automerge
2020-01-03 16:34:58 -08:00
Ryo Onodera c8fe4043b6
Rename slot_hash => bank_hash in AcoountsDB (#7579)
* Rename slot_hash => bank_hash in AcoountsDB
2019-12-23 10:50:31 +09:00
sakridge 7dec934bb3
Optimize lock_accounts mutex use (#7593)
Use the lock for the whole batch instead of per-tx
Optimize the critical section to pre-generate the keys necessary
before taking the lock.
2019-12-21 10:43:22 -08:00
Ryo Onodera 3c361eb759
Check account hashes in snapshot (#7559)
* Check for incorrect hash value

* Finish up checking for incorrect hash value

* Fix comment a bit

Co-authored-by: sakridge <sakridge@gmail.com>
2019-12-20 09:39:30 +09:00
Trent Nelson 1ffd6b4b4d
Add program and runtime support for Durable Transaction Nonces (#6845)
* Rework transaction processing result forwarding

Durable nonce prereq

* Add Durable Nonce program API

* Add runtime changes for Durable Nonce program

* Register Durable Nonce program

* Concise comments and bad math

* Fix c/p error

* Add rent sysvar to withdraw ix

* Remove rent exempt required balance from Meta struct

* Use the helper
2019-12-07 12:54:10 -07:00
Justin Starry b7d4330dd4
Fail fast if account paths cannot be canonicalized (#7300)
* Canonicalize account paths to avoid symlink issues

* fixes
2019-12-05 21:41:29 -05:00
Sagar Dhawan 4485b978c1
Clean up accounts hash internal state api (#7090) 2019-11-22 08:56:00 -08:00
Parth 9f6a2e51b2
add credit-debit rent handling (#6947)
* add credit-debit rent handling

* add tests

* charge rent for validator account for fee credit

* rent is stored per tx instead of account
2019-11-21 01:57:02 +05:30
Sagar Dhawan 42da1ce4e2
Fix bank hash not changing when no internal state has changed (#7052)
* Fix bank hash not changing when no internal state has changed

* Fix unnecessary call to hash_internal_state

* Add blockhash into the bank_hash

* Add blockhash into the bank_hash and update tests

* Refactor accounts_db slot_hashes

* More clarity in comments

* Add clippy suggestion

* Grammar

* Fix compile after clippy made me break it

* Schooled by clippy
2019-11-19 20:19:43 -08:00
Michael Vines e7f63cd336
Upgrade to rust 1.39.0 (#6939)
* Upgrade to rust 1.39.0

* 1.39.0 clippy
2019-11-14 12:27:01 -07:00
Parth 7b05b3dbb3
rent collector improvments (#6888)
* avoid account copying + pre-empt rent

* adding support for base rent
2019-11-14 10:56:49 +05:30
Tyera Eulberg c6931dcb07
Remove credit-only account handling (#6726)
* Renaming
- credit-only/credit-debit to read-only/read-write
- debitable to writable

* Remove credit handling, making credit-only accounts read-only

* Update programs to remove deprecated credit-only account designation

* Use readonly and writable instead of underscored types
2019-11-05 09:38:35 -07:00