Commit Graph

734 Commits

Author SHA1 Message Date
Justin Starry ef3af104ae
Return appropriate error for invalid program account (#9047)
automerge
2020-03-25 13:23:05 -07:00
carllin 9dc69d9843
Store and compute node/stake state in EpochStakes struct (#8958)
* Store and compute needed bank state in EpochStakes struct
2020-03-25 12:19:15 -07:00
sakridge 45348b2c83
Remove accounts unwrap (#9062)
automerge
2020-03-25 10:21:30 -07:00
Justin Starry eed8087d87
Respect confirmations param for signature subscription notifications (#9019)
automerge
2020-03-23 17:00:34 -07:00
sakridge c530fbd22b
Remove thread-priority crate which is not cross-platform (#9023) 2020-03-23 12:18:52 -07:00
Tyera Eulberg 1b8f9e75dd
Update getSignatureStatus: support multiple signatures, include slot in each response item (#9022)
* Rename enable-rpc-get-confirmed-block

* Rename RpcTransactionStatus -> RpcTransactionStatusMeta

* Return simplified RpcTransactionStatus; Add support for multiple transactions

* Update docs

* typo
2020-03-23 11:25:39 -06:00
sakridge 4b397d15b3
Accounts cleanup service and perf improvements (#8799)
* Use atomic for ref count instead of taking rwlock

* Accounts cleanup service

* Review comments
2020-03-23 08:50:23 -07:00
Justin Starry a0ffcc61ae
Add slot info to Bank::get_signature_confirmation_status (#9018) 2020-03-23 21:55:15 +08:00
Michael Vines 88ba8439fc
Add frozen account support (#8989)
automerge
2020-03-22 11:10:04 -07:00
sakridge 4dd0367136
Rwlock storage opt (#9006)
* Remove unecessary account paths rwlock

* Remove path rwlock in accounts_db and optimize storage critical section
2020-03-22 10:04:03 -07:00
sakridge 3ea556bc24
Drop storage lock (#8997) 2020-03-21 13:37:52 -07:00
Jack May e28368ff1b
Move address creation with seed into pubkey (#8991) 2020-03-20 15:20:48 -07:00
Michael Vines f78a90bce2
Vote InitializeAccount and UpdateNode instructions now need a signature from the validator identity (#8947)
automerge
2020-03-19 01:58:52 -07:00
Michael Vines 306a5c849e Use into_iter() 2020-03-18 16:11:57 -07:00
Michael Vines bb92184085 Refactor distribute_rent_to_validators() for clarity 2020-03-18 16:11:57 -07:00
sakridge 2435c3ce0c
Add accounts-bench, a benchmark to test the accounts store speed (#8866) 2020-03-17 11:02:07 -07:00
Michael Vines b7b36bb0a4
Upgrade to Rust 1.42 (#8836)
* Upgrade to Rust 1.42

* deref

* parens

Co-authored-by: Trent Nelson <trent@solana.com>
2020-03-13 14:15:22 -06:00
Trent Nelson fbf2dd1672
CLI: Error message cleanup (#8804)
automerge
2020-03-12 23:20:49 -07:00
Ryo Onodera 4bbf09f582
Enable conservative out-of-bound snapshot cleaning (#8811)
* Enable conservative out-of-bound snapshot cleaning

* Add tests
2020-03-13 14:44:00 +09:00
Ryo Onodera 952cd38b7b
Avoid early clean and bad snapshot by ref-counting (#8724)
* Avoid early clean and bad snapshot by ref-counting

* Add measure

* Clean ups

* clean ups
2020-03-13 14:14:37 +09:00
Jack May deaf3cb416
Instruction member function (#8801) 2020-03-12 09:08:39 -07:00
Jack May 6eb4973780
Don't use move semantics if not needed (#8793) 2020-03-11 14:37:23 -07:00
carllin 9872430bd2
Add VoteTracker for tracking cluster's votes in gossip (#8327)
Track votes by slot in cluster_vote_listener
2020-03-09 22:03:09 -07:00
Trent Nelson 4db074a5aa
RPC: Add `getFeeCalculatorForBlockhash` method call (#8687)
Returns the `FeeCalculator` associated with the given blockhash, or
`null` if said blockhash has expired
2020-03-06 17:01:31 -07:00
Jack May 8ad6a8767f
Simplify runtime account handling (#8674) 2020-03-05 16:17:31 -08:00
Grimes 7fad53b112
Use iterated account (#8673)
automerge
2020-03-05 13:10:20 -08:00
Grimes 9d667db634
SDK: Allow `RecentBlockhashes` to hold the entire `BlockhashQueue` (#8632)
automerge
2020-03-05 11:03:21 -08:00
Jack May 5e3ce30d02
Pass the correct program_id to programs (#8630) 2020-03-05 10:57:35 -08:00
Jack May 97c5fb8141
Allow passing of program_ids to programs (#8639) 2020-03-05 10:57:12 -08:00
Michael Vines f146c92e88
Always and fully normalize stored 0-lamport accts. (#8657) 2020-03-05 09:14:40 -07:00
Trent Nelson fd00e5cb35
Store FeeCalculator with blockhash in nonce accounts (#8650)
* Copy current state version to v0

* Add `FeeCalculator` to nonce state

* fixup compile

* Dump v0 handling...

Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...

* Create more representative test `RecentBlockhashes`

* Improve CLI nonce account display

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix that last bank test...

* clippy/fmt

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-03-05 07:40:26 -07:00
Grimes b321da00b4
Nit: Use accessor function (#8647)
automerge
2020-03-04 18:26:58 -08:00
Grimes 80aae18794
Nit: Update native loader to iterate accounts (#8640)
automerge
2020-03-04 17:10:22 -08:00
Jack May 408d5da50f
Add test for program_ids passed in metas (#8618) 2020-03-04 11:13:33 -08:00
Trent Nelson 561808cf90
SDK: Store FeeCalculator in recent_blockhashes sysvar (#8609)
* SDK: Store FeeCalculators in recent_blockhashes sysvar

* nits
2020-03-04 12:01:32 -07:00
Jack May b85d7c1f70
Fix account tests (#8615) 2020-03-04 10:40:41 -08:00
Jack May 642720a2fe
nit: describe the root program id (#8621) 2020-03-04 08:55:01 -08:00
Trent Nelson 1cc7131bb7
Consolidate Nonce state under one struct (#8624)
automerge
2020-03-04 08:51:48 -08:00
sakridge b084c1d437
Remove accounts hack and correctly restore accounts store counts (#8569)
* Remove accounts hack and correctly restore append-vec counts

* Add test
2020-03-03 20:48:55 -08:00
Trent Nelson 1cb6101c6a
SDK: Add versioning to nonce state (#8607) 2020-03-03 19:39:09 -07:00
Trent Nelson be0cc0273f
SDK: Re-org nonce state module to facilitate versioning (#8603)
automerge
2020-03-03 17:00:39 -08:00
sakridge 4f05f08f5d
Use fs::rename which is much faster than move_items (#8579) 2020-03-03 10:03:17 -08:00
Trent Nelson 0c76b89e55
Fix c/p error. We want a rent sysvar account here (#8559) 2020-03-03 09:49:02 -07:00
Ryo Onodera d86103383a
Do periodic inbound cleaning for rooted slots (#8436)
* Do periodic inbound compaction for rooted slots

* Add comment

* nits

* Consider not_compacted_roots in cleanup_dead_slot

* Renames in AccountsIndex

* Rename to reflect expansion of removed accounts

* Fix a comment

* rename

* Parallelize clean over AccountsIndex

* Some niceties

* Reduce locks and real chunked parallelism

* Measure each step for sampling opportunities

* Just noticed par iter is maybe lazy

* Replace storage scan with optimized index scan

* Various clean-ups

* Clear uncleared_roots even if no updates
2020-03-03 14:57:25 +09:00
Ryo Onodera 5d9130a3c4 Hack to skip cleanup_dead_slots upon snapshot load 2020-03-02 10:24:12 -07:00
Trent Nelson 90bedd7e06
Split signature throughput tracking out of `FeeCalculator` (#8447)
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`

Leaving `FeeCalculator` to *only* calculate transaction fees

* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate

* Expose recent `FeeRateGovernor` to clients

* Move `burn()` back into `FeeCalculator`

Appease BPF tests

* Revert "Move `burn()` back into `FeeCalculator`"

This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.

* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`

* Make `FeeRateGovernor`'s `lamports_per_signature` private

* rebase artifacts

* fmt

* Drop 'Recent'

* Drop _with_commitment variant

* Use a more portable integer for `target_signatures_per_slot`

* Add docs for `getReeRateCalculator` JSON RPC method

* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
2020-02-28 13:27:01 -07:00
Michael Vines 7d27be2a73 Upgrade to Rust 1.41.1 2020-02-28 10:10:42 -07:00
carllin d821fd29d6
Add versioning (#8348)
automerge
2020-02-25 17:12:01 -08:00
Ryo Onodera 0b48c8eb35
Promote dangerous cond. from just warning to panic (#8439) 2020-02-26 05:09:57 +09:00
Michael Vines 73063544bd Move shred_version module to sdk/ 2020-02-24 14:46:12 -07:00
sakridge 947a339714
Add snapshot hash of full accounts state (#8295)
* Add snapshot hash of full accounts state

* Use normal hashing for the accounts delta state

* Add merkle
2020-02-22 13:46:40 -08:00
Pankaj Garg aa80f69171
Promote some datapoints to `info` to fix dashboard (#8381)
automerge
2020-02-21 13:41:49 -08:00
Jack May 0ace22d03f
Optimize account verification (#8385) 2020-02-21 13:28:35 -08:00
Jack May 0e6aca5a7e
Reorganize message processor in prep for cross-program-invocation (#8338) 2020-02-21 11:30:00 -08:00
Ryo Onodera d238371b0c
Correct missing entry handling to avoid bad warns (#8339)
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360)
automerge
2020-02-20 13:28:55 -08:00
Greg Fitzgerald e8124324ff
Support transaction signing by heterogenous lists of keypairs (#8342)
automerge
2020-02-20 12:13:23 -08:00
carllin 73a278dc64
Factor out creating genesis with vote accounts into a utility function (#8315)
automerge
2020-02-18 02:39:47 -08:00
Ryo Onodera 027ec71aa9
Remove AccountInfo's (De)Serialize (#8313)
automerge
2020-02-17 03:07:36 -08:00
Michael Vines 1bf2285fa2 ledger-tool: Add print-accounts command 2020-02-14 19:59:48 -07:00
Jack May 940519ea5a
Remove Native Loader's use of GenericError (#8285) 2020-02-14 13:58:48 -08:00
Ryo Onodera 58727463e1
Remove needless last_root for better reclaims (#8148)
* Restore last_root to fix unintended storage delete

* Remove last_root thing altogether

* Remove unneeded test...
2020-02-13 08:19:53 +09:00
Greg Fitzgerald 127553ce4b
Wrap ed25519_dalek::Keypair (#8247) 2020-02-12 14:15:12 -07:00
sakridge a8028fbb93
Fix accounts_db store counts in purging accounts logic (#8218)
* Show insufficient purge_zero_lamport_account logic

* Add another pass to detect non-deleted values and increment the count

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-02-12 08:51:03 -08:00
Michael Vines 72b11081a4 Report validator rewards in getConfirmedBlock JSON RPC 2020-02-11 17:25:45 -07:00
Ryo Onodera 7614af2a45
Verify frozen bank from snapshot by hashing (#8184) 2020-02-11 16:46:33 +09:00
Michael Vines 84b3e12e1f Minor logging improvements 2020-02-10 10:42:42 -07:00
Ryo Onodera 485806c488 Just define BnakSlotDelta type alias (#8186)
automerge
2020-02-10 03:11:37 -08:00
Trent Nelson 5b070ad014
CLI: Support offline fee payers (#8009)
* CLI: Support offline fee-payer

* Add some knobs to test genesis/validator helpers

* Add tests
2020-02-07 09:14:26 -07:00
François Garillot f016c9a669
Maintenance : simplify a few patterns, remove unneeded dependencies (#8137)
* Simplify a few pattern matches

* Removing unneeded dependencies, upgrading internal version #s

 net-shaper: Removing log, semver, serde_derive
 bench-tps: Removing serde, serde_derive
 banking-bench: Removing solana
 ledger-tool: Removing bincode, serde, serde_derive
 librapay: Removing solana, language_e2e_tests
 log-analyzer: Removing log, semver, serde_derive
 exchange: Removing solana
 core: Removing crc, memmap, symlink, untrusted
 perf: Removing serde_derive
 genesis: Removing hex, serde_derive
 sdk-c: Removing sha2
 sys-tuner: Removing semver
 bench-exchange: Removing bincode, bs58, env_logger, serde, serde_derive, untrusted, ws
 btc_spv_bin: Removing serde_json
 btc_spv: Removing chrono
 bpf_loader: Removing serde
 ledger: Removing dlopen, dlopen_derive, serde_derive
 move_loader: Removing byteorder, libc, language_e2e_tests
 ownable: Removing serde, serde_derive
 client: Removing rand
 archiver-utils: Removing rand_chacha
 validator: Removing serde_json, tempfile
 param_passing_dep: Removing solana
 failure: Removing log
 vest: Removing log
 vote-signer: Removing bs58, serde
 local-cluster: Removing symlink
 keygen: Removing rpassword
 install: Removing bs58, log
 upload-perf: Removing log
 runtime: Removing serde_json
 stake: Removing rand

* Add modified Cargo.lock

* fixup! Simplify a few pattern matches

* fixup! Simplify a few pattern matches
2020-02-06 10:02:38 -07:00
Jack May 8436457e75
Rename program_utils.rs (#8127) 2020-02-05 12:48:30 -08:00
Michael Vines 3ac0192d40 Better surface bank hash verification failures 2020-02-05 11:39:47 -07:00
Jack May 15ab966ed1
Move native program entrypoint out of instruction_processor_utils (#8122) 2020-02-04 14:54:49 -08:00
sakridge 78f6ddc5b7
Fix spelling of verification in accounts_db (#8117)
automerge
2020-02-04 11:45:47 -08:00
Michael Vines 3bc9789e8d Remove support for 0.22.3 snapshots 2020-01-30 23:34:15 -07:00
Jack May 83718a3b3e
Cleanup runtime use syntax (#8002) 2020-01-28 17:03:20 -08:00
Michael Vines 912aafcefd Reduce epoch duration from 2 weeks to 2 days 2020-01-27 10:34:55 -07:00
Michael Vines 989355e885 Add ability to hard fork at any slot (#7801)
automerge
2020-01-24 17:27:04 -08:00
Ryo Onodera 2c7447b73e
Secure sysvars under hash by freezing all strictly (#7892)
* Secure sysvars under hash by freezing all strictly

* Fix hash's non-idempotnet and add new test

* Clean up

* More cleanups
2020-01-24 16:10:32 +09:00
Michael Vines 8b8033c72b Set BankRc slot correctly when restoring a bank snapshot 2020-01-23 13:37:13 -07:00
Michael Vines 25dfed207c Remove dead code (#7940)
automerge
2020-01-23 00:38:46 -08:00
Jack May c95e5346a4
Boot the mut (#7926) 2020-01-22 17:54:06 -08:00
Jack May 8f79327190 Test account doesn't need RefCell (#7932)
automerge
2020-01-22 17:06:11 -08:00
Trent Nelson 964ff522be
Verb-noun-ify Nonce API (#7925)
* Verb-noun-ify Nonce API

* Unify instruction naming with API naming

The more verbose nonce_account/NonceAccount was chosen for clarity
that these instructions work on a unique species of system account
2020-01-22 16:31:39 -07:00
Michael Vines 934c32cbc6 Add mechanism to load v0.22.3 snapshots on newer Solana versions 2020-01-22 15:40:32 -07:00
Rob Walker ce70d6eedc
Add redeem_vote_credits to runtime (#7910)
* Move redeem_vote_credits into runtime

* fixup

* test

* move stake manipulation to stake program

* chugga for less indentation
2020-01-22 12:21:31 -08:00
Jack May 023074650f
Allow the same account to be passed multiple times to a single instruction (#7795) 2020-01-22 09:11:56 -08:00
Greg Fitzgerald 3aabeb2b81
Rename bootstrap leader (#7906)
* Rename bootstrap leader to bootstrap validator

It's a normal validator as soon as other validators enter the
leader schedule.

* cargo fmt

* Fix build

Thanks @CriesofCarrots!
2020-01-22 09:22:09 -07:00
Rob Walker 470d9cd752
Add system_instruction::{allocate, allocate_with_seed, assign_with_seed}, (#7847)
* cleanup test checks cargo audit

* Add system_instruction allocate

* fixup

* fixup
2020-01-17 09:29:15 -08:00
Justin Starry 87598c7612 Consolidate tx error counters and update metrics dashboard (#7724)
automerge
2020-01-16 23:26:50 -08:00
Rob Walker 4c08184379
no check if no change (#7824) 2020-01-15 15:13:11 -08:00
Rob Walker a6d083d69d
Remove create_account bandaid now that `to`'s signature is required (#7776)
* Remove create account bandaid now that  requires signature

* shrink scope of this PR to bandaid
2020-01-15 13:03:22 -08:00
Ryo Onodera b16c30b4c6
Fix cluster collapse due to no proper shifted read (#7797)
* Fix cluster collapse due to no proper shifted read

* Add test for bank hash mismatch

Co-authored-by: sakridge <sakridge@gmail.com>
2020-01-15 11:45:19 +09:00
Justin Starry ff1ca1e0d3
Consolidate entry tick verification into one function (#7740)
* Consolidate entry tick verification into one function

* Mark bad slots as dead in blocktree processor

* more feedback

* Add bank.is_complete

* feedback
2020-01-15 09:15:26 +08:00
Jack May 5f4e0c7e3e Naming nits (#7798)
automerge
2020-01-14 13:38:17 -08:00
sakridge 3b78be83cf Add hash stats information to check hashes between validators (#7780)
automerge
2020-01-14 11:57:29 -08:00
Trent Nelson 81ae44f858 Nonce: Rename instructions with VerbNoun scheme (#7775)
automerge
2020-01-13 15:34:43 -08:00
Trent Nelson 9754fc789e
Manage durable nonce stored value in runtime (#7684)
* Bank: Return nonce pubkey/account from `check_tx_durable_nonce`

* Forward account with HashAgeKind::DurableNonce

* Add durable nonce helper for HashAgeKind

* Add nonce util for advancing stored nonce in runtime

* Advance nonce in runtime

* Store rolled back nonce account on TX InstructionError

* nonce: Add test for replayed InstErr fee theft
2020-01-10 16:57:31 -07:00
Jack May 08ba27627d
Direct entrypoint for execution (#7746) 2020-01-10 13:20:15 -08:00
Ryo Onodera 865c42465a
Cap file size for snapshot data files (#7182)
* save limit deserialize

* save

* Save

* Clean up

* rustfmt

* rustfmt

* Just comment out to please CI

* Fix ci...

* Move code

* Rustfmt

* Crean up control flow

* Add another comment

* Introduce predetermined constant limit on snapshot data files (deserialize side)

* Introduce predetermined constant limit on snapshot data files (serialize side)

* rustfmt

* Tweak message

* Revert dynamic memory limit

* Limit size of snapshot data file (de)serialization

* Fix test breakage

* Clean up

* Fix uses formatting

* Rename: deserialize_{for,from}_snapshot

* Simplify comment

* Use Slot

* Provide slot for status cache

* Align variable name with snapshot_status_cache_file_path

* Define serialize_snapshot_data_file_with_metrics

* Fix build.......

* De-marco serialize_snapshot_data_file_with_metrics

* Revert u64 => Slot
2020-01-10 09:49:36 +09:00