Commit Graph

776 Commits

Author SHA1 Message Date
Ryo Onodera 13bc3f8094
Fix unstable test after eager rent collection (#10031)
automerge
2020-05-13 10:35:58 -07:00
Ryo Onodera 1eb40c3fe0
Introduce eager rent collection (#9527)
* Switch AccountsIndex.account_maps from HashMap to BTreeMap

* Introduce eager rent collection

* Start to add tests

* Avoid too short eager rent collection cycles

* Add more tests

* Add more tests...

* Refacotr!!!!!!

* Refactoring follow up

* More tiny cleanups

* Don't rewrite 0-lamport accounts to be deterministic

* Refactor a bit

* Do hard fork, restore tests, and perf. mitigation

* Fix build...

* Refactor and add switch over for testnet (TdS)

* Use to_be_bytes

* cleanup

* More tiny cleanup

* Rebase cleanup

* Set Bank::genesis_hash when resuming from snapshot

* Reorder fns and clean ups

* Better naming and commenting

* Yet more naming clarifications

* Make prefix width strictly uniform for 2-base partition_count

* Fix typo...

* Revert cluster-dependent gate

* kick ci?

* kick ci?

* kick ci?
2020-05-13 16:22:14 +09:00
Justin Starry 5cc252d471
Remove hash field from account (#9915) 2020-05-12 23:39:46 +08:00
Jack May 97e17f9b32 Programs can only sign their accounts 2020-05-11 09:06:05 -07:00
Tyera Eulberg 3ee702a922
Rpc: Add getCirculatingSupply endpoint, redux (#9953)
* Add Bank.clock() helper

* Add non-circulating calculations

* Plumb getSupply rpc endpoint

* Add docs for getSupply, and remove getTotalSupply from docs

* Add pubkeys! procedural macro

* Use procedural macro in non_circulating_supply
2020-05-09 12:05:29 -06:00
Ryo Onodera 00e45ec935
Maintain sysvar balances for consistent market cap. (#9936)
* Maintain sysvar balances for consistent market cap.

* Unindent
2020-05-09 02:42:32 +09:00
Ryo Onodera 2ac50177a6
Include account.owner into account hash (#9917)
automerge
2020-05-07 13:01:11 -07:00
Tyera Eulberg 754c65c066
Refactor RPC subscriptions account handling (#9888)
* Switch subscriptions to use commitment instead of confirmations

* Add bank method to return account and last-modified slot

* Add last_modified_slot to subscription data and use to filter account subscriptions

* Update tests to non-zero last_notified_slot

* Add accounts subscriptions to test; fails at higher tx load

* Pass BankForks to RpcSubscriptions

* Use BankForks on add_account_subscription to properly initialize last_notified_slot

* Bundle subscriptions

* Check for non-equality

* Use commitment to initialize last_notified_slot; revert context.slot chage
2020-05-07 00:23:06 -06:00
carllin 3442f36f8a
Repair alternate versions of dead slots (#9805)
Co-authored-by: Carl <carl@solana.com>
2020-05-05 14:07:21 -07:00
Tyera Eulberg f5b0d13f08
Rpc: add getLargestAccounts endpoint (#9869)
automerge
2020-05-04 16:46:10 -07:00
sakridge f37f83fd12
Fuzzer test and fixes (#9853) 2020-05-02 08:07:52 -07:00
sakridge 894549f002
Put empty accounts in the accounts list on load (#9840)
Indexing into accounts array does not match account_keys otherwise.
Also enforce program accounts not at index 0
Enforce at least 1 Read-write signing fee-payer account.
2020-05-01 17:23:33 -07:00
Michael Vines 8dfe0affd4
Add incinerator sysvar (#9815) 2020-04-30 22:04:08 -07:00
Michael Vines 230df0ec0c
Upgrade to Rust 1.43.0 (#9754) 2020-04-29 18:02:05 -07:00
Jack May 068f12fd6f
Add Cross-program invocations (#9582) 2020-04-28 14:33:56 -07:00
sakridge aac580686f
fix test compilation from add_static_program change (#9755) 2020-04-27 23:37:08 -07:00
Jack May efad193180
Make default programs static (#9717) 2020-04-27 21:05:12 -07:00
anatoly yakovenko 8ef097bf6f
Input values are not sanitized after they are deserialized, making it far too easy for Leo to earn SOL (#9706)
* sanitize gossip protocol messages
* sanitize transactions
* crds protocol sanitize
2020-04-27 11:06:00 -07:00
Ryo Onodera 9918539229
Introduce type alias Ancestors (#9699)
* Introduce type alias AncestorList

* Rename AncestorList => Ancestors
2020-04-27 11:07:03 +09:00
sakridge e907c0e650
Filter program ids to store (#9721)
automerge
2020-04-26 00:11:37 -07:00
Greg Fitzgerald 76b1c2baf0
One less alloc per transaction (#9705)
* One less alloc per transaction

* Fix benches

* Fix compiler warnings in bench build

* Fix move build

* Fix bench
2020-04-24 13:03:46 -06:00
Jack May b1a0abc7a6
Bump libloading to v0.6.1 (#9615)
automerge
2020-04-20 16:46:06 -07:00
Jack May d08517db8c
Nit picks (#9580) 2020-04-18 22:39:08 -07:00
Jack May 58887c591b
Add and update tests (#9566) 2020-04-18 17:04:13 -07:00
Jack May 241a05fc52
Add native loader entry points (#9486) 2020-04-15 09:41:29 -07:00
sakridge 217828a849
Reduce accounts_db prints (#9469) 2020-04-15 09:05:54 -07:00
Ryo Onodera c77ed82caa
Use same max_age regardless of leader/not-leader (#9423)
automerge
2020-04-14 00:34:41 -07:00
sakridge 99655206c8
Calculate account refs fix (#9447) 2020-04-11 12:52:10 -07:00
Jack May ad0482be73
Revert "Add native loader entry points (#9275)" Breaks genesis_config abi (#9377)
This reverts commit ed86d8d1fc.
2020-04-08 14:36:18 -07:00
carllin 4522e85ac4
Add Metrics/Dashboards tracking block production (#9342)
* Add metric tracking blocks/dropped blocks

Co-authored-by: Carl <carl@solana.com>
2020-04-08 14:35:24 -07:00
Ryo Onodera b28ec430e4
Introduce background stale AppendVec shrink mechanism (#9219)
* Introduce background AppendVec shrink mechanism

* Support ledger tool

* Clean up

* save

* save

* Fix CI

* More clean up

* Add tests

* Clean up yet more

* Use account.hash...

* Fix typo....

* Add comment

* Rename accounts_cleanup_service
2020-04-06 17:30:23 +09:00
Jack May ed86d8d1fc
Add native loader entry points (#9275) 2020-04-03 17:40:59 -07:00
Jack May 268e04cb4a
Rename CustomError to Custom (#9207) 2020-04-01 09:01:11 -07:00
Jack May 130c0b484d
Enforce an executable's rent exemption in the runtime (#9134) 2020-03-31 10:07:38 -07:00
Tyera Eulberg 50fa577af8
Use cluster confirmations in rpc and pubsub (#9138)
* Add runtime methods to simply get status and slot

* Add helper function to get slot confirmation_count from BlockCommitmentCache

* Return cluster confirmations in getSignatureStatus

* Remove use of invalid get_signature_confirmation_status

* Remove unused methods

* Update pubsub to use cluster confirmations

* Fix test_check_signature_subscribe failure

* Refactor confirmations to read commitment cache only once

* Review comments

* Use bank, root from BlockCommitmentCache

* Update docs

* Add metric for block-commitment aggregations

Co-authored-by: Justin Starry <justin@solana.com>
2020-03-30 17:53:25 -06:00
sakridge b1771b92ec
Calculate ref counts earlier to prevent bad clean (#9147) 2020-03-29 14:42:34 -07:00
Ryo Onodera 729cc4e04f
Sanitize zero lamport accounts in append vecs (#9083) 2020-03-29 15:45:45 +09:00
Jack May e2491c6322
Prevent add/subtract from executable account (#9132) 2020-03-27 16:43:25 -07:00
Justin Starry 5d9298543f
Exclude all executable accounts from rent collection (#9116)
* Whitelist executable accounts for rent exemption

* nudge
2020-03-27 23:28:18 +08:00
carllin f3d556e3f9
Refactor VoteTracker (#9084)
* Refactor VoteTracker

Co-authored-by: Carl <carl@solana.com>
2020-03-26 17:55:17 -07:00
Jack May 8d4cecdb77
Account data may not change once the executable bit is set (#9099)
automerge
2020-03-26 17:10:11 -07:00
Ryo Onodera 2623c71ed3
Use type aliases/resulting var names consistently (#9060) 2020-03-26 13:08:56 +09:00
Justin Starry ef3af104ae
Return appropriate error for invalid program account (#9047)
automerge
2020-03-25 13:23:05 -07:00
carllin 9dc69d9843
Store and compute node/stake state in EpochStakes struct (#8958)
* Store and compute needed bank state in EpochStakes struct
2020-03-25 12:19:15 -07:00
sakridge 45348b2c83
Remove accounts unwrap (#9062)
automerge
2020-03-25 10:21:30 -07:00
Justin Starry eed8087d87
Respect confirmations param for signature subscription notifications (#9019)
automerge
2020-03-23 17:00:34 -07:00
sakridge c530fbd22b
Remove thread-priority crate which is not cross-platform (#9023) 2020-03-23 12:18:52 -07:00
Tyera Eulberg 1b8f9e75dd
Update getSignatureStatus: support multiple signatures, include slot in each response item (#9022)
* Rename enable-rpc-get-confirmed-block

* Rename RpcTransactionStatus -> RpcTransactionStatusMeta

* Return simplified RpcTransactionStatus; Add support for multiple transactions

* Update docs

* typo
2020-03-23 11:25:39 -06:00
sakridge 4b397d15b3
Accounts cleanup service and perf improvements (#8799)
* Use atomic for ref count instead of taking rwlock

* Accounts cleanup service

* Review comments
2020-03-23 08:50:23 -07:00
Justin Starry a0ffcc61ae
Add slot info to Bank::get_signature_confirmation_status (#9018) 2020-03-23 21:55:15 +08:00
Michael Vines 88ba8439fc
Add frozen account support (#8989)
automerge
2020-03-22 11:10:04 -07:00
sakridge 4dd0367136
Rwlock storage opt (#9006)
* Remove unecessary account paths rwlock

* Remove path rwlock in accounts_db and optimize storage critical section
2020-03-22 10:04:03 -07:00
sakridge 3ea556bc24
Drop storage lock (#8997) 2020-03-21 13:37:52 -07:00
Jack May e28368ff1b
Move address creation with seed into pubkey (#8991) 2020-03-20 15:20:48 -07:00
Michael Vines f78a90bce2
Vote InitializeAccount and UpdateNode instructions now need a signature from the validator identity (#8947)
automerge
2020-03-19 01:58:52 -07:00
Michael Vines 306a5c849e Use into_iter() 2020-03-18 16:11:57 -07:00
Michael Vines bb92184085 Refactor distribute_rent_to_validators() for clarity 2020-03-18 16:11:57 -07:00
sakridge 2435c3ce0c
Add accounts-bench, a benchmark to test the accounts store speed (#8866) 2020-03-17 11:02:07 -07:00
Michael Vines b7b36bb0a4
Upgrade to Rust 1.42 (#8836)
* Upgrade to Rust 1.42

* deref

* parens

Co-authored-by: Trent Nelson <trent@solana.com>
2020-03-13 14:15:22 -06:00
Trent Nelson fbf2dd1672
CLI: Error message cleanup (#8804)
automerge
2020-03-12 23:20:49 -07:00
Ryo Onodera 4bbf09f582
Enable conservative out-of-bound snapshot cleaning (#8811)
* Enable conservative out-of-bound snapshot cleaning

* Add tests
2020-03-13 14:44:00 +09:00
Ryo Onodera 952cd38b7b
Avoid early clean and bad snapshot by ref-counting (#8724)
* Avoid early clean and bad snapshot by ref-counting

* Add measure

* Clean ups

* clean ups
2020-03-13 14:14:37 +09:00
Jack May deaf3cb416
Instruction member function (#8801) 2020-03-12 09:08:39 -07:00
Jack May 6eb4973780
Don't use move semantics if not needed (#8793) 2020-03-11 14:37:23 -07:00
carllin 9872430bd2
Add VoteTracker for tracking cluster's votes in gossip (#8327)
Track votes by slot in cluster_vote_listener
2020-03-09 22:03:09 -07:00
Trent Nelson 4db074a5aa
RPC: Add `getFeeCalculatorForBlockhash` method call (#8687)
Returns the `FeeCalculator` associated with the given blockhash, or
`null` if said blockhash has expired
2020-03-06 17:01:31 -07:00
Jack May 8ad6a8767f
Simplify runtime account handling (#8674) 2020-03-05 16:17:31 -08:00
Grimes 7fad53b112
Use iterated account (#8673)
automerge
2020-03-05 13:10:20 -08:00
Grimes 9d667db634
SDK: Allow `RecentBlockhashes` to hold the entire `BlockhashQueue` (#8632)
automerge
2020-03-05 11:03:21 -08:00
Jack May 5e3ce30d02
Pass the correct program_id to programs (#8630) 2020-03-05 10:57:35 -08:00
Jack May 97c5fb8141
Allow passing of program_ids to programs (#8639) 2020-03-05 10:57:12 -08:00
Michael Vines f146c92e88
Always and fully normalize stored 0-lamport accts. (#8657) 2020-03-05 09:14:40 -07:00
Trent Nelson fd00e5cb35
Store FeeCalculator with blockhash in nonce accounts (#8650)
* Copy current state version to v0

* Add `FeeCalculator` to nonce state

* fixup compile

* Dump v0 handling...

Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...

* Create more representative test `RecentBlockhashes`

* Improve CLI nonce account display

Co-Authored-By: Michael Vines <mvines@gmail.com>

* Fix that last bank test...

* clippy/fmt

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-03-05 07:40:26 -07:00
Grimes b321da00b4
Nit: Use accessor function (#8647)
automerge
2020-03-04 18:26:58 -08:00
Grimes 80aae18794
Nit: Update native loader to iterate accounts (#8640)
automerge
2020-03-04 17:10:22 -08:00
Jack May 408d5da50f
Add test for program_ids passed in metas (#8618) 2020-03-04 11:13:33 -08:00
Trent Nelson 561808cf90
SDK: Store FeeCalculator in recent_blockhashes sysvar (#8609)
* SDK: Store FeeCalculators in recent_blockhashes sysvar

* nits
2020-03-04 12:01:32 -07:00
Jack May b85d7c1f70
Fix account tests (#8615) 2020-03-04 10:40:41 -08:00
Jack May 642720a2fe
nit: describe the root program id (#8621) 2020-03-04 08:55:01 -08:00
Trent Nelson 1cc7131bb7
Consolidate Nonce state under one struct (#8624)
automerge
2020-03-04 08:51:48 -08:00
sakridge b084c1d437
Remove accounts hack and correctly restore accounts store counts (#8569)
* Remove accounts hack and correctly restore append-vec counts

* Add test
2020-03-03 20:48:55 -08:00
Trent Nelson 1cb6101c6a
SDK: Add versioning to nonce state (#8607) 2020-03-03 19:39:09 -07:00
Trent Nelson be0cc0273f
SDK: Re-org nonce state module to facilitate versioning (#8603)
automerge
2020-03-03 17:00:39 -08:00
sakridge 4f05f08f5d
Use fs::rename which is much faster than move_items (#8579) 2020-03-03 10:03:17 -08:00
Trent Nelson 0c76b89e55
Fix c/p error. We want a rent sysvar account here (#8559) 2020-03-03 09:49:02 -07:00
Ryo Onodera d86103383a
Do periodic inbound cleaning for rooted slots (#8436)
* Do periodic inbound compaction for rooted slots

* Add comment

* nits

* Consider not_compacted_roots in cleanup_dead_slot

* Renames in AccountsIndex

* Rename to reflect expansion of removed accounts

* Fix a comment

* rename

* Parallelize clean over AccountsIndex

* Some niceties

* Reduce locks and real chunked parallelism

* Measure each step for sampling opportunities

* Just noticed par iter is maybe lazy

* Replace storage scan with optimized index scan

* Various clean-ups

* Clear uncleared_roots even if no updates
2020-03-03 14:57:25 +09:00
Ryo Onodera 5d9130a3c4 Hack to skip cleanup_dead_slots upon snapshot load 2020-03-02 10:24:12 -07:00
Trent Nelson 90bedd7e06
Split signature throughput tracking out of `FeeCalculator` (#8447)
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`

Leaving `FeeCalculator` to *only* calculate transaction fees

* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate

* Expose recent `FeeRateGovernor` to clients

* Move `burn()` back into `FeeCalculator`

Appease BPF tests

* Revert "Move `burn()` back into `FeeCalculator`"

This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.

* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`

* Make `FeeRateGovernor`'s `lamports_per_signature` private

* rebase artifacts

* fmt

* Drop 'Recent'

* Drop _with_commitment variant

* Use a more portable integer for `target_signatures_per_slot`

* Add docs for `getReeRateCalculator` JSON RPC method

* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
2020-02-28 13:27:01 -07:00
Michael Vines 7d27be2a73 Upgrade to Rust 1.41.1 2020-02-28 10:10:42 -07:00
carllin d821fd29d6
Add versioning (#8348)
automerge
2020-02-25 17:12:01 -08:00
Ryo Onodera 0b48c8eb35
Promote dangerous cond. from just warning to panic (#8439) 2020-02-26 05:09:57 +09:00
Michael Vines 73063544bd Move shred_version module to sdk/ 2020-02-24 14:46:12 -07:00
sakridge 947a339714
Add snapshot hash of full accounts state (#8295)
* Add snapshot hash of full accounts state

* Use normal hashing for the accounts delta state

* Add merkle
2020-02-22 13:46:40 -08:00
Pankaj Garg aa80f69171
Promote some datapoints to `info` to fix dashboard (#8381)
automerge
2020-02-21 13:41:49 -08:00
Jack May 0ace22d03f
Optimize account verification (#8385) 2020-02-21 13:28:35 -08:00
Jack May 0e6aca5a7e
Reorganize message processor in prep for cross-program-invocation (#8338) 2020-02-21 11:30:00 -08:00
Ryo Onodera d238371b0c
Correct missing entry handling to avoid bad warns (#8339)
* Correct missing entry handling to avoid bad warns

* Pass storage entries to AccountStorageSerialize

* Fix CI.....

* Add tests and reorder condition for cheapest first

* Remove unneeded reference
2020-02-21 15:27:55 +09:00
Tyera Eulberg ab361a8073
Rename KeypairUtil to Signer (#8360)
automerge
2020-02-20 13:28:55 -08:00
Greg Fitzgerald e8124324ff
Support transaction signing by heterogenous lists of keypairs (#8342)
automerge
2020-02-20 12:13:23 -08:00
carllin 73a278dc64
Factor out creating genesis with vote accounts into a utility function (#8315)
automerge
2020-02-18 02:39:47 -08:00