Commit Graph

1430 Commits

Author SHA1 Message Date
Jeff Washington (jwash) abd01553d5
tests: Keypair::new().pubkey() -> pubkey::new_rand (#27705) 2022-09-10 13:56:45 -07:00
Tao Zhu ced8f6a512
Add feature gate to remove support for RequestUnitsDeprecated instruction (#27503)
* feature gate: remove support for RequestUnitsDeprecated instruction #27500

* review update: stop support deprecated ix for prioritization

* Apply suggestions from code review

Co-authored-by: Justin Starry <justin.m.starry@gmail.com>

Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
2022-09-09 22:24:21 +00:00
Brooks Prumo fc396f965c
Fixup log of Epoch Accounts Hash when freezing bank (#27688) 2022-09-09 16:33:12 -04:00
Jeff Washington (jwash) d3ca364e9e
reuse shared function (#27642) 2022-09-07 13:11:00 -07:00
Jeff Washington (jwash) a31d4a597d
serialize epoch_accounts_hash (#27516) 2022-09-07 10:07:00 -07:00
Brooks Prumo 1cf9077d06
Include the Epoch Accounts Hash in the Bank hash (#27541) 2022-09-07 10:10:49 -04:00
Alexander Meißner 12d2147efa
Adds `IndexOfAccount` type (#27599)
Adds the type `IndexOfAccount`.
2022-09-06 11:31:40 +02:00
Alexander Meißner 0b94d5af18
Refactor: `InstructionContext::configure()` (#27400)
* Replaces InstructionContext::new() by InstructionContext::configure().
Adds TransactionContext::get_next_instruction_context().

* Switch back to using references as parameters.

* Hoists InstructionContext::configure() from TransactionContext::push() into InvokeContext::push().

* Moves InstructionContext::config() to the beginning of InvokeContext::push().

* Hoists InstructionContext::configure() from InvokeContext::push() into InvokeContext::process_instruction().

* Review feedback: Updates docu comments.
2022-09-03 10:34:57 +02:00
Jeff Washington (jwash) 3e40fad46b
deserialize(or eof) snapshot epoch_accounts_hash (#27513)
deserialize(or eof) snapshot prior_epoch_accounts_hash
2022-09-01 08:35:52 -07:00
Brooks Prumo 757e46c3c7
Set cap for new allocations per transaction (#27385) 2022-08-29 14:30:48 -04:00
Brooks Prumo 3c7cd62030
Move pruned_banks_receiver into PrunedBanksRequestHandler (#27445) 2022-08-29 13:30:06 -04:00
Michael Vines d0983c3cf7 Update runtime background hash verifier thread name 2022-08-26 13:44:11 -07:00
Brennan Watt 46a48760db
Switch concurrent replay from feature to param (#27401)
* Switch concurrent replay from feature to param
2022-08-26 12:36:02 -07:00
Alexander Meißner b2ae7def06
Remove `total_data_size` and `data_size_changed` from `ExecuteDetailsTimings` (#27051)
* Removes total_data_size and data_size_changed from ExecuteDetailsTimings.
2022-08-23 14:58:32 +02:00
Brennan Watt e4a7d01e10
Rust v1.63 (#27303)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

* Update quinn-udp crate

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-22 18:01:03 -07:00
Brooks Prumo 04eb5d414d
Remove unused `Bank::_clean_accounts()` (#27262) 2022-08-22 18:36:25 -04:00
Brooks Prumo 9daa41024d
Call AccountsDb::clean_accounts() directly, inside Bank::verify_snapshot_bank() (#27258) 2022-08-22 13:56:13 -04:00
HaoranYi 389bedda5e
Refactor epoch reward 4 (#27261)
* refactor: extract store_stake_accounts fn

* refactor: extract store_vote_account fn

* refactor: extract reward history update fn

* remove avg point value from pay_valiator fn. not used

* clippy: slice

* clippy: slice

* remove abort() from test-validator (#27124)

* chore: bump bytes from 1.1.0 to 1.2.1 (#27172)

* chore: bump bytes from 1.1.0 to 1.2.1

Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.1.0 to 1.2.1.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.1)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>

* Share Ancestors API get with contains_key (#27161)

consolidate similar fns

* Rename to `MAX_BLOCK_ACCOUNTS_DATA_SIZE_DELTA` (#27175)

* chore: bump libc from 0.2.129 to 0.2.131 (#27162)

* chore: bump libc from 0.2.129 to 0.2.131

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.129 to 0.2.131.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.129...0.2.131)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>

* reverts wide fanout in broadcast when the root node is down (#26359)

A change included in
https://github.com/solana-labs/solana/pull/20480
was that when the root node in turbine broadcast tree is down, the
leader will broadcast the shred to all nodes in the first layer.
The intention was to mitigate the impact of dead nodes on shreds
propagation, because if the root node is down, then the entire cluster
will miss out the shred.
On the other hand, if x% of stake is down, this will cause 200*x% + 1
packets/shreds ratio at the broadcast stage which might contribute to
line-rate saturation and packet drop.
To avoid this bandwidth saturation issue, this commit reverts that logic
and always broadcasts shreds from the leader only to the root node.
As before we rely on erasure codes to recover shreds lost due to staked
nodes being offline.

* add getTokenLargestAccounts rpc method to rust client (#26840)

* add get token largest accounts rpc call to client

* split to include with commitment

* Bump spl-token-2022 (#27181)

* Bump token-2022 to 0.4.3

* Allow cargo to bump stuff to v1.11.5

* VoteProgram.safeWithdraw function to safeguard against accidental vote account closures (#26586)

feat: safe withdraw function

Co-authored-by: aschonfeld <andrew@proofofalpha.io>

* chore: bump futures from 0.3.21 to 0.3.23 (#27182)

* chore: bump futures from 0.3.21 to 0.3.23

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.21 to 0.3.23.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.21...0.3.23)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>

* chore: bump nix from 0.24.2 to 0.25.0 (#27179)

* chore: bump nix from 0.24.2 to 0.25.0

Bumps [nix](https://github.com/nix-rust/nix) from 0.24.2 to 0.25.0.
- [Release notes](https://github.com/nix-rust/nix/releases)
- [Changelog](https://github.com/nix-rust/nix/blob/master/CHANGELOG.md)
- [Commits](https://github.com/nix-rust/nix/compare/v0.24.2...v0.25.0)

---
updated-dependencies:
- dependency-name: nix
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>

* Parse ConfidentialTransaction instructions (#26825)

Parse ConfidentialTransfer instructions

* snapshots: serialize version file first (#27192)

serialize version file first

* serialize incremental_snapshot_hash (#26839)

* serialize incremental_snapshot_hash

* pr feedback

* derives Error trait for ClusterInfoError and core::result::Error (#27208)

* Add clean_accounts_for_tests() (#27200)

* Rust v1.63.0 (#27148)

* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

Co-authored-by: Michael Vines <mvines@gmail.com>

* docs: updated "transaction fees" page (#26861)

* docs: transaction fees, compute units, compute budget

* docs: added messages definition

* Revert "docs: added messages definition"

This reverts commit 3c56156dfaaf17158c5eafbc5877080a83607a06.

* docs: added messages definition

* Update docs/src/transaction_fees.md

Co-authored-by: Jacob Creech <82475023+jacobcreech@users.noreply.github.com>

* fix: updates from feedback

Co-authored-by: Jacob Creech <82475023+jacobcreech@users.noreply.github.com>

* sdk: Fix args after "--" in build-bpf and test-bpf (#27221)

* Flaky Unit Test test_rpc_subscriptions (#27214)

Increase unit test timeout from 5 seconds to 10 seconds

* chore: only buildkite pipelines use sccache in docker-run.sh (#27204)

chore: only buildkite ci use sccache

* clean feature: `prevent_calling_precompiles_as_programs` (#27100)

* clean feature: prevent_calling_precompiles_as_programs

* fix tests

* fix test

* remove comment

* fix test

* feedback

* Add get_account_with_commitment to BenchTpsClient (#27176)

* Fix a corner-case panic in get_entries_in_data_block() (#27195)

#### Problem
get_entries_in_data_block() panics when there's inconsistency between
slot_meta and data_shred.

However, as we don't lock on reads, reading across multiple column families is
not atomic (especially for older slots) and thus does not guarantee consistency
as the background cleanup service could purge the slot in the middle.  Such
panic was reported in #26980 when the validator serves a high load of RPC calls.

#### Summary of Changes
This PR makes get_entries_in_data_block() panic only when the inconsistency
between slot-meta and data-shred happens on a slot older than lowest_cleanup_slot.

* Verify snapshot slot deltas (#26666)

* store-tool: log lamports for each account (#27168)

log lamports for each account

* add an assert for a debug feature to avoid wasted time (#27210)

* remove redundant call that bumps age to future (#27215)

* Use from_secs api to create duration (#27222)

use from_secs api to create duration

* reorder slot # in debug hash data path (#27217)

* create helper fn for clarity (#27216)

* Verifying snapshot bank must always specify the snapshot slot (#27234)

* Remove `Bank::ensure_no_storage_rewards_pool()` (#26468)

* cli: Add subcommands for address lookup tables (#27123)

* cli: Add subcommand for creating address lookup tables

* cli: Add additional subcommands for address lookup tables

* short commands

* adds hash domain to ping-pong protocol (#27193)

In order to maintain backward compatibility, for now the responding node
will hash the token both with and without domain so that the other node
will accept the response regardless of its upgrade status.
Once the cluster has upgraded to the new code, we will remove the legacy
domain = false case.

* Revert "Rust v1.63.0 (#27148)" (#27245)

This reverts commit a2e7bdf50a.

* correct double negation (#27240)

* Enable QUIC client by default. Add arg to disable QUIC client. (Forward port #26927) (#27194)

Enable QUIC client by default. Add arg to disable QUIC client.

* Enable QUIC client by default. Add arg to disable QUIC client.
* Deprecate --disable-quic-servers arg
* Add #[ignore] annotation to failing tests

* slots_connected: check if the range is connected (>= ending_slot) (#27152)

* create-snapshot check if snapshot slot exists (#27153)

* Add Bank::clean_accounts_for_tests() (#27209)

* Call `AccountsDb::shrink_all_slots()` directly (#27235)

* add ed25519_program to built-in instruction cost list (#27199)

* add ed25519_program to built-in instruction cost list

* Remove unnecessary and stale comment

* simple refactorings to disk idx (#27238)

* add _inclusive for clarity (#27239)

* eliminate unnecessary ZERO_RAW_LAMPORTS_SENTINEL (#27218)

* make test code more clear (#27260)

* banking stage: actually aggregate tracer packet stats (#27118)

* aggregated_tracer_packet_stats_option was alwasys None

* Actually accumulate tracer packet stats

* Refactor epoch reward 1 (#27253)

* refactor: extract store_stake_accounts fn

* clippy: slice

Co-authored-by: haoran <haoran@mbook>

* recovers merkle shreds from erasure codes (#27136)

The commit
* Identifies Merkle shreds when recovering from erasure codes and
  dispatches specialized code to reconstruct shreds.
* Coding shred headers are added to recovered erasure shards.
* Merkle tree is reconstructed for the erasure batch and added to
  recovered shreds.
* The common signature (for the root of Merkle tree) is attached to all
  recovered shreds.

* Simplify `Bank::clean_accounts()` by removing params (#27254)

* Account files remove (#26910)

* Create a new function cleanup_accounts_paths, a trivial change

* Remove account files asynchronously

* Update and simplify the implementation after the validator test runs.

* Fixes after testing on the dev device

* Discard tokio.  Use thread instead

* Fix comments format

* Fix config type to pass the github test

* Fix failed tests.  Handle the case of non-existing path

* Final cleanup, addressing the review comments
Avoided OsString.
Made the function more generic with "impl AsRef<Path>"

Co-authored-by: Jeff Washington <jeff.washington@solana.com>

* Refactor: Flattens `TransactionContext::instruction_trace` (#27109)

* Flattens TransactionContext::instruction_trace.

* Stop the search at transaction level.

* Renames get_instruction_context_at => get_instruction_context_at_nesting_level.

* Removes TransactionContext::get_instruction_trace().
Adds TransactionContext::get_instruction_trace_length() and TransactionContext::get_instruction_context_at_index().

* Have TransactionContext::instruction_accounts_lamport_sum() accept an iterator instead of a slice.

* Removes instruction_trace from ExecutionRecord.

* make InstructionContext::new() private

* Parallel insertion of dirty store keys during clean (#27058)

parallelize dirty store key insertion

* Refactor epoch reward 2 (#27257)

* refactor: extract store_stake_accounts fn

* refactor: extract store_vote_account fn

* clippy: slice

* clippy: slice

* fix merge error

Co-authored-by: haoran <haoran@mbook>

* Standardize thread names

Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case

* cleanup comment on filter_zero_lamport_clean_for_incremental_snapshots (#27273)

* remove inaccurate log (#27255)

* patches metrics for invalid cached vote/stake accounts (#27266)

patches invalid cached vote/stake accounts metrics

Invalid cached vote accounts is overcounting actual mismatches, and
invalid cached stake accounts is undercounting.

* Refactor epoch reward 3 (#27259)

* refactor: extract store_stake_accounts fn

* refactor: extract store_vote_account fn

* refactor: extract reward history update fn

* clippy: slice

* clippy: slice

Co-authored-by: haoran <haoran@mbook>

* fix merges

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: haoran <haoran@mbook>
Co-authored-by: Jeff Biseda <jbiseda@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Brooks Prumo <brooks@solana.com>
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: AJ Taylor <anthonjtaylor@gmail.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
Co-authored-by: Andrew Schonfeld <andrew.schonfeld1@gmail.com>
Co-authored-by: aschonfeld <andrew@proofofalpha.io>
Co-authored-by: apfitzge <apfitzge@users.noreply.github.com>
Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
Co-authored-by: Brennan Watt <brennan.watt@solana.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
Co-authored-by: Nick Frostbutter <75431177+nickfrosty@users.noreply.github.com>
Co-authored-by: Jacob Creech <82475023+jacobcreech@users.noreply.github.com>
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
Co-authored-by: Yihau Chen <a122092487@gmail.com>
Co-authored-by: Justin Starry <justin@solana.com>
Co-authored-by: kirill lykov <kirill.lykov@solana.com>
Co-authored-by: Yueh-Hsuan Chiang <93241502+yhchiang-sol@users.noreply.github.com>
Co-authored-by: leonardkulms <42893075+leonardkulms@users.noreply.github.com>
Co-authored-by: Will Hickey <will.hickey@solana.com>
Co-authored-by: Tao Zhu <82401714+taozhu-chicago@users.noreply.github.com>
Co-authored-by: Xiang Zhu <xzhu70@gmail.com>
Co-authored-by: Jeff Washington <jeff.washington@solana.com>
Co-authored-by: Alexander Meißner <AlexanderMeissner@gmx.net>
2022-08-22 08:19:50 -05:00
HaoranYi f617374b0b
Refactor epoch reward 3 (#27259)
* refactor: extract store_stake_accounts fn

* refactor: extract store_vote_account fn

* refactor: extract reward history update fn

* clippy: slice

* clippy: slice

Co-authored-by: haoran <haoran@mbook>
2022-08-21 16:20:51 -05:00
behzad nouri 544a957ad7
patches metrics for invalid cached vote/stake accounts (#27266)
patches invalid cached vote/stake accounts metrics

Invalid cached vote accounts is overcounting actual mismatches, and
invalid cached stake accounts is undercounting.
2022-08-20 19:10:22 +00:00
HaoranYi c17f15a34f
Refactor epoch reward 2 (#27257)
* refactor: extract store_stake_accounts fn

* refactor: extract store_vote_account fn

* clippy: slice

* clippy: slice

* fix merge error

Co-authored-by: haoran <haoran@mbook>
2022-08-20 09:37:02 -05:00
Alexander Meißner 55d18e8463
Refactor: Flattens `TransactionContext::instruction_trace` (#27109)
* Flattens TransactionContext::instruction_trace.

* Stop the search at transaction level.

* Renames get_instruction_context_at => get_instruction_context_at_nesting_level.

* Removes TransactionContext::get_instruction_trace().
Adds TransactionContext::get_instruction_trace_length() and TransactionContext::get_instruction_context_at_index().

* Have TransactionContext::instruction_accounts_lamport_sum() accept an iterator instead of a slice.

* Removes instruction_trace from ExecutionRecord.

* make InstructionContext::new() private
2022-08-20 11:20:47 +02:00
Brooks Prumo 510d195620
Simplify `Bank::clean_accounts()` by removing params (#27254) 2022-08-19 18:15:04 -04:00
HaoranYi a54ea4d62d
Refactor epoch reward 1 (#27253)
* refactor: extract store_stake_accounts fn

* clippy: slice

Co-authored-by: haoran <haoran@mbook>
2022-08-19 21:04:24 +00:00
Jeff Washington (jwash) 1b441f7a2d
make test code more clear (#27260) 2022-08-19 14:39:20 -05:00
Brooks Prumo b3c72bc8a2
Call `AccountsDb::shrink_all_slots()` directly (#27235) 2022-08-19 12:09:47 -04:00
Brooks Prumo 2184d0ff26
Add Bank::clean_accounts_for_tests() (#27209) 2022-08-19 12:09:35 -04:00
Brennan Watt 7573000d87
Revert "Rust v1.63.0 (#27148)" (#27245)
This reverts commit a2e7bdf50a.
2022-08-19 09:19:44 +01:00
Brooks Prumo 0b54b22f58
Remove `Bank::ensure_no_storage_rewards_pool()` (#26468) 2022-08-18 16:24:36 -04:00
Brooks Prumo fa4122f9a8
Verifying snapshot bank must always specify the snapshot slot (#27234) 2022-08-18 17:24:23 +00:00
HaoranYi 4634fb944c
Use from_secs api to create duration (#27222)
use from_secs api to create duration
2022-08-18 11:06:52 -05:00
Justin Starry 7d765e3d67
clean feature: `prevent_calling_precompiles_as_programs` (#27100)
* clean feature: prevent_calling_precompiles_as_programs

* fix tests

* fix test

* remove comment

* fix test

* feedback
2022-08-18 05:21:16 +00:00
Brennan Watt a2e7bdf50a
Rust v1.63.0 (#27148)
* Upgrade to Rust v1.63.0

* Add nightly_clippy_allows

* Resolve some new clippy nightly lints

* Increase QUIC packets completion timeout

Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-17 15:48:33 -07:00
Jeff Washington (jwash) 225cddcffb
serialize incremental_snapshot_hash (#26839)
* serialize incremental_snapshot_hash

* pr feedback
2022-08-17 15:14:31 -05:00
Justin Starry bdce208fe5
clean feature: `request_units_deprecated` (#27102)
clean feature: request_units_deprecated
2022-08-13 13:12:35 +01:00
Justin Starry 6c796727df
clean-feature: `default_units_per_instruction` (#27101)
clean-feature: default_units_per_instruction
2022-08-13 09:04:40 +01:00
Brooks Prumo 35c87c3888
Fix test_accounts_data_size_and_resize_transactions (#27105) 2022-08-12 17:06:54 +00:00
Justin Starry 5618e9fd07
Allow overriding the runtime transaction account lock limit (#26948)
* Add --transaction-account-lock-limit cli arg to test-validator

* Allow overriding the tx account lock limit in ProgramTest
2022-08-12 15:07:48 +01:00
Michael Vines ccfbc54195 Move vote program state and instructions to solana-program 2022-08-09 20:52:47 -07:00
Justin Starry 69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank (#26946)
* Refactor: Simplify arguments for bank constructor methods

* Refactor: Add RuntimeConfig to Bank fields

* Arc wrap runtime_config

* Arc wrap all runtime config usages

* Remove Copy trait derivation from RuntimeConfig

* Remove some arc wrapping
2022-08-05 20:49:00 +01:00
Brooks Prumo 06a610130e
Remove bank test_max_accounts_data_size_exceeded() (#26772)
bank: remove test_max_accounts_data_size_exceeded()
2022-08-05 09:55:29 -04:00
Tyera Eulberg 2dca239480
Remove runtime dependency from solana-transaction-status (#26930)
* Move RewardType out of runtime

* Move collect_token_balances to solana-ledger

* Remove solana-runtime dependency
2022-08-05 00:20:27 -06:00
Brennan Watt f24de9d254
Make tx error accounting more granular (#26718) 2022-08-02 16:43:03 -07:00
Jeff Washington (jwash) c7462b7a52
ledger tool verify can store debug info on hash calc (#26837) 2022-07-29 15:54:56 -05:00
Brennan Watt 467cb5def5
Concurrent slot replay (#26465)
* Concurrent replay slots

* Split out concurrent and single bank replay paths

* Sub function processing of replay results for readability

* Add feature switch for concurrent replay
2022-07-28 11:33:19 -07:00
Tao Zhu 5e622f82ac
Add feature gate to use default units per instruction for fee calculation (#26786)
* Add feature gate to use default units per instruction for fee calculation

* fix bpf tests
2022-07-26 19:39:24 +00:00
Brooks Prumo 01e367f19b
Add Bank::update_accounts_data_size_delta_off_chain_for_tests() (#26771) 2022-07-25 18:58:04 +00:00
Jeff Washington (jwash) d9c7bc7e78
Revert "cleanup feature: default units per instruction (#26684)" (#26750)
This reverts commit 39a34db52a.
2022-07-23 11:03:46 -05:00
Jack May 39a34db52a
cleanup feature: default units per instruction (#26684) 2022-07-20 19:13:34 +00:00
Justin Starry f8e9af5f1e
Cap the number of accounts passed to a bpf program ix (#26630)
* Cap the number of accounts passed to a bpf program ix

* update bank abi hash

* fix ci failures
2022-07-20 14:12:43 +02:00
Justin Starry fd6038884b Clean up forgotten error log 2022-07-17 12:21:52 +02:00
Jeff Washington (jwash) 47716a5e01
async hash verify on load (#26208)
* verify accounts hash in bg on startup

* fix some tests and loading from genesis

* add extra state for when background thread has completed
2022-07-15 14:29:56 -05:00
Alexander Meißner 038da82b6f
Feature: Early verification of account modifications in `BorrowedAccount` (#25899)
* Adjusts test cases for stricter requirements.

* Removes account reset in deserialization test.

* Removes verify related test cases.

* Replicates account modification verification logic of PreAccount in BorrowedAccount.

* Adds TransactionContext::account_touched_flags.

* Adds account modification verification to the BPF ABIv0 and ABIv1 deserialization, CPI syscall and program-test.

* Replicates the total sum of all lamports verification of PreAccounts in InstructionContext

* Check that the callers instruction balance is maintained during a call / push.

* Replicates PreAccount statistics in TransactionContext.

* Disable verify() and verify_and_update() if the feature enable_early_verification_of_account_modifications is enabled.

* Moves Option<Rent> of enable_early_verification_of_account_modifications into TransactionContext::new().

* Relaxes AccountDataMeter related test cases.

* Don't touch the account if nothing changes.

* Adds two tests to trigger InstructionError::UnbalancedInstruction.

Co-authored-by: Justin Starry <justin@solana.com>
2022-07-15 09:31:34 +02:00
Jeff Washington (jwash) 3e9bd6170c
use OnceCell instead of RwLock for rent_paying_accounts_by_partition (#26601) 2022-07-14 08:52:41 -05:00
Jeff Washington (jwash) 6b0eb5a42b
handle rent paying accounts differently (#26487)
* handle rent paying accounts differently

* restore collection

* downgrade assert to metric and warn until we get more runtime
2022-07-12 17:51:33 -05:00
Brooks Prumo 1dfbb225e3
Fix test_accounts_data_size_and_rent_collection (#26576) 2022-07-12 17:05:21 -05:00
Jeff Washington (jwash) f54f528148
make bank test creation standard (#26519) 2022-07-12 09:07:55 -05:00
Nicholas Clarke ee0a40937e
Add validator argument log_messages_bytes_limit to change log truncation limit.
Add new cli argument log_messages_bytes_limit to solana-validator to control how long program logs can be before truncation
2022-07-11 10:53:18 -05:00
behzad nouri ba785cf8ab
removes erroneous uses of std::mem::swap (#26536)
All instances should be replace by std::mem::{replace,take},
or just plain assignment.
2022-07-11 11:33:15 +00:00
Jeff Washington (jwash) 602da5e51f
add accounts db config to bank tests (#26517) 2022-07-10 19:42:06 -05:00
Jeff Washington (jwash) 1babf460ab
collapse some bank test creation fns (#26532) 2022-07-10 17:51:38 -05:00
behzad nouri 105c7e19e1
moves check for feature activation out of accounts loop (#26535) 2022-07-10 17:39:19 +00:00
Jeff Washington (jwash) 8576832f2a
rename Bank::new_with_config_for_tests (#26533) 2022-07-10 09:33:13 -05:00
Brooks Prumo 785a7a5936
Track more accounts data size changes (#26467) 2022-07-08 19:17:15 -05:00
behzad nouri c99d9f00a9
preserves rent_epoch for rent exempt accounts (#26479)
https://github.com/solana-labs/solana/pull/22292
prevents rent paying account creation going forward. As a result
rent_epoch field for rent exempt accounts is redundant, and advancing
this field will incur expensive account rewrites and cause discrepancy
between accounts-db and cached vote/stake accounts.

This commit adds a feature which upon activation preserves rent_epoch
field for rent exempt accounts so that the field is frozen and is no
longer advanced.
2022-07-08 20:04:08 +00:00
Brooks Prumo 78b8a8a6db
Remove `get_total_resize_remaining()` and `total_resize_limit` from `TransactionContext` (#26504) 2022-07-08 20:26:54 +02:00
Brooks Prumo 3b759bcffb
Add fn to get max accounts data size from Bank (#26478) 2022-07-07 18:08:42 -05:00
Jeff Washington (jwash) 8e64b5883e
restore population of 'accounts' metric in rent collection (#26454) 2022-07-07 13:19:23 -05:00
Justin Starry 8416090732
Clean up `enable_durable_nonce` feature (#26470)
Clean up enable_durable_nonce feature
2022-07-07 17:14:46 +01:00
Justin Starry eb6a722eba
Clean up `nonce_must_be_writable` feature (#26444)
Clean up nonce_must_be_writable feature
2022-07-07 15:45:19 +01:00
Brooks Prumo f8bdedb596
Make accounts data size tests more robust (#26466) 2022-07-07 00:01:13 -05:00
carllin 90ef2cd02a
Parse snapshot for bank fields (#26016) 2022-07-06 17:30:30 -05:00
behzad nouri d7201a8d1a
names fields in RentResullt::CollectRent enum variant (#26449)
Avoiding ambiguous raw tuple:
    CollectRent((Epoch, u64))

Using named fields instead:
    CollectRent {
        new_rent_epoch: Epoch,
        rent_due: u64,
    },
2022-07-06 20:01:16 +00:00
Justin Starry f4718be22f
Clean up `leave_nonce_on_success` feature (#26447)
Clean up leave_nonce_on_success feature
2022-07-06 17:59:34 +00:00
Justin Starry ff1e6fcbed
Clean up `nonce_must_be_advanceable` feature (#26446)
Clean up nonce_must_be_advanceable feature
2022-07-06 17:36:33 +00:00
Alexander Meißner 06ebfa1eb2
Replicates `AccountsDataMeter` in `TransactionContext` (#26438)
Replicates AccountsDataMeter in TransactionContext.
2022-07-06 19:27:42 +02:00
Justin Starry 611ac33718
Clean up `nonce_must_be_authorized` feature (#26445)
Clean up nonce_must_be_authorized feature
2022-07-06 17:17:31 +00:00
Jeff Washington (jwash) 17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… (#26441)
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936)"

This reverts commit e24cc537a4.
2022-07-06 11:32:45 -05:00
Justin Starry f8dccd4602
Clean up `max_tx_account_locks` feature (#26440)
Clean up max_tx_account_locks feature
2022-07-06 15:06:03 +00:00
behzad nouri 6f5857a5db
removes feature gate code separating durable nonce from blockhash domain (#26055) 2022-07-06 12:03:13 +00:00
carllin ce39c14025
Add end-to-end replay slot metrics (#25752) 2022-07-05 13:58:51 -05:00
Jeff Washington (jwash) 5f3b7bdd16
prevent ledger tool from calculating hash on non-rooted slots (#26355) 2022-07-02 11:50:01 -05:00
Jack May 4563bf40f6
cleanup feature: tx-wide-compute-cap (#26326) 2022-06-29 23:54:45 -07:00
behzad nouri d053ce79d4
removes RwLock+Once in favor of OnceCell in caching staked-nodes (#26313)
VoteAccounts uses std::sync::{RwLock, Once} to lazily compute and cache
staked_nodes:
https://github.com/solana-labs/solana/blob/032bee13a/runtime/src/vote_account.rs#L89-L104

This commit instead switches to using once_cell::sync::OnceCell which
provides this exact intended functionality by design.
2022-06-29 22:22:22 +00:00
Jeff Washington (jwash) 557bf6e656
allow initial hash calc to occur in bg (#26271)
* allow initial hash calc to occur in bg

* validator_initialized -> startup_verification_complete

* add infos for leader and vote

* rework snapshot for startup verification

* change to assert
2022-06-29 16:48:33 -05:00
Jeff Washington (jwash) 2deb396490
remove noisy and useless 'error!' and add test (#26138) 2022-06-29 16:26:23 -05:00
behzad nouri af7f08eba4
uses OnceCell instead of RwLock+Once to cache vote-state in vote-account (#26257)
RwLock seems excessive since only the very 1st call to VoteAccount::vote_state
will write-lock the inner field. Future calls would also incur overhead of an
RwLockReadGuard.
once_cell::sync::OnceCell provides a matching api to the desired functionality.
2022-06-29 11:45:53 +00:00
Jeff Washington (jwash) d8516b5c88
comment and expand on test (#26231) 2022-06-27 11:35:46 -05:00
Brooks Prumo 662818ef0d
Use `VoteAccount::node_pubkey()` (#26207) 2022-06-27 09:09:06 -05:00
Jeff Washington (jwash) cbefc581c0
test_rent_collect_rent_from_accounts (#26204) 2022-06-25 19:04:28 -05:00
Justin Starry 0ca963a869
Refactor: Add separate methods for retrieving stake info from VoteAccounts (#26224) 2022-06-25 20:03:15 +00:00
Justin Starry 44d1e62007
Refactor: No need to return stake in Bank::get_vote_account (#26220) 2022-06-25 16:27:43 +00:00
Jeff Washington (jwash) 18397527ad
retain prior metric behavior for invalid_cached_stake_accounts (#26181) 2022-06-24 14:12:16 -05:00
Jeff Washington (jwash) 054b25729d
eliminate vec creation in stake rewards (#26191) 2022-06-24 13:40:53 -05:00
HaoranYi fbcb4d8216
extract get_stake_reward fn from StakeReward (#26183) 2022-06-24 08:44:48 -05:00
Brooks Prumo 877fedadac
Remove StatusCacheRc type and use StatusCache directly (#26184) 2022-06-24 08:38:56 -05:00
Jeff Washington (jwash) a36abfda28
don't skip rewrite when account was written IN the current slot already (#26197) 2022-06-24 07:28:37 -05:00
Brooks Prumo 23c50a2389
Add StatusCache::root_slot_deltas() and use it (#26170) 2022-06-23 15:19:06 -05:00
Jeff Washington (jwash) 102fb98070
impl StorableAccounts for [StakeReward] (#26165) 2022-06-23 08:34:28 -05:00
Jeff Washington (jwash) 19b79aa764
add metrics for invalid_cached_stake_accounts_rent_epoch (#26143) 2022-06-23 08:33:24 -05:00
Jeff Washington (jwash) 9db44ce352
cleanup some AccountSharedData::from(Account (#26120) 2022-06-23 08:32:48 -05:00
Jeff Washington (jwash) aea75e82e3
Bank::store_accounts takes StorableAccounts (#26113) 2022-06-22 21:11:01 -05:00
Jeff Washington (jwash) a32523c43b
cleanup some tests (#26139) 2022-06-22 21:10:46 -05:00
HaoranYi c4efb9f19e
Stake cache size (#26071)
* reduce threadpool for stake processing

* wip

* Revert "reduce threadpool for stake processing"

This reverts commit 004a4f872ea7a3ef53e38d145b6350c3f57c680c.

* batch update stake_cache

* fix deadlock

* add test

* code review feedbacks

* more review feedbacks

* fix conflicts

* report stake account len and vote account len at epoch boundary

* report num_staked_nodes

* remove batch store, no atomic for redeem timing
2022-06-22 16:22:22 -05:00
Jeff Washington (jwash) c5c2b44ecc
adjust cached stake account compare for rent_epoch (#26135) 2022-06-22 15:11:25 -05:00
apfitzge f4189c0305
ledger-tool minimized snapshots (#25334)
* working on local snapshot

* Parallelization for slot storage minimization

* Additional clean-up and fixes

* make --minimize an option of create-snapshot

* remove now unnecessary function

* Parallelize parts of minimized account set generation

* clippy fixes

* Add rent collection accounts and voting node_pubkeys

* Simplify programdata_accounts generation

* Loop over storages to get slot set

* Parallelize minimized slot set generation

* Parallelize adding owners and programdata_accounts

* Remove some now unncessary checks on the blockstore

* Add a warning for minimized snapshots across epoch boundary

* Simplify ledger-tool minimize

* Clarify names of bank's minimization helper functions

* Remove unnecesary funciton, fix line spacing

* Use DashSets instead of HashSets for minimized account and slot sets

* Filter storages uses all threads instead of thread_pool

* Add some additional comments on functions for minimization

* Moved more into bank and parallelized

* Update programs/bpf/Cargo.lock for dashmap in ledger

* Clippy fix

* ledger-tool: convert minimize_bank_for_snapshot Measure into measure!

* bank.rs: convert minimize_bank_for_snapshot Measure into measure!

* accounts_db.rs: convert minimize_accounts_db Measure into measure!

* accounts_db.rs: add comment about use of minimize_accounts_db

* ledger-tool: CLI argument clarification

* minimization functions: make infos unique

* bank.rs: Add test_get_rent_collection_accounts_between_slots

* bank.rs: Add test_minimization_add_vote_accounts

* bank.rs: Add test_minimization_add_stake_accounts

* bank.rs: Add test_minimization_add_owner_accounts

* bank.rs: Add test_minimization_add_programdata_accounts

* accounts_db.rs: Add test_minimize_accounts_db

* bank.rs: Add negative case and comments in test_get_rent_collection_accounts_between_slots

* bank.rs: Negative test in test_minimization_add_programdata_accounts

* use new static runtime and sdk ids

* bank comments to doc comments

* Only need to insert the maximum slot a key is found in

* rename remove_pubkeys to purge_pubkeys

* add comment on builtins::get_pubkeys

* prevent excessive logging of removed dead slots

* don't need to remove slot from shrink slot candidates

* blockstore.rs: get_accounts_used_in_range shouldn't return Result

* blockstore.rs: get_accounts_used_in_range: parallelize slot loop

* report filtering progress on time instead of count

* parallelize loop over snapshot storages

* WIP: move some bank minimization functionality into a new class

* WIP: move some accounts_db minimization functionality into SnapshotMinimizer

* WIP: Use new SnapshotMinimizer

* SnapshotMinimizer: fix use statements

* remove bank and accounts_db minimization code, where possible

* measure! doesn't take a closure

* fix use statement in blockstore

* log_dead_slots does not need pub(crate)

* get_unique_accounts_from_storages does not need pub(crate)

* different way to get stake accounts/nodes

* fix tests

* move rent collection account functionality to snapshot minimizer

* move accounts_db minimize behavior to snapshot minimizer

* clean up

* Use bank reference instead of Arc. Additional comments

* Add a comment to blockstore function

* Additional clarifying comments

* Moved all non-transaction account accumulation into the SnapshotMinimizer.

* transaction_account_set does not need to be mutable now

* Add comment about load_to_collect_rent_eagerly

* Update log_dead_slots comment

* remove duplicate measure/print of get_minimized_slot_set
2022-06-22 13:17:43 -04:00
Jeff Washington (jwash) cf5ec13f59
Accounts::store_accounts_cached uses StorableAccounts (#26112) 2022-06-22 12:13:56 -05:00
Michael Vines f3639b76ce Remove some clippy lints 2022-06-22 09:23:22 -07:00
HaoranYi 5624ab0d4c
Optimize stake accounts store (#26004)
* reduce threadpool for stake processing

* wip

* Revert "reduce threadpool for stake processing"

This reverts commit 004a4f872ea7a3ef53e38d145b6350c3f57c680c.

* batch update stake_cache

* fix deadlock

* add test

* code review feedbacks

* more review feedbacks

* clippy

* update comments

* fix conflicts

* remove batch store, no atomic for redeem timing

* refactor stake_reward struct
2022-06-22 10:45:58 -05:00
Brooks Prumo 8a18e02f80
Call StatusCache::append() directly (#26125) 2022-06-22 07:42:01 -05:00
Brooks Prumo 97bd81a32b
Add bench for StatusCache::slot_deltas() (#26123) 2022-06-22 07:41:06 -05:00
Brooks Prumo d35c1a7d04
Use BankStatusCache in bank.rs (#26114) 2022-06-21 23:16:16 +00:00
Tyera Eulberg 752c851070
Clean up feature preventing new rent-paying accounts (#26000)
* Clean up require_rent_exempt_accounts feature

* Fix tests in bank_client, system_instruction_processor

* Fix tests in snapshot_utils

* Fix bank tests

* Ignore rent tests

* Fix stake-accounts tests
2022-06-16 15:35:25 -06:00
Alexander Meißner bf9ca9827e
Refactor: instruction account index (#25825)
* Adds methods based on instruction_account_index to InstructionContext.
Removes methods which are based on index_in_instruction.

* Adjusts program-runtime.

* Adjusts runtime.

* Adjusts bpf loader.

* Adjusts built-in programs.

* Adjusts program-test and bpf tests.
2022-06-16 18:46:17 +02:00
Brooks Prumo 54675b2dd6
Add bank tests for accounts_data_size and resize transactions / rent collection (#25884) 2022-06-15 19:40:43 -05:00
Jeff Washington (jwash) 631ea93259
parallel rent collection avoids overlapping ranges (#25991) 2022-06-15 16:04:38 -05:00
Tyera Eulberg 99cf1e280e
Clean up warp-timestamp features (#25994) 2022-06-15 14:11:41 -06:00
Alexander Meißner 4652377e5b
Removes the feature gate of reject_empty_instruction_without_program. (#25870) 2022-06-14 18:00:12 +02:00
Jeff Washington (jwash) e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936) 2022-06-14 10:10:44 -05:00
Jeff Washington (jwash) 3eff7648ff
collect rent from multiple partitions in parallel (#25774) 2022-06-13 07:53:49 -05:00
Brooks Prumo bea35d5fbe
Add ApplyFeatureActivationsCaller enum (#25858) 2022-06-10 10:15:13 -05:00
Brooks Prumo fedd3f26d0
Fix new cost tracker with accounts data size limit (#25843) 2022-06-10 07:39:43 -05:00
behzad nouri 3c1ce3cc93
permanently disables durable nonces with chain blockhash domain (#25788)
https://github.com/solana-labs/solana/pull/25744
separated durable nonce and blockhash domains, which will stop double
execution going forward. However it is possible that a durable
transaction has *already* been executed once as a normal transaction and
it is now a valid durable transaction. #25744 cannot stop such
transactions to be re-executed until the nonce accounts are advanced.

This commit adds a new nonce version indicating that the nonce is moved
out of the blockhash domain, and permanently disables durable
transactions for legacy nonces which are in the blockhash domain.
2022-06-09 15:28:37 +00:00
Justin Starry 6a7edc02c6
Reject durable nonce txs that don't use an advanceable nonce (#25832)
* Reject durable nonce txs that use the latest durable nonce hash

* feedback
2022-06-08 21:34:57 +00:00
Justin Starry b2b426d4bf
Reject durable nonce transactions not signed by authority (#25831) 2022-06-08 14:43:09 -05:00
Brooks Prumo d4e4871c47
Add tests to Bank for accounts data size (#25816) 2022-06-07 18:28:53 +00:00
apfitzge e6c21a3036
Convert Measure::this to measure! and remove Measure::this (#25776)
* Remove the args param from Measure::this since we don't ever use it

* banking_stage.rs: convert to measure!

* poh_recorder.rs: convert to measure!

* cost_update_service.rs: convert to measure!

* poh_service.rs: convert to measure!

* bank.rs: convert to measure!

* measure.rs: Remove Measure::this now that all have been converted to measure!
2022-06-06 20:21:05 -05:00
Ashwin Sekar 8caced68ce
Serialize lamports per signature (#25364)
* Serialize lamports per signature

* Add full snapshot archive test, enable features in previous tests
2022-06-06 13:07:02 -07:00
Brooks Prumo ec64d5261f
Avoid RMWs on shared data inside parallel loops: collect_rent_from_accounts() (#25790) 2022-06-06 13:32:22 -05:00
Justin Starry 61ad8fcc55
Fix bank executor stats and remove copy-on-write semantics (#25621)
* Fix bank executor stats and remove copy-on-write semantics

* Remove clone implementation for CachedExecutors

* feedback
2022-06-04 13:54:26 +00:00
behzad nouri 9851774133 adds feature gate enabling durable nonce
Previous commit separates durable nonce and blockhash domains with a
feature gate. A 2nd feature added in this commit enables durable nonce
at least one epoch after the 1st feature.
By the time 2nd feature is activated, some nonce accounts will have an
old blockhash, but no nonce account can have a recent blockhash.
As a result no transaction (durable or normal) can be executed twice.
2022-06-04 12:39:31 +00:00
behzad nouri 5ee157f43d separates durable nonce and blockhash domains
AdvanceNonceAccount instruction updates nonce to blockhash. This makes it
possible that a durable transaction is executed twice both as a normal
transaction and a nonce transaction if it uses blockhash (as opposed to nonce)
for its recent_blockhash field.

The commit prevents this double execution by separating nonce and blockhash
domains; when advancing nonce account, blockhash is hashed with a fixed string.
As a result a blockhash cannot be a valid nonce value; and if transaction was
once executed as a normal transaction it cannot be re-executed as a durable
transaction again and vice-versa.
2022-06-04 12:39:31 +00:00
Jeff Washington (jwash) 8b0d5b2485
parallelize rent collection (#25767) 2022-06-03 18:00:02 -05:00
Christian Kamm 0da0e0da63
Bank: Batch account stores in collect_rent_eagerly (#25707)
* Bank: Batch account stores in collect_rent_eagerly

Previously store() calls were done one-by-one, which leads to suboptimal
performance.

* Accounts: Remove store_slow_cached()

* clippy

Co-authored-by: Jeff Washington (jwash) <wash678@gmail.com>
2022-06-03 15:00:39 -05:00
Jeff Washington (jwash) 646bd2e488
extract prefix_from_pubkey (#25766) 2022-06-03 14:03:15 -05:00
Jeff Washington (jwash) a9a94166ef
load_account_into_read_cache (#25760) 2022-06-03 13:34:17 -05:00
Justin Starry 6fc057c1d9
Remove activated evict_invalid_stakes_cache_entries feature (#25739) 2022-06-02 21:04:54 +00:00
apfitzge ed68c0b889
epoch_stakes_reation -> epoch_stakes_creation in Bank::_new_from_parent (#25660) 2022-06-02 12:26:01 -05:00
Jeff Washington (jwash) 96f71c7d4d
add metrics on rent collection pieces (#25713) 2022-06-02 09:08:53 -05:00
apfitzge 369ff0858d
Helper functions for getting rent partitions between slots (#25669) 2022-06-01 20:20:44 -05:00
Jeff Washington (jwash) 891dcb2084
move rent collection time into datapoint (#25714) 2022-06-01 17:28:47 -05:00
Trent Nelson b80a82a9e5 runtime: gate nonce disable at restart slot 2022-06-01 12:00:10 -06:00
Trent Nelson 635bd40f09 runtime: disable durable nonces 2022-06-01 11:15:16 -06:00
Tyera Eulberg 8584a3a331
Add feature to increase timestamp bounding on the slow side to 150% of poh estimate (1s/slot) (#25666) 2022-05-31 16:53:06 -06:00
Christian Kamm 387d9d9c77 Store missing unupdated executors earlier 2022-05-28 13:42:35 +10:00
carllin 740ee0c502
Fix datapoint name (#25614) 2022-05-28 01:01:29 +00:00
Trent Nelson 4e6b276f6c
bump executor cache max entries (#25600)
* executor-cache: unify `remove` calls

* executor-cache: metrics - add one-hit-wonder counter

* executor-cache: bump max entries to 256
2022-05-28 01:54:10 +10:00
Brooks Prumo 7384c41ea7
Speed up bank::tests::test_max_accounts_data_size_exceeded() (#25523) 2022-05-25 22:22:30 -04:00
Brooks Prumo b7149f7f54
Add helper fn to set accounts_data_size_initial for tests (#25522) 2022-05-25 02:43:54 +00:00
Brooks Prumo 0dd3c6bf1f
Split up accounts data size field (#25495) 2022-05-24 20:26:32 -04:00
Justin Starry e66ea7cb6a Clean up Bank::commit_transactions parameters 2022-05-24 20:24:42 +08:00
Jeff Washington (jwash) 41f30a2383
stop logging misleading bank hash mismatch (#25427) 2022-05-23 08:43:25 -05:00
Michael Vines b05c7d91ed Fix derive_partial_eq_without_eq clippy lint 2022-05-22 22:22:21 -07:00
Alexander Meißner 7eae98c6d9
Introduces `Result` return types to mutable accessors of `BorrowedAccount` (#25380)
* Introduces result return types to get_data_mut(), set_data() and set_data_length() of BorrowedAccount.

* Introduces result return types to set_owner(), set_lamports() and set_executable() of BorrowedAccount.
2022-05-21 17:47:09 +02:00
Jack May 851958f77a
Disallow deployment of deprecated _sol_alloc_free syscall (#25179) 2022-05-20 13:19:41 -07:00
Jeff Washington (jwash) 9ef0acdf5d
allow hash calc to use all threads at startup (#25369) 2022-05-20 11:45:01 -05:00
Jeff Washington (jwash) 467431de89
add ledger-tool accounts-db-skip-initial-hash-calc (#25354) 2022-05-20 10:27:00 -05:00
Brooks Prumo 021276c955
Move updating accounts data len from execute to commit (#25256) 2022-05-19 12:24:13 -05:00
Justin Starry a1522d0024
Use consistent naming for compute unit limit (#25229)
* Use consistent naming for compute unit limit

* feedback
2022-05-18 13:14:31 +08:00
Jack May 21e066ef26
Merge pull request from GHSA-wxrh-ff9f-fc6w 2022-05-16 12:30:37 -07:00
Justin Starry f81c5df1f0
Include account index in rent paying account tx error (#25189) 2022-05-16 23:35:34 +08:00
Tao Zhu b1b3702e6d
Prioritize transactions in banking stage by their compute unit price (#25178)
* - get prioritization fee from compute_budget instruction;
- update compute_budget::process_instruction function to take instruction iter to support sanitized versioned message;
- updated runtime.md

* update transaction fee calculation for prioritization fee rate as lamports per 10K CUs

* review changes

* fix test

* fix a bpf test

* fix bpf test

* patch feedback

* fix clippy

* fix bpf test

* feedback

* rename prioritization fee rate to compute unit price

* feedback

Co-authored-by: Justin Starry <justin@solana.com>
2022-05-16 12:06:33 +08:00
Ryo Onodera 592aa4c21b
Improve hard fork message with more detail (#25214) 2022-05-15 01:14:23 +09:00
Jack May cde15ff687
support 64-bit prioritization fee (#25027) 2022-05-12 11:07:36 -07:00
Jon Cinque 542a14d9b7
test-validator: Always use overridden compute budget in bank (#25128) 2022-05-11 15:46:51 +02:00
Pankaj Garg c838e15234
Unset needs_unlock for rebatched transactions batches (#25095)
* Unset needs_unlock for rebatched transactions batches

* address review comments
2022-05-10 13:39:08 -07:00
Jeff Washington (jwash) b91cc87680
assert when rewrites skipped (#25110) 2022-05-10 13:55:48 -05:00
Justin Starry 082502d4f3
Fail tx sanitization when ix program id uses lookup table (#25035)
* Fail tx sanitization when ix program id uses lookup table

* feedback
2022-05-07 03:19:50 +08:00
Jeff Washington (jwash) df9a4a0c9d
Bank::new_for_benches uses bench config (#25017) 2022-05-06 13:35:18 -05:00
behzad nouri 492f89a170
checks account owner when initializing a vote-account (#25018)
A VoteAccount may only wrap an account if the account owner is
solana_vote_program:id or equivalently this check returns true:
solana_vote_program::check_id(account.owner())
2022-05-06 16:22:49 +00:00
Jeff Washington (jwash) 0cc97689f9
add stats for stakes_cache_check_and_store_us (#24349)
* add stats for stakes_cache_check_and_store_us

* remove redundant metrics
2022-05-05 17:22:48 -05:00
Jeff Washington (jwash) e0092902ea
maybe_update_rent_epoch_on_load uses correct EpochSchedule (#24999) 2022-05-05 13:34:03 -05:00
Jeff Washington (jwash) e8bf812a8a
allow ledger-tool halt at slot to calc hash using write cache (#24974) 2022-05-04 16:27:49 -05:00
apfitzge 0ca54db524
use bank's epoch_schedule accessor internally (#24918) 2022-05-04 13:52:48 -05:00
Jack May e070c5ca38
default compute units per instruction (#24899) 2022-05-03 09:50:06 -07:00
apfitzge 112a0b475a
Revert "Refactor to use EpochSchedule from within RentCollector struct" (#24893)
* Revert "Ran cargo fmt"

This reverts commit 9052e41b32.

* Revert "Fix build error introduced by my editor setup, part 2"

This reverts commit 4dfeab3b38.

* Revert "Fix build error introduced by my editor setup"

This reverts commit 87fb78dc56.

* Revert "Remove redundant epoch_schedule from AccountsPackage"

This reverts commit c2f7f2fff8.

* Revert "Fix a test"

This reverts commit 36c0bdaa78.

* Revert "Fixes to initial code"

This reverts commit ed7813e698.

* Revert "Removing redundant EpochSchedule param from fns"

This reverts commit 5472d2e605.
2022-05-02 13:46:17 -05:00
Jeff Washington (jwash) 546730cf42
retire deprecate_rewards_sysvar (#24651)
* retire deprecate_rewards_sysvar

* Update expected hash for bank after eliminating reqards sysvar

* Hack test_bank_update_vote_stake_rewards to pass after removing sysvar rewards account

* formatting change

* formatting cleanup

* Remove legacy validator Rewards code from unit test. Change expected inflation computation

* Update bank.rs

* Update bank.rs

* Clean up inflation abstraction function

* Formatting cleanup

* rename vars. Add struct for inflation rewards return

* format cleanup + comment

* Fix format warnings

Co-authored-by: Brennan Watt <brennan.watt@solana.com>
2022-05-02 10:10:51 -05:00
Brooks Prumo a73f99823d
Do not hold lock unnecessarily when hashing (#24815) 2022-04-29 08:40:28 +00:00
Jon Cinque b4503d4110
token: Add features for new program releases (#24742)
* token: Add features for new program releases

* Add PR number
2022-04-28 23:43:57 +02:00
Tyera Eulberg e189603d17
Make test less brittle (#24803) 2022-04-28 12:09:01 -06:00
Brooks Prumo 7b6880f652
Make test_bank_cloned_stake_delegations() aware of stake minimum delegation (#24753) 2022-04-28 09:35:07 -05:00
Jeff Washington (jwash) f22738fd7b
fix test_rent_eager_collect_rent_in_partition (#24722) 2022-04-26 23:06:57 -05:00
Ryo Onodera 412a5a0d33
Credits auto rewind on vote recreation (#22546)
* Credits auto rewind on vote recreation

* Update comment

* Improve comments and tests

* Recommended fn rename

* Restore old feature, and replace new feature key

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-04-26 15:49:35 -06:00
Steven Luscher 3007f233f4
Log subscription for `All` no longer clobbers `Some` subscription for pubkey (#24215)
* fix: subscribing for all logs no longer clobbers pubkey-specific subscriptions
* test: write tests to cover logsSubscribe
2022-04-26 10:31:11 -07:00
Jeff Washington (jwash) 534a666153
hookup 'just_rewrites' (#24662) 2022-04-26 12:14:25 -05:00
Justin Starry b6bc5d2fa1 Aggregate executor update timing metrics 2022-04-26 23:15:22 +08:00
behzad nouri be0bdd2261
explicitly ignores struct fields which do not impact PartialEq impl (#24624)
Unless struct fields are explicitly ignored in PartialEq implementation,
there are no compile-time checks that if more fields are added to the
struct, PartialEq implementation is accordingly updated.
2022-04-24 14:33:42 +00:00
carllin 8a062273de
Move error counters to be reported by leader only at end of slot (#24581)
* Add error counters to leader metrics only

* Add dependencies
2022-04-23 18:10:47 -05:00
Michael Vines 83e041299a Run real snapshot packager while processing blockstore at validator startup 2022-04-22 21:17:49 -07:00
Jon Cinque 0d51596224
sim: Override slot hashes account on simulation bank (#24543)
* sim: Override slot hashes during simulation

* Add simulation test program

* Address feedback

* Add AccountOverrides explicit type

* Cargo fmt
2022-04-22 12:32:31 +02:00
behzad nouri d524ae293f collects metrics for cached vote/stake accounts consistency
Tracking mismatches between cached vote/stake accounts and accounts-db
in preparation of activating feature for updating rewards at
epoch-boundary using cached vote/stake accounts.
2022-04-21 15:28:41 +00:00
behzad nouri 108aa23d90 enforces type-safety in Stakes<StakeAccount> using phantom data
StakeAccount<Delegation> can only wrap a stake-state which is a
Delegation; whereas StakeAccount<()> wraps any account with stake state.

As a result, StakeAccount::<Delegation>::delegation() will return
Delegation instead of Option<Delegation>.
2022-04-21 15:28:41 +00:00
behzad nouri f937fcbd95 updates rewards at epoch boundary using cached accounts
Loading vote and stake accounts from accounts-db takes a significant
portion of time updating rewards at epoch boundary.

This commit bypasses accounts-db and instead uses vote and stake
accounts cached in bank stakes:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/stakes.rs#L148-L152

These cached accounts are synchronized with accounts-db after each
transaction, and so there should not be any change in the resulting
computation:
https://github.com/solana-labs/solana/blob/d2702201c/runtime/src/bank.rs#L4526

Nevertheless, to avoid any chances of introducing a consensus issue, the
switch to cached account is feature gated.
2022-04-21 15:28:41 +00:00
behzad nouri b4491ff4ba adds StakesEnum type representing Stakes<StakeAccount|Delegation>
For backward compatibility, we can only serialize and deserialize
Stakes<Delegation>. However Bank caches Stakes<StakeAccount>. This type
mismatch incurs a conversion cost at epoch boundary when updating
EpochStakes.

This commit adds StakesEnum which allows EpochStakes to include either a
Stakes<StakeAccount> or Stakes<Delegation> and so bypass the conversion
cost between the two at the epoch boundary.
2022-04-21 15:28:41 +00:00
behzad nouri 454ef38e43 caches StakeAccount instead of Delegation in Stakes
The commit makes values in stake_delegations map in Stakes struct
generic. Stakes<Delegation> is equivalent to the old code and is used
for backward compatibility in BankFieldsTo{Serialize,Deserialize}.

But banks cache Stakes<StakeAccount> which includes the entire stake
account and StakeState deserialized from account. Doing so, will remove
the need to load stake account from accounts-db when working with
stake-delegations.
2022-04-21 15:28:41 +00:00
behzad nouri 8c4d6357fe adds StakeAccount type embodying an Account and a StakeState
The added type does sanity checks on the Account and stores deserialized
StakeState. Following commits will use this type instead of Delegation
in Stakes.
2022-04-21 15:28:41 +00:00
Justin Starry d5127abf46
Only add hashes for completed blocks to recent blockhashes (#24389)
* Only add hashes for completed blocks to recent blockhashes

* feedback
2022-04-21 21:05:29 +08:00
Tao Zhu a21fc3f303
Apply transaction actual execution units to cost_tracker (#24311)
* Pass the sum of consumed compute units to cost_tracker

* cost model tracks builtins and bpf programs separately, enabling adjust block cost by actual bpf programs execution costs

* Copied nightly-only experimental `checked_add_(un)signed` implementation to sdk

* Add function to update cost tracker with execution cost adjustment

* Review suggestion - using enum instead of struct for CommitTransactionDetails
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>

* review - rename variable to distinguish accumulated_consumed_units from individual compute_units_consumed

* not to use signed integer operations

* Review - using saturating_add_assign!(), and checked_*().unwrap_or()

* Review - using Ordering enum to cmp

* replace checked_ with saturating_

* review - remove unnecessary Option<>

* Review - add function to report number of non-zero units account to metrics
2022-04-21 07:38:07 +00:00
Justin Starry 79923c3b58
Refactor: Rename BlockhashQueue fields and methods for clarity (#24426) 2022-04-21 11:57:17 +08:00
Jeff Washington (jwash) 52f43a986d
skip rewrites in bank (#23985) 2022-04-20 14:08:46 -05:00