* Adds a CLI option to the validator to enable just-in-time compilation of BPF.
* Refactoring to use bpf_loader_program instead of feature_set to pass JIT flag from the validator CLI to the executor.
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.
This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.
Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
* runtime: Add `FeeCalculator` resolution method to `HashAgeKind`
* runtime: Plumb fee-collected accounts for failed nonce tx rollback
* runtime: Use fee-collected nonce/fee account for nonced TX error rollback
* runtime: Add test for failed nonced TX accounts rollback
* Fee payer test
* fixup: replace nonce account when it pays the fee
* fixup: nonce fee-payer collect test
* fixup: fixup: clippy/fmt for replace...
* runtime: Test for `HashAgeKind::fee_calculator()`
* Clippy
Co-authored-by: Trent Nelson <trent@solana.com>
* Add new inflation feature-ids, and full_inflation default values
* Compute inflation start from full_inflation activation
* Include pico_inflation in inflation start computation
* Add full-inflation constructor
* Align inflation taper with rewards accrual start and catch overflow edge case
* Fix fragile tests in prep of stake rewrite pr
* Restore BOOTSTRAP_VALIDATOR_LAMPORTS where appropriate
* Further clean up
* Further clean up
* Aligh with other call site change
* Remove false warn!
* fix ci!
* Fix slow/stuck unstaking due to toggling in epoch
* nits
* nits
* Add stake_program_v2 feature status check to cli
Co-authored-by: Tyera Eulberg <tyera@solana.com>
* Test for different ancestors with mismatch bank hash
* Test cleanup
* Remove nondeterministic ancestor check
* Update timestamp bounding feature key
* Update design doc
* Filter recent_timestamps to nodes voting within the last epoch
Co-authored-by: Stephen Akridge <sakridge@gmail.com>
* Output more inflation calc details in ledger-tool
* Fix broken ci...
* Rename confusing variables
* Fix panic by wrapping PointValue with Opiton...
* Minor modifications
* Remove explict needless flush; Drop already does
* Yet another csv field adjustments
* Add data_size and rename epochs to earned_epochs
* Introduce null_tracer
* Unwrap Option in new_from_parent_with_tracer
* Don't shorten identifiers
* Allow irrefutable_let_patterns temporalily
* More null_tracer
* More field adjustments
* Re-use accounts_db stores
Creating files and dropping mmap areas can be expensive
* Add test for storage finder
Can encounter an infinite loop when the store is too small, but
smaller than the normal store size.
* Fix storage finding
* Check for strong_count == 1
* try_recycle helper
* Move timestamp helper to sdk
* Add Bank method for getting timestamp estimate
* Return sysvar info from Bank::clock
* Add feature-gated timestamp correction
* Rename unix_timestamp method to be more descriptive
* Review comments
* Add timestamp metric
* introduce store program logs in blockstore / bigtable
* fix test, transaction logs created for successful transactions
* fix test for legacy bincode implementation around log_messages
* only api nodes should record logs
* truncate transaction logs to 100KB
* refactor log truncate for improved coverage
* Include post balance information for rewards
* Add post-balance to stored Reward struct
* Handle extended Reward in bigtable
Co-authored-by: Michael Vines <mvines@gmail.com>
* Save/restore Tower
* Avoid unwrap()
* Rebase cleanups
* Forcibly pass test
* Correct reconcilation of votes after validator resume
* d b g
* Add more tests
* fsync and fix test
* Add test
* Fix fmt
* Debug
* Fix tests...
* save
* Clarify error message and code cleaning around it
* Move most of code out of tower save hot codepath
* Proper comment for the lack of fsync on tower
* Clean up
* Clean up
* Simpler type alias
* Manage tower-restored ancestor slots without banks
* Add comment
* Extract long code blocks...
* Add comment
* Simplify returned tuple...
* Tweak too aggresive log
* Fix typo...
* Add test
* Update comment
* Improve test to require non-empty stray restored slots
* Measure tower save and dump all tower contents
* Log adjust and add threshold related assertions
* cleanup adjust
* Properly lower stray restored slots priority...
* Rust fmt
* Fix test....
* Clarify comments a bit and add TowerError::TooNew
* Further clean-up arround TowerError
* Truly create ancestors by excluding last vote slot
* Add comment for stray_restored_slots
* Add comment for stray_restored_slots
* Use BTreeSet
* Consider root_slot into post-replay adjustment
* Tweak logging
* Add test for stray_restored_ancestors
* Reorder some code
* Better names for unit tests
* Add frozen_abi to SavedTower
* Fold long lines
* Tweak stray ancestors and too old slot history
* Re-adjust error conditon of too old slot history
* Test normal ancestors is checked before stray ones
* Fix conflict, update tests, adjust behavior a bit
* Fix test
* Address review comments
* Last touch!
* Immediately after creating cleaning pr
* Revert stray slots
* Revert comment...
* Report error as metrics
* Revert not to panic! and ignore unfixable test...
* Normalize lockouts.root_slot more strictly
* Add comments for panic! and more assertions
* Proper initialize root without vote account
* Clarify code and comments based on review feedback
* Fix rebase
* Further simplify based on assured tower root
* Reorder code for more readability
Co-authored-by: Michael Vines <mvines@gmail.com>
* Check bank capitalization
* Simplify and unify capitalization calculation
* Improve and add tests
* Avoid overflow and inhibit automatic restart
* Fix test
* Tweak checked sum for cap. and add tests
* Fix broken build after merge conflicts..
* Rename to ClusterType
* Rename confusing method
* Clarify comment
* Verify cap. in rent and inflation tests
Co-authored-by: Stephen Akridge <sakridge@gmail.com>
* Prevent unbound memory growth by blockstore_processor
* Promote log to info! considering infrequency
* Exclude the time of freeing from interval...
* Skip not-shrinkable slots even if forced
* Add comment
* Implement Debug for MessageProcessor
* Switch from delta-based gating to whole-set gating
* Remove dbg!
* Fix clippy
* Clippy
* Add test
* add loader to stable operating mode at proper epoch
* refresh_programs_and_inflation after ancestor setup
* Callback via snapshot; avoid account re-add; Debug
* Fix test
* Fix test and fix the past history
* Make callback management stricter and cleaner
* Fix test
* Test overwrite and frozen for native programs
* Test epoch callback with genesis-programs
* Add assertions for parent bank
* Add tests and some minor cleaning
* Remove unsteady assertion...
* Fix test...
* Fix DOS
* Skip ensuring account by dual (whole/delta) gating
* Fix frozen abi implementation...
* Move compute budget constatnt init back into bank
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
* wip: re-do rent collection check on rent-exempt account
* Let's see how the ci goes
* Restore previous code
* Well, almost all new changes are revertable
* Update doc
* Add test and gating
* Fix tests
* Fix tests, especially avoid to change abi...
* Fix more tests...
* Fix snapshot restore
* Align to _new_ with better uninitialized detection
* Fix bad rent in Bank::deposit as if since epoch 0
* Remove redundant predicate
* Rename
* Start to add tests with some cleanup
* Forgot to add refactor code...
* Enchance test
* Really fix rent timing in deposit with robust test
* Simplify new behavior by disabling rent altogether
* fix rewards points
* fixups
* * verify that we don't spend more in rewards than we've allocated for rewards
* purge f64s from calculations that could be done with integers
* test typical values
* simplify iteration over delegations some
* fixups
* Use try_from
* Add a comment for commission_split()
* Add assertion to detect inconsistent reward dist.
* Fix vote_balance_and_staked
* Don't overwrite accounts with stale copies
* Fix CI...
* Add tests for vote_balance_and_staked
* Add test for the determinism of update_rewards
* Revert "Don't overwrite accounts with stale copies"
This reverts commit 9886d085a69d695f48970f48f2f567e62ba05414.
* Make stake_delegation_accounts to return hashmap
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
* Move bank (de)serialisation logic from bank and snapshot_utils to serde_snapshot.
Add sanity assertions between genesis config and bank fields on deserialisation.
Atomically update atomic bool in quote_for_specialization_detection().
Use same genesis config when restoring snapshots in test cases.
* Tidy up namings and duplicate structs to version
* Apply struct renames to tests
* Update abi hashes
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
* Dont skip eager rent collect across gapped epochs
* Adjust style and comment
* Adjust ascii chart and comment a bit
* Moar assert
* Relax the partition_count assert for completeness
* Tweak comment...
* tweak a bit
* Add gating logic
* Address reviews
* small formatting
* Clarify the code by replacing auto_generated...
* small formatting
* small formatting
* small formatting
* small formatting
* Narrow down conditional compilation scope
* Make Message::new_with_payer the default constructor
* Remove Transaction::new_[un]signed_instructions
These guess the fee-payer instead of stating it explicitly
* Prioritize shrinking of previously cleaned roots
* measure time of stale slot shrinking
* Disable shrink for test
* shrink: budgeting, store count, force for snapshot
* Polish implementation and fix tests
* Fix ci..
* Clean up a bit
* Further polish implementation and fix/add tests
* Rebase fixes
* Remove unneeded Default for AccountStorageEntry
* Address review comments
* More cleanup
* More cleanup
* Multi-version snapshot support
* rustfmt
* Remove CLI options and runtime support for selection output snapshot version.
Address some clippy complaints.
* Muzzle clippy type complexity warning.
Despite clippy's suggestion, it is not currently possible to create type aliases
for traits and so everything within the 'Box<...>' cannot be type aliased.
This then leaves creating full blown traits, and either implementing
said traits by closure (somehow) or moving the closures into new structs
implementing said traits which seems a bit of a palaver.
Alternatively it is possible to define and use the type alias 'type ResultBox<T> = Result<Box<T>>'
which does seems rather pointless and not a great reduction in complexity but is enough to keep clippy happy.
In the end I simply went with squelching the clippy warning.
* Remove now unused Serialize/Deserialize trait implementations for AccountStorageEntry and AppendVec
* refactor versioned de/serialisers
* rename serde_utils to serde_snapshot
* move call to accounts_db.generate_index() back down to context_accountsdb_from_stream()
* update version 1.1.1 to 1.2.0
remove nested use of serialize_bytes
* cleanups
* Add back measurement of account storage entry serialization.
Remove construction of Vec and HashMap temporaries during serialization.
* consolidate serialisation test cases into serde_snapshot.
clean up leakage of implementation details in serde_snapshot.
* move short term / legacy snapshot code into child module
* add serialize_iter_as_tuple
* preliminary integration of following commit
commit 6d58b73c47294bfb93465d5a83cd2175660b6e6d
Author: Ryo Onodera <ryoqun@gmail.com>
Date: Wed May 20 14:02:02 2020 +0900
Confine snapshot 1.1 relic to versioned codepath
* refactored serde_snapshot, rustfmt
legacy accounts_db format now "owns" both leading u64s, legacy bank_rc format has none
* reduce type complexity (clippy)
* Switch AccountsIndex.account_maps from HashMap to BTreeMap
* Introduce eager rent collection
* Start to add tests
* Avoid too short eager rent collection cycles
* Add more tests
* Add more tests...
* Refacotr!!!!!!
* Refactoring follow up
* More tiny cleanups
* Don't rewrite 0-lamport accounts to be deterministic
* Refactor a bit
* Do hard fork, restore tests, and perf. mitigation
* Fix build...
* Refactor and add switch over for testnet (TdS)
* Use to_be_bytes
* cleanup
* More tiny cleanup
* Rebase cleanup
* Set Bank::genesis_hash when resuming from snapshot
* Reorder fns and clean ups
* Better naming and commenting
* Yet more naming clarifications
* Make prefix width strictly uniform for 2-base partition_count
* Fix typo...
* Revert cluster-dependent gate
* kick ci?
* kick ci?
* kick ci?
* Switch subscriptions to use commitment instead of confirmations
* Add bank method to return account and last-modified slot
* Add last_modified_slot to subscription data and use to filter account subscriptions
* Update tests to non-zero last_notified_slot
* Add accounts subscriptions to test; fails at higher tx load
* Pass BankForks to RpcSubscriptions
* Use BankForks on add_account_subscription to properly initialize last_notified_slot
* Bundle subscriptions
* Check for non-equality
* Use commitment to initialize last_notified_slot; revert context.slot chage
Indexing into accounts array does not match account_keys otherwise.
Also enforce program accounts not at index 0
Enforce at least 1 Read-write signing fee-payer account.
* Introduce background AppendVec shrink mechanism
* Support ledger tool
* Clean up
* save
* save
* Fix CI
* More clean up
* Add tests
* Clean up yet more
* Use account.hash...
* Fix typo....
* Add comment
* Rename accounts_cleanup_service
* Add runtime methods to simply get status and slot
* Add helper function to get slot confirmation_count from BlockCommitmentCache
* Return cluster confirmations in getSignatureStatus
* Remove use of invalid get_signature_confirmation_status
* Remove unused methods
* Update pubsub to use cluster confirmations
* Fix test_check_signature_subscribe failure
* Refactor confirmations to read commitment cache only once
* Review comments
* Use bank, root from BlockCommitmentCache
* Update docs
* Add metric for block-commitment aggregations
Co-authored-by: Justin Starry <justin@solana.com>
* Copy current state version to v0
* Add `FeeCalculator` to nonce state
* fixup compile
* Dump v0 handling...
Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...
* Create more representative test `RecentBlockhashes`
* Improve CLI nonce account display
Co-Authored-By: Michael Vines <mvines@gmail.com>
* Fix that last bank test...
* clippy/fmt
Co-authored-by: Michael Vines <mvines@gmail.com>
* Do periodic inbound compaction for rooted slots
* Add comment
* nits
* Consider not_compacted_roots in cleanup_dead_slot
* Renames in AccountsIndex
* Rename to reflect expansion of removed accounts
* Fix a comment
* rename
* Parallelize clean over AccountsIndex
* Some niceties
* Reduce locks and real chunked parallelism
* Measure each step for sampling opportunities
* Just noticed par iter is maybe lazy
* Replace storage scan with optimized index scan
* Various clean-ups
* Clear uncleared_roots even if no updates
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`
Leaving `FeeCalculator` to *only* calculate transaction fees
* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate
* Expose recent `FeeRateGovernor` to clients
* Move `burn()` back into `FeeCalculator`
Appease BPF tests
* Revert "Move `burn()` back into `FeeCalculator`"
This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.
* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`
* Make `FeeRateGovernor`'s `lamports_per_signature` private
* rebase artifacts
* fmt
* Drop 'Recent'
* Drop _with_commitment variant
* Use a more portable integer for `target_signatures_per_slot`
* Add docs for `getReeRateCalculator` JSON RPC method
* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
Ryo Onodera
Jack May
Alexander Meißner
Trent Nelson
behzad nouri
Michael Vines
sakridge
Tyera Eulberg
carllin
Justin Starry
Tyera Eulberg
Greg Fitzgerald
Eric Williams
Josh
Rob Walker
Kristofer Peterson
Boqin Qin
Ryo Onodera
anatoly yakovenko
Pankaj Garg