* Multi-version snapshot support
* rustfmt
* Remove CLI options and runtime support for selection output snapshot version.
Address some clippy complaints.
* Muzzle clippy type complexity warning.
Despite clippy's suggestion, it is not currently possible to create type aliases
for traits and so everything within the 'Box<...>' cannot be type aliased.
This then leaves creating full blown traits, and either implementing
said traits by closure (somehow) or moving the closures into new structs
implementing said traits which seems a bit of a palaver.
Alternatively it is possible to define and use the type alias 'type ResultBox<T> = Result<Box<T>>'
which does seems rather pointless and not a great reduction in complexity but is enough to keep clippy happy.
In the end I simply went with squelching the clippy warning.
* Remove now unused Serialize/Deserialize trait implementations for AccountStorageEntry and AppendVec
* refactor versioned de/serialisers
* rename serde_utils to serde_snapshot
* move call to accounts_db.generate_index() back down to context_accountsdb_from_stream()
* update version 1.1.1 to 1.2.0
remove nested use of serialize_bytes
* cleanups
* Add back measurement of account storage entry serialization.
Remove construction of Vec and HashMap temporaries during serialization.
* consolidate serialisation test cases into serde_snapshot.
clean up leakage of implementation details in serde_snapshot.
* move short term / legacy snapshot code into child module
* add serialize_iter_as_tuple
* preliminary integration of following commit
commit 6d58b73c47294bfb93465d5a83cd2175660b6e6d
Author: Ryo Onodera <ryoqun@gmail.com>
Date: Wed May 20 14:02:02 2020 +0900
Confine snapshot 1.1 relic to versioned codepath
* refactored serde_snapshot, rustfmt
legacy accounts_db format now "owns" both leading u64s, legacy bank_rc format has none
* reduce type complexity (clippy)
* Switch AccountsIndex.account_maps from HashMap to BTreeMap
* Introduce eager rent collection
* Start to add tests
* Avoid too short eager rent collection cycles
* Add more tests
* Add more tests...
* Refacotr!!!!!!
* Refactoring follow up
* More tiny cleanups
* Don't rewrite 0-lamport accounts to be deterministic
* Refactor a bit
* Do hard fork, restore tests, and perf. mitigation
* Fix build...
* Refactor and add switch over for testnet (TdS)
* Use to_be_bytes
* cleanup
* More tiny cleanup
* Rebase cleanup
* Set Bank::genesis_hash when resuming from snapshot
* Reorder fns and clean ups
* Better naming and commenting
* Yet more naming clarifications
* Make prefix width strictly uniform for 2-base partition_count
* Fix typo...
* Revert cluster-dependent gate
* kick ci?
* kick ci?
* kick ci?
* Switch subscriptions to use commitment instead of confirmations
* Add bank method to return account and last-modified slot
* Add last_modified_slot to subscription data and use to filter account subscriptions
* Update tests to non-zero last_notified_slot
* Add accounts subscriptions to test; fails at higher tx load
* Pass BankForks to RpcSubscriptions
* Use BankForks on add_account_subscription to properly initialize last_notified_slot
* Bundle subscriptions
* Check for non-equality
* Use commitment to initialize last_notified_slot; revert context.slot chage
Indexing into accounts array does not match account_keys otherwise.
Also enforce program accounts not at index 0
Enforce at least 1 Read-write signing fee-payer account.
* Introduce background AppendVec shrink mechanism
* Support ledger tool
* Clean up
* save
* save
* Fix CI
* More clean up
* Add tests
* Clean up yet more
* Use account.hash...
* Fix typo....
* Add comment
* Rename accounts_cleanup_service
* Add runtime methods to simply get status and slot
* Add helper function to get slot confirmation_count from BlockCommitmentCache
* Return cluster confirmations in getSignatureStatus
* Remove use of invalid get_signature_confirmation_status
* Remove unused methods
* Update pubsub to use cluster confirmations
* Fix test_check_signature_subscribe failure
* Refactor confirmations to read commitment cache only once
* Review comments
* Use bank, root from BlockCommitmentCache
* Update docs
* Add metric for block-commitment aggregations
Co-authored-by: Justin Starry <justin@solana.com>
* Copy current state version to v0
* Add `FeeCalculator` to nonce state
* fixup compile
* Dump v0 handling...
Since we new account data is all zeros, new `Current` versioned accounts
look like v0. We could hack around this with some data size checks, but
the `account_utils::*State` traits are applied to `Account`, not the
state data, so we're kind SOL...
* Create more representative test `RecentBlockhashes`
* Improve CLI nonce account display
Co-Authored-By: Michael Vines <mvines@gmail.com>
* Fix that last bank test...
* clippy/fmt
Co-authored-by: Michael Vines <mvines@gmail.com>
* Do periodic inbound compaction for rooted slots
* Add comment
* nits
* Consider not_compacted_roots in cleanup_dead_slot
* Renames in AccountsIndex
* Rename to reflect expansion of removed accounts
* Fix a comment
* rename
* Parallelize clean over AccountsIndex
* Some niceties
* Reduce locks and real chunked parallelism
* Measure each step for sampling opportunities
* Just noticed par iter is maybe lazy
* Replace storage scan with optimized index scan
* Various clean-ups
* Clear uncleared_roots even if no updates
* SDK: Split new `FeeRateGovernor` out of `FeeCalculator`
Leaving `FeeCalculator` to *only* calculate transaction fees
* Replace `FeeCalculator` with `FeeRateGovernor` as appropriate
* Expose recent `FeeRateGovernor` to clients
* Move `burn()` back into `FeeCalculator`
Appease BPF tests
* Revert "Move `burn()` back into `FeeCalculator`"
This reverts commit f3035624307196722b62ff8b74c12cfcc13b1941.
* Adjust BPF `Fee` sysvar test to reflect removal of `burn()` from `FeeCalculator`
* Make `FeeRateGovernor`'s `lamports_per_signature` private
* rebase artifacts
* fmt
* Drop 'Recent'
* Drop _with_commitment variant
* Use a more portable integer for `target_signatures_per_slot`
* Add docs for `getReeRateCalculator` JSON RPC method
* Don't return `lamports_per_signature` in `getFeeRateGovernor` JSONRPC reply
* Verb-noun-ify Nonce API
* Unify instruction naming with API naming
The more verbose nonce_account/NonceAccount was chosen for clarity
that these instructions work on a unique species of system account
* Rename bootstrap leader to bootstrap validator
It's a normal validator as soon as other validators enter the
leader schedule.
* cargo fmt
* Fix build
Thanks @CriesofCarrots!
* Consolidate entry tick verification into one function
* Mark bad slots as dead in blocktree processor
* more feedback
* Add bank.is_complete
* feedback
* Bank: Return nonce pubkey/account from `check_tx_durable_nonce`
* Forward account with HashAgeKind::DurableNonce
* Add durable nonce helper for HashAgeKind
* Add nonce util for advancing stored nonce in runtime
* Advance nonce in runtime
* Store rolled back nonce account on TX InstructionError
* nonce: Add test for replayed InstErr fee theft
* save limit deserialize
* save
* Save
* Clean up
* rustfmt
* rustfmt
* Just comment out to please CI
* Fix ci...
* Move code
* Rustfmt
* Crean up control flow
* Add another comment
* Introduce predetermined constant limit on snapshot data files (deserialize side)
* Introduce predetermined constant limit on snapshot data files (serialize side)
* rustfmt
* Tweak message
* Revert dynamic memory limit
* Limit size of snapshot data file (de)serialization
* Fix test breakage
* Clean up
* Fix uses formatting
* Rename: deserialize_{for,from}_snapshot
* Simplify comment
* Use Slot
* Provide slot for status cache
* Align variable name with snapshot_status_cache_file_path
* Define serialize_snapshot_data_file_with_metrics
* Fix build.......
* De-marco serialize_snapshot_data_file_with_metrics
* Revert u64 => Slot
* Rework transaction processing result forwarding
Durable nonce prereq
* Add Durable Nonce program API
* Add runtime changes for Durable Nonce program
* Register Durable Nonce program
* Concise comments and bad math
* Fix c/p error
* Add rent sysvar to withdraw ix
* Remove rent exempt required balance from Meta struct
* Use the helper
* add investor stake placeholders
fixups
fixups
review comments, fixups
make more data-looky for easier management
rent may be zero
rework with more tables, derived keys
fixups
rebase-fix
fixups
fixups
* genesis is now too big to boot in 10 seconds
* Pass blocktree into execute_batch, if persist_transaction_status
* Add validator arg to enable persistent transaction status store
* Pass blocktree into banking_stage, if persist_transaction_status
* Add validator params to bash scripts
* Expose actual transaction statuses outside Bank; add tests
* Fix benches
* Offload transaction status writes to a separate thread
* Enable persistent transaction status along with rpc service
* nudge
* Review comments
* Fix bank hash not changing when no internal state has changed
* Fix unnecessary call to hash_internal_state
* Add blockhash into the bank_hash
* Add blockhash into the bank_hash and update tests
* Refactor accounts_db slot_hashes
* More clarity in comments
* Add clippy suggestion
* Grammar
* Fix compile after clippy made me break it
* Schooled by clippy
* vote array
wip
wip
wip
update
gossip index should match tower index
tests build
clippy
test index after expired vote
test
bank specific last vote sync time
* verify
* we are likely to see many more warnings about old votes now
* SDK: Add sysvar to expose recent block hashes to programs
* Blockhashes is one word
* Missed one
* Avoid allocs on update
* unwrap_or_else
* Use iterators
* Add microbench
* Revert "unwrap_or_else"
This reverts commit a8f8c3bfbe0d88e2229e67a29e9d2b489b93ff62.
* Revert "Avoid allocs on update"
This reverts commit 486f01790c271cf09ac3dbffba0e40e526e81324.
* credit_only_credits_forwarding
* whack transfer_now()
* fixup
* bench should retry the airdrop TX
* fixup
* try to make bench-exchange a bit more robust, informative
* Stabilize some banking stage tests
Fixes#5660
* Fix CI...
* clean up
* Fix ci
* Address review nits
* Use bank.max_tick_height due to off-by-one for no PohRecord's clearing bank
* Fix CI...
* Use bank.max_tick_height() instead for clarity
* collect rent from credit debit accounts
* collect rent from credit only account
* rent_collector now can deduct partial rent + no mem copy + improved design
* adding a test to test credit only rent
* add bank level test for rent deduction
* add test to check if hash value changes or not
* adding test scenario for lamport circulation
* collect rent from credit-debit account
* collect rent from credit only account
* improved design for rent collection
* only process if collected rent is non zero
* rent_collector now can deduct partial rent + no mem copy
* adding a test to test credit only rent
* add bank level test for rent deduction
* add test to check if hash value changes or not
* adding test scenario for lamport circulation
* combining rent debtors into credit only locks
* add missing convenience method
* require vote account to be exempt
* make stake account rent exempt
* making executable rent exempt
* rent will be initialized in genesis
* add test for update_rent
* server side new rpc endpoint
* client side rpc
* take data_len as usize
Co-Authored-By: Tyera Eulberg <teulberg@gmail.com>
* add test and documentation
* Move status cache serialization to the Snapshot Packager service
* Minor comment updates
* use ok_or_else instead of ok_or
* satus cache
* Remove assert when snapshot format is wrong
* Fix compile
* Remove slots_to_snapshot from bank forks
* Address review comment
* Remove unused imports
* Change confidence parameters
* Add status_cache_ancestors to get all relevant ancestors of a bank including roots from status cache
* Fix and add tests
* Clippy
* require vote account to be exempt
* make stake account rent exempt
* add rent exempted system instruction
* use rent exemption instruction in vote and stake api
* use rent exempted account while creating executable account
* updating chacha golden hash as instruction data has changed
* rent will be initialized for genesis bank too
* rename rent.rs to rent_calculator.rs
* add rent sysvar
* integrate rent_calculator with bank
* rent_calculator integration with genesis
* add test for rent sysvar
Summary of Changes:
This change adds functionality to randomize tx execution for every entry. It does this by implementing OrderedIterator that iterates tx slice as per the order specified. The order is generated randomly for every entry.
* Remove unnecessary entry_height from BankInfo
* Refactor process_blocktree to support process_blocktree_from_root
* Refactor to process blocktree after loading from snapshot
* On restart make sure bank_forks contains all the banks between the root and the tip of each fork, not just the head of each fork
* Account for 1 tick_per_slot in bank 0 so that blockhash of bank0 matches the tick
* Rate limit transaction counters
* @sakridge feedback
* Set default high metrics rate for multinode demo
* Fix tests
* Swap defaults and fix env var tests
* Only set metrics rate if not already set
* change paths to something accounts_db (the singleton) owns, fixes SIGILL
* fail deserialize if paths don't work
serialize paths, too
* test that paths are populated from a bank snapshot
* Implement CreditOnlyLocks
* Update credit-only atomic on account load
* Update credit-only atomic after bank.freeze_lock; store credits if all credit-only lock references are dropped
* Commit credit-only credits on bank freeze
* Update core to CreditAccountLocks
* Impl credit-only in System Transfer
* Rework CreditAccountLocks, test, and fix bugs
* Review comments: Pass CreditAccountLocks by reference; Tighten up insert block
* Only store credits on completed slot
* Check balance in bench_exchange funding to ensure commit_credits has completed
* Add is_debitable info to KeyedAccount meta to pass into programs
* Reinstate CreditOnlyLocks check on lock_account
* Rework CreditAccountLocks to remove strong_count usage
* Add multi-threaded credit-only locks test
* Improve RwLocks usage
* Review comments: panic if bad things happen; tighter code
* Assert lock_accounts race does not happen
* Revert panic if bad things happen; not a bad thing
* Add storage point tracking and tie in storage rewards to epochs and economics
* Prevent validators from updating their validations for a segment
* Fix test
* Retain syscall scoping for readability
* Update Credits to own epoch tracking
Jack May
carllin
Boqin Qin
Trent Nelson
Kristofer Peterson
Michael Vines
Justin Starry
Greg Fitzgerald
Tyera Eulberg
sakridge
Ryo Onodera
anatoly yakovenko
Ryo Onodera
Pankaj Garg
Rob Walker
sakridge
Parth
Sagar Dhawan
Ryo Onodera
Sunny Gleason
Justin Starry
Tyera Eulberg
TristanDebrunner
Sathish
Sathish Ambley