Commit Graph

327 Commits

Author SHA1 Message Date
Lucas Steuernagel b97b3dd4ab
Use BankForks on tests - Part 3 (#34248)
* Add BankForks to core tests

* Refactor functions under DCOU
2023-12-01 13:47:22 -03:00
Andrew Fitzgerald 2294801954
Do not derive Copy for EpochSchedule and Rent (#32767) 2023-12-01 07:57:25 -08:00
steviez 230779d459
Revert " Remove redundant bounds check from getBlock and getBlockTime… (#33996)
Revert " Remove redundant bounds check from getBlock and getBlockTime (#33901)"

This reverts commit 03a456e7bb.
2023-11-08 18:16:51 -06:00
steviez 03a456e7bb
Remove redundant bounds check from getBlock and getBlockTime (#33901)
JsonRpcRequestProcessor::check_blockstore_root() contained some logic
that performed duplicate sanity checking on a Blockstore fetch result.
The checking involved creating rocksdb iterators, which has non-trivial
overhead.

This PR removes the duplicate checking, and also adds comments to help
reason about how JsonRpcRequestProcessor interprets the Blockstore
result.
2023-11-08 12:09:10 -06:00
Lijun Wang eba1b2d3e3
Remove RwLock on TransactionNotifier (#33962)
* Remove RwLock on TransactionNotifier
2023-11-07 10:28:56 -08:00
steviez ee29647f67
Remove Option<_> from Blockstore::get_rooted_block_time() return type (#33955)
Instead of returning Result<Option<UnixTimestamp>>, return
Result<UnixTimestamp> and map None to an error. This makes the return
type similar to that of Blockstore::get_rooted_block().
2023-11-06 12:56:10 -06:00
Tyera 7048e72d81
Blockstore: only return block times for rooted slots (#33871)
* Add Blockstore::get_rooted_block_time method and use in RPC

* Un-pub get_block_time
2023-10-26 11:38:58 -06:00
Pankaj Garg 78c31aa6b8
Use program cache fork graph in extract() (#33806)
* Use program cache fork graph instead of WorkingSlot trait

* Fix deadlocked tests

* keep WorkingSlot trait for now
2023-10-25 06:04:38 -07:00
Pankaj Garg 9d42cd7efe
Initialize fork graph in program cache during bank_forks creation (#33810)
* Initialize fork graph in program cache during bank_forks creation

* rename BankForks::new to BankForks::new_rw_arc

* fix compilation

* no need to set fork_graph on insert()

* fix partition tests
2023-10-23 09:32:41 -07:00
steviez 56ccffdaa5
Replace get_tmp_ledger_path!() with self cleaning version (#33702)
This macro is used a lot for tests to create a ledger path in order to
open a Blockstore. Files will be left on disk unless the test remembers
to call Blockstore::destroy() on the directory. So, instead of requiring
this, use the get_tmp_ledger_path_auto_delete!() macro that creates a
TempDir (which automatically deletes itself when it goes out of scope).
2023-10-21 11:38:31 +02:00
Ryo Onodera 5a963529a8
Add BankWithScheduler for upcoming scheduler code (#33704)
* Add BankWithScheduler for upcoming scheduler code

* Remove too confusing insert_without_scheduler()

* Add doc comment as a bonus

* Simplify BankForks::banks()

* Add derive(Debug) on BankWithScheduler
2023-10-21 15:56:43 +09:00
steviez 8bd0e4cd95
Change getHealth to compare optimistically confirmed slots (#33651)
The current getHealth mechanism checks a local accounts hash slot vs.
those of other nodes as specified by --known-validator. This is a
very coarse comparison given that the default for this value is 100
slots. More so, any nodes using a value larger than the default
(ie --incremental-snapshot-interval 500) will likely see getHealth
return status behind at some point.

Change the underlying mechanism of how health is computed. Instead of
using the accounts hash slots published in gossip, use the latest
optimistically confirmed slot from the cluster. Even when a node is
behind, it is able to observe cluster optimistically confirmed by slots
by viewing votes published in gossip.

Thus, the latest cluster optimistically confirmed slot can be compared
against the latest optimistically confirmed bank from replay to
determine health. This new comparison is much more granular, and not
needing to depend on individual known validators is also a plus.
2023-10-16 11:21:33 -05:00
steviez 73a9a14731
Fold noisy metric into struct of metrics that is reported every 10s (#33635) 2023-10-10 16:35:16 -05:00
Tyera 509d6acd2b
Remove primary index from Blockstore special-column keys (#33419)
* Add helper trait for column key deprecation

* Add WriteBatch::delete_raw

* Add ProtobufColumn::get_raw_protobuf_or_bincode

* Add ColumnIndexDeprecation iterator methods

* Impl ColumnIndexDeprecation for TransactionStatus (doesn't build)

* Update TransactionStatus put

* Update TransactionStatus purge_exact

* Fix read_transaction_status

* Fix get_transaction_status_with_counter

* Fix test_all_empty_or_min (builds except tests)

* Fix test_get_rooted_block

* Fix test_persist_transaction_status

* Fix test_get_transaction_status

* Fix test_get_rooted_transaction

* Fix test_get_complete_transaction

* Fix test_lowest_cleanup_slot_and_special_cfs

* Fix test_map_transactions_to_statuses

* Fix test_transaction_status_protobuf_backward_compatability

* Fix test_special_columns_empty

* Delete test_transaction_status_index

* Delete test_purge_transaction_status

* Ignore some tests until both special columns are dealt with (all build)

* Impl ColumnIndexDeprecation for AddressSignatures (doesn't build)

* Add BlockstoreError variant

* Update AddressSignatures put

* Remove unneeded active_transaction_status_index column lock

* Update AddressSignatures purge_exact

* Fix find_address_signatures_for_slot methods

* Fix get_block_signatures methods

* Fix get_confirmed_signatures_for_address2

* Remove unused method

* Fix test_all_empty_or_min moar (builds except tests)

* Fix tests (all build)

* Fix test_get_confirmed_signatures_for_address

* Fix test_lowest_cleanup_slot_and_special_cfs moar

* Unignore tests (builds except tests)

* Fix test_purge_transaction_status_exact

* Fix test_purge_front_of_ledger

* Fix test_purge_special_columns_compaction_filter (all build)

* Move some test-harness stuff around

* Add test cases for purge_special_columns_with_old_data

* Add test_read_transaction_status_with_old_data

* Add test_get_transaction_status_with_old_data

* Review comments

* Move rev of block-signatures into helper

* Improve deprecated_key impls

* iter_filtered -> iter_current_index_filtered

* Add comment to explain why use the smallest (index, Signature) to seed the iterator

* Impl ColumnIndexDeprecation for TransactionMemos (doesn't build)

* Update TransactionMemos put

* Add LedgerColumn::get_raw

* Fix read_transaction_memos

* Add TransactionMemos to purge_special_columns_exact

* Add TransactionMemos to compaction filter

* Take find_address_signatures out of service

* Remove faulty delete_new_column_key logic

* Simplify comments
2023-10-10 10:40:36 -06:00
steviez 3eae980293
Minor cleanup on some snapshot related tests (#33485) 2023-10-02 20:25:30 +02:00
Andrew Fitzgerald 660e41a8e1
Remove entry shuffling (#33378) 2023-10-02 09:03:12 -07:00
Jon Cinque de38b05ad1
spl: Bump token-2022 and friends (#33453)
* token: Update to 4.0.0

* token-2022: Bump and support new account and instruction types

* Update token-2022 in fetch_spl / program-test

* Fixup downstream uses

* Mint and destination were flipped in 0.9.0

* Don't use `convert_pubkey`

* Bump spl dependencies to versions which avoid recompilations
2023-09-29 19:12:06 +02:00
Tyera 3e8ccbe196
Add comment to OptimisticallyConfirmedBankTracker (#33238)
Add comment
2023-09-21 09:59:51 -06:00
Pankaj Garg f50342a790
Split vote related code from runtime to its own crate (#32882)
* Move vote related code to its own crate

* Update imports in code and tests

* update programs/sbf/Cargo.lock

* fix check errors

* update abi_digest

* rebase fixes

* fixes after rebase
2023-09-19 10:46:37 -07:00
bji bdf7207eec
Implement timely vote credits feature. (#32957) 2023-09-12 10:40:04 -07:00
Alexander Meißner bbb57be0a5
Refactor - Move interfaces of address-lookup-table into the program SDK (#33165)
* Adds a module `address_lookup_table` to the SDK.

* Adds a module `address_lookup_table::instruction` to the SDK.

* Adds a module `address_lookup_table::error` to the SDK.

* Adds a module `address_lookup_table::state` to the SDK.

* Moves AddressLookupTable into SDK as well.

* Moves AddressLookupTableAccount into address_lookup_table.

* Adds deprecation messages.

* Disentangles dependencies across cargo files.
2023-09-11 21:10:40 +02:00
Tao Zhu 4f4ce69f5f
purge duplicated bank prioritization fee from cache (#33062)
* purge duplicated bank prioritization fee from cache

* add test for purge dup bank

* Added metrics counts to monitor anomalies

* fix a flaky test
2023-09-11 10:08:55 -05:00
Tao Zhu 904b2a7672
finalize prioritization_fee_cache after oc-ed bank is frozen (#33100) 2023-09-06 09:58:58 -05:00
Tyera 17b1b5646d
rpc: turn off getStakeActivation epoch parameter (#33156)
* Return error on past-epoch parameter

* Update docs
2023-09-05 16:52:49 -06:00
Alexander Meißner 9e703f85de
Upgrades Rust to 1.72.0 & nightly-2023-08-25 (#32961)
* allow pedantic invalid cast lint

* allow lint with false-positive triggered by `test-case` crate

* nightly `fmt` correction

* adapt to rust layout changes

* remove dubious test

* Use transmute instead of pointer cast and de/ref when check_aligned is false.

* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.

* bump rust nightly to 2023-08-25

* Upgrades Rust to 1.72.0

---------

Co-authored-by: Trent Nelson <trent@solana.com>
2023-09-01 07:26:13 +00:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
Lijun Wang 2642b8a552
Get slot info along tpu leader (#32942)
Created an overload of get_leader_tpus: get_leader_tpus_with_slots to get the tpu address along with the slot. And put that information into the debug log. This information can be compared with where the the transaction confirmed to measure leader accuracy. This help us better understand the txn latency.
2023-08-27 03:13:41 -07:00
steviez a4c8cc3ce0
Remove improper uses of &Arc<Bank> (#32802)
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy

This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
2023-08-18 16:46:34 -05:00
steviez f86b6edc4d
Simplify get_parent_chains() test function (#32892)
The previous implementation is a little tricky to follow, and repeats
some functionality that is already implemented in
Bank::parents_inclusive(). So, swap in bank.ancestors.
2023-08-18 14:10:03 -05:00
steviez 427b8b1332
Revert "Implement timely vote credits feature (#31291)" (#32890)
This reverts commit 35ec7bf804.
2023-08-18 17:04:20 +00:00
Jon Cinque c73a56faf8
stake: Rename `StakeStateWithFlags` -> `StakeStateV2` (#32795) 2023-08-11 01:07:21 +02:00
bji 35ec7bf804
Implement timely vote credits feature (#31291) 2023-08-10 14:07:51 -07:00
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
Jon Cinque 8e4a9a94ed
sdk: Add new version of `StakeState` to avoid breaking downstream users (#32736)
* sdk: Rename `StakeState` -> `StakeStateWithFlags`

* Add back `StakeFlags` with a deprecation warning
2023-08-09 00:05:40 +02:00
Tao Zhu ef6af307a4
improve prioritization fee cache accuracy (#32692)
* improve prioritization cache accuracy
2023-08-07 19:27:28 -05:00
Ashwin Sekar fa3506631a
stake: deprecate on chain warmup/cooldown rate and config (#32723)
* stake: deprecate on chain warmup/cooldown rate and config

* Pr feedback: Deprecate since 1.16.7

Co-authored-by: Jon Cinque <me@jonc.dev>

---------

Co-authored-by: Jon Cinque <me@jonc.dev>
2023-08-07 13:23:24 -07:00
Tao Zhu 6eea38d803
Remove unnecessary ref of arc (#32729)
* remove unnecessary ref of arc
2023-08-07 09:33:57 -05:00
steviez 226d7d986b
Simplify root slot lookup from BankForks (#32717)
No need to get an Arc<Bank> when we want the root slot from BankForks;
can just use BankForks::root().
2023-08-04 12:03:22 -06:00
steviez 20fc3a5ded
Remove improper &Arc<Blockstore> instances (#32698)
Update to either &Blockstore if the function just needs a ref, or
Arc<Blockstore> if the function needs to hang onto a copy.
2023-08-03 15:10:25 -06:00
Brooks 7143667dbe
Removes unnecessary mut (#32476) 2023-07-13 13:14:33 -04:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
Pankaj Garg b060f62004
Move transaction results data structures out of bank.rs (#32386)
* Move transaction results data structures out of bank.rs

* Fix CI check failures
2023-07-06 09:45:01 -07:00
Pankaj Garg 356a827087
Move NonceInfo, NonceFull and NoncePartial out of bank.rs (#32375)
* Move NonceInfo, NonceFull and NoncePartial out of bank.rs

* fix imports

* move test_nonce_info to nonce_info.rs
2023-07-05 10:44:54 -07:00
Alexander Meißner ee2c2ef6c7
Cleanup - require_static_program_ids_in_transaction (#31767)
require_static_program_ids_in_transaction
2023-06-07 17:12:41 +02:00
Lijun Wang 4b0514d9b1
Fixing send-transaction-service using quic, tpu address is wrong (#31899)
* Fixing send-transaction-service using quic, tpu address is wrong

* Use Protocol field instead of bool for passing protocol info

* Address some code review comment from Behzad: get_leader_tpus per protocol
2023-06-02 09:25:23 -07:00
Illia Bobyr 4353ac6797
Pass Arc<AtomicBool> by value, not by reference. (#31916)
`Arc` is already a reference internally, so it does not seem to be
beneficial to pass a reference to it.  Just adds an extra layer of
indirection.

Functions that need to be able to increment `Arc` reference count need
to take `Arc<AtomicBool>`, but those that just want to read the
`AtomicBool` value can accept `&AtomicBool`, making them a bit more
generic.

This change focuses specifically on `Arc<AtomicBool>`.  There are other
uses of `&Arc<T>` in the code base that could be converted in a similar
manner.  But it would make the change even larger.
2023-06-01 17:25:48 -07:00
Tyera 6bd4ae6955
Clean up Spl type conversions (#31845)
* Use spl-token ids directly in program-id checks

* Remove id redefinitions

* Deprecate pubkey_from_spl_token and remove usage

* Deprecate spl_token_pubkey and remove usage

* Deprecate native mint helpers and remove usage

* Deprecate spl_token_instruction and remove usage
2023-05-30 14:34:38 -06:00
behzad nouri 9281ab7d97
separates out connection-cache metrics for different protocols (#31803) 2023-05-25 14:48:22 +00:00
Lijun Wang 917f3d2586
Use unwrap_or_else for efficiency (#31747)
Use unwrap_or_else for efficiency.
2023-05-22 09:58:24 -07:00
steviez c5d72fc480
Condense and rename pubsub counters into a single datapoint (#31691)
Condense and rename pubsub counters into a single datapoint

The counters have been combined into fields within a struct that are
accumulated locally and submitted on a set interval (10s). Doing so
decreases overhead on the system as well as makes it easier to compare
these similar fields in the metrics explorer by having them all under
the same measurement.
2023-05-18 01:25:39 -05:00