Commit Graph

25207 Commits

Author SHA1 Message Date
Alexander Meißner d448c4e5f0
Fix - `ProgramCache::assign_program()` Closed => Loaded in same slot (#673)
* Refactors test_add_builtin() to use LoadedProgram::new_builtin() and ProgramCache::extract().

* Revert Closed => Loaded transition

* Adds more loaded programs deployment tests
2024-04-12 14:59:51 +02:00
Alexander Meißner 81519fab2a
Fix - `test_add_builtin()` should use `LoadedProgram::new_builtin()` and `ProgramCache::extract()` (#772)
Refactors test_add_builtin() to use LoadedProgram::new_builtin() and ProgramCache::extract().
2024-04-12 14:58:23 +02:00
Ashwin Sekar 499d36e354
vote: update benches and tests to TowerSync (#725) 2024-04-11 22:15:02 -07:00
Illia Bobyr 4f7e45bb24
metrics: Submit metrics when exiting. Refactor `MetricsAgent::run()`. (#718)
There are a few minor issues this change addresses:

1. When we send points to the `MetricsWriter` we are calling
   `Instant::now()` twice, using the first result in the metrics stats,
   and using the seconds value for `last_write_time`.  Yet, on the next
   upload, we would use `last_write_time` as a reference point.

   We upload metrics using a network call, so it is far from
   instantaneous.  This creates a minor discrepancy in our time
   reporting.

   Good news is that we do not really need to call `Instant::now()`
   twice at all, as we can use the same value for both stats and
   `last_write_time`.

2. We did not report metrics stats if we did not have any points
   accumulated.  It seems better to always report metric stats,
   including when no points have been accumulated.  In practice, this
   does not happen for the validator, as validators always report
   something during a 10-second accumulation interval.

3. We did not upload any points when the metrics thread was existing.
   This may cause a short number of metrics not to be reported.

4. `collect_points()` was always converting both `points` and `counters`
   into a vector of `DataPoint`, even if the final length was over the
   specified `max_points`.  In the `mainnet-beta` we have values of up
   to 5m points lost, so it could be a small optimization if we drop
   them sooner.
2024-04-11 22:02:44 -07:00
Illia Bobyr dabcc39819
cli: Move `allow(clippy::arithmetic_side_effects)` into modules (#758)
The rest of the codebase has `clippy::arithmetic_side_effects` enabled.
While CLI is not as critical as the rest of the validator code base, it
seems nice to be precise about the arithmetic operations, and clippy
helps with that if `arithmetic_side_effects` is enabled.

A blanket disable at the module level is suboptimal.  As we have a lot
of operations already written, that generate warnings under
`arithmetic_side_effects`, the first step is to move the lint control
into individual modules, that we can then inspect one by one.
2024-04-11 22:01:53 -07:00
Justin Starry c4996d6e52
Remove stale poh recorder validator id (#700) 2024-04-12 12:09:02 +08:00
samkim-crypto b046c12cdc
[zk-token-sdk] Refactor `AuthenticatedEncryptionError` and `ElGamalError` to errors module (#589)
* make the `errors` module available in sbf targets

* move `AuthenticatedEncryptionError` to the `errors` module

* move `ElGamalError` to the `errors` module
2024-04-12 11:26:40 +09:00
dmakarov ad94732d4f
SVM: rename TransactionCheckResult variable for consistency (#759) 2024-04-11 12:46:23 -07:00
Jeff Washington (jwash) e9f8fcb4e6
use previously calculated `stored_size` (#757) 2024-04-11 14:36:53 -05:00
behzad nouri df1b8f983e
expands test coverage of chained Merkle shreds (#55) 2024-04-11 19:17:14 +00:00
Jeff Washington (jwash) baed522d2b
call StorableAccounts.pubkey once (#752) 2024-04-11 11:22:50 -05:00
carllin d5c291a934
Remove send snapshot hard unwrap (#326) 2024-04-11 12:18:52 -04:00
dependabot[bot] f6cac1e20f
build(deps): bump anyhow from 1.0.81 to 1.0.82 (#749)
* build(deps): bump anyhow from 1.0.81 to 1.0.82

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-04-11 16:01:46 +00:00
Kevin Heavey e04dbaa305
move ThreadExecuteTimings to solana-ledger (#690)
* move ThreadExecuteTimings to solana-ledger
* run cargo-for-all-lock-files tree
* move ThreadExecuteTimings below BatchExecutionTiming
* remove ReplaySlotStats to solana-ledger
* remove  `#![recursion_limit = "2048"]` from program-runtime
2024-04-11 17:59:16 +02:00
Brooks 8aa32172d6
Stops copying new PathBuf when querying storage files (#751) 2024-04-11 11:00:16 -04:00
behzad nouri 293414f482
pads last erasure batch with empty data shreds (#639)
For duplicate blocks prevention we want to verify that the last erasure
batch was sufficiently propagated through turbine. This requires
additional bookkeeping because, depending on the erasure coding schema,
the entire batch might be recovered from only a few coding shreds.

In order to simplify above, this commit instead ensures that the last
erasure batch has >= 32 data shreds so that the batch cannot be
recovered unless 32+ shreds are received from turbine or repair.
2024-04-11 14:50:43 +00:00
Ashwin Sekar 4fc6b0bd96
banking_stage: track VoteStorage size instead of costly iterator (#743)
* banking_stage: track VoteStorage size instead of costly iterator

* missed one

---------

Co-authored-by: steviez <steven@anza.xyz>
2024-04-11 07:16:33 -07:00
Lucas Steuernagel 4e2fc2772b
Add unit test for `add_builtin` (#720) 2024-04-11 11:13:23 -03:00
Brooks 2871c59863
Removes write version from store-tool (#722) 2024-04-11 10:09:36 -04:00
Tyera 9706a64646
Fix typo (#745) 2024-04-10 22:52:51 -06:00
Yihau Chen 0e6d42e613
bump nix to 0.28.0 (#628)
* bump nix to 0.28.0

* enable 'socket' for net-utils

* enable 'signal' for install

* enable 'user' for perf

* enable 'net' for streamer
2024-04-11 12:03:23 +08:00
Ashwin Sekar c4734ad127
vote: plumb TowerSync ix (#584) 2024-04-10 20:33:45 -07:00
ripatel-fd 16efe510cb
Don't panic vote program if epoch number overflows (#730)
Helps with fuzzing

Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2024-04-11 02:56:19 +00:00
Alessandro Decina 55ab7fadbc
quic: use smallvec to aggregate chunks, save 1 alloc per packet (#735)
quic: use smallvec, save one allocation per packet

Use smallvec to hold chunks. Streams are packet-sized so we don't expect
them to have many chunks. This saves us an allocation for each packet.
2024-04-11 12:25:35 +10:00
Alessandro Decina 85c6e412e0
quic: do ordered reads, save 1 BTreeMap allocation per packet (#736)
quic: switch to ordered reads

Unordered reads cause a BTreeMap allocation for each packet inside quinn
in Assembler::ensure_ordering.

Most streams will fit in one datagram and will therefore be ordered by
definition. Switch to ordered reads to avoid the allocation.
2024-04-11 12:25:06 +10:00
Lijun Wang 92ebf0f80c
Treat super low staked as unstaked in streamer QOS (#701)
* Treat super low staked with QOS of unstaked

* simplify

* address some comment from Pankaj
2024-04-10 17:13:41 -06:00
samkim-crypto 006cf28dec
[zk-keygen] Remove deprecated functions (#599)
* add `deprecated` feature to produce warnings on use of deprecated functions

* replace `is_present` with `try_contains_id`

* replace validator functions

* replace `possible_values` with `value_parser`

* replace `value_of` function

* remove `deprecated` feature
2024-04-11 07:52:57 +09:00
Ashwin Sekar 411fdc9312
blockstore: scaffolding for chained merkle root conflict detection (#719)
* blockstore: scaffolding for chained merkle root conflict detection

* pr feedback: use dummy feature key until final plumbing
2024-04-10 22:46:33 +00:00
dmakarov d4fe07c68c
SVM: explain fields of TransactionBatchProcessor struct in spec (#727) 2024-04-10 15:23:27 -07:00
ripatel-fd 4b6d274887
Fix crash if vote state is malformed (#715)
* Fix crash if vote state is malformed

* Fix vote program crash if clock rewinds

Return an instruction error instead of panicking if the vote
program detects the vote program's vote authority history is
newer than the current clock.

---------

Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2024-04-10 17:12:56 -04:00
Andrew Fitzgerald e91a5e2744
default staked client in LocalCluster (#716)
* default staked client in LocalCluster

* fix underflow
2024-04-10 15:33:07 -05:00
dependabot[bot] 51f99727b9
build(deps): bump quote from 1.0.35 to 1.0.36 (#706)
* build(deps): bump quote from 1.0.35 to 1.0.36

Bumps [quote](https://github.com/dtolnay/quote) from 1.0.35 to 1.0.36.
- [Release notes](https://github.com/dtolnay/quote/releases)
- [Commits](https://github.com/dtolnay/quote/compare/1.0.35...1.0.36)

---
updated-dependencies:
- dependency-name: quote
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-04-10 23:49:56 +08:00
Kevin Heavey ff8a25b97f
svm: move bpf-loader-program to dev-dependencies (#711) 2024-04-10 10:22:51 -05:00
Kevin Heavey d06e3e95cb
program-test: move test-case to dev-dependencies (#710) 2024-04-10 10:22:28 -05:00
behzad nouri d308acdc84
removes feature gated code for dropping legacy shreds (#677) 2024-04-10 14:28:34 +00:00
Jeff Washington (jwash) 9778e489b0
add ScanAccountStorageData (#694) 2024-04-10 09:22:37 -05:00
Justin Starry 72ee270943
Update cargo metadata to Anza (#635) 2024-04-10 14:29:48 +08:00
Tyera dd8e1f4c73
Remove overly restrictive check_account_access for partitioned epoch rewards (#631)
* Remove rewards-interval-related check_account_access implementation

* Move RewardsInterval to tests module

* Update test to new StakeProgram functionality
2024-04-09 21:55:45 -06:00
Jeff Washington (jwash) 4535ea60a9
remove ReadableAccount trait from LoadedAccount (#692) 2024-04-09 22:14:52 -05:00
Jeff Washington (jwash) 0c5d56bd33
pass &LoadedAccount scan_account_storage (#693) 2024-04-09 22:13:10 -05:00
Ashwin Sekar 70c4cb0ba1
consensus: add dev-context-only-utils to tower (#687) 2024-04-09 16:39:57 -07:00
Tyera 2470b45468
Check EpochRewards::active within the Stake Program (#617)
* Add new StakeError variant

* Add closure to return error if EpochRewards::active

* Use error_during_epoch_rewards for Instructions that mutate stake accounts

* Use try instead of manually matching Ok/Err

* Consolidate error_during_epoch_rewards check

* Add new test helper

* Add test demonstrating which ix return StakeError::EpochRewardsActive

* Remove single-use fn
2024-04-09 17:29:50 -06:00
Andrew Fitzgerald 1744e9efd7
BankingStage Forwarding Filter (#685)
* add PacketFlags::FROM_STAKED_NODE

* Only forward packets from staked node

* fix local-cluster test forwarding

* review comment

* tpu_votes get marked as from_staked_node
2024-04-09 23:12:26 +00:00
Lijun Wang 592107a907
corrected to not use hardcoded connections count for unstaked (#633)
* corrected to not use hardcoded connections count for unstaked

* Fixed a math problem on max_unstaked_load_in_throttling_window

* Fixed a unit test failure
2024-04-09 15:20:24 -07:00
Jeff Washington (jwash) d9d11cda4d
break out parameters to hash_account (#691) 2024-04-09 17:03:37 -05:00
Jeff Washington (jwash) f62d79b8be
add `data_len()` to `LoadedAccount` (#689) 2024-04-09 16:32:07 -05:00
Jeff Washington (jwash) 9d700b8749
in get_pubkey_hash_account_for_slot use take_account earlier (#688) 2024-04-09 16:05:46 -05:00
Brooks 4546e79cbc
Fixes nits in streamer perf tracking (#648) 2024-04-09 16:47:18 -04:00
Jeff Washington (jwash) 374705293c
implement get_account_sizes for hot storages (#676) 2024-04-09 15:21:09 -05:00
Lucas Steuernagel 4753a8aefa
Move `add_builtin` to SVM (#547) 2024-04-09 16:55:09 -03:00