Commit Graph

25229 Commits

Author SHA1 Message Date
Mike Schem 0288e0db29
[anza migration]: update repo path (#811)
update repo path
2024-04-15 11:43:46 -06:00
behzad nouri 50f10284bb
allows gossip pull requests with new contact-info (#803)
Current code is only allowing gossip pull requests with legacy
contact-info:
https://github.com/anza-xyz/agave/blob/8c5a33a81/gossip/src/cluster_info.rs#L1958-L1966

Working towards migrating to the new contact-info, the commit allows
gossip pull requests with both legacy and new contact-infos.
2024-04-15 17:37:26 +00:00
Tyera 180a186c7d
Populate EpochRewards with correct data (#763)
* Populate EpochRewards::parent_blockhash correctly

* Populate EpochRewards::num_partitions correctly

* Add new internal struct to be extended

* Add CalculateValidatorRewardsResult::total_points

* Add PartitionedRewardsCalculation::total_points

* Add CalculateRewardsAndDistributeVoteRewardsResult::total_points

* Populate EpochRewards::total_points correctly

* Nit: reorder fields to match struct definition

* Use Self::last_blockhash() to get parent_blockhash
2024-04-15 10:16:02 -06:00
Jeff Washington (jwash) aed1a5e452
AccountAddressRange is byval (#807) 2024-04-15 10:09:10 -05:00
steviez 7138ea7517
Plumb CLI arg to control number of TVU receive threads/sockets (#550)
The parameter directly controls the number of sockets that are created;
the sockets later have one thread created per socket to listen.
2024-04-15 16:56:10 +02:00
Jeff Washington (jwash) e0b0bcc803
hot storage AccountIndexWriterEntry needs values (#808) 2024-04-15 09:43:52 -05:00
behzad nouri 2c0bea045c
skips some Merkle shreds tests in coverage CI (#804)
skips some Merkle shreds test in coverage CI
2024-04-15 14:34:15 +08:00
Brooks 2ffc7f64b7
Renames fns to get_account_shared_data() (#782) 2024-04-14 18:25:39 -04:00
Jeff Washington (jwash) 970d925a2e
hot owners accumulation needs to be byval (#779) 2024-04-14 17:12:31 -05:00
Kevin Heavey 76502b40d0
program: remove unused serde_json dep (#800)
remove unused serde_json dep from solana-program
2024-04-14 16:08:22 -06:00
Brooks 8c5a33a81a
Renames fns to get_stored_account_meta() (#783) 2024-04-13 14:01:51 -04:00
Tyera 97799e4826
Use same stakes for everything in get_epoch_reward_calculate_param_info (#787) 2024-04-13 10:30:28 -06:00
Brooks 864e044ea3
Refactors tiered storage tests (#776) 2024-04-13 10:11:34 -04:00
Jeff Washington (jwash) 1221dcf480
use `is_zero_lamport()` trait (#781) 2024-04-13 08:45:54 -05:00
Jeff Washington (jwash) 4e43820a0c
decouple StorableAccounts pubkey & hash from account (#780) 2024-04-13 08:45:35 -05:00
Justin Starry 09241ae9c3
Use reserved account keys list to restrict tx write locks (#541)
* Plumb through reserved account keys set

* Plumb through tests
2024-04-13 09:37:58 +08:00
Justin Starry cb13b39118
Track reserved account keys in bank (#769) 2024-04-13 09:12:00 +08:00
Brooks 45e09396b9
Renames tiered storage get_account() to get_stored_account_meta() (#774) 2024-04-12 15:46:27 -04:00
Tyera cf1299a871
Assortment of small partitioned-epoch rewards cleanups (#760)
* Limit pub of calculate_rewards_for_partitioning

* Wrap long string literal

* Remove sysvar account balance from log

* Update EpochRewards in account_decoder

* Add and use EpochRewards sysvar getter

* Restore update_epoch_rewards_sysvar panic
2024-04-12 12:55:53 -06:00
behzad nouri 5b4f1a0e6d
patches flaky Merkle shreds test (#775) 2024-04-12 18:22:39 +00:00
Brooks 3715830bcc
Adds get_account_shared_data() to tiered storage (#773) 2024-04-12 12:42:42 -04:00
HaoranYi 518a5b7df8
Change read-only-cache to be keyed on pubkey only (#576)
* change read-only-cache to keyed on pubkey only

* implement review feedbacks

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2024-04-12 09:25:19 -05:00
Alexander Meißner d448c4e5f0
Fix - `ProgramCache::assign_program()` Closed => Loaded in same slot (#673)
* Refactors test_add_builtin() to use LoadedProgram::new_builtin() and ProgramCache::extract().

* Revert Closed => Loaded transition

* Adds more loaded programs deployment tests
2024-04-12 14:59:51 +02:00
Alexander Meißner 81519fab2a
Fix - `test_add_builtin()` should use `LoadedProgram::new_builtin()` and `ProgramCache::extract()` (#772)
Refactors test_add_builtin() to use LoadedProgram::new_builtin() and ProgramCache::extract().
2024-04-12 14:58:23 +02:00
Ashwin Sekar 499d36e354
vote: update benches and tests to TowerSync (#725) 2024-04-11 22:15:02 -07:00
Illia Bobyr 4f7e45bb24
metrics: Submit metrics when exiting. Refactor `MetricsAgent::run()`. (#718)
There are a few minor issues this change addresses:

1. When we send points to the `MetricsWriter` we are calling
   `Instant::now()` twice, using the first result in the metrics stats,
   and using the seconds value for `last_write_time`.  Yet, on the next
   upload, we would use `last_write_time` as a reference point.

   We upload metrics using a network call, so it is far from
   instantaneous.  This creates a minor discrepancy in our time
   reporting.

   Good news is that we do not really need to call `Instant::now()`
   twice at all, as we can use the same value for both stats and
   `last_write_time`.

2. We did not report metrics stats if we did not have any points
   accumulated.  It seems better to always report metric stats,
   including when no points have been accumulated.  In practice, this
   does not happen for the validator, as validators always report
   something during a 10-second accumulation interval.

3. We did not upload any points when the metrics thread was existing.
   This may cause a short number of metrics not to be reported.

4. `collect_points()` was always converting both `points` and `counters`
   into a vector of `DataPoint`, even if the final length was over the
   specified `max_points`.  In the `mainnet-beta` we have values of up
   to 5m points lost, so it could be a small optimization if we drop
   them sooner.
2024-04-11 22:02:44 -07:00
Illia Bobyr dabcc39819
cli: Move `allow(clippy::arithmetic_side_effects)` into modules (#758)
The rest of the codebase has `clippy::arithmetic_side_effects` enabled.
While CLI is not as critical as the rest of the validator code base, it
seems nice to be precise about the arithmetic operations, and clippy
helps with that if `arithmetic_side_effects` is enabled.

A blanket disable at the module level is suboptimal.  As we have a lot
of operations already written, that generate warnings under
`arithmetic_side_effects`, the first step is to move the lint control
into individual modules, that we can then inspect one by one.
2024-04-11 22:01:53 -07:00
Justin Starry c4996d6e52
Remove stale poh recorder validator id (#700) 2024-04-12 12:09:02 +08:00
samkim-crypto b046c12cdc
[zk-token-sdk] Refactor `AuthenticatedEncryptionError` and `ElGamalError` to errors module (#589)
* make the `errors` module available in sbf targets

* move `AuthenticatedEncryptionError` to the `errors` module

* move `ElGamalError` to the `errors` module
2024-04-12 11:26:40 +09:00
dmakarov ad94732d4f
SVM: rename TransactionCheckResult variable for consistency (#759) 2024-04-11 12:46:23 -07:00
Jeff Washington (jwash) e9f8fcb4e6
use previously calculated `stored_size` (#757) 2024-04-11 14:36:53 -05:00
behzad nouri df1b8f983e
expands test coverage of chained Merkle shreds (#55) 2024-04-11 19:17:14 +00:00
Jeff Washington (jwash) baed522d2b
call StorableAccounts.pubkey once (#752) 2024-04-11 11:22:50 -05:00
carllin d5c291a934
Remove send snapshot hard unwrap (#326) 2024-04-11 12:18:52 -04:00
dependabot[bot] f6cac1e20f
build(deps): bump anyhow from 1.0.81 to 1.0.82 (#749)
* build(deps): bump anyhow from 1.0.81 to 1.0.82

Bumps [anyhow](https://github.com/dtolnay/anyhow) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/dtolnay/anyhow/releases)
- [Commits](https://github.com/dtolnay/anyhow/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: anyhow
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2024-04-11 16:01:46 +00:00
Kevin Heavey e04dbaa305
move ThreadExecuteTimings to solana-ledger (#690)
* move ThreadExecuteTimings to solana-ledger
* run cargo-for-all-lock-files tree
* move ThreadExecuteTimings below BatchExecutionTiming
* remove ReplaySlotStats to solana-ledger
* remove  `#![recursion_limit = "2048"]` from program-runtime
2024-04-11 17:59:16 +02:00
Brooks 8aa32172d6
Stops copying new PathBuf when querying storage files (#751) 2024-04-11 11:00:16 -04:00
behzad nouri 293414f482
pads last erasure batch with empty data shreds (#639)
For duplicate blocks prevention we want to verify that the last erasure
batch was sufficiently propagated through turbine. This requires
additional bookkeeping because, depending on the erasure coding schema,
the entire batch might be recovered from only a few coding shreds.

In order to simplify above, this commit instead ensures that the last
erasure batch has >= 32 data shreds so that the batch cannot be
recovered unless 32+ shreds are received from turbine or repair.
2024-04-11 14:50:43 +00:00
Ashwin Sekar 4fc6b0bd96
banking_stage: track VoteStorage size instead of costly iterator (#743)
* banking_stage: track VoteStorage size instead of costly iterator

* missed one

---------

Co-authored-by: steviez <steven@anza.xyz>
2024-04-11 07:16:33 -07:00
Lucas Steuernagel 4e2fc2772b
Add unit test for `add_builtin` (#720) 2024-04-11 11:13:23 -03:00
Brooks 2871c59863
Removes write version from store-tool (#722) 2024-04-11 10:09:36 -04:00
Tyera 9706a64646
Fix typo (#745) 2024-04-10 22:52:51 -06:00
Yihau Chen 0e6d42e613
bump nix to 0.28.0 (#628)
* bump nix to 0.28.0

* enable 'socket' for net-utils

* enable 'signal' for install

* enable 'user' for perf

* enable 'net' for streamer
2024-04-11 12:03:23 +08:00
Ashwin Sekar c4734ad127
vote: plumb TowerSync ix (#584) 2024-04-10 20:33:45 -07:00
ripatel-fd 16efe510cb
Don't panic vote program if epoch number overflows (#730)
Helps with fuzzing

Co-authored-by: Richard Patel <ripatel@jumptrading.com>
2024-04-11 02:56:19 +00:00
Alessandro Decina 55ab7fadbc
quic: use smallvec to aggregate chunks, save 1 alloc per packet (#735)
quic: use smallvec, save one allocation per packet

Use smallvec to hold chunks. Streams are packet-sized so we don't expect
them to have many chunks. This saves us an allocation for each packet.
2024-04-11 12:25:35 +10:00
Alessandro Decina 85c6e412e0
quic: do ordered reads, save 1 BTreeMap allocation per packet (#736)
quic: switch to ordered reads

Unordered reads cause a BTreeMap allocation for each packet inside quinn
in Assembler::ensure_ordering.

Most streams will fit in one datagram and will therefore be ordered by
definition. Switch to ordered reads to avoid the allocation.
2024-04-11 12:25:06 +10:00
Lijun Wang 92ebf0f80c
Treat super low staked as unstaked in streamer QOS (#701)
* Treat super low staked with QOS of unstaked

* simplify

* address some comment from Pankaj
2024-04-10 17:13:41 -06:00
samkim-crypto 006cf28dec
[zk-keygen] Remove deprecated functions (#599)
* add `deprecated` feature to produce warnings on use of deprecated functions

* replace `is_present` with `try_contains_id`

* replace validator functions

* replace `possible_values` with `value_parser`

* replace `value_of` function

* remove `deprecated` feature
2024-04-11 07:52:57 +09:00
Ashwin Sekar 411fdc9312
blockstore: scaffolding for chained merkle root conflict detection (#719)
* blockstore: scaffolding for chained merkle root conflict detection

* pr feedback: use dummy feature key until final plumbing
2024-04-10 22:46:33 +00:00