Commit Graph

21875 Commits

Author SHA1 Message Date
Benno Fünfstück e8c43aa0d1
ledger-tool: Notify geyser of transactions (#29933)
ledger-tool: Notify geyser of transactions
2023-02-09 12:34:20 -08:00
ivandzen 2f9146e8c8
Replace txn_signature in ReplicaAccountInfo with transaction (#30189)
#### Current implementation of ReplicaAccountInfo restricts
possibilities for inflight account filtration
Current implementation includes transaction signature in
ReplicaAccountInfo. This approach does not allow to filter accounts by
matching other accounts in transaction in-flight (e. g. accept only
those accounts included in transactions for specific programs). Current
implementation forces to collect ALL accounts and transactions for some
period of time and perform such complex filtration afterwards.

#### Pass reference to transaction object instead of transaction
signature into ReplicaAccountInfo
Advanced in-flight filtration can be implemented in plugins by passing
reference to transaction for every update_account event. This change
doesn't bring any overhead comparing to current implementation (only
data type of reference is changed) and brings only minor changes in
source code.
2023-02-09 12:00:33 -08:00
Brennan da34f2edd0
Fix duplicate broadcast test (#30209) 2023-02-09 11:58:54 -08:00
Jeff Washington (jwash) c6695996b8
add test parameters to fix build (#30227)
#### Problem
build broken due to pr conflicts

#### Summary of Changes


Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:56:12 -08:00
Jeff Washington (jwash) 62b93df1c7
change ancient tests to correct size (#30221)
#### Problem
fix tests to be more realistic

#### Summary of Changes
allocate correctly sized append vecs

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:41:36 -08:00
Jeff Washington (jwash) 99937af69d
test fn append_single_account_with_default_hash can add to index (#30219)
#### Problem
When adding test accounts to storages, it is useful to be able to make
the accounts alive in the index, too.

#### Summary of Changes
test fn append_single_account_with_default_hash can add to index

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:37:02 -08:00
Jeff Washington (jwash) e0d354f542
fix bug in test fn compare_all_accounts (#30217)
#### Problem
test code compares sets of accounts after clean/shrink operations.
The indexing was off and new tests were giving false positives.
The test code was equivalent as long as everything was in the same
order.

#### Summary of Changes
Fix test code comparison indexes.

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:28:53 -08:00
dependabot[bot] 57dced05f0
Bump sha2 from 0.10.5 to 0.10.6 (#30215)
Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.10.5 to
0.10.6.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-02-09 12:18:09 -07:00
Jeff Washington (jwash) 68b2b1badd
add ancient calc_accounts_to_combine (#30195)
#### Problem
Building new algorithm for packing ancient storage. Packing will occur
in 1 pass across multiple ancient slots.
This will be put in 1 dead code piece at a time with tests until all
pieces are present. Switch between current packing algorithm and this
new one is in a validator cli argument. Resulting append vecs are
correct and compatible (as a set) either way. When a new storage format
optimized for cold storage becomes available, it will only work with
this new packing algorithm, so the change will need to be complete prior
to the new storage format.

#### Summary of Changes
Add `ancient calc_accounts_to_combine` to separate accounts to prepare
for creating packed ancient append vecs. This will be used soon.

Fixes #
<!-- OPTIONAL: Feature Gate Issue: # -->
<!-- Don't forget to add the "feature-gate" label -->
2023-02-09 11:06:17 -08:00
dependabot[bot] 2b4a6a4dd9
Bump libloading from 0.7.3 to 0.7.4 (#30216)
Bumps [libloading](https://github.com/nagisa/rust_libloading) from 0.7.3
to 0.7.4.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="6e284984ae"><code>6e28498</code></a>
Fix CI</li>
<li><a
href="95a0f62e8f"><code>95a0f62</code></a>
Placate clippy</li>
<li><a
href="224a3def35"><code>224a3de</code></a>
Release 0.7.4</li>
<li><a
href="bd17713dcc"><code>bd17713</code></a>
Support AIX dyld constants</li>
<li><a
href="6e07929736"><code>6e07929</code></a>
fix typo</li>
<li><a
href="6b33651a90"><code>6b33651</code></a>
Construct a PathBuf</li>
<li><a
href="964dd8b4c0"><code>964dd8b</code></a>
Simplify lib_path</li>
<li><a
href="8009f51a9f"><code>8009f51</code></a>
Honor Cargo's target directory when building LIBPATH</li>
<li><a
href="842e377ae4"><code>842e377</code></a>
Fix a typo in the CI scripts</li>
<li>See full diff in <a
href="https://github.com/nagisa/rust_libloading/compare/0.7.3...0.7.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=libloading&package-manager=cargo&previous-version=0.7.3&new-version=0.7.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-02-09 10:53:18 -07:00
Andrew Fitzgerald 058738424d
BankingStage Refactor: transaction recorder record transactions (#30106) 2023-02-09 08:34:02 -08:00
behzad nouri b3887af7c6
flattens the internal buffer in the duplicate-shreds handler (#30196)
Duplicate-shreds handler is using a nested hash-map for the incomplete
chunks buffered. This is resulting in a convoluted logic to limit the
number of entries:
https://github.com/solana-labs/solana/blob/427bd6264/gossip/src/duplicate_shred_handler.rs#L62

This commit instead uses a flat buffer mapping (Slot, Pubkey) pairs to
the respective duplicate shreds chunks. The buffer is allowed to grow to
twice the intended capacity, at which point the extraneous entries are
removed in linear time, resulting an amortized O(1) performance.
2023-02-09 16:33:20 +00:00
Jeff Washington (jwash) a4fb1834d1
AliveAccounts includes source slot (#30208) 2023-02-09 05:57:58 -08:00
Alexander Meißner f7dfe8337a
Moves `if use_jit` outside of `#[cfg()]`. (#30202) 2023-02-09 09:14:26 +01:00
dependabot[bot] 4d599a8362
Bump rand_core from 0.6.3 to 0.6.4 (#30114)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-02-08 21:27:26 -07:00
dependabot[bot] 46293ec43f
Bump openssl from 0.10.41 to 0.10.45 (#30115)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-02-08 21:26:43 -07:00
Nick Frostbutter 4a10ebdf0b
[docs] style: prettier force line length to 80 chars (#30133) 2023-02-09 03:23:21 +00:00
Nick Frostbutter ae59195ed0
[docs] fix: rpc api redirect (#30212) 2023-02-08 20:40:05 -06:00
steviez d3dab24bbe
chore: Use `i` over `ix` variable name when naming worker threads (#30206) 2023-02-09 01:24:57 +00:00
behzad nouri 1ad69cfc38
removes dynamic cast and dynamic dispatch from connection-cache (#30128)
Dynamic dispatch forces heap allocation and adds extra overhead.
Dynamic casting as in the ones below, lacks compile-time type safety:
https://github.com/solana-labs/solana/blob/eeb622c4e/quic-client/src/lib.rs#L172-L175
https://github.com/solana-labs/solana/blob/eeb622c4e/udp-client/src/lib.rs#L52-L55

The commit removes all instances of Any, Box<dyn ...>, and Arc<dyn ...>,
and instead uses generic and associated types.

There are only two protocols QUIC and UDP; and the code which has to
work with both protocols can use a trivial thin enum wrapper.

With respect to connection-cache specifically:
* connection-cache/ConnectionCache is a single protocol cache which
  allows to use either QUIC or UDP without any build dependency on the
  other protocol.
* client/ConnectionCache is an enum wrapper around both protocols and
  can be used in the code which has to work with both QUIC and UDP.

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2023-02-09 00:50:44 +00:00
Jon Cinque 0481ce3069
sdk: Make `declare_id` mostly const (#30177) 2023-02-09 01:26:09 +01:00
steviez 55c8963462
Append integer to solBigTGetBlk threads to uniquify them (#30184) 2023-02-09 00:23:29 +00:00
Tao Zhu 5569946a8f
correct a comment with right info (#30201) 2023-02-08 17:33:28 -06:00
Pankaj Garg 375f9ae41d
LoadedPrograms cache implementation and tests (#30139) 2023-02-08 13:24:44 -08:00
Dmitri Makarov 1896d4110f
Bump sbf-tools version (#30186) 2023-02-08 13:20:26 -08:00
Jeff Washington (jwash) 0ca638648b
add shrink_can_be_active to remove (#30191) 2023-02-08 11:32:18 -08:00
Jeff Washington (jwash) b4fe1280b3
add Debug to some structs (#30193) 2023-02-08 10:17:07 -08:00
Jeff Washington (jwash) 427bd62642
handle_dropped_roots_for_ancient takes iter (#30190) 2023-02-08 09:40:49 -08:00
Jeff Washington (jwash) 0da4697b2f
add ancient write_ancient_accounts (#30182) 2023-02-08 09:40:17 -08:00
behzad nouri 544fbded07
removes wallclock from duplicate-shreds handler (#30187) 2023-02-08 17:29:30 +00:00
Jeff Washington (jwash) 5cbd6b3a74
minor cleanup (#30181) 2023-02-08 08:00:14 -08:00
Jeff Washington (jwash) 70d635127e
add `ancient collect_sort_filter_info` (#30169) 2023-02-08 07:08:20 -08:00
kirill lykov 10dde654e7
add client-type cli option documentation (#30170) 2023-02-08 10:56:25 +01:00
kirill lykov 9f35ffc639
add http to the url in remote-client script (#30167) 2023-02-08 10:56:10 +01:00
Illia Bobyr cf77f5dbb8
doc: ledger: Document `completed_data_sets_service` module (#30001) 2023-02-07 21:20:09 -08:00
Illia Bobyr 6bc566d802
doc: ledger: Document `CompletedDataSetInfo` (#29998) 2023-02-07 21:19:07 -08:00
Jeff Washington (jwash) e14d0638e7
add ancient get_unique_accounts_from_storage_for_combining_ancient_slots (#30176) 2023-02-07 18:22:54 -08:00
Pankaj Garg d0623a38c8
API to match account owner against a set of owners (#30154) 2023-02-07 17:48:10 -08:00
Jeff Washington (jwash) bfc29a6b2f
remove storage_finder from write_accounts_to_storage (#30172)
Co-authored-by: Trent Nelson <trent@solana.com>
2023-02-07 14:44:40 -08:00
Trent Nelson 5494146413
ci: update openssl-src to 111.25 as per RUSTSEC-2023-0007 (#30173) 2023-02-07 15:22:48 -07:00
Jeff Washington (jwash) 1eb304de28
NonZeroU64 for ideal_storage_size (#30168) 2023-02-07 13:34:27 -08:00
Athar Mohammad 78f9d8b66d
Fix camelCase response for RpcPrioritizationFee (#30162) 2023-02-07 13:42:58 -07:00
Jeff Washington (jwash) 927da903a8
StoreTo::Storage holds storage (#30156) 2023-02-07 11:54:26 -08:00
Jeff Washington (jwash) d53c49c230
add ancient filter_ancient_slots (#30152) 2023-02-07 10:35:40 -08:00
Andrew Fitzgerald 2b99756b3e
BankingStage Refactor: Move counters out of record_transactions (#30093)
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2023-02-07 07:45:50 -08:00
HaoranYi 3f913e49fc
refactor rent payment order cmp (#30132) 2023-02-07 09:30:48 -06:00
HaoranYi 5dd90268b1
comments fix (#30134)
Co-authored-by: haoran <haoran@mbook>
2023-02-07 09:22:08 -06:00
Alexander Meißner d387f15587
Refactor - Remove redundant `use_jit` flag from `BpfExecutor` (#30116) 2023-02-07 10:05:54 +01:00
Michael Vines b8a8ca7c2b
Mergify: include destination branch in backport PR title (#30153) 2023-02-06 16:13:54 -08:00
Jeff Washington (jwash) db4174cbcb
ancient choose_storages_to_shrink (#30146) 2023-02-06 14:29:54 -08:00