Commit Graph

26091 Commits

Author SHA1 Message Date
GroovieGermanikus 0432ba384f
Merge tag 'v2.0.18' into v2.0.18_mango_rpc_optimizations 2024-12-06 09:59:11 +01:00
GroovieGermanikus df46c3f598
RPC: allow IPv6 binding 2024-12-05 23:28:31 +01:00
GroovieGermanikus 83752afe17
RPC: add method getProgramAccountsCompressed 2024-12-05 23:28:23 +01:00
mergify[bot] f77014dcdf
v2.0: Marks old storages as dirty and uncleaned in clean_accounts() (backport of #3737) (#3747)
* Marks old storages as dirty and uncleaned in clean_accounts() (#3737)

(cherry picked from commit 31742ca61e7eff18570d7987b3fc7af9050d3303)

# Conflicts:
#	accounts-db/src/accounts_db.rs
#	accounts-db/src/accounts_db/tests.rs
#	runtime/src/bank.rs

* fixup merge conflict

---------

Co-authored-by: Brooks <brooks@anza.xyz>
2024-11-25 15:27:59 -05:00
github-actions[bot] ef8a9f504e
Bump version to v2.0.18 (#3753)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-22 19:29:11 -06:00
mergify[bot] 93e6e33ea5
v2.0: remove ntapi patch (backport of #3584) (#3665)
remove ntapi patch (#3584)

* use anza-xyz/ntapi instead of the labs one

* remove ntapi patch

* fix shellcheck

(cherry picked from commit da90cbee49d0e08d5124f0888d82b2f6774c27be)

Co-authored-by: Yihau Chen <yihau.chen@icloud.com>
2024-11-21 13:13:32 +08:00
mergify[bot] 4f8cf99e8e
v2.0: Fix broken parallelism in quic-client (backport of #2526) (#3709)
Fix broken parallelism in quic-client (#2526)

* Fix broken parallelism in quic-client

Fixes excessive fragmentation by TPU clients leading to a large
number of streams per conn in 'sending' state simultaneously.
This, in turn, requires excessive in-memory buffering server-side
to reassemble fragmented transactions.

- Simplifies QuicClient::send_batch to enqueue send operations
  in sequential order
- Removes the "max_parallel_streams" config option

The quic-client now produces an ordered fragment stream when
scheduling send operations from a single-thread.

* quic-client: remove outdated test

---------

Co-authored-by: Richard Patel <ripatel@jumptrading.com>
Co-authored-by: Alessandro Decina <alessandro.d@gmail.com>
(cherry picked from commit 76cbf1a91c)

Co-authored-by: ripatel-fd <ripatel+git@jumptrading.com>
2024-11-21 12:46:14 +11:00
mergify[bot] 7104d713cc
v2.0: rolls out chained Merkle shreds to devnet (backport of #3722) (#3725)
rolls out chained Merkle shreds to devnet (#3722)

(cherry picked from commit 8030d2dfd977cf3897500728149537118d2869cc)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-11-20 22:10:31 +00:00
mergify[bot] aab790b872
v2.0: Startup only sets latest_full_snapshot_slot if generating snapshots (backport of #3633) (#3667)
* Startup only sets latest_full_snapshot_slot if generating snapshots (#3633)

(cherry picked from commit 4ea10c224ae4a360db0011fb08d4b58d60d3a5a6)

# Conflicts:
#	accounts-db/src/accounts_db.rs
#	ledger/src/bank_forks_utils.rs

* fix backport merge conflict

---------

Co-authored-by: Brooks <brooks@anza.xyz>
2024-11-19 09:25:40 -05:00
github-actions[bot] 78ea8ffa14
Bump version to v2.0.17 (#3678)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-19 14:48:52 +08:00
mergify[bot] f5a88ce904
v2.0: simplify forward packet batches (backport of #3642) (#3648)
* simplify forward packet batches (#3642)

simplify forward packet batches - nonzero, no branching

(cherry picked from commit 2cb11a7e2cbe533d01f80cfa08f2c3b2a06ee01e)

# Conflicts:
#	core/src/banking_stage/forward_packet_batches_by_accounts.rs

* resolve conflicts

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-11-15 15:07:00 -06:00
mergify[bot] 79f23b6709
v2.0: Invoke ancient slots shrinking only if skipping rewrites is enabled (backport of #3266) (#3580)
Invoke ancient slots shrinking only if skipping rewrites is enabled (#3266)

(cherry picked from commit 8ff5dae418)

Co-authored-by: dmakarov <dmakarov@users.noreply.github.com>
2024-11-13 12:08:16 -05:00
mergify[bot] 2408949818
v2.0: program-runtime: double program cache size (backport of #3481) (#3494)
program-runtime: double program cache size (#3481)

The cache is currently getting thrashed and programs are getting
reloaded pretty much at every single slot. Double the cache size, which
makes reloading happen only due to random eviction sometimes picking a
popular entry.

The JIT code size with the new cache size is about 800MB.

This change reduces jit time 15x.

(cherry picked from commit fb4adda5a8a59d7eafe942157260c023b962925a)

Co-authored-by: Alessandro Decina <alessandro.d@gmail.com>
2024-11-09 10:16:09 +07:00
github-actions[bot] 2a3de521ce
Bump version to v2.0.16 (#3530)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-11-08 16:00:46 +08:00
Alexander Meißner f8f3fe31d9
Feature - disable account loader special case (#3514)
Feature gate disable_account_loader_special_case.
2024-11-07 21:36:42 +01:00
mergify[bot] 0348991a2c
v2.0: Store epoch in MaxAge (backport of #3485) (#3501)
* Store epoch in MaxAge (#3485)

(cherry picked from commit 5a6f518c6018ab0ebd101777045c5a1470cde5db)

# Conflicts:
#	core/src/banking_stage/consumer.rs
#	core/src/banking_stage/scheduler_messages.rs
#	core/src/banking_stage/transaction_scheduler/transaction_state.rs
#	core/src/banking_stage/transaction_scheduler/transaction_state_container.rs

* resolve conflicts

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-11-07 09:01:18 -06:00
mergify[bot] e1b66ab4f0
v2.0: Revert "rolls back chained Merkle shreds for testnet downgrade (#3194)" (backport of #3503) (#3504)
Revert "rolls back chained Merkle shreds for testnet downgrade (#3194)" (#3503)

This reverts commit 69916f1077.

(cherry picked from commit db4f2d651683262eb133a4c1802876904e4fc130)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-11-07 13:33:39 +00:00
Alexander Meißner 204d282e44
Revert - #879 and #768 (#3511)
* Revert "Cleanup - Removes the owner form the result of `filter_executable_program_accounts()` (#879)"

This reverts commit 5fe30cb788.

* Revert "Refactor - Remove `program_accounts_map` from account_loader (#768)"

This reverts commit e7617a1b1f.
2024-11-07 11:32:44 +01:00
Jon C 6f2b6f2471
CI: Pin downstream SPL job to v2.0 branch (#3384)
CI: Pin downstrem SPL job to v2.0 branch

##### Problem

SPL is about to update to use the v2.1 crates, which will cause the
downstream jobs to fail for branch v2.0.

#### Summary of changes

Checkout the `v2.0` branch from the SPL repo for the downstream tests.
2024-10-30 18:16:48 +00:00
Will Hickey 7feb24da35
Pin SPL Token CLI version to v4.1.1 (#3344) 2024-10-30 22:13:17 +08:00
mergify[bot] 6d74d13749
v2.0: runtime: use leader schedule epoch to serve `sol_get_epoch_stake` (backport of #3279) (#3312)
runtime: use leader schedule epoch to serve `sol_get_epoch_stake` (#3279)

Co-authored-by: Joe Caulfield <joe.caulfield@anza.xyz>
2024-10-29 10:36:11 +04:00
mergify[bot] 2788c950aa
v2.0: ci: remove deprecated macos-12 image (backport of #3202) (#3207)
ci: remove deprecated macos-12 image (#3202)

ci: use macos-13 for client_targets pipeline
(cherry picked from commit c6e8239843)

Co-authored-by: Yihau Chen <yihau.chen@icloud.com>
2024-10-24 17:09:18 +08:00
mergify[bot] aa44dc87c5
v2.0: Update latest release variable for agave docs (backport of #3220) (#3221)
Update latest release variable for agave docs (#3220)

Update latest release variable for agave

(cherry picked from commit df97398f0d)

Co-authored-by: Will Hickey <will.hickey@anza.xyz>
2024-10-21 09:53:54 -05:00
github-actions[bot] 135742fe3c
Bump version to v2.0.15 (#3210)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-18 21:19:04 +08:00
mergify[bot] 8561902444
v2.0: Scheduler: Improve TTL (backport of #3161) (#3179)
* Scheduler: Improve TTL (#3161)

Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
(cherry picked from commit 7b0a57316d)

# Conflicts:
#	accounts-db/src/accounts.rs
#	core/src/banking_stage/consumer.rs
#	core/src/banking_stage/immutable_deserialized_packet.rs
#	core/src/banking_stage/latest_unprocessed_votes.rs
#	core/src/banking_stage/transaction_scheduler/scheduler_controller.rs
#	runtime/src/bank/address_lookup_table.rs

* resolve conflicts

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-10-17 12:52:12 -05:00
mergify[bot] 192460ba9e
v2.0: rolls back chained Merkle shreds for testnet downgrade (backport of #3194) (#3195)
rolls back chained Merkle shreds for testnet downgrade (#3194)

(cherry picked from commit 69916f1077)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-10-16 16:44:14 +00:00
mergify[bot] 654cf17c68
v2.0: ledger-tool: Fix create-snapshot default value for output_directory (backport of #3148) (#3153)
ledger-tool: Fix create-snapshot default value for output_directory (#3148)

The arguments to specify full and incremental snapshot archives paths
used to be a global argument; these were moved to only be instantiated
on commands that needed them in #1773.

But, when the arguments were moved from app-level to subcommand-level,
the code that matches the arguments was not updated to look at
subcommand-matches instead of app-matches.

(cherry picked from commit 1d9947cd68)

Co-authored-by: steviez <steven@anza.xyz>
2024-10-15 10:41:18 -10:00
mergify[bot] b271f1ffbf
v2.0: token-2022: Use mainnet-beta binary in all tools (backport of #3165) (#3166)
token-2022: Use mainnet-beta binary in all tools (#3165)

#### Problem

The binary of token-2022 bundled in program-test and fetch-spl.sh is
very out of date with what's currently running on mainnet.

#### Summary of changes

Update the shared object and version to fetch.

(cherry picked from commit 96955661f7)

Co-authored-by: Jon C <me@jonc.dev>
2024-10-15 18:52:21 +02:00
mergify[bot] c70d9d3f31
v2.0: link openssl statically for Windows (backport of #3118) (#3156)
link openssl statically for Windows (#3118)

* ci: fix windows pipeline

* vendor openssl for windows

* add comment for the workaround

(cherry picked from commit 49d2298762)

Co-authored-by: Yihau Chen <yihau.chen@icloud.com>
2024-10-15 13:15:34 +08:00
github-actions[bot] 0b643611ac
Bump version to v2.0.14 (#3056)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-03 10:58:03 -05:00
Yihau Chen 4cc7b90484
v2.0: ignore tonic audit report (#3051)
* fix audit

* add more comment
2024-10-02 23:56:43 +08:00
github-actions[bot] 1715788830
Bump version to v2.0.13 (#3048)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-10-02 22:49:41 +08:00
mergify[bot] 65233160c4
v2.0: increase writeable accounts cost hashmap size (backport of #3009) (#3019)
increase writeable accounts cost hashmap size (#3009)

(cherry picked from commit 9f822c2f2c)

Co-authored-by: Brennan <brennan.watt@anza.xyz>
2024-10-01 10:49:23 -07:00
mergify[bot] 4f423a512a
v2.0: reworks max number of outgoing push messages (backport of #3016) (#3038)
reworks max number of outgoing push messages (#3016)

max_bytes for outgoing push messages is pretty outdated and does not
allow gossip to function properly with current testnet cluster size.

In particular it does not allow to clear out queue of pending push
messages unless the new_push_messages function is called very frequently
which involves repeatedly locking/unlocking CRDS table.

Additionally leaving gossip entries in the queue for the next round will
add delay to propagating push messages which can compound as messages go
through several hops.

(cherry picked from commit 489f483e1d)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-10-01 01:01:37 +00:00
mergify[bot] 573db9c5d8
v2.0: banking_stage: do not insert legacy vote ixs, refactor & unstaked (backport of #2888) (#2901)
* banking_stage: do not insert legacy vote ixs, refactor & unstaked (#2888)

* banking_stage: do not insert legacy vote ixs, refactor & unstaked

* pr feedback: use matches instead of separate fn

(cherry picked from commit 1334fb5248)

# Conflicts:
#	core/src/banking_stage/latest_unprocessed_votes.rs

* fix conflicts

* rekey feature to indicate it must not be activated

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2024-09-30 19:31:55 -04:00
mergify[bot] 246fe45d49
v2.0: removes early return if prune_messages are empty (backport of #3006) (#3011)
removes early return if prune_messages are empty (#3006)

Even if there are no outgoing prune messages we still need to generate
outgoing push messages for packets just received, so the code should not
early return here:
https://github.com/anza-xyz/agave/blob/d2cc71f0d/gossip/src/cluster_info.rs#L2400-L2402

(cherry picked from commit ce158213fd)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-09-30 20:36:04 +00:00
mergify[bot] 3aa2c0a6ef
v2.0: excludes node's pubkey from bloom filter of pruned origins (backport of #2990) (#3015)
excludes node's pubkey from bloom filter of pruned origins (#2990)

Bloom filter of pruned origins can return false positive for a node's
own pubkey but a node should always be able to push its own values to
other nodes in the cluster.

(cherry picked from commit bce28c0282)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-09-30 15:17:25 +00:00
Yihau Chen 0ab59bff26
v2.0: ci: increase timeout for check3 (#3017) 2024-09-28 10:05:04 -05:00
github-actions[bot] 3ade64bd13
Bump version to v2.0.12 (#3012)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-27 16:53:32 -05:00
mergify[bot] d868218e08
v2.0: Fix: Corrected the derivation command format (backport of #2952) (#3005)
Fix: Corrected the derivation command format (#2952)

(cherry picked from commit d2cc71f0d4)

Co-authored-by: Asten <dandelionlion@protonmail.com>
2024-09-27 11:59:00 -05:00
mergify[bot] 77ed6dbe84
v2.0: rolls out chained Merkle shreds to 100% of testnet slots (backport of #2858) (#2995)
rolls out chained Merkle shreds to 100% of testnet slots (#2858)

(cherry picked from commit a15fef1b26)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-09-27 14:13:49 +00:00
mergify[bot] 9e90b001e9
v2.0: fix: set allocation size to 0 for transactions known to fail (backport of #2966) (#2989)
fix: set allocation size to 0 for transactions known to fail (#2966)

(cherry picked from commit 443246dee0)

Co-authored-by: Justin Starry <justin@anza.xyz>
2024-09-27 14:20:12 +08:00
github-actions[bot] 4255f52d4a
Bump version to v2.0.11 (#2942)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-25 14:08:51 -05:00
mergify[bot] 57227ea058
v2.0: Publish Agave docs (backport of #2887) (#2899)
Publish Agave docs (#2887)

* Add publish-docs.sh to build.sh

* Add debugging echos

* Fake being in CI

* Update vercel doc project names

* Remove debugging echos. Remove CI var.

(cherry picked from commit 383b49fdc8)

Co-authored-by: Will Hickey <will.hickey@anza.xyz>
2024-09-18 21:31:11 -05:00
mergify[bot] 1b5c6140e2
v2.0: remove disable_block_production_forwarding cli flag (backport of #2687) (#2909)
remove disable_block_production_forwarding cli flag (#2687)

(cherry picked from commit df892c4241)

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-09-12 12:33:40 -05:00
mergify[bot] c01560e136
v2.0: scheduler opt-in forwarding (backport of #1801) (#2285)
* scheduler opt-in forwarding (#1801)

(cherry picked from commit 61d8be0d6f)

* Scheduler: buffer packets for forwarding if forwarding is enabled (#2305)

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-09-12 10:43:21 -05:00
mergify[bot] 1b4dfb3a03
v2.0: extends Turbine fanout experiment to wider fanout values (backport of #2373) (#2450)
extends Turbine fanout experiment to wider fanout values (#2373)

Based on previous Turbine fanout experiment, wider fanouts are more
effective in propagating shreds and reducing repairs:
https://discord.com/channels/428295358100013066/478692221441409024/1265782094211321897

In order to identify optimal fanout value, this commit extends the
experiment with wider fanout values.

(cherry picked from commit 57144b0cea)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-09-10 14:39:26 +00:00
mergify[bot] 81c310a28a
v2.0: Remove explicit stream finish (backport of #2760) (#2766)
Remove explicit stream finish (#2760)

(cherry picked from commit 61d6a22a47)

Co-authored-by: Lijun Wang <83639177+lijunwangs@users.noreply.github.com>
2024-09-08 22:53:43 +07:00
github-actions[bot] b55220fd88
Bump version to v2.0.10 (#2864)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-08 14:52:38 +08:00
mergify[bot] ced98f1ebe
v2.0: rolls out chained Merkle shreds to ~50% of testnet slots (backport of #2660) (#2833)
rolls out chained Merkle shreds to ~50% of testnet slots (#2660)

(cherry picked from commit f2b7ef4aea)

Co-authored-by: behzad nouri <behzadnouri@gmail.com>
2024-09-04 19:43:21 +00:00