Commit Graph

4819 Commits

Author SHA1 Message Date
GroovieGermanikus 0432ba384f
Merge tag 'v2.0.18' into v2.0.18_mango_rpc_optimizations 2024-12-06 09:59:11 +01:00
GroovieGermanikus 83752afe17
RPC: add method getProgramAccountsCompressed 2024-12-05 23:28:23 +01:00
mergify[bot] f77014dcdf
v2.0: Marks old storages as dirty and uncleaned in clean_accounts() (backport of #3737) (#3747)
* Marks old storages as dirty and uncleaned in clean_accounts() (#3737)

(cherry picked from commit 31742ca61e7eff18570d7987b3fc7af9050d3303)

# Conflicts:
#	accounts-db/src/accounts_db.rs
#	accounts-db/src/accounts_db/tests.rs
#	runtime/src/bank.rs

* fixup merge conflict

---------

Co-authored-by: Brooks <brooks@anza.xyz>
2024-11-25 15:27:59 -05:00
mergify[bot] 79f23b6709
v2.0: Invoke ancient slots shrinking only if skipping rewrites is enabled (backport of #3266) (#3580)
Invoke ancient slots shrinking only if skipping rewrites is enabled (#3266)

(cherry picked from commit 8ff5dae418)

Co-authored-by: dmakarov <dmakarov@users.noreply.github.com>
2024-11-13 12:08:16 -05:00
Alexander Meißner f8f3fe31d9
Feature - disable account loader special case (#3514)
Feature gate disable_account_loader_special_case.
2024-11-07 21:36:42 +01:00
mergify[bot] 6d74d13749
v2.0: runtime: use leader schedule epoch to serve `sol_get_epoch_stake` (backport of #3279) (#3312)
runtime: use leader schedule epoch to serve `sol_get_epoch_stake` (#3279)

Co-authored-by: Joe Caulfield <joe.caulfield@anza.xyz>
2024-10-29 10:36:11 +04:00
mergify[bot] 8561902444
v2.0: Scheduler: Improve TTL (backport of #3161) (#3179)
* Scheduler: Improve TTL (#3161)

Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
(cherry picked from commit 7b0a57316d)

# Conflicts:
#	accounts-db/src/accounts.rs
#	core/src/banking_stage/consumer.rs
#	core/src/banking_stage/immutable_deserialized_packet.rs
#	core/src/banking_stage/latest_unprocessed_votes.rs
#	core/src/banking_stage/transaction_scheduler/scheduler_controller.rs
#	runtime/src/bank/address_lookup_table.rs

* resolve conflicts

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-10-17 12:52:12 -05:00
mergify[bot] 573db9c5d8
v2.0: banking_stage: do not insert legacy vote ixs, refactor & unstaked (backport of #2888) (#2901)
* banking_stage: do not insert legacy vote ixs, refactor & unstaked (#2888)

* banking_stage: do not insert legacy vote ixs, refactor & unstaked

* pr feedback: use matches instead of separate fn

(cherry picked from commit 1334fb5248)

# Conflicts:
#	core/src/banking_stage/latest_unprocessed_votes.rs

* fix conflicts

* rekey feature to indicate it must not be activated

---------

Co-authored-by: Ashwin Sekar <ashwin@anza.xyz>
Co-authored-by: Ashwin Sekar <ashwin@solana.com>
2024-09-30 19:31:55 -04:00
mergify[bot] a829deae22
v2.0: SIMD-0118: fix `total_rewards` for recalculation (backport of #2780) (#2794)
* SIMD-0118: fix `total_rewards` for recalculation (#2780)

* Add new feature key

* Wrap existing code with new feature

* Extend test harness

* Make test fail

* Populate EpochRewards::total_rewards from PointValue

* Remove superfluous struct field

* Fixup tests

(cherry picked from commit 4470f6d96e)

# Conflicts:
#	programs/bpf_loader/src/syscalls/mod.rs
#	sdk/src/feature_set.rs

* Fix conflicts

---------

Co-authored-by: Tyera <tyera@anza.xyz>
2024-09-03 10:31:12 -06:00
mergify[bot] 3b0cdbf373
v2.0: Runtime: Core BPF: Add test for CPI post-migration (backport of #2531) (#2665)
Runtime: Core BPF: Add test for CPI post-migration (#2531)

(cherry picked from commit 8f675ebcfe)

Co-authored-by: Joe C <joe.caulfield@anza.xyz>
2024-08-20 08:45:31 -04:00
mergify[bot] e00d83b0cc
v2.0: Runtime: Core BPF Migration: Add checks for executable program account (backport of #2483) (#2516)
Runtime: Core BPF Migration: Add checks for executable program account (#2483)

* Runtime: Core BPF: check `executable` on program load

* Runtime: Core BPF: set `executable` on migration

(cherry picked from commit 33119c5df7)

Co-authored-by: Joe C <joe.caulfield@anza.xyz>
2024-08-19 17:02:12 -04:00
mergify[bot] e86b3cdcfc
v2.0: fix: borrow stakes delegation during snapshot serialization (backport of #2455) (#2500) 2024-08-14 07:41:36 +08:00
mergify[bot] b57b883355
v2.0: Fix BankForks::new_rw_arc memory leak (backport of #1893) (#2066)
* Fix BankForks::new_rw_arc memory leak (#1893)

(cherry picked from commit d441c0f577)

* Fix flaky test test_banking_stage_entries_only_central_scheduler (#2082)

---------

Co-authored-by: Andrei Silviu Dragnea <andreisilviudragnea@gmail.com>
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-07-17 16:11:40 -07:00
mergify[bot] e24b410429
v2.0: Clean up disable_fees_sysvar feature (backport of #2003) (#2043)
Clean up disable_fees_sysvar feature (#2003)

* Clean up disable_fees_sysvar

* Remove bank fees test

* Remove unused rpc response types

* Fixup sysvar_cache tests

* Remove fees-sysvar from bootstrap_validator_stake_lamports; no longer included

* Update account counts, since fees no longer touched

* Update bank hashes in test, since fees sysvar is not populated

* Remove test case passing fees sysvar as account

(cherry picked from commit 1c34908d03)

Co-authored-by: Tyera <tyera@anza.xyz>
2024-07-08 18:14:05 -06:00
mergify[bot] ec08d81213
v2.0: Remove deprecated symbols from recent_blockhashes_account (backport of #2004) (#2009)
Remove deprecated symbols from recent_blockhashes_account (#2004)

* Move recent_blockhashes_account to runtime

* Fixup imports

* Reduce pub

* Remove unused method

* Duplicate code to limit pub and dependency complexity

* Move test-only fn into tests module

(cherry picked from commit f77658b9e4)

Co-authored-by: Tyera <tyera@anza.xyz>
2024-07-08 10:35:24 -06:00
mergify[bot] 3aee1eaf5e
v2.0: Remove deprecated CommitmentLevel variants (backport of #1903) (#1989)
Remove deprecated CommitmentLevel variants (#1903)

* Remove deprecated CommitmentLevel variants and CommitmentConfig methods

* Remove deprecated variants from BlockCommitmentCache usage

* Remove deprecated variants from rpc

* Remove maybe_map_commitment from RpcClient

(cherry picked from commit 2e3c2c78e9)

Co-authored-by: Tyera <tyera@anza.xyz>
2024-07-08 10:34:13 -06:00
mergify[bot] 5d106ec0b1
v2.0: Remove deprecated, unused methods from Bank (backport of #1980) (#1988)
Remove deprecated, unused methods from Bank (#1980)

Remove deprecated methods from Bank (not in use)

(cherry picked from commit 0313047046)

Co-authored-by: Tyera <tyera@anza.xyz>
2024-07-03 19:33:57 +00:00
mergify[bot] 6d8549e3d6
v2.0: Remove deprecated SyncClient methods (backport of #1902) (#1909)
Remove deprecated SyncClient methods (#1902)

* Remove deprecated methods from SyncClient trait

* Add line to changelog

(cherry picked from commit 3d3faf5dee)

Co-authored-by: Tyera <tyera@anza.xyz>
2024-06-28 16:34:51 -06:00
mergify[bot] a0934f5d15
v2.0: Avoid unneeded start_session() with cleanups (bp: #1815, #1861) (#1854)
* Avoid unneeded start_session() when spawning (#1815)

* Avoid unneeded start_session() when spawning

* Add comments

(cherry picked from commit 40a9851c82)

* Apply cosmetic changes to unified scheduler (#1861)

* Apply cosmetic changes to unified scheduler

* Use first instead of old-fashioned firstly

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>

---------

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Co-authored-by: Andrew Fitzgerald <apfitzge@gmail.com>
2024-06-28 12:19:04 +09:00
Joe C 0f956c8ba7
SVM: Refactor program match criteria (#1784)
* SVM: hoist `program_modification_slot` up from bank

* SVM: add `check_program_modification_slot` to processing config

* SVM: hoist `program_match_criteria` up from bank

* SVM: drop `get_program_match_critera` from callbacks
2024-06-21 12:21:26 -05:00
Ryo Onodera 114d94a25e
Improve SchedulerStatus code and test as follow-up (#1797)
* Improve SchedulerStatus code and test as follow-up

* Don't use wait_timeout_while with magic number
2024-06-21 23:21:32 +09:00
Justin Starry 66e3acce45
Rename tx counts from committed to executed (#1807) 2024-06-20 13:13:47 -07:00
Justin Starry 8da4062644
refactor: consolidate fee deduction for failed transactions (#1636) 2024-06-20 08:03:40 -07:00
Alexander Meißner eceaf3433b
CI - SBPF program build script (#1581)
* Replaces the rust build script by a makefile
2024-06-20 13:40:00 +02:00
Alessandro Decina 2107adcf35
bpf_loader: use an explicit thread-local pool for stack and heap memory (#1370)
* Rename ComputeBudget::max_invoke_stack_height to max_instruction_stack_depth

The new name is consistent with the existing
ComputeBudget::max_instruction_trace_length.

Also expose compute_budget:MAX_INSTRUCTION_DEPTH.

* bpf_loader: use an explicit thread-local pool for stack and heap memory

Use a fixed thread-local pool to hold stack and heap memory. This
mitigates the long standing issue of jemalloc causing TLB shootdowns to
serve such frequent large allocations.

Because we need 1 stack and 1 heap region per instruction, and the
current max instruction nesting is hardcoded to 5, the pre-allocated
size is (MAX_STACK + MAX_HEAP) * 5 * NUM_THREADS. With the current
limits that's about 2.5MB per thread. Note that this is memory that
would eventually get allocated anyway, we're just pre-allocating it now.

* programs/sbf: add test for stack/heap zeroing

Add TEST_STACK_HEAP_ZEROED which tests that stack and heap regions are
zeroed across reuse from the memory pool.
2024-06-20 08:57:11 +07:00
Pankaj Garg ad0a48c731
SVM: Reduce public visibility of sysvar_cache (#1783) 2024-06-18 10:53:57 -07:00
Joe C 2dc95087ae
SVM: Move `fee_structure` to environment input (#1771)
* SVM: add `fee_structure` to environment arg

* runtime: add `fee_structure` to bank

* SVM: drop `fee_structure` from global configs
2024-06-18 10:07:22 -05:00
Ryo Onodera d9fcfeb12a
Don't panic when pausing stale unified schedulers (#1761) 2024-06-18 11:44:16 +09:00
samkim-crypto 563d1c04d6
[sdk] Add ZK Elgamal Proof program feature gate (#1679)
* add `zk-elgamal-proof-program-enabled` feature gate

* reserve account for zk elgamal proof program

* add zk elgamal proof as builtin program

* cargo lock

* cargo sort

* replace feature gate key to start with `zk..`

* add simd number

* update `elgamal_program` to `zk_elgamal_proof_program`
2024-06-18 11:34:34 +09:00
Tyera 034cd7396a
Add num_partitions to Blockstore rewards (#1601)
* Add num_partitions field to Rewards proto definition

* Add type to hold rewards plus num_partitions

* Add Bank method to get rewards plus num_partitions for recording

* Update Blockstore::write_rewards to use num_partitions

* Update RewardsRecorderService to handle num_partitions

* Populate num_partitions in ReplayStage::record_rewards

* Write num_partitions to Bigtable

* Reword KeyedRewardsAndNumPartitions method

* Clone immediately

* Determine epoch boundary by checking parent epoch

* Rename UiConfirmedBlock field

* nit: fix comment typo

* Add test_get_rewards_and_partitions

* Add pre-activation test

* Add should_record unit test
2024-06-17 16:42:33 -06:00
Pankaj Garg 2e54ed280b
SVM: Unify different instances of epoch_schedule in SVM and Bank into one (#1736) 2024-06-17 15:18:34 -07:00
Joe C 3af8f0cfdb
Program Runtime: Unify transaction batch program caches (#1399)
* local program cache: add `modified_entries` field

* use `modified_entries` for modified program cache

* invoke context: make `program_cache_for_tx_batch` mutable

* invoke context: unify local program cache instances

* remove `find_program_in_cache` alias
2024-06-17 17:08:13 -05:00
Tyera 863137135a
Simd-118: [HAL-04] simplify assertion that epoch contains enough slots for partitions (#1769)
Simplify assertion that epoch contains enough slots for partitions
2024-06-17 15:41:47 -06:00
Tyera c1bfb28d83
Simd-118: [HAL-01] add log to unreachable macro (#1767)
Add log to unreachable macro
2024-06-17 20:22:45 +00:00
Tyera 24d0bd850e
Simd-118: [HAL-02] add lamports burned to error log (#1768)
Add lamports burned to error log
2024-06-17 20:03:57 +00:00
galactus a6b2283142
ScanConfig defaults no longer sort results (#1539) 2024-06-14 08:27:33 -04:00
Joe C a108498af1
SVM: Group transaction processing environment (#1650)
* SVM: introduce `TransactionProcessingEnvironment`

* SVM: fold `blockhash` and `lamports_per_signature` into env

* SVM: fold `rent_collector` into env

* SVM: fold `feature_set` into env

* SVM: fold `epoch_stake` into env
2024-06-14 00:02:50 -05:00
Tyera d319d0d1be
Reorganize partitioned_epoch_rewards::tests fixtures (#1719)
* Move test fixtures up

* Update test_rewards_computation_and_partitioned_distribution_two_blocks

* Update test_rewards_computation_and_partitioned_distribution_one_block
2024-06-13 11:45:44 -06:00
Brooks 29a36b66c1
Fixes test_add_builtin_account() (#1727) 2024-06-13 13:36:37 -04:00
Ryo Onodera 334221503c
Return back stale out-of-pool scheduler by timeout (#1690)
* Return back stale out-of-pool scheduler by timeout

* Add tests

* Document and proper panic messages

* Avoid recursion

* Tweak comments
2024-06-13 12:48:49 +09:00
Joe C 5148dcccf7
SVM: change modified programs result to hash map (#1391)
* SVM: change modified programs result to hash map

* take entries
2024-06-12 22:54:29 +00:00
Joe C 6357f4ee76
Feature-Gate: Migrate Address Lookup Table program to Core BPF (#1380)
feature-gate: migrate address lookup table to Core BPF
2024-06-12 16:09:07 -05:00
Jeff Washington (jwash) 1bd9bd1a10
file based append vecs (#1394)
* file based storages

* pr feedback

* add comments

* add tests for buffered reader and file reader

* fix windows clippy

* pr: update tests for jeff comments

* pr feedback

* fix a bug - read at least default min bytes

* Revert "fix a bug - read at least default min bytes"

This reverts commit 52ccd8e022dd8a047596b95b4ab2abc91c7c983c.

* add test coverages flush, reset, reopen functions for append_vec.rs

* one more dead code

* renames

* renames

* renames

* wee wah wee wah grammar police

* rename

* add debug assert

* use const

* const

* reorder

* reorders and renames

---------

Co-authored-by: HaoranYi <haoran.yi@solana.com>
2024-06-12 15:26:47 -05:00
Joe C ecf72523ab
SVM: Dissolve `RuntimeConfig` (#1590)
* SVM: add `compute_budget` to processing config

* bank: add `compute_budget` field

* SVM: add `transaction_account_lock_limit` to processing config

* bank: add `transaction_account_lock_limit` field

* bank: drop `runtime_config` getter

* SVM: require `compute_budget` for `prepare_program_cache_for_upcoming_feature_set`

* SVM: drop `runtime_config` from batch processor
2024-06-12 12:04:39 -05:00
Sean Young 873808cf21
Add option to record transactions to ledger-tool (#181) 2024-06-12 11:43:53 +01:00
Pankaj Garg 3e077b7350
Use try_read() in get_environments_for_epoch() to prevent recursive read (#1621)
* move load_program to dev-context-only-utils

* more changes to use dev-context-only-utils
2024-06-11 12:53:03 -07:00
Brooks 52b82eb355
bank hash details supports skipped rewrites (#1648) 2024-06-11 12:16:31 -04:00
Justin Starry e4b0055e52
Validate transaction fee payer earlier (#1529) 2024-06-11 07:49:11 -07:00
Joe C b1508010c0
Syscall: `GetEpochStake` (#1152)
* SVM: add `get_vote_accounts` callback

* program-runtime: invoke context: add `vote_accounts`

* sdk: add feature gate for sol-get-epoch-stake

* syscall: get-epoch-stake

* syscall: get-epoch-stake: tests

* drop `get_compute_meter`

* refactor for total stake

* refactor API for total stake
2024-06-11 09:28:31 -05:00
Andrew Fitzgerald 10e0479f40
Prepare unlocked batch from single tx - reduce alloc (#1680)
prepare_unlocked_batch_from_single_tx - remove allocation
2024-06-11 05:38:52 +00:00