blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
18,689 Commits 181 Branches 498 Tags 574 MiB
Commit Graph

2663 Commits

Author SHA1 Message Date
Yueh-Hsuan Chiang 634f4eb37d
(LedgerStore) Use different path for different blockstore storage type. (#23236) 
#### Summary of Changes
To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.

This PR still keeps the RocksFifo setting hidden.  The default ShredStorageType and
blockstore directory are still RocksLevel and `rocksdb`.

Will follow-up with PRs on making FIFO option public in ledger-tool and validator.

#### Test Plan
* Added a new test to verify the existence of `rocksdb-fifo` directory when FIFO compaction is used.
* Updated existing test to verify the current setting still store ledger under `rocksdb` directory.
* Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
* Ran a validator with this PR.
 2022-03-02 18:30:22 -08:00
dependabot[bot] 39387e8446
chore: bump blake3 from 1.3.0 to 1.3.1 (#23460) 
* chore: bump blake3 from 1.3.0 to 1.3.1

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.3.0...1.3.1)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-03-02 18:20:30 -07:00
Jack May 97d40ba3da Resized accounts must be rent exempt 2022-03-02 13:02:02 -08:00
Will Hickey 1a99251498
Bump version to 1.10.1 (#23453) 2022-03-02 13:47:01 -06:00
dependabot[bot] da00d29de0
chore: bump bytemuck from 1.7.3 to 1.8.0 (#23437) 
* chore: bump bytemuck from 1.7.3 to 1.8.0

Bumps [bytemuck](https://github.com/Lokathor/bytemuck) from 1.7.3 to 1.8.0.
- [Release notes](https://github.com/Lokathor/bytemuck/releases)
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md)
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.7.3...v1.8.0)

---
updated-dependencies:
- dependency-name: bytemuck
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-03-02 19:04:54 +00:00
Jeff Washington (jwash) e88da2ec0a
minor test cleanup (#23447) 
* minor test cleanup

* fmt
 2022-03-02 12:10:05 -06:00
Jeff Washington (jwash) e630eb73d7
rework test to avoid multi-epoch gap (#23393) 2022-03-02 11:12:10 -06:00
Jeff Washington (jwash) ef8b7d9c62
ledger tool halt at slot verify hash (#23424) 2022-03-02 11:11:18 -06:00
Jeff Washington (jwash) d909b7c80b
log hash and lamport result of calculate_accounts_hash_without_index (#23425) 2022-03-02 11:10:40 -06:00
Jeff Washington (jwash) 7b7fdb42d9
minor test refactoring (#23423) 2022-03-02 08:31:14 -06:00
Jeff Washington (jwash) c8cb940b4e
tweak 2 rent tests to narrow epoch ranges (#23420) 2022-03-02 08:22:33 -06:00
Jeff Washington (jwash) 454e82683e
refactor rent test (#23392) 2022-03-01 08:47:09 -06:00
Jeff Washington (jwash) 6b2683f7da
refactor test (#23384) 2022-03-01 08:22:10 -06:00
Tyera Eulberg 19448ba078
Allow sub-rent-exempt-minimum transfers to `1nc1nerator` (#23382) 
* Add failing test

* Allow small burns to incinerator

* Use check_id method
 2022-02-28 23:56:34 -07:00
Jeff Washington (jwash) f0a235d16f
add comment (#23378) 2022-02-28 15:18:42 -06:00
Jeff Washington (jwash) 4eeb9f4648
acct idx comments (#23377) 2022-02-28 15:00:53 -06:00
Jeff Washington (jwash) 611d745241
refactor rent_due for normal case of exempt (#23350) 2022-02-28 09:42:42 -06:00
Jeff Washington (jwash) 30dafc7135
list -> slot_list (#23355) 2022-02-27 23:08:58 -06:00
Tyera Eulberg 36484f4f08
Prevent new RentPaying state created by paying fees (#23358) 
* Add failing test

* Check fee-payer rent-state change on load

* Add more test cases

* Review comments
 2022-02-26 12:10:01 -07:00
Jeff Washington (jwash) 0ad4757159
plumbing for 'other_slot' in 'update_index' (#23330) 2022-02-25 12:58:08 -06:00
Trent Nelson 5e0086c1ee followup safety checks for #23295 2022-02-24 17:50:58 -07:00
dependabot[bot] db12d90735
chore: bump ouroboros from 0.13.0 to 0.14.2 (#23328) 
* chore: bump ouroboros from 0.13.0 to 0.14.2

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.13.0 to 0.14.2.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits/0.14.2)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-02-24 15:56:13 -07:00
Jeff Washington (jwash) 2b0f16e7c3
slot per account in StorableAccounts (#23313) 2022-02-24 10:34:38 -06:00
Jeff Washington (jwash) 017170c99d
DiskIdx: new items are not dirty by default (#23123) 2022-02-24 10:17:35 -06:00
Jeff Washington (jwash) c0d0724be0
clippy (#23310) 2022-02-23 18:07:00 -06:00
Jeff Washington (jwash) 7ee549e5ae
fix params for store_accounts_frozen (#23312) 2022-02-23 17:40:51 -06:00
Jeff Washington (jwash) 227df52213
update comment (#23314) 2022-02-23 17:33:06 -06:00
Jeff Washington (jwash) cafc18c3f9
update_index uses ReadableAccount to reduce params (#23305) 2022-02-23 17:01:23 -06:00
Jeff Washington (jwash) a245efe83d
trait for acct data with slot per item (#23285) 2022-02-23 15:22:29 -06:00
Trent Nelson c81dd602c4 Hack fix for ICE as seen in CI 2022-02-23 11:17:39 -07:00
Jeff Washington (jwash) 2996f1f783
add comments (#23275) 2022-02-23 10:53:11 -06:00
Justin Starry d0e85c293f
Fix rustfmt check (#23296) 2022-02-23 16:38:53 +08:00
Gavin Chan 20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array (#23085) 2022-02-22 14:46:56 -08:00
Brooks Prumo 88e1192c6b
Refactor new() and default() for CostTracker (#23267) 2022-02-22 10:44:23 -06:00
Jeff Washington (jwash) 8b32e80ee2
DiskIdx: rename 'remove' to 'evict' (#23188) 2022-02-22 09:40:25 -06:00
Jeff Washington (jwash) 7ebf398ed7
AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB" (#23194) 
* AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB"

* ignore env var when starting as validator

* Update runtime/src/bucket_map_holder.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
 2022-02-22 09:40:12 -06:00
Justin Starry bcda74f42f
Fix builtin handling on epoch boundaries (#23256) 2022-02-22 20:54:08 +08:00
Michael Vines 72c68695b5 Do not keep oldest full snapshot 2022-02-21 17:10:31 -07:00
Tyera Eulberg 7e08ae1d0c
Revert "Add simulation detection countermeasure (#22880)" (#23261) 
This reverts commit c42b80f099.
 2022-02-21 21:15:37 +00:00
HaoranYi 04d23a1597
fix memory ordering in append_vec (#23215) 2022-02-20 20:30:49 -06:00
buffalu 70ebab2c82
Add rustfmt.toml and `cargo fmt` (#23238) 
* fmt

* formatted

Co-authored-by: Lucas B <buffalu@jito.network>
 2022-02-19 13:32:29 +08:00
Justin Starry 1719d2349f
Skip adding builtins if they will be removed (#23233) 
* Add failing test for precompile transition

* Skip adding builtins if they will be removed

* cargo clean

* nits

* fix abi check

* remove workaround

Co-authored-by: Jack May <jack@solana.com>
 2022-02-18 18:36:59 -08:00
Alexander Meißner ee7e411d68
Replaces `KeyedAccount` by `BorrowedAccount` in `system_instruction_processor`. (#23217) 
* Adds InstructionContext::check_number_of_instruction_accounts() and InstructionContext::get_instruction_account_key().
Bases check_sysvar_account() on instuction account indices.

* Adds instruction_account_indices enums to system_instruction_processor.

* Reorders parameters and adds InstructionContext.

* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 1).

* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 2).

* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 3).

* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 4).

* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 5).

* Code cleanup
 2022-02-19 02:06:33 +01:00
Jack May 970f543ef6
Precompiles owned by the native loader (#23237) 2022-02-18 15:41:59 -08:00
Alexander Meißner 1a68f81f89
Replaces `KeyedAccount` by `BorrowedAccount` in `nonce_keyed_account`. (#23214) 
* Adds get_sysvar_with_account_check2 for ABIv2.

* Replaces get_signers() and get_sysvar_with_account_check() in system_instruction_processor.

* Replaces KeyedAccount by BorrowedAccount in nonce_keyed_account.
 2022-02-17 17:36:55 +01:00
Alexander Meißner da00b39f4f
Cleanup: get_program_key() and get_loader_key() in TransactionContext (#23191) 
* Moves TransactionContext::get_program_key() to InstructionContext::get_program_key().

* Removes TransactionContext::get_loader_key().

* Test full program and loader executable account chain in BPF loader.
 2022-02-17 10:16:28 +01:00
Michael Vines a102453bae
accounts_index: Add SPL Token account indexing for token-2022 accounts (#23043) 2022-02-16 16:23:25 -08:00
Justin Starry 813725dfec
Fix ed25519 builtin program handling (#23182) 
* Fix ed25519 builtin program handling

* Fix tests

* Add integration tests for processing transactions with ed25519 ixs

* Fix another test

* fix formatting
 2022-02-16 18:48:55 +00:00
Jeff Washington (jwash) c2435363f3
DiskIdx: fix race condition with holding ranges in mem (#23158) 2022-02-16 09:25:00 -06:00
Michael Vines 577fa4ec0c Generate full snapshots 4x faster to keep incremental snapshots nice and small 2022-02-15 21:09:21 -08:00
Jeff Washington (jwash) 43393ea45d
assert on AcctInfo.store_id with cached store (#21952) 2022-02-15 15:22:36 -06:00
dependabot[bot] 197f42d346
chore: bump zstd from 0.9.2+zstd.1.5.1 to 0.10.0+zstd.1.5.2 (#23089) 
* chore: bump zstd from 0.9.2+zstd.1.5.1 to 0.10.0+zstd.1.5.2

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.9.2+zstd.1.5.1 to 0.10.0+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/compare/v0.9.2...v0.10.0)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-02-15 10:56:04 -07:00
Michael Vines c42b80f099
Add simulation detection countermeasure (#22880) 
* Add simulation detection countermeasures

* Add program and test using TestValidator

* Remove incinerator deposit

* Remove incinerator

* Update Cargo.lock

* Add more features to simulation bank

* Update Cargo.lock per rebase

Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
 2022-02-15 13:09:59 +01:00
Alexander Meißner 20bc37c9c9
Removes KeyedAccount in tests of system_instruction_processor. (#23118) 2022-02-15 00:00:49 +01:00
Jack May 3d9874b95a
Add fees to tx-wide caps (#22081) 2022-02-11 16:23:16 -08:00
Brooks Prumo 0a1ab945bc
Move cap_accounts_data_len feature gate only around new error (#23048) 2022-02-10 11:57:00 -06:00
Jeff Washington (jwash) 47e4291303
add Debug to RentResult (#23044) 2022-02-09 18:07:36 -06:00
Alexander Meißner 226a71f073
Refactor: Use instruction_trace instead of instruction_context_stack (#22988) 
* Moves stack_height of the instruction_trace into InstructionContext.

* Uses nesting_level instead of stack_height internally in transaction_context.rs

* Uses the instruction_trace instead of the instruction_context_stack internally.

* Adds feature gate: record_instruction_in_transaction_context_push.
 2022-02-09 20:04:49 +01:00
Jeff Washington (jwash) 205cd4609b
bank::get_rent_collector_from_parent (#23016) 2022-02-09 12:31:58 -06:00
Jeff Washington (jwash) f334931903
rent: refactor test function (#23014) 2022-02-08 22:11:47 -06:00
Jeff Washington (jwash) fb47c1f0a3
consolidate Rent construction in test code (#23018) 2022-02-08 21:51:52 -06:00
Brooks Prumo 869cfc9a1c
Return the accounts data len delta after processing messages (#22986) 2022-02-08 19:24:47 -06:00
Tao Zhu 7aa1fb4e24 1. Persist to blockstore less frequently; 
2. reduce alpha for EMA to 1 percent to have roughly 200 data points for estimatio
 2022-02-08 16:18:23 -06:00
Tao Zhu 6587dbfa47 use EMA in place of Welford 2022-02-08 16:18:23 -06:00
Tao Zhu a25ac1c988 - estimate a program cost as 2 standard deviation above mean 
- replaced get_average / get_mode with get_default to assign max units to unknown program
 2022-02-08 16:18:23 -06:00
Jeff Washington (jwash) 86cf226395
eliminate separate max root member (#22943) 2022-02-08 09:31:32 -06:00
Brooks Prumo f0f4042680
Put accounts data len updates behind feature gate (#22918) 2022-02-08 08:51:16 -06:00
Jeff Washington (jwash) b2e475b5c4
clarify that RollingBitField max is exclusive (#22947) 2022-02-07 13:40:31 -06:00
Jeff Washington (jwash) a160fc30f2
make max_roots_inclusive clear (#22942) 2022-02-07 13:26:53 -06:00
Justin Starry ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys (#22960) 2022-02-05 20:00:31 +08:00
Alexander Meißner 660f6981c6
Cleanup: TransactionContext (#22910) 
* Adds BorrowedAccount::check_sysvar().

* Adds BorrowedAccount::get_data_mut().

* Implements account resizing in BorrowedAccount.

* Exposes is_signer() and is_writable() in InstructionContext.

* Removes AccountMeta and get_instruction_accounts_metas().

* Makes throwing errors in BorrowedAccount optional.

* Removes result return values from BorrowedAccount.
 2022-02-03 17:19:42 +01:00
Alexander Meißner c16cf9cf8a
Refactor: Use `SysvarCache` in all builtin programs (#22864) 
* Replaces from_keyed_account() by SysvarCache in stake instruction.

* Replaces from_keyed_account() by SysvarCache in system instruction processor.

* Removes from_keyed_account().
Moves check_sysvar_keyed_account() into sysvar_cache.rs

* Removes tests which test for incorrectly serialized sysvars.
 2022-02-03 13:03:50 +01:00
Justin Starry 60af1a4cce
Refactor: Add trait for loading addresses (#22903) 2022-02-03 11:00:12 +00:00
Jack May cc94a93b56
Safer invoke context (#22898) 
* Safer invoke context

* feedback and rebase with master
 2022-02-03 02:34:51 -08:00
carllin bd1850df25
Return actual committed transactions from process_transactions() (#22802) 2022-02-03 03:56:36 -05:00
Jack May ab02dba96f
Add get_processed_sibling_instruction syscall (#22859) 2022-02-02 16:45:57 -08:00
Will Hickey 75563f6c7b
Reject close of active vote accounts (#22651) 
* 10461 Reject close of vote accounts unless it earned no credits in the previous epoch. This is checked by comparing current epoch (from clock sysvar) with the most recent epoch with credits in vote state.
 2022-02-02 14:16:24 -06:00
Brooks Prumo fb0e71946f
Cleanup use declaration (#22875) 2022-02-01 23:34:11 +00:00
Brooks Prumo 22f6db2e4f
Cleanup serde snapshot common.rs (#22854) 2022-02-01 05:39:53 -06:00
Jack May 551c24da57
nit: consistent name (#22857) 2022-02-01 07:46:04 +00:00
Jeff Washington (jwash) d60dac9749
add comment (#22846) 2022-01-31 11:40:19 -06:00
Jeff Washington (jwash) fb95fa68a2
cleanup api (#22845) 2022-01-31 11:39:23 -06:00
Alexander Meißner bc800a8d5a
Refactor: Unify `SysvarCache` (#22843) 
* Unifies SysvarCache filling in the runtime and tests.
Removes new_mock_with_sysvars_and_features()
Removes mock_process_instruction_with_sysvars().
Replaces from_keyed_account() by SysvarCache in vote processor.

* Replaces from_keyed_account() by SysvarCache in BPF loader.
 2022-01-31 17:53:50 +01:00
dependabot[bot] cc74693176
chore: bump serde from 1.0.134 to 1.0.136 (#22817) 
* chore: bump serde from 1.0.134 to 1.0.136

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.134 to 1.0.136.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.134...v1.0.136)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-28 12:20:37 -07:00
carllin 94a5aee484
Add new_from_parent() timings (#22744) 2022-01-27 20:35:13 -05:00
Jeff Washington (jwash) 89c42ebcbe
refactor rent collection (#22803) 2022-01-27 18:48:04 -06:00
Jeff Washington (jwash) 350207b05f
impl ReadableAccount for LoadedAccount (#22793) 2022-01-27 16:59:30 -06:00
Jeff Washington (jwash) 18d69c8d2b
SortedStoragesIter to remove random access requirement (#22713) 2022-01-27 10:25:13 -06:00
Justin Starry d9c259a231
Set the correct root in block commitment cache initialization (#22750) 
* Set the correct root in block commitment cache initialization

* clean up test

* bump
 2022-01-27 00:48:00 +08:00
Justin Starry 071e97053f
Perf: Reduce write locks on blockhash queue (#22729) 
* Perf: Reduce write locks on blockhash queue

* Add comment about thread safety

* Add comment about write starvation
 2022-01-26 16:24:27 +08:00
Jeff Washington (jwash) 8b90084ebc
fix comment typos (#22723) 2022-01-25 11:10:42 -06:00
Jon Cinque fc21af4e6e
spl-associated-token-account: Add feature for new program (#22648) 
* spl-associated-token-account: Add feature for new program

* Address feedback
 2022-01-24 22:54:41 +01:00
Michael Vines 6d5bbca630 Pacify clippy 2022-01-21 19:12:57 -08:00
dependabot[bot] 0cf886302d
chore: bump serde from 1.0.133 to 1.0.134 (#22650) 
* chore: bump serde from 1.0.133 to 1.0.134

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.133 to 1.0.134.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.133...v1.0.134)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-21 17:14:02 -07:00
Jeff Washington (jwash) e3915e4b7a
add comment (#22636) 2022-01-21 10:51:58 -06:00
Jeff Washington (jwash) 4abf7a7f88
add ActiveStats to track when long running, expensive bg processes are running (#22608) 2022-01-21 10:51:23 -06:00
Brooks Prumo 9977396d8f
Remove unused fields from Bank (#22491) 2022-01-21 06:03:41 -06:00
Justin Starry 7d34a7acac
Perf: Only check executors cache for executable bpf program ids (#22624) 
* Only check executors cache for executable bpf program ids

* switch to native loader check

* clean up tests

* fix tests

* clippy
 2022-01-21 18:32:30 +08:00
Jeff Washington (jwash) 0eb488580d
minor cleanup (#22610) 2022-01-20 18:11:00 -06:00
Justin Starry 7ba57e7a7c
Refactor: move instructions sysvar serialization out of Message (#22544) 2022-01-20 17:33:49 +08:00
Justin Starry 7f20c6149e
Refactor: move simple vote parsing to runtime (#22537) 2022-01-20 10:39:21 +08:00
anatoly yakovenko d343713f61
Optimize packet dedup (#22571) 
* Use bloom filter to dedup packets

* dedup first

* Update bloom/src/bloom.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* Update core/src/sigverify_stage.rs

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>

* fixup

* fixup

* fixup

Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
 2022-01-19 13:58:20 -08:00
Alexander Meißner b448472037
Refactor: Move `InstructionRecorder` into `TransactionContext` (#22578) 
* Moves InstructionRecorder into TransactionContext.

* Adds assertions for number_of_instructions_at_transaction_level.
 2022-01-19 22:40:09 +01:00
Justin Starry cc76a73c49
Refactor: move compute budget runtime logic into solana-program-runtime (#22543) 2022-01-17 20:48:00 +08:00
Justin Starry 2c38a9213f
Revert "Refactor: move compute budget runtime logic into solana-program-runtime (#22533)" (#22542) 
This reverts commit b27976626a.
 2022-01-17 17:43:17 +08:00
Justin Starry b27976626a
Refactor: move compute budget runtime logic into solana-program-runtime (#22533) 2022-01-17 15:52:33 +08:00
Brooks Prumo 6edeed888d
Remove unnecessary & from AsRef params (#22523) 2022-01-15 04:51:05 +00:00
Brooks Prumo 70633b5c2f
Add Bank::accounts_data_len to SetRootMetrics (#22509) 2022-01-14 20:00:07 -06:00
Tao Zhu a724fa2347
Add hidden cli option to allow validator reports replayed transaction cost metrics (#22369) 
* add hidden cli option to allow validator reports replayed transaction cost detail metrics

* Update validator/src/main.rs

Co-authored-by: Michael Vines <mvines@gmail.com>

* - rebase master, using unbounded instead of channel; dowgrade to datapoint_trace

* removed cli arg, prefer log at trace

Co-authored-by: Michael Vines <mvines@gmail.com>
 2022-01-15 00:31:21 +00:00
Jeff Washington (jwash) 7d7228c356
cleanup assert (#22495) 2022-01-14 11:53:24 -06:00
Tao Zhu 9c9f2dd5bd port counting vote CUs to block cost (#22477) 2022-01-14 10:50:29 -06:00
Justin Starry cddab635ff
Cleanup activated rent_for_sysvars feature (#22454) 2022-01-14 20:34:09 +08:00
Justin Starry ae6c511f13
Refactor: Split vote_instruction.rs into multiple files (#22502) 2022-01-14 17:25:15 +08:00
Justin Starry 93a7b94507
Add benchmark for vote processing (#22486) 2022-01-14 17:10:17 +08:00
Michael Vines 4c577d7f8c `Bank::get_fee_for_message` is now nonce aware 2022-01-13 17:27:38 -08:00
Jeff Washington (jwash) d3b95b9226
Cleanup AccountStorage struct (#22463) 
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)"

This reverts commit 8aa3d690b5.

* Cleanup AccountStorage struct
 2022-01-13 17:51:53 -06:00
Jeff Washington (jwash) 7711cd74c3
partition_from_pubkey (#22430) 
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)"

This reverts commit 8aa3d690b5.

* partition_from_pubkey
 2022-01-13 17:02:42 -06:00
Jeff Washington (jwash) e291342c4a
Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)" (#22488) 
This reverts commit 8aa3d690b5.
 2022-01-13 13:06:39 -06:00
Brooks Prumo b82d71d22a
Inline DEFAULT_SNAPSHOT_VERSION constant string (#22487) 2022-01-13 18:19:15 +00:00
Brooks Prumo 2756abce39
More serde snapshot cleanup (#22449) 2022-01-13 09:20:20 -06:00
Brooks Prumo 9c3144e286
Refactor serde snapshot's "future" to "newer" (#22431) 2022-01-13 07:12:09 -06:00
Justin Starry 2370e61431
Perf: Store deserialized sysvars in the sysvars cache (#22455) 
* Perf: Store deserialized sysvars in sysvars cache

* add bench
 2022-01-13 05:36:21 +00:00
Justin Starry 7171c95bdd Refactor: move sysvar cache to new module 2022-01-12 12:35:28 -07:00
Tyera Eulberg 3ca16de851
Only examine explicit tx accounts for rent state (#22441) 
* Add failing test

* Fix: only examine accounts explicitly included in a tx
 2022-01-11 20:23:51 -07:00
Brooks Prumo c45dde6164
Handle accounts data size changes due to rent-collected accounts (#22412) 2022-01-11 17:20:28 -06:00
dependabot[bot] 157f165a3d
Bump memmap2 from 0.5.1 to 0.5.2 (#22414) 
* Bump memmap2 from 0.5.1 to 0.5.2

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.1 to 0.5.2.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.1...v0.5.2)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-11 15:08:13 -07:00
Tyera Eulberg 637e366b18
Prevent rent-paying account creation (#22292) 
* Fixup typo

* Add new feature

* Add new TransactionError

* Add framework for checking account state before and after transaction processing

* Fail transactions that leave new rent-paying accounts

* Only check rent-state of writable tx accounts

* Review comments: combine process_result success behavior; log and metrics before feature activation

* Fix tests that assume rent-exempt accounts are okay

* Remove test no longer relevant

* Remove native/sysvar special case

* Move metrics submission to report legacy->legacy rent paying transitions as well
 2022-01-11 11:32:25 -07:00
behzad nouri a49ef49f87
optimizes ReadOnlyAccountsCache LRU eviction implementation (#22403) 
ReadOnlyAccountsCache is using a background thread, table scan and sort
to implement LRU eviction policy:
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L66-L73
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L186-L191
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L222

DashMap internally locks each shard when accessed; so a table scan in
the background thread can create a lot of lock contention.

This commit adds an index-list queue containing cached keys in the order
that they are accessed. Each hash-map entry also includes its index into
this queue.
When an item is first entered into the cache, it is added to the end of
the queue. Also each time an entry is looked up from the cache it is
moved to the end of queue. As a result, items in the queue are always
sorted in the order that they have last been accessed. When doing LRU
eviction, cache entries are evicted from the front of the queue.
Using index-list, all queue operations above are O(1) with low overhead
and so above achieves an efficient implementation of LRU cache eviction
policy.
 2022-01-11 17:25:28 +00:00
Jeff Biseda 8b66625c95
convert std::sync::mpsc to crossbeam_channel (#22264) 2022-01-11 02:44:46 -08:00
dependabot[bot] 04a2c19c87
Bump memmap2 from 0.5.0 to 0.5.1 (#22409) 
* Bump memmap2 from 0.5.0 to 0.5.1

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.0...v0.5.1)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-10 11:14:56 -07:00
Brooks Prumo 9bc2592da1
Use lazy_rent_collection directly (#22410) 2022-01-10 10:47:08 -06:00
dependabot[bot] 28275a33d6
Bump tempfile from 3.2.0 to 3.3.0 (#22401) 
* Bump tempfile from 3.2.0 to 3.3.0

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.2.0...v3.3.0)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-09 12:26:08 -07:00
Trent Nelson 3b4aad9df1 bank: fix executor cache metrics 2022-01-09 04:22:39 +00:00
dependabot[bot] eaa52bc935
Bump zstd from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1 (#22388) 
* Bump zstd from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/compare/0.9.1...v0.9.2)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-08 22:21:56 +00:00
dependabot[bot] d8ec8ca2e6
Bump blake3 from 1.2.0 to 1.3.0 (#22373) 
* Bump blake3 from 1.2.0 to 1.3.0

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.2.0...1.3.0)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-08 20:28:04 +00:00
Brooks Prumo d90d5ee9b6
Refactor Rent::due() with RentDue enum (#22346) 2022-01-08 09:03:46 -06:00
Jack May 4a9f4e2505 improve multi executor cache addition 2022-01-08 03:47:23 -07:00
Trent Nelson ad3cb0bc93 --amend 2022-01-08 08:19:27 +00:00
Trent Nelson 4ce48307bb bank: prime new executor cache entry use-counts 2022-01-08 08:19:27 +00:00
Trent Nelson 6d76db1de5 bank: Add executors cache metrics 2022-01-07 16:53:27 -07:00
behzad nouri c2389fc209
removes CowCachedExecutors (#22343) 
Copy-on-write semantics for cached executors can be implemented by a
simple Arc<CachedExecutors> as opposed to CowCachedExecutors:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L244-L247

This will also avoid the need for double locking as in:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3490-L3491
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3525-L3526
 2022-01-07 14:06:29 +00:00
Justin Starry 52d12cc802
Add runtime support for address table lookups (#22223) 
* Add support for address table lookups in runtime

* feedback

* feedback
 2022-01-07 11:59:09 +08:00
Jack May f1e2598baa
Retain executor cache counts (#22322) 2022-01-06 08:56:00 -08:00
Jeff Washington (jwash) e0c091a9f4
factor out code in do_shrink_slot_stores (#22306) 2022-01-06 10:49:24 -06:00
Jeff Washington (jwash) 100293c4b5
refactor get_store_for_shrink (#22307) 2022-01-06 10:49:01 -06:00
Trent Nelson 390ef0fbcd Consolidate process instruction execution timings to own struct 2022-01-06 03:56:46 -07:00
Trent Nelson 72fc6096a0 Use saturating_add_assign macro 2022-01-06 03:56:46 -07:00
Carl Lin b25e4a200b Add execute metrics 2022-01-06 03:56:46 -07:00
Trent Nelson 7d32909e17 move `ExecuteTimings` from `runtime::bank` to `program_runtime::timings` 2022-01-06 03:56:46 -07:00
Jack May 12e160269e
cache executors on failed transactions (#22308) 2022-01-05 22:09:03 -08:00
Brooks Prumo ab13e39518
Only sum accounts data len from non-zero lamport accounts (#22309) 2022-01-05 23:23:37 +00:00
Justin Starry f2ed6f09ee Skip updating already cached executors if unmodified 2022-01-05 16:08:10 -07:00
Michael Vines bb3a1b6b31 Add zk_token_sdk_enabled feature to gate Zk Token proof program and `sol_zk_token_elgamal_op` syscalls 2022-01-05 11:57:37 -08:00
Alexander Meißner 9f63493789
Refactor: Remove KeyedAccounts (2) (#22274) 
* Adds InstructionContext::get_signers().
Improves error messages when modifying borrowed accounts.

* Removes keyed_accounts from InvokeContext tests.

* Removes keyed_accounts from message_processor.rs

* Removes keyed_accounts from bank.rs

* Removes keyed_accounts from bpf serialization.
 2022-01-05 09:39:37 +01:00
Justin Starry 45458e7139
Refactor: Improve type safety and readability of transaction execution (#22215) 
* Refactor Bank::load_and_execute_transactions

* Refactor: improve type safety of TransactionExecutionResult

* Add enum for extra type safety in execution results

* feedback
 2022-01-05 10:15:15 +08:00
Brooks Prumo 1460f00e0f
Consume from AccountsDataMeter (#21994) 2022-01-04 10:00:21 -06:00
Justin Starry 2b5e00d36d
Limit number of accounts that a transaction can lock (#22201) 2022-01-04 14:25:23 +08:00
Alexander Meißner 73e6038986
Refactor: Remove `KeyedAccount` from program runtime (#22226) 
* Makes error handling in BorrowedAccount optional.
Adds BorrowedAccount ::get_rent_epoch().
Exposes InstructionContext::get_index_in_transaction().
Turns accounts and account_keys into pinned boxed slices.

* Introduces "unsafe" to InvokeContext::push().

* Turns &TransactionContext into &mut TransactionContext in InvokeContext.

* Push and pop InstructionContext in InvokeContext.
Makes test_process_cross_program and test_native_invoke symmetric.
Removes the borrow check from test_invoke_context_verify.

* Removes keyed_accounts from prepare_instruction()

* Removes usage of invoke_stack.

* Removes keyed_accounts from program-test.

* Removes caller_write_privileges.

* Removes keyed_accounts from BPF parameter (de-)serialization.
 2022-01-03 23:30:56 +01:00
dependabot[bot] 672fed04cb
Bump serde from 1.0.132 to 1.0.133 (#22233) 
* Bump serde from 1.0.132 to 1.0.133

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.132 to 1.0.133.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.132...v1.0.133)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2022-01-03 22:19:39 +00:00
Brooks Prumo 04b76eb066
Use AtomicUsize for next_bucket_to_flush (#22095) 2022-01-03 10:35:35 -06:00
Jeff Washington (jwash) fb62407232
AcctIdx: appendvecid: u32 (#21842) 2022-01-03 10:35:17 -06:00
Tao Zhu a2a7e91ad6 set secp256k1 cost similar to sigverify 2021-12-31 17:45:50 -06:00
Tao Zhu d743c2917c re-calibrate limit based on mainnet data, see issue #21917 2021-12-31 17:45:50 -06:00
Alexander Meißner e529d03c11
Cleanup #22182 (#22205) 
* Turns compute_units_consumed of ProcessInstructionResult into a &mut parameter.

* Removes second nesting level from test_process_instruction_compute_budget().

* Makes test_process_cross_program and test_native_invoke symmetric.

* Unifies test_process_cross_program(), test_native_invoke() and test_process_instruction_compute_budget() into test_process_instruction().
 2021-12-31 17:55:27 +01:00
carllin d06e6c7425
Count compute units even when transaction errors (#22182) 2021-12-30 21:21:42 -05:00
Alexander Meißner edb20d6909
Splits index of InstructionAccount into index_in_transaction and index_in_caller. (#22165) 2021-12-30 15:46:36 +01:00
carllin 33d0b5e011
Revert "Count compute units even when transaction errors (#22059)" (#22174) 
This reverts commit eaa8c67bde.
 2021-12-30 02:42:32 -05:00
Justin Starry b1d9a2e60e
Don't forward packets received from TPU forwards port (#22078) 
* Don't forward packets received from TPU forwards port

* Add banking stage test
 2021-12-29 19:34:31 +01:00
carllin eaa8c67bde
Count compute units even when transaction errors (#22059) 2021-12-28 17:05:11 -05:00
Tyera Eulberg 422a095647
Add (preflight) simulation to BanksClient (#22084) 
* Add more-legitimate conversion from legacy Transaction to SanitizedTransaction

* Add Banks method with preflight checks

* Expose BanksClient method with preflight checks

* Unwrap simulation err

* Add Bank simulation method that works on unfrozen Banks

* Add simpler api

* Better name: BanksTransactionResultWithSimulation
 2021-12-28 19:25:46 +00:00
Brooks Prumo e3b20c443a
Use load_accounts_data_len() instead of Atomic .load() (#22144) 2021-12-28 13:38:20 +00:00
Brooks Prumo 800472ddf5
Add AccountsDataMeter to InvokeContext (#21813) 2021-12-28 05:14:48 -06:00
Alexander Meißner a06646631c
Feature: TransactionContext, InstructionContext and BorrowedAccount (#21706) 
* Adds TransactionContext, InstructionContext and BorrowedAccount.

* Redirects the usage of accounts in InvokeContext through TransactionContext.
Also use the types declared in transaction_context.rs everywhere.

* Adjusts all affected tests.
 2021-12-27 18:49:32 +01:00
dependabot[bot] bb97c8fdcd
Bump zstd from 0.9.0+zstd.1.5.0 to 0.9.1+zstd.1.5.1 (#22105) 
* Bump zstd from 0.9.0+zstd.1.5.0 to 0.9.1+zstd.1.5.1

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.9.0+zstd.1.5.0 to 0.9.1+zstd.1.5.1.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/compare/0.9.0...0.9.1)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-25 21:26:55 -07:00
Alexander Meißner cc947cad03
Refactor: CPI Instruction Recording (#22111) 
* Unifies all InstructionRecorders of a transaction into one.

* Stops explicitly compiling CPI instructions for recording,
uses the indices gathered from instruction_accounts instead.
 2021-12-25 13:35:43 +01:00
Alexander Meißner 2ab4f34c02
Refactor: Remove `Message` and `CompiledInstruction` from `InvokeContext` interfaces (#22102) 
* Introduces InstructionAccount which is like AccountMeta but uses an index instead of a Pubkey

* Renames InvokeContext::create_message() to InvokeContext::prepare_instruction()

* Removes Message and CompiledInstruction from InvokeContext interfaces.

* Resolves TODOs of sol_invoke_signed() in program-test.

* Moves CompiledInstruction::visit_each_account() into invoke_context.rs
 2021-12-24 16:17:55 +01:00
behzad nouri f67ecd5c18
removes unused Cargo dependencies (#22022) 
Also moving some to [dev-dependencies] so that they are not propagated
to other packages which depend on the package.
 2021-12-22 17:01:33 +00:00
Jeff Washington (jwash) 61cc7b10a9
AcctIdx: respect disk idx mem size param (#22050) 2021-12-22 09:54:05 -06:00
Jeff Washington (jwash) bdae2993e0
AcctIdx: hold ranges in memory uses multiple threads (#22031) 2021-12-21 17:31:48 -06:00
dependabot[bot] 41ec7c8be9
chore: bump num_cpus from 1.13.0 to 1.13.1 (#22044) 
* chore: bump num_cpus from 1.13.0 to 1.13.1

Bumps [num_cpus](https://github.com/seanmonstar/num_cpus) from 1.13.0 to 1.13.1.
- [Release notes](https://github.com/seanmonstar/num_cpus/releases)
- [Changelog](https://github.com/seanmonstar/num_cpus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/num_cpus/compare/v1.13.0...v1.13.1)

---
updated-dependencies:
- dependency-name: num_cpus
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-21 00:06:14 -07:00
dependabot[bot] 8f547a6c98
chore: bump serde from 1.0.131 to 1.0.132 (#21989) 
* chore: bump serde from 1.0.131 to 1.0.132

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.131 to 1.0.132.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.131...v1.0.132)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-18 07:45:20 +00:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787) 
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
 2021-12-17 16:03:09 -07:00
mooori 0f6e8d3385
Check file size of `snapshot_version` when unarchiving snapshot (#21925) 2021-12-17 12:27:54 -06:00
Jeff Washington (jwash) 8ed7ad5fa7
AcctIdx: hold range recognizes already held ranges (#21937) 2021-12-17 10:04:41 -06:00
Jeff Washington (jwash) 729698e815
AcctIdx: items() uses held ranges (#21954) 2021-12-17 09:59:29 -06:00
Jeff Washington (jwash) af53d2f692
simplify api on reconstruct_single_storage (#21970) 2021-12-17 09:06:23 -06:00
Alexander Meißner 66fa8f9667
Refactor: Removes `Rc` from `Refcell<AccountSharedData>` in the program-runtime (#21927) 
* Removes Rc from Rc<RefCell<AccountSharedData>> in the program-runtime.

* Adjusts tests in bpf_loader, system_instruction_processor, config_processor, vote_instruction and stake_instruction
 2021-12-17 14:01:12 +01:00
Jeff Washington (jwash) c4389a6675
AcctIdx: held ranges search in lifo order (#21964) 2021-12-16 23:25:22 -06:00
Jeff Washington (jwash) 5d40da5688
log "next_id" to track append vec ids (#21971) 2021-12-16 22:41:34 -06:00
Jeff Washington (jwash) 6374995522
AcctIdx: share bucket map size for perf (#21935) 2021-12-16 21:25:54 -06:00
Jeff Washington (jwash) ba777f4f56
AcctIdx: remove Option from held ranges (#21958) 2021-12-16 21:22:04 -06:00
Jeff Washington (jwash) e11a1911ad
load_accounts_index_for_shrink ignores cached entries (#21951) 2021-12-16 16:37:08 -06:00
Justin Starry 6ff0be6a82
Clean up demote program write lock feature (#21949) 
* Clean up demote program write lock feature

* fix test
 2021-12-16 17:27:22 -05:00
dependabot[bot] a5769c029f
chore: bump tar from 0.4.37 to 0.4.38 (#21921) 
* chore: bump tar from 0.4.37 to 0.4.38

Bumps [tar](https://github.com/alexcrichton/tar-rs) from 0.4.37 to 0.4.38.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases)
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.37...0.4.38)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-16 15:06:56 -07:00
Jeff Washington (jwash) 347323cbb2
use bg thread pool for shrink (#21950) 2021-12-16 15:54:38 -06:00
Jeff Washington (jwash) e97da0ea15
AcctIdx: define type for serialized AppendVecId (#21938) 2021-12-16 08:41:01 -06:00
Jeff Washington (jwash) 18417e410e
AcctIdx: remove troublesome assert (#21947) 2021-12-16 08:37:05 -06:00
Alexander Meißner 49cb161203
Fixes the calculation of the "compute_meter_consumption" across process_instruction() and process_message(). (#21944) 2021-12-16 15:15:58 +01:00
Jeff Washington (jwash) 82672b40fd
AcctIdx: streamline metric update (#21936) 2021-12-15 19:52:23 -06:00
Jeff Washington (jwash) 1e0d3f13e6
AcctIdx: fix metrics bug (#21934) 2021-12-15 17:05:38 -06:00
Brooks Prumo 635337d2ff
Bank gets accounts data len delta from MessageProcessor::process_message() 2021-12-15 16:41:38 -06:00
Jeff Washington (jwash) 46e5350d8c
AcctInfo: store offset in AccountInfo as u32 (#21895) 2021-12-15 15:41:11 -06:00
Jeff Washington (jwash) e5be96d8bf
AcctIdx: consolidate next_id calls (#21929) 2021-12-15 15:39:54 -06:00
Jeff Washington (jwash) 41dd31e5f4
AcctIdx: move cached store id to bit (#21892) 
* AcctIdx: move cached store id to bit

* add comments/rename
 2021-12-15 11:49:24 -06:00
Jeff Washington (jwash) 02fa135815
AcctIdx: create test fn get_test() to isolate changes to AcctIdx::get() (#21909) 2021-12-15 09:09:56 -06:00
Jeff Washington (jwash) f4308bdb64
AcctIdx: StoredSize is u32 (#21894) 2021-12-14 13:50:19 -06:00
Jeff Washington (jwash) e694acaf5f
AcctIdx: better types for AccountInfo (#21893) 2021-12-14 13:08:49 -06:00
Jeff Washington (jwash) ec583bd12d
AcctIdex: use StorageLocation (#21853) 2021-12-14 10:27:17 -06:00
Jeff Washington (jwash) a86fe899ac
AcctIdx: move zero lamport out of accounts index (#21526) 2021-12-14 09:31:42 -06:00
Justin Starry c92c09a8be
Remove activated feature for removing inactive delegations from stakes cache (#21732) 
* Remove activated feature for removing inactive delegations from stakes cache

* Fix builtin purging
 2021-12-14 09:23:36 -05:00
Justin Starry e5476913fe
Remove activated feature that checks tx signature len (#21747) 2021-12-14 09:23:05 -05:00
Jeff Washington (jwash) 6fc6673ead
AcctInfo: store_id private and add accessor (#21839) 2021-12-13 21:35:30 -06:00
Jeff Washington (jwash) 98e5ea9dce
AcctIdx: simplify AccountIndexGetResult (#21857) 2021-12-13 21:16:17 -06:00
Jeff Washington (jwash) 50f26ea9c0
AcctInfo: create AcctInfo with cache explicitly (#21847) 2021-12-13 17:35:07 -06:00
Jeff Washington (jwash) 6155ef6377
AcctInfo: make offset private, add accessor (#21838) 2021-12-13 14:43:26 -06:00
dependabot[bot] 8aa3d690b5
chore: bump dashmap from 4.0.2 to 5.0.0 (#21824) 
* chore: bump dashmap from 4.0.2 to 5.0.0

Bumps [dashmap](https://github.com/xacrimon/dashmap) from 4.0.2 to 5.0.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.0.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-13 13:23:58 -07:00
Jeff Washington (jwash) 9e9a1a4876
AcctIdx: AccountInfo.lamports private (#21819) 2021-12-13 10:59:33 -06:00
Jeff Washington (jwash) c9e0bde407
AcctIdx: AccountInfo::stored_size private (#21821) 2021-12-13 10:10:06 -06:00
Jeff Washington (jwash) 90f41fd9b7
use cost model to limit new account creation (#21369) 
* use cost model to limit new account creation

* handle every system instruction

* remove &

* simplify match

* simplify match

* add datapoint for account data size

* add postgres error handling

* handle accounts:unlock_accounts
 2021-12-12 14:57:18 -06:00
Jeff Washington (jwash) 025a5a3b9c
AccountInfo: construct with new() (#21788) 2021-12-12 14:36:05 -06:00
Jeff Washington (jwash) 825f8bcea4
use AppendVecId instead of usize (#21792) 2021-12-11 21:38:13 -06:00
Jeff Washington (jwash) c5b6ea74ef
replace some .lamports checks with trait (#21783) 2021-12-11 21:37:33 -06:00
Jeff Washington (jwash) 4c0373b72a
introduce AtomicAppendVecId (#21793) 2021-12-11 21:34:35 -06:00
Jeff Washington (jwash) ef7f49a21d
move AccountInfo to its own file (#21784) 2021-12-11 11:47:05 -06:00
Brooks Prumo eeb97fe7ce
Add accounts_data_len to Bank (#21781) 2021-12-11 09:34:38 -06:00
Justin Starry 2bbe1d875a
Deserialize accounts before acquiring stakes cache lock (#21733) 
* Deserialize stored accounts before locking stakes cache

* fix test
 2021-12-10 16:02:35 -05:00
Justin Starry 9b41ddd9ba
Add address lookup table program (#21616) 
* Add address lookup table program

* feedback
 2021-12-10 16:02:16 -05:00
Brooks Prumo ec7e17787e
Compute accounts data len during generate_index() (#21757) 2021-12-10 13:27:59 -06:00
Tyera Eulberg 80eae20610 Fix master fmt 2021-12-10 11:33:39 -07:00
Tyera Eulberg 350845c513
Move type alias and use it more broadly (#21763) 2021-12-10 10:46:21 -07:00
Justin Starry fd175c1ea9
Add StakesCache struct to abstract away locking (#21738) 2021-12-10 10:51:33 -05:00
axleiro 622fd7c7ec
testing for the latest changes 2021-12-10 19:44:22 +05:30
dependabot[bot] d870f566ef
chore: bump serde from 1.0.130 to 1.0.131 (#21758) 
* chore: bump serde from 1.0.130 to 1.0.131

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.130 to 1.0.131.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.130...v1.0.131)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-10 08:47:46 +00:00
Tyera Eulberg c1386d66e6
Nits in message-processor (#21755) 
* Fixup typo

* Simplify types slightly
 2021-12-09 23:38:36 -07:00
Jeff Washington (jwash) 16a6dceb6b
impl better debug for Ancestors (#21737) 2021-12-09 18:36:13 -06:00
Brooks Prumo 1528f85112
Add return types to generate_index() (#21735) 2021-12-09 17:15:13 -06:00
Justin Starry 6fc329180b
Add more reporting for invalid stake cache members and prune them (#21654) 
* Add more reporting for invalid stake cache members

* feedback
 2021-12-09 14:12:11 -05:00
Jeff Washington (jwash) a2df1eb502
AcctIdx: disk generate index and filler accounts use more threads (#21566) 2021-12-09 11:54:14 -06:00
Jeff Washington (jwash) 58a7d3fc0e
AcctIdx: disable direct to disk for filler account upsert (#21710) 2021-12-09 10:14:57 -06:00
Jeff Washington (jwash) 0090916735
skip clean on startup, too (#21718) 2021-12-09 09:36:02 -06:00
Jeff Washington (jwash) e020960f49
AcctIdx: generate index goes to mem not disk (#21709) 2021-12-08 19:48:12 -06:00
Jeff Washington (jwash) 54862eba0d
AcctIdx: env var to enable testing of disk buckets (#21494) 2021-12-08 19:47:25 -06:00
Jeff Washington (jwash) 181c0092d6
AcctIdx: resize in-mem after startup for disk index (#21676) 2021-12-08 16:52:22 -06:00
Jeff Washington (jwash) 8d1e5ac294
refactor and test scan abort code (#21390) 2021-12-08 14:09:34 -06:00
Ashwin Sekar f0acf7681e
Add vote instructions that directly update on chain vote state (#21531) 
* Add vote state instructions

UpdateVoteState and UpdateVoteStateSwitch

* cargo tree

* extract vote state version conversion to common fn
 2021-12-07 16:47:26 -08:00
Alexander Meißner 1df88837c8
- Implicitly fixes invoke_context.return_data not being reset between instructions in process_message. (#21671) 
- Lets InvokeContext::process_cross_program_instruction() handle the first invocation depth too.
- Marks InvokeContext::verify(), InvokeContext::verify_and_update() and InvokeContext::process_executable_chain() private.
- Renames InvokeContext::process_cross_program_instruction() to InvokeContext::process_instruction().
- Removes InvokeContext::new_mock_with_sysvars().
 2021-12-07 23:00:04 +01:00
dependabot[bot] f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 (#21643) 
* chore: bump itertools from 0.10.1 to 0.10.3

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-12-06 23:23:26 -07:00
Brooks Prumo da4015a959
Ensure that StakeDelegations and StakeHistory serde (#21640) 
Add tests to StakeDelegations and StakeHistory to ensure that the outer
types serialize and deserialize correctly to/from the inner types.
 2021-12-06 17:34:10 -06:00
Justin Starry a1adcb23b6
Remove activated feature for filtering invalid stakes from rewards (#21641) 2021-12-06 17:56:05 -05:00
Justin Starry 3dab1e711d
Move transaction error code into new module (#21635) 2021-12-06 12:45:33 -05:00
Trent Nelson d6f22433d0 Bump version to v1.10.0 2021-12-04 20:17:54 +00:00
Brooks Prumo 46fe56171b
Make StakeHistory clone-on-write (#21573) 2021-12-03 12:10:29 -06:00
Justin Starry 1430b58a6d
Remove deprecated slow epoch boundary methods (#21568) 2021-12-03 17:59:10 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports 
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
 2021-12-03 09:19:13 -08:00
Brooks Prumo 0ef1b25e4b
Make StakeDelegations clone-on-write (#21542) 2021-12-03 08:54:38 -06:00
Alexander Meißner 1a4a039913
Adds feature reject_empty_instruction_without_program. (#21591) 2021-12-03 15:47:18 +01:00
Alexander Meißner a9d5ef2055
Cleanup: InvokeContext accessors (#21574) 
* Removes blockhash accessors from InvokeContext.

* Removes lamports_per_signature accessors from InvokeContext.

* Removes return_data accessors from InvokeContext.

* Removes feature_set accessor from InvokeContext.

* Removes instruction_recorders and instruction_index accessors from InvokeContext.

* Moves get_sysvars() into InvokeContext.

* Removes compute_meter parameter from InvokeContext::new().

* Removes InvokeContext::new_mock_with_sysvars_and_features().

* Removes InvokeContext::update_timing().
 2021-12-03 12:15:22 +01:00
Brooks Prumo bdc33ba0a1
Remove unnecessary Option (#21569) 2021-12-02 20:19:22 +00:00
Tao Zhu bf5b7f5d7f
report compute units without tx_wide_compute_cap feature enabled (#21421) 2021-12-02 12:14:57 -06:00
Jeff Washington (jwash) f0b32b75ab
AcctIdx: use ZeroLamport trait (#21552) 2021-12-02 12:10:11 -06:00
Jeff Washington (jwash) 314605e149
AcctIdx: test cleanup (#21550) 2021-12-02 12:09:37 -06:00
Jack May 976eb81d4f
Cleanup the bank's use of nonces (#21246) 2021-12-02 09:57:05 -08:00
Alexander Meißner bfdb775ffc
Unifies ThisInvokeContext and dyn trait InvokeContext. (#21563) 2021-12-02 18:47:16 +01:00
Jeff Washington (jwash) 8dfa83c579
AcctIdx: minor code cleanup (#21547) 2021-12-02 11:05:54 -06:00
Anton Lazarev 6bb884836c address review feedback - use u64 instead of usize 2021-12-02 08:12:30 -08:00
Anton Lazarev adbf31b98c avoid usize overflow for MAXIMUM_APPEND_VEC_FILE_SIZE on 32-bit platforms 2021-12-02 08:12:30 -08:00
Jeff Washington (jwash) c8a52337c8
AcctIdx: remove unused function (#21546) 2021-12-01 17:14:49 -06:00
Trent Nelson 1fae3d24de bank: reuse last hash/fees helper to instatiate `InvokeContext` 2021-12-01 12:59:20 -07:00
Trent Nelson 91c6a10cda bank: get hash and fee rate from the same source when storing accounts 2021-12-01 12:59:20 -07:00
Jeff Washington (jwash) 308d7d40d0
aggressively flush cache based on cache size (#21332) 
* aggressively flush cache based on cache size

* find_older_frozen_slots -> cached_frozen_slots

* remove 'WRITE_CACHE_LIMIT_BYTES_DEFAULT'

* tweaks to stats

* fix tests
 2021-12-01 13:10:48 -06:00
Jeff Washington (jwash) 1eefdeba85
AcctIdx: cleanup trait (#21522) 2021-12-01 12:19:06 -06:00
Brooks Prumo 4a1ef12bd9
Refactor fns that calculate stake and voter balances (#21529) 2021-12-01 10:29:24 -06:00
Jeff Washington (jwash) b108d7ddaa
AcctIdx: insert goes directly to disk to avoid unnecessary allocations (#21490) 
* AcctIdx: upsert avoids unnecessary allocation (during startup)

* feedback
 2021-12-01 10:21:01 -06:00
Brooks Prumo cb368e6554
Pass Epoch by value in StakeHistory::get() (#21523) 2021-12-01 08:57:29 -06:00
Alexander Meißner b78f5b6032
Refactor: Cleanup InstructionProcessor (#21404) 
* Moves create_message(), native_invoke() and process_cross_program_instruction()
from the InstructionProcessor to the InvokeContext so that they can have a useful "self" parameter.

* Moves InstructionProcessor into InvokeContext and Bank.

* Moves ExecuteDetailsTimings into its own file.

* Moves Executor into invoke_context.rs

* Moves PreAccount into its own file.

* impl AbiExample for BuiltinPrograms
 2021-12-01 08:54:42 +01:00
Michael Vines e922c2da9d Update to Rust 1.56.1 2021-11-30 23:28:07 -08:00
Michael Vines dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
ryleung-solana 8cf36e5cb0
Add GPU sigverify for verify path (#20851) 
Allows the use of GPU acceleration in verifying the signatures in Entry's after deserialization in the replay stage

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
Co-authored-by: Ryan Leung <ryan.leung@solana.com>
 2021-11-30 21:16:13 -05:00
Jeff Washington (jwash) d8fb7ce511
AcctIdx: upsert avoids unnecessary allocation (#21488) 2021-11-30 11:36:46 -06:00
Jeff Washington (jwash) 7ec88226ee
AcctIdx: stats are usize (#21493) 
* AcctIdx: stats are usize

* rename test function
 2021-11-30 11:35:21 -06:00
Jeff Washington (jwash) 2be859d304
add test for aborting index scan (#21487) 2021-11-29 16:13:39 -06:00
Michael Vines ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
Trent Nelson 41cc7af7cd bank: `TransactionLogCollector` - only push address mappings after pushing a log message 2021-11-25 04:50:51 +00:00
Trent Nelson e558ad4c71 bank: don't panic when getting logs from malformed TransactionLogCollector 2021-11-25 04:50:51 +00:00
Trent Nelson dafdc15dfe bank: Add failing test for `TransactionLogCollector::get_logs_for_address()` 2021-11-25 04:50:51 +00:00
Trent Nelson efedb55705 bank: factor tx log getter out to method on `TransactionLogCollector` 2021-11-25 04:50:51 +00:00
Tao Zhu 9edfc5936d
Refactor accounts.rs with Justin's comments to improve lock accounts (#21406) 
with results code path.
- fix a bug that could unlock accounts that weren't locked
- add test to the refactored function
- skip enumerating transaction accounts if qos results is an error
- add #[must_use] annotation
- avoid clone error in results
- add qos error code to unlock_accounts match statement
- remove unnecessary AbiExample
 2021-11-23 21:17:55 +00:00
Alexander Meißner 22a2537aac
Refactor: Cleanup program-runtime dyn Traits (#21395) 
* Unifies dyn Trait ComputeMeter, ThisComputeMeter and TransactionComputeMeter.

* Unifies dyn Trait Logger and ThisLogger.

* Moves Logger to log_collector.rs

* Unifies Logger and LogCollector.

* Removes inner RefCell from LogCollector.

* Adds the log::debug!() message to ic_logger_msg!() again.
 2021-11-23 13:23:40 +01:00
Jeff Washington (jwash) b87ebf9e58
AcctIdx: PreAllocatedAccountMapEntry does not make unnecessary Arc (#21364) 2021-11-22 15:52:45 -06:00
Jeff Washington (jwash) a762066310
AcctIdx: don't log disk stats when disk isn't enabled (#21371) 2021-11-22 15:26:25 -06:00
sakridge c4d68063c7
Add timing for accounts add_root (#21379) 2021-11-22 18:29:45 +01:00
sakridge cd6f931223
Add timing metrics to bank creation and rewards calculation (#21066) 2021-11-21 15:11:02 +01:00
Jon Cinque 02bc4e3fc1
spl-token: New program feature flag (#21354) 
* spl-token: Add feature flag for new release

* Remove all spl token version declarations
 2021-11-21 14:27:03 +01:00
Jeff Washington (jwash) ebea3297f9
AcctIdx: generate index inserts/updates directly to disk (#21363) 
* when initially creating account index, write directly to disk

* AcctIdx: generate index inserts/updates directly to disk
 2021-11-19 17:17:07 -06:00
Brooks Prumo 0bb059185c
Add lamports_in_rent_paying_accounts to TotalAccountsStats (#21368) 2021-11-19 17:02:23 -06:00
Alexander Meißner 8a50b6302f
Fix #21346 (#21362) 
Fixes the empty transaction bug in ThisInvokeContext::push() and adds a test for it to the bank.
 2021-11-19 20:43:42 +01:00
Jeff Washington (jwash) 79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort (#21327) 
* ScanConfig -> &ScanConfig

* add --accounts-index-scan-results-limit-mb to allow scans to abort

* feedback
 2021-11-19 09:00:19 -06:00
Brooks Prumo c24e30f689
Constrain memory ordering on AccountsDb::next_id (#21311) 2021-11-18 20:34:37 -06:00
Brooks Prumo f7152c889c
Constrain memory ordering on AppendVec::id (#21310) 2021-11-18 20:34:20 -06:00
Brooks Prumo 5788973153
Constrain memory ordering on AccountStorageEntry::alive_bytes (#21309) 2021-11-18 20:34:08 -06:00