fa7926580a
minor VoteAccount refactoring ( #23686 )
2022-03-16 09:50:12 -05:00
3252dc7203
uses structural sharing for stake-delegations hash-map ( #23585 )
...
StakeDelegations is using Arc to implement copy-on-write semantics:
https://github.com/solana-labs/solana/blob/58c0db970/runtime/src/stake_delegations.rs#L14-L16
However a single delegation change will still clone the entire hash-map,
resulting in excessive memory use as observed in:
https://github.com/solana-labs/solana/issues/23061#issuecomment-1063444072
This commit instead uses immutable hash-map implementing structural
sharing:
> which means that if two data structures are mostly copies of each
> other, most of the memory they take up will be shared between them.
https://docs.rs/im/latest/im/
2022-03-16 12:58:05 +00:00
584ac80b1e
Revert: `KeyedAccount` refactoings in builtin programs ( #23649 )
...
* Revert "Replaces KeyedAccount by BorrowedAccount in the BPF loader. (#23056 )"
6c56eb9663#23217 )"
ee7e411d68#23214 )"
1a68f81f89#23302 )"
a14c7c37ee#23348 )"
e2fa6a0f7a#23375 )"
ee3fc39f1c
2022-03-16 11:30:01 +01:00
2d3501dff9
make upsert infallible op
2022-03-15 17:05:41 -05:00
b5a99b9b09
avoid data copies in writing to cache ( #23674 )
2022-03-15 16:42:26 -05:00
29af42f428
chore: bump zstd from 0.11.0+zstd.1.5.2 to 0.11.1+zstd.1.5.2 ( #23679 )
...
* chore: bump zstd from 0.11.0+zstd.1.5.2 to 0.11.1+zstd.1.5.2
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.11.0+zstd.1.5.2 to 0.11.1+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/commits )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-15 15:31:14 -06:00
8c4f010b8d
fix logging: only walk remote dir if it exist ( #23663 )
2022-03-15 08:56:22 -05:00
8c8f9694e0
Refactor: Sanitized transaction creation ( #23558 )
...
* Refactor: SanitizedTransaction::try_create optionally computes hash
* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
f05ac7a899
move to get_slots_in_epoch ( #23657 )
2022-03-14 22:47:45 -05:00
102dd68a03
Rename AccountsDb plugins to Geyser plugins ( #23604 )
2022-03-14 19:18:46 -06:00
c68c0e881e
consolidate use-jit flag ( #23652 )
2022-03-14 15:00:00 -07:00
0c684721d8
Separate remotely downloaded snapshot archives ( #23510 )
...
* seperate remotely downloaded snapshot archives
* add str const for snapshot download dir
* only walk remote sub directory
* move directory creation outside of loop
* move is_remote to traits
* clippy simplify
* clippy
* clippy
* add unittest
* fix local cluster tests
* look for remote snapshot archive in remote foler
* create remote dir in tests
* use snapshot download dir constant
* extract build_remote_dir fn
* fix build
* code review - walking snapshot archived dirs explicitly
* fix build
* fix build
* fix comments
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* clippy
* borrow to avoid copy
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-14 14:03:59 -05:00
a3242b6b86
remove unnecessary namespaces ( #23646 )
2022-03-14 12:28:55 -05:00
63bf0f66af
Bump version to 1.10.3 ( #23648 )
2022-03-14 11:18:45 -05:00
11be3fb7fa
Add FlushGuard to ensure `flushing_active` is used safely ( #23571 )
2022-03-14 10:35:00 -05:00
e8a8f4e9e2
fix filler acct slots in epoch ( #23536 )
2022-03-14 09:57:44 -05:00
7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit ( #23537 )
2022-03-13 15:58:57 +00:00
ccff006948
Add more logs to `purge_old_snapshot_archives` ( #23531 )
2022-03-11 15:01:48 -06:00
d20dd21600
Sort tables in Cargo.toml files ( #23602 )
2022-03-11 11:05:57 -06:00
9b591286d7
Revert "chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )" ( #23592 )
...
This reverts commit 3a0271c113
2022-03-11 09:12:27 +01:00
e6a67cd091
Remove unused dependencies ( #23595 )
2022-03-10 22:20:52 -06:00
631be1ffdd
chore: bump zstd from 0.10.0+zstd.1.5.2 to 0.11.0+zstd.1.5.2 ( #23581 )
...
* chore: bump zstd from 0.10.0+zstd.1.5.2 to 0.11.0+zstd.1.5.2
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.10.0+zstd.1.5.2 to 0.11.0+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/compare/v0.10.0...v0.11.0 )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-10 20:01:05 -07:00
b444836a97
Bump version to 1.10.2 ( #23597 )
2022-03-10 16:41:06 -06:00
3c6840050c
Ensure blocks do not exceed the max accounts data size during Replay Stage ( #23422 )
2022-03-10 10:24:31 -06:00
17b00ad3a4
Add quic-client module ( #23166 )
...
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
2022-03-09 21:33:05 -05:00
1fe0d6eeeb
Set ordering flushing_active.swap() to AcqRel ( #23567 )
2022-03-10 01:22:42 +00:00
1eddb6d1e9
Move ArchiveFormat into own module ( #23562 )
2022-03-09 16:09:34 -06:00
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
ba54b30101
clippy ( #23563 )
2022-03-09 15:33:50 -06:00
5a0cd05866
Revert "- estimate a program cost as 2 standard deviation above mean"
...
This reverts commit a25ac1c988
2022-03-08 17:18:44 -08:00
9acbfa5eb1
Revert "use EMA in place of Welford"
...
This reverts commit 6587dbfa47
2022-03-08 17:18:44 -08:00
c878c9e2cb
Revert "1. Persist to blockstore less frequently;"
...
This reverts commit 7aa1fb4e24
2022-03-08 17:18:44 -08:00
cc4d75a16f
fix typo ( #23535 )
2022-03-08 18:28:00 -06:00
5599bd9442
Remove BankFromArchiveTimings from ledger/
2022-03-08 08:11:50 -08:00
3114c199bd
Add RPC support for versioned transactions ( #22530 )
...
* Add RPC support for versioned transactions
* fix doc tests
* Add rpc test for versioned txs
* Switch to preflight bank
2022-03-08 15:20:34 +08:00
ddbf5c782f
Fix broken doc links in solana-runtime ( #23504 )
2022-03-07 22:21:40 +00:00
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
3a0271c113
chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )
...
* chore: bump dashmap from 4.0.2 to 5.1.0
Bumps [dashmap](https://github.com/xacrimon/dashmap ) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases )
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0 )
---
updated-dependencies:
- dependency-name: dashmap
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-07 00:00:26 -05:00
463cd564cf
fix typos ( #23495 )
...
* fix typos
2022-03-05 20:46:46 -06:00
38db1dead4
Refactor RPC tests ( #23483 )
2022-03-04 18:20:11 +08:00
b28acd2d4d
Reduce fn visibility
2022-03-04 09:52:46 +01:00
afda8c4020
chore: bump memmap2 from 0.5.2 to 0.5.3 ( #23475 )
...
* chore: bump memmap2 from 0.5.2 to 0.5.3
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.5.2 to 0.5.3.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.2...v0.5.3 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-03 14:45:07 -07:00
7f608965ef
new counter data point for unexpected rent_epoch ( #23449 )
2022-03-03 09:09:31 -06:00
ddfd4f86f3
exit early from acct bg svc if no shrink candidates ( #23459 )
2022-03-03 08:47:39 -06:00
a99fd09c16
allow index update to change storage slot # ( #23311 )
2022-03-03 08:40:48 -06:00
634f4eb37d
(LedgerStore) Use different path for different blockstore storage type. ( #23236 )
...
#### Summary of Changes
To avoid mixing the use of different shred storage types, each shred storage type
will have its blockstore in a different directory.
This PR still keeps the RocksFifo setting hidden. The default ShredStorageType and
blockstore directory are still RocksLevel and `rocksdb`.
Will follow-up with PRs on making FIFO option public in ledger-tool and validator.
#### Test Plan
* Added a new test to verify the existence of `rocksdb-fifo` directory when FIFO compaction is used.
* Updated existing test to verify the current setting still store ledger under `rocksdb` directory.
* Manually ran ledger_cleanup_test with both level and fifo compaction and verified the resulting ledger.
* Ran a validator with this PR.
2022-03-02 18:30:22 -08:00
39387e8446
chore: bump blake3 from 1.3.0 to 1.3.1 ( #23460 )
...
* chore: bump blake3 from 1.3.0 to 1.3.1
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.3.0...1.3.1 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-02 18:20:30 -07:00
97d40ba3da
Resized accounts must be rent exempt
2022-03-02 13:02:02 -08:00
1a99251498
Bump version to 1.10.1 ( #23453 )
2022-03-02 13:47:01 -06:00
da00d29de0
chore: bump bytemuck from 1.7.3 to 1.8.0 ( #23437 )
...
* chore: bump bytemuck from 1.7.3 to 1.8.0
Bumps [bytemuck](https://github.com/Lokathor/bytemuck ) from 1.7.3 to 1.8.0.
- [Release notes](https://github.com/Lokathor/bytemuck/releases )
- [Changelog](https://github.com/Lokathor/bytemuck/blob/main/changelog.md )
- [Commits](https://github.com/Lokathor/bytemuck/compare/v1.7.3...v1.8.0 )
---
updated-dependencies:
- dependency-name: bytemuck
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-02 19:04:54 +00:00
e88da2ec0a
minor test cleanup ( #23447 )
...
* minor test cleanup
* fmt
2022-03-02 12:10:05 -06:00
e630eb73d7
rework test to avoid multi-epoch gap ( #23393 )
2022-03-02 11:12:10 -06:00
ef8b7d9c62
ledger tool halt at slot verify hash ( #23424 )
2022-03-02 11:11:18 -06:00
d909b7c80b
log hash and lamport result of calculate_accounts_hash_without_index ( #23425 )
2022-03-02 11:10:40 -06:00
7b7fdb42d9
minor test refactoring ( #23423 )
2022-03-02 08:31:14 -06:00
c8cb940b4e
tweak 2 rent tests to narrow epoch ranges ( #23420 )
2022-03-02 08:22:33 -06:00
454e82683e
refactor rent test ( #23392 )
2022-03-01 08:47:09 -06:00
6b2683f7da
refactor test ( #23384 )
2022-03-01 08:22:10 -06:00
19448ba078
Allow sub-rent-exempt-minimum transfers to `1nc1nerator` ( #23382 )
...
* Add failing test
* Allow small burns to incinerator
* Use check_id method
2022-02-28 23:56:34 -07:00
f0a235d16f
add comment ( #23378 )
2022-02-28 15:18:42 -06:00
4eeb9f4648
acct idx comments ( #23377 )
2022-02-28 15:00:53 -06:00
611d745241
refactor rent_due for normal case of exempt ( #23350 )
2022-02-28 09:42:42 -06:00
30dafc7135
list -> slot_list ( #23355 )
2022-02-27 23:08:58 -06:00
36484f4f08
Prevent new RentPaying state created by paying fees ( #23358 )
...
* Add failing test
* Check fee-payer rent-state change on load
* Add more test cases
* Review comments
2022-02-26 12:10:01 -07:00
0ad4757159
plumbing for 'other_slot' in 'update_index' ( #23330 )
2022-02-25 12:58:08 -06:00
5e0086c1ee
followup safety checks for #23295
2022-02-24 17:50:58 -07:00
db12d90735
chore: bump ouroboros from 0.13.0 to 0.14.2 ( #23328 )
...
* chore: bump ouroboros from 0.13.0 to 0.14.2
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.13.0 to 0.14.2.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits/0.14.2 )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-02-24 15:56:13 -07:00
2b0f16e7c3
slot per account in StorableAccounts ( #23313 )
2022-02-24 10:34:38 -06:00
017170c99d
DiskIdx: new items are not dirty by default ( #23123 )
2022-02-24 10:17:35 -06:00
c0d0724be0
clippy ( #23310 )
2022-02-23 18:07:00 -06:00
7ee549e5ae
fix params for store_accounts_frozen ( #23312 )
2022-02-23 17:40:51 -06:00
227df52213
update comment ( #23314 )
2022-02-23 17:33:06 -06:00
cafc18c3f9
update_index uses ReadableAccount to reduce params ( #23305 )
2022-02-23 17:01:23 -06:00
a245efe83d
trait for acct data with slot per item ( #23285 )
2022-02-23 15:22:29 -06:00
c81dd602c4
Hack fix for ICE as seen in CI
2022-02-23 11:17:39 -07:00
2996f1f783
add comments ( #23275 )
2022-02-23 10:53:11 -06:00
d0e85c293f
Fix rustfmt check ( #23296 )
2022-02-23 16:38:53 +08:00
20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array ( #23085 )
2022-02-22 14:46:56 -08:00
88e1192c6b
Refactor new() and default() for CostTracker ( #23267 )
2022-02-22 10:44:23 -06:00
8b32e80ee2
DiskIdx: rename 'remove' to 'evict' ( #23188 )
2022-02-22 09:40:25 -06:00
7ebf398ed7
AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB" ( #23194 )
...
* AcctIdx: env var "SOLANA_TEST_ACCOUNTS_INDEX_MEMORY_LIMIT_MB"
* ignore env var when starting as validator
* Update runtime/src/bucket_map_holder.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-02-22 09:40:12 -06:00
bcda74f42f
Fix builtin handling on epoch boundaries ( #23256 )
2022-02-22 20:54:08 +08:00
72c68695b5
Do not keep oldest full snapshot
2022-02-21 17:10:31 -07:00
7e08ae1d0c
Revert "Add simulation detection countermeasure ( #22880 )" ( #23261 )
...
This reverts commit c42b80f099
2022-02-21 21:15:37 +00:00
04d23a1597
fix memory ordering in append_vec ( #23215 )
2022-02-20 20:30:49 -06:00
70ebab2c82
Add rustfmt.toml and `cargo fmt` ( #23238 )
...
* fmt
* formatted
Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
1719d2349f
Skip adding builtins if they will be removed ( #23233 )
...
* Add failing test for precompile transition
* Skip adding builtins if they will be removed
* cargo clean
* nits
* fix abi check
* remove workaround
Co-authored-by: Jack May <jack@solana.com>
2022-02-18 18:36:59 -08:00
ee7e411d68
Replaces `KeyedAccount` by `BorrowedAccount` in `system_instruction_processor`. ( #23217 )
...
* Adds InstructionContext::check_number_of_instruction_accounts() and InstructionContext::get_instruction_account_key().
Bases check_sysvar_account() on instuction account indices.
* Adds instruction_account_indices enums to system_instruction_processor.
* Reorders parameters and adds InstructionContext.
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 1).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 2).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 3).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 4).
* Replaces KeyedAccount by BorrowedAccount in system_instruction_processor (part 5).
* Code cleanup
2022-02-19 02:06:33 +01:00
970f543ef6
Precompiles owned by the native loader ( #23237 )
2022-02-18 15:41:59 -08:00
1a68f81f89
Replaces `KeyedAccount` by `BorrowedAccount` in `nonce_keyed_account`. ( #23214 )
...
* Adds get_sysvar_with_account_check2 for ABIv2.
* Replaces get_signers() and get_sysvar_with_account_check() in system_instruction_processor.
* Replaces KeyedAccount by BorrowedAccount in nonce_keyed_account.
2022-02-17 17:36:55 +01:00
da00b39f4f
Cleanup: get_program_key() and get_loader_key() in TransactionContext ( #23191 )
...
* Moves TransactionContext::get_program_key() to InstructionContext::get_program_key().
* Removes TransactionContext::get_loader_key().
* Test full program and loader executable account chain in BPF loader.
2022-02-17 10:16:28 +01:00
a102453bae
accounts_index: Add SPL Token account indexing for token-2022 accounts ( #23043 )
2022-02-16 16:23:25 -08:00
813725dfec
Fix ed25519 builtin program handling ( #23182 )
...
* Fix ed25519 builtin program handling
* Fix tests
* Add integration tests for processing transactions with ed25519 ixs
* Fix another test
* fix formatting
2022-02-16 18:48:55 +00:00
c2435363f3
DiskIdx: fix race condition with holding ranges in mem ( #23158 )
2022-02-16 09:25:00 -06:00
577fa4ec0c
Generate full snapshots 4x faster to keep incremental snapshots nice and small
2022-02-15 21:09:21 -08:00
43393ea45d
assert on AcctInfo.store_id with cached store ( #21952 )
2022-02-15 15:22:36 -06:00
197f42d346
chore: bump zstd from 0.9.2+zstd.1.5.1 to 0.10.0+zstd.1.5.2 ( #23089 )
...
* chore: bump zstd from 0.9.2+zstd.1.5.1 to 0.10.0+zstd.1.5.2
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.9.2+zstd.1.5.1 to 0.10.0+zstd.1.5.2.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/compare/v0.9.2...v0.10.0 )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-02-15 10:56:04 -07:00
c42b80f099
Add simulation detection countermeasure ( #22880 )
...
* Add simulation detection countermeasures
* Add program and test using TestValidator
* Remove incinerator deposit
* Remove incinerator
* Update Cargo.lock
* Add more features to simulation bank
* Update Cargo.lock per rebase
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-15 13:09:59 +01:00
20bc37c9c9
Removes KeyedAccount in tests of system_instruction_processor. ( #23118 )
2022-02-15 00:00:49 +01:00
3d9874b95a
Add fees to tx-wide caps ( #22081 )
2022-02-11 16:23:16 -08:00
0a1ab945bc
Move cap_accounts_data_len feature gate only around new error ( #23048 )
2022-02-10 11:57:00 -06:00
47e4291303
add Debug to RentResult ( #23044 )
2022-02-09 18:07:36 -06:00
226a71f073
Refactor: Use instruction_trace instead of instruction_context_stack ( #22988 )
...
* Moves stack_height of the instruction_trace into InstructionContext.
* Uses nesting_level instead of stack_height internally in transaction_context.rs
* Uses the instruction_trace instead of the instruction_context_stack internally.
* Adds feature gate: record_instruction_in_transaction_context_push.
2022-02-09 20:04:49 +01:00
205cd4609b
bank::get_rent_collector_from_parent ( #23016 )
2022-02-09 12:31:58 -06:00
f334931903
rent: refactor test function ( #23014 )
2022-02-08 22:11:47 -06:00
fb47c1f0a3
consolidate Rent construction in test code ( #23018 )
2022-02-08 21:51:52 -06:00
869cfc9a1c
Return the accounts data len delta after processing messages ( #22986 )
2022-02-08 19:24:47 -06:00
7aa1fb4e24
1. Persist to blockstore less frequently;
...
2. reduce alpha for EMA to 1 percent to have roughly 200 data points for estimatio
2022-02-08 16:18:23 -06:00
6587dbfa47
use EMA in place of Welford
2022-02-08 16:18:23 -06:00
a25ac1c988
- estimate a program cost as 2 standard deviation above mean
...
- replaced get_average / get_mode with get_default to assign max units to unknown program
2022-02-08 16:18:23 -06:00
86cf226395
eliminate separate max root member ( #22943 )
2022-02-08 09:31:32 -06:00
f0f4042680
Put accounts data len updates behind feature gate ( #22918 )
2022-02-08 08:51:16 -06:00
b2e475b5c4
clarify that RollingBitField max is exclusive ( #22947 )
2022-02-07 13:40:31 -06:00
a160fc30f2
make max_roots_inclusive clear ( #22942 )
2022-02-07 13:26:53 -06:00
ba215e94f6
Refactor: Add AccountKeys struct for static and dynamic message keys ( #22960 )
2022-02-05 20:00:31 +08:00
660f6981c6
Cleanup: TransactionContext ( #22910 )
...
* Adds BorrowedAccount::check_sysvar().
* Adds BorrowedAccount::get_data_mut().
* Implements account resizing in BorrowedAccount.
* Exposes is_signer() and is_writable() in InstructionContext.
* Removes AccountMeta and get_instruction_accounts_metas().
* Makes throwing errors in BorrowedAccount optional.
* Removes result return values from BorrowedAccount.
2022-02-03 17:19:42 +01:00
c16cf9cf8a
Refactor: Use `SysvarCache` in all builtin programs ( #22864 )
...
* Replaces from_keyed_account() by SysvarCache in stake instruction.
* Replaces from_keyed_account() by SysvarCache in system instruction processor.
* Removes from_keyed_account().
Moves check_sysvar_keyed_account() into sysvar_cache.rs
* Removes tests which test for incorrectly serialized sysvars.
2022-02-03 13:03:50 +01:00
60af1a4cce
Refactor: Add trait for loading addresses ( #22903 )
2022-02-03 11:00:12 +00:00
cc94a93b56
Safer invoke context ( #22898 )
...
* Safer invoke context
* feedback and rebase with master
2022-02-03 02:34:51 -08:00
bd1850df25
Return actual committed transactions from process_transactions() ( #22802 )
2022-02-03 03:56:36 -05:00
ab02dba96f
Add get_processed_sibling_instruction syscall ( #22859 )
2022-02-02 16:45:57 -08:00
75563f6c7b
Reject close of active vote accounts ( #22651 )
...
* 10461 Reject close of vote accounts unless it earned no credits in the previous epoch. This is checked by comparing current epoch (from clock sysvar) with the most recent epoch with credits in vote state.
2022-02-02 14:16:24 -06:00
fb0e71946f
Cleanup use declaration ( #22875 )
2022-02-01 23:34:11 +00:00
22f6db2e4f
Cleanup serde snapshot common.rs ( #22854 )
2022-02-01 05:39:53 -06:00
551c24da57
nit: consistent name ( #22857 )
2022-02-01 07:46:04 +00:00
d60dac9749
add comment ( #22846 )
2022-01-31 11:40:19 -06:00
fb95fa68a2
cleanup api ( #22845 )
2022-01-31 11:39:23 -06:00
bc800a8d5a
Refactor: Unify `SysvarCache` ( #22843 )
...
* Unifies SysvarCache filling in the runtime and tests.
Removes new_mock_with_sysvars_and_features()
Removes mock_process_instruction_with_sysvars().
Replaces from_keyed_account() by SysvarCache in vote processor.
* Replaces from_keyed_account() by SysvarCache in BPF loader.
2022-01-31 17:53:50 +01:00
cc74693176
chore: bump serde from 1.0.134 to 1.0.136 ( #22817 )
...
* chore: bump serde from 1.0.134 to 1.0.136
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.134 to 1.0.136.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.134...v1.0.136 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-28 12:20:37 -07:00
94a5aee484
Add new_from_parent() timings ( #22744 )
2022-01-27 20:35:13 -05:00
89c42ebcbe
refactor rent collection ( #22803 )
2022-01-27 18:48:04 -06:00
350207b05f
impl ReadableAccount for LoadedAccount ( #22793 )
2022-01-27 16:59:30 -06:00
18d69c8d2b
SortedStoragesIter to remove random access requirement ( #22713 )
2022-01-27 10:25:13 -06:00
d9c259a231
Set the correct root in block commitment cache initialization ( #22750 )
...
* Set the correct root in block commitment cache initialization
* clean up test
* bump
2022-01-27 00:48:00 +08:00
071e97053f
Perf: Reduce write locks on blockhash queue ( #22729 )
...
* Perf: Reduce write locks on blockhash queue
* Add comment about thread safety
* Add comment about write starvation
2022-01-26 16:24:27 +08:00
8b90084ebc
fix comment typos ( #22723 )
2022-01-25 11:10:42 -06:00
fc21af4e6e
spl-associated-token-account: Add feature for new program ( #22648 )
...
* spl-associated-token-account: Add feature for new program
* Address feedback
2022-01-24 22:54:41 +01:00
6d5bbca630
Pacify clippy
2022-01-21 19:12:57 -08:00
0cf886302d
chore: bump serde from 1.0.133 to 1.0.134 ( #22650 )
...
* chore: bump serde from 1.0.133 to 1.0.134
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.133 to 1.0.134.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.133...v1.0.134 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-21 17:14:02 -07:00
e3915e4b7a
add comment ( #22636 )
2022-01-21 10:51:58 -06:00
4abf7a7f88
add ActiveStats to track when long running, expensive bg processes are running ( #22608 )
2022-01-21 10:51:23 -06:00
9977396d8f
Remove unused fields from Bank ( #22491 )
2022-01-21 06:03:41 -06:00
7d34a7acac
Perf: Only check executors cache for executable bpf program ids ( #22624 )
...
* Only check executors cache for executable bpf program ids
* switch to native loader check
* clean up tests
* fix tests
* clippy
2022-01-21 18:32:30 +08:00
0eb488580d
minor cleanup ( #22610 )
2022-01-20 18:11:00 -06:00
7ba57e7a7c
Refactor: move instructions sysvar serialization out of Message ( #22544 )
2022-01-20 17:33:49 +08:00
7f20c6149e
Refactor: move simple vote parsing to runtime ( #22537 )
2022-01-20 10:39:21 +08:00
d343713f61
Optimize packet dedup ( #22571 )
...
* Use bloom filter to dedup packets
* dedup first
* Update bloom/src/bloom.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* fixup
* fixup
* fixup
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-01-19 13:58:20 -08:00
b448472037
Refactor: Move `InstructionRecorder` into `TransactionContext` ( #22578 )
...
* Moves InstructionRecorder into TransactionContext.
* Adds assertions for number_of_instructions_at_transaction_level.
2022-01-19 22:40:09 +01:00
cc76a73c49
Refactor: move compute budget runtime logic into solana-program-runtime ( #22543 )
2022-01-17 20:48:00 +08:00
2c38a9213f
Revert "Refactor: move compute budget runtime logic into solana-program-runtime ( #22533 )" ( #22542 )
...
This reverts commit b27976626a
2022-01-17 17:43:17 +08:00
b27976626a
Refactor: move compute budget runtime logic into solana-program-runtime ( #22533 )
2022-01-17 15:52:33 +08:00
6edeed888d
Remove unnecessary & from AsRef params ( #22523 )
2022-01-15 04:51:05 +00:00
70633b5c2f
Add Bank::accounts_data_len to SetRootMetrics ( #22509 )
2022-01-14 20:00:07 -06:00
a724fa2347
Add hidden cli option to allow validator reports replayed transaction cost metrics ( #22369 )
...
* add hidden cli option to allow validator reports replayed transaction cost detail metrics
* Update validator/src/main.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* - rebase master, using unbounded instead of channel; dowgrade to datapoint_trace
* removed cli arg, prefer log at trace
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-01-15 00:31:21 +00:00
7d7228c356
cleanup assert ( #22495 )
2022-01-14 11:53:24 -06:00
9c9f2dd5bd
port counting vote CUs to block cost ( #22477 )
2022-01-14 10:50:29 -06:00
cddab635ff
Cleanup activated rent_for_sysvars feature ( #22454 )
2022-01-14 20:34:09 +08:00
ae6c511f13
Refactor: Split vote_instruction.rs into multiple files ( #22502 )
2022-01-14 17:25:15 +08:00
93a7b94507
Add benchmark for vote processing ( #22486 )
2022-01-14 17:10:17 +08:00
4c577d7f8c
`Bank::get_fee_for_message` is now nonce aware
2022-01-13 17:27:38 -08:00
d3b95b9226
Cleanup AccountStorage struct ( #22463 )
...
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824 )"
This reverts commit 8aa3d690b5
2022-01-13 17:51:53 -06:00
7711cd74c3
partition_from_pubkey ( #22430 )
...
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824 )"
This reverts commit 8aa3d690b5
2022-01-13 17:02:42 -06:00
e291342c4a
Revert "chore: bump dashmap from 4.0.2 to 5.0.0 ( #21824 )" ( #22488 )
...
This reverts commit 8aa3d690b5
2022-01-13 13:06:39 -06:00
b82d71d22a
Inline DEFAULT_SNAPSHOT_VERSION constant string ( #22487 )
2022-01-13 18:19:15 +00:00
2756abce39
More serde snapshot cleanup ( #22449 )
2022-01-13 09:20:20 -06:00
9c3144e286
Refactor serde snapshot's "future" to "newer" ( #22431 )
2022-01-13 07:12:09 -06:00
2370e61431
Perf: Store deserialized sysvars in the sysvars cache ( #22455 )
...
* Perf: Store deserialized sysvars in sysvars cache
* add bench
2022-01-13 05:36:21 +00:00
7171c95bdd
Refactor: move sysvar cache to new module
2022-01-12 12:35:28 -07:00
3ca16de851
Only examine explicit tx accounts for rent state ( #22441 )
...
* Add failing test
* Fix: only examine accounts explicitly included in a tx
2022-01-11 20:23:51 -07:00
c45dde6164
Handle accounts data size changes due to rent-collected accounts ( #22412 )
2022-01-11 17:20:28 -06:00
157f165a3d
Bump memmap2 from 0.5.1 to 0.5.2 ( #22414 )
...
* Bump memmap2 from 0.5.1 to 0.5.2
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.5.1 to 0.5.2.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.1...v0.5.2 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-11 15:08:13 -07:00
637e366b18
Prevent rent-paying account creation ( #22292 )
...
* Fixup typo
* Add new feature
* Add new TransactionError
* Add framework for checking account state before and after transaction processing
* Fail transactions that leave new rent-paying accounts
* Only check rent-state of writable tx accounts
* Review comments: combine process_result success behavior; log and metrics before feature activation
* Fix tests that assume rent-exempt accounts are okay
* Remove test no longer relevant
* Remove native/sysvar special case
* Move metrics submission to report legacy->legacy rent paying transitions as well
2022-01-11 11:32:25 -07:00
a49ef49f87
optimizes ReadOnlyAccountsCache LRU eviction implementation ( #22403 )
...
ReadOnlyAccountsCache is using a background thread, table scan and sort
to implement LRU eviction policy:
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L66-L73
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L186-L191
https://github.com/solana-labs/solana/blob/eaa52bc93/runtime/src/read_only_accounts_cache.rs#L222
DashMap internally locks each shard when accessed; so a table scan in
the background thread can create a lot of lock contention.
This commit adds an index-list queue containing cached keys in the order
that they are accessed. Each hash-map entry also includes its index into
this queue.
When an item is first entered into the cache, it is added to the end of
the queue. Also each time an entry is looked up from the cache it is
moved to the end of queue. As a result, items in the queue are always
sorted in the order that they have last been accessed. When doing LRU
eviction, cache entries are evicted from the front of the queue.
Using index-list, all queue operations above are O(1) with low overhead
and so above achieves an efficient implementation of LRU cache eviction
policy.
2022-01-11 17:25:28 +00:00
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
04a2c19c87
Bump memmap2 from 0.5.0 to 0.5.1 ( #22409 )
...
* Bump memmap2 from 0.5.0 to 0.5.1
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.5.0 to 0.5.1.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.0...v0.5.1 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-10 11:14:56 -07:00
9bc2592da1
Use lazy_rent_collection directly ( #22410 )
2022-01-10 10:47:08 -06:00
28275a33d6
Bump tempfile from 3.2.0 to 3.3.0 ( #22401 )
...
* Bump tempfile from 3.2.0 to 3.3.0
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases )
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.2.0...v3.3.0 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-09 12:26:08 -07:00
3b4aad9df1
bank: fix executor cache metrics
2022-01-09 04:22:39 +00:00
eaa52bc935
Bump zstd from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1 ( #22388 )
...
* Bump zstd from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1
Bumps [zstd](https://github.com/gyscos/zstd-rs ) from 0.9.1+zstd.1.5.1 to 0.9.2+zstd.1.5.1.
- [Release notes](https://github.com/gyscos/zstd-rs/releases )
- [Commits](https://github.com/gyscos/zstd-rs/compare/0.9.1...v0.9.2 )
---
updated-dependencies:
- dependency-name: zstd
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-08 22:21:56 +00:00
d8ec8ca2e6
Bump blake3 from 1.2.0 to 1.3.0 ( #22373 )
...
* Bump blake3 from 1.2.0 to 1.3.0
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 1.2.0 to 1.3.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.2.0...1.3.0 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-08 20:28:04 +00:00
d90d5ee9b6
Refactor Rent::due() with RentDue enum ( #22346 )
2022-01-08 09:03:46 -06:00
4a9f4e2505
improve multi executor cache addition
2022-01-08 03:47:23 -07:00
ad3cb0bc93
--amend
2022-01-08 08:19:27 +00:00
4ce48307bb
bank: prime new executor cache entry use-counts
2022-01-08 08:19:27 +00:00
6d76db1de5
bank: Add executors cache metrics
2022-01-07 16:53:27 -07:00
c2389fc209
removes CowCachedExecutors ( #22343 )
...
Copy-on-write semantics for cached executors can be implemented by a
simple Arc<CachedExecutors> as opposed to CowCachedExecutors:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L244-L247
This will also avoid the need for double locking as in:
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3490-L3491
https://github.com/solana-labs/solana/blob/f1e2598ba/runtime/src/bank.rs#L3525-L3526
2022-01-07 14:06:29 +00:00
52d12cc802
Add runtime support for address table lookups ( #22223 )
...
* Add support for address table lookups in runtime
* feedback
* feedback
2022-01-07 11:59:09 +08:00
f1e2598baa
Retain executor cache counts ( #22322 )
2022-01-06 08:56:00 -08:00
e0c091a9f4
factor out code in do_shrink_slot_stores ( #22306 )
2022-01-06 10:49:24 -06:00
100293c4b5
refactor get_store_for_shrink ( #22307 )
2022-01-06 10:49:01 -06:00
390ef0fbcd
Consolidate process instruction execution timings to own struct
2022-01-06 03:56:46 -07:00
72fc6096a0
Use saturating_add_assign macro
2022-01-06 03:56:46 -07:00
b25e4a200b
Add execute metrics
2022-01-06 03:56:46 -07:00
7d32909e17
move `ExecuteTimings` from `runtime::bank` to `program_runtime::timings`
2022-01-06 03:56:46 -07:00
12e160269e
cache executors on failed transactions ( #22308 )
2022-01-05 22:09:03 -08:00
ab13e39518
Only sum accounts data len from non-zero lamport accounts ( #22309 )
2022-01-05 23:23:37 +00:00
f2ed6f09ee
Skip updating already cached executors if unmodified
2022-01-05 16:08:10 -07:00
bb3a1b6b31
Add zk_token_sdk_enabled feature to gate Zk Token proof program and `sol_zk_token_elgamal_op` syscalls
2022-01-05 11:57:37 -08:00
9f63493789
Refactor: Remove KeyedAccounts (2) ( #22274 )
...
* Adds InstructionContext::get_signers().
Improves error messages when modifying borrowed accounts.
* Removes keyed_accounts from InvokeContext tests.
* Removes keyed_accounts from message_processor.rs
* Removes keyed_accounts from bank.rs
* Removes keyed_accounts from bpf serialization.
2022-01-05 09:39:37 +01:00
45458e7139
Refactor: Improve type safety and readability of transaction execution ( #22215 )
...
* Refactor Bank::load_and_execute_transactions
* Refactor: improve type safety of TransactionExecutionResult
* Add enum for extra type safety in execution results
* feedback
2022-01-05 10:15:15 +08:00
Jeff Washington (jwash)
behzad nouri
Alexander Meißner
Tao Zhu
dependabot[bot]
HaoranYi
Justin Starry
Tyera Eulberg
Jack May
Will Hickey
Brooks Prumo
Giorgio Gambino
sakridge
ryleung-solana
Carl Lin
Michael Vines
Brian Anderson
Yueh-Hsuan Chiang
Trent Nelson
Gavin Chan
Brooks Prumo
Jeff Washington (jwash)
buffalu
Will Hickey
Jon Cinque
anatoly yakovenko
Tao Zhu
Tyera Eulberg
Jeff Biseda