f5388cfe71
tests which assert or panic in other threads now will report an error ( #19532 )
...
* tests which assert or panic in other threads now will report an error
* rename, add assert detecting time out
2021-09-02 11:26:57 -05:00
7eb793a55e
add debugging to test ( #19563 )
...
* add debugging to test
* update cargo
2021-09-02 02:52:16 +00:00
ed2c071fe1
remove unused is_startup ( #19561 )
2021-09-01 18:39:00 -05:00
8378e8790f
Accountsdb replication installment 2 ( #19325 )
...
This is the 2nd installment for the AccountsDb replication.
Summary of Changes
The basic google protocol buffer protocol for replicating updated slots and accounts. tonic/tokio is used for transporting the messages.
The basic framework of the client and server for replicating slots and accounts -- the persisting of accounts in the replica-side will be done at the next PR -- right now -- the accounts are streamed to the replica-node and dumped. Replication for information about Bank is also not done in this PR -- to be addressed in the next PR to limit the change size.
Functionality used by both the client and server side are encapsulated in the replica-lib crate.
There is no impact to the existing validator by default.
Tests:
Observe the confirmed slots replicated to the replica-node.
Observe the accounts for the confirmed slot are received at the replica-node side.
2021-09-01 14:10:16 -07:00
a5f74d86c8
make test pass with 1 bin to remove noisy failure during testing ( #19554 )
2021-09-01 15:58:56 -05:00
dd9481c403
parallelize index_read in shrink ( #19506 )
2021-09-01 15:40:59 -05:00
82a6bbe068
add Debug trait to range for accounts index ( #19531 )
2021-09-01 08:13:56 -05:00
12dc8749cf
add Copy trait to AccountInfo for fast copies to mmapped file ( #19524 )
2021-09-01 08:13:08 -05:00
0088aefa24
Fix tests that make assumptions about tx fee rate ( #19537 )
2021-08-31 22:02:14 -06:00
1d5a8ebc6a
Revert "Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )" ( #19529 )
...
This reverts commit 4d361af976
2021-08-31 22:03:19 -05:00
2df96cd81e
remove type specifics to make funciton more flexible ( #19536 )
2021-08-31 20:03:42 -05:00
fe9ee9134a
Make background services aware of incremental snapshots ( #19401 )
...
AccountsBackgroundService now knows about incremental snapshots. It is
now also in charge of deciding if an AccountsPackage is destined to be a
SnapshotPackage or not (or just used by AccountsHashVerifier).
!!! New behavior changes !!!
Taking snapshots (both bank and archive) **MUST** succeed.
This is required because of how the last full snapshot slot is
calculated, which is used by AccountsBackgroundService when calling
`clean_accounts()`.
File system calls are now unwrapped and will result in a crash. As Trent told me:
>Well I think if a snapshot fails due to some IO error, it's very likely that the operator is going to have to intervene before it works. We should exit error in this case, otherwise the validator might happily spin for several more hours, never successfully writing a complete snapshot, before something else brings it down. This would leave the validator's last local snapshot many more slots behind than it would be had we exited outright and potentially force the operator to abandon ledger continuity in favor of a quick catchup
Other errors will set the `exit` flag to `true`, and the node will gracefully shutdown.
Fixes #19167
Fixes #19168
2021-08-31 18:33:27 -05:00
718ab7c12e
Revert "Add snapshot_runtime_info module ( #18199 )"
...
This reverts commit c1b9d40a64
2021-08-31 17:50:18 -05:00
5cf8d8795b
Feature cleanup ( #19528 )
2021-08-31 14:51:26 -07:00
e1939688c2
parallel shrink ( #19507 )
2021-08-31 11:57:34 -05:00
0e94b8fd4e
remove redundant cancelled_shrink ( #19522 )
2021-08-31 11:51:48 -05:00
9d3afba045
Debug shrink stats ( #19505 )
...
* add some shrink metrics
* renames
2021-08-31 09:04:56 -05:00
09458cc802
Shrink mmap leakage ( #19373 )
...
* Putting stores satisfying future shrink into the candidate list.
* Fixed unit tests for select_shrink_candidates
* Added metrics on stores readded back for next round
* Ehance the metrics on accountsdb to report the total bytes and alive bytes and alive ratio
* Enhance select_shrink_candidates metrics
2021-08-31 00:55:16 -07:00
a655b01700
some basic accounts index refactoring ( #19510 )
2021-08-30 18:40:10 -05:00
6cfd44b811
chore: bump flate2 from 1.0.20 to 1.0.21 ( #19500 )
...
* chore: bump flate2 from 1.0.20 to 1.0.21
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.20 to 1.0.21.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.20...1.0.21 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-30 16:48:02 -06:00
6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 ( #19497 )
...
* chore: bump serde from 1.0.129 to 1.0.130
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 18:19:34 +00:00
8ad52fa095
implements copy-on-write for vote-accounts ( #19362 )
...
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186
This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
2021-08-30 15:54:01 +00:00
2d7f036afd
Add solana-program-runtime crate ( #19438 )
2021-08-27 00:30:36 +00:00
02b050e0f5
replace AccountsIndex btree with hashmap of 8k bins ( #19212 )
2021-08-26 18:12:43 -05:00
98bc694606
hash calculation adds really old slots to dirty_stores ( #19434 )
2021-08-26 14:32:43 -05:00
535de3b302
add test for test_clean_nonrooted ( #19409 )
2021-08-26 13:56:09 -05:00
e492638b7d
chore: bump libc from 0.2.100 to 0.2.101 ( #19442 )
...
* chore: bump libc from 0.2.100 to 0.2.101
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-26 12:38:34 -06:00
46890ac197
chore: bump serde from 1.0.128 to 1.0.129 ( #19395 )
...
* chore: bump serde from 1.0.128 to 1.0.129
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-24 12:32:30 -06:00
c0c95e88d8
chore: bump libc from 0.2.99 to 0.2.100 ( #19371 )
...
* chore: bump libc from 0.2.99 to 0.2.100
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-23 09:34:32 -06:00
aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 ( #19370 )
...
* chore: bump serde from 1.0.127 to 1.0.128
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-23 09:33:34 -06:00
1a4bede918
Remove filter_snapshot_storages_for_incremental_snapshot() ( #19349 )
...
Now that we can get just the subset of snapshot storages we want, the extra
filtering afterwards is redundant.
2021-08-23 09:45:29 -05:00
6d939811e9
Name snapshots consistently ( #19346 )
...
#### Problem
Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:
```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```
#### Summary of Changes
For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
234461f779
Add base_slot to get_snapshot_storages() ( #19348 )
2021-08-20 16:23:43 -05:00
4d361af976
Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )
2021-08-20 17:06:53 +00:00
3e5ba8dcaa
bug: `sysvar::Instructions` is not owned by `Sysvar1111111111111111111111111111111111111` ( #19242 )
...
* Fix instructions sysvar owner
* Update feature switch address
Co-authored-by: Justin Starry <justin@solana.com>
2021-08-20 08:32:28 -07:00
73aa004c59
stake: Remove v2 program references ( #19308 )
...
* stake: Remove v2 program references
* Remove stake v2 feature, along with stake rewrite
2021-08-20 01:08:44 -04:00
3ec33e7d02
Fail secp256k1 if the instruction data looks incorrect ( #19300 )
2021-08-19 13:13:54 -07:00
89a31ff473
change untar to use unpack instead of unpack_in ( #19216 )
...
* change untar to use unpack instead of unpack_in
* hacky, but maybe passes tests
* chore: bump tar from 0.4.35 to 0.4.37
Bumps [tar](https://github.com/alexcrichton/tar-rs ) from 0.4.35 to 0.4.37.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases )
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.35...0.4.37 )
---
updated-dependencies:
- dependency-name: tar
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* cleanup
* cleanup, add validate_inside_dst
* collapse use
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* delete comment line
* add comments
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-08-18 15:49:02 -05:00
6327e006df
CalculateHashIntermediate::new_without_slot -> new ( #19274 )
2021-08-18 09:07:34 -05:00
c167211611
Expose genesis block time via rpc ( #19267 )
...
* Expose genesis_creation_time from Bank
* Backfill genesis_creation_time for block/block-time requests of slot 0
2021-08-17 16:29:34 -06:00
c50b01cb60
Store versioned transactions in the ledger, disabled by default ( #19139 )
...
* Add support for versioned transactions, but disable by default
* merge conflicts
* trent's feedback
* bump Cargo.lock
* Fix transaction error encoding
* Rename legacy_transaction method
* cargo clippy
* Clean up casts, int arithmetic, and unused methods
* Check for duplicates in sanitized message conversion
* fix clippy
* fix new test
* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
098e2b2de3
chore: bump memmap2 from 0.3.0 to 0.3.1 ( #19255 )
...
* chore: bump memmap2 from 0.3.0 to 0.3.1
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v.0.3.0...v0.3.1 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-17 15:49:50 -06:00
7c70f2158b
accounts_index_bins to AccountsIndexConfig ( #19257 )
...
* accounts_index_bins to AccountsIndexConfig
* rename param bins -> config
* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
f9986c66b8
Make SnapshotPackagerService aware of Incremental Snapshots ( #19254 )
...
Add a field to SnapshotPackage that is an enum for SnapshotType, so archive_snapshot_package() will do the right thing.
Fixes #19166
2021-08-17 13:01:59 -05:00
7481d9b66b
exhaustively_free_unused_resource uses more threads ( #19264 )
2021-08-17 12:38:41 -05:00
39a3d5b8a9
implements AsRef (instead of Borrow) for VoteAccounts ( #19252 )
...
According to the docs:
> if you want to borrow only a single field of a struct you can
> implement AsRef, but not Borrow.
https://doc.rust-lang.org/std/convert/trait.AsRef.html
2021-08-16 18:21:33 +00:00
176036aa58
Rename AccountsPacakge to SnapshotPackage and AccountsPackagePre to AccountsPackage ( #19231 )
...
Renaming these types to better communicate their usages, which will
further diverge as incremental snapshot support is added.
With the new names, AccountsPacakge now refers to the type between
AccountsBackgroundProcess and AccountsHashVerifier, and SnapshotPackage
refers to the type between AccountsHashVerifier and
SnapshotPackagerService.
2021-08-13 16:08:09 -05:00
1a3f29e22d
rework bank test to specifically use BTreeMap ( #19225 )
2021-08-13 13:30:48 -05:00
01b00bc593
Fix crossed PRs; remove deprecated fee_calculator usage ( #19228 )
2021-08-13 11:14:00 -06:00
0b50bb2b20
Deprecate FeeCalculator returning APIs ( #19120 )
2021-08-13 09:08:20 -07:00
26e963f436
remove par iter for 32k wide code path ( #18764 )
2021-08-13 10:07:47 -05:00
772341dccb
fixup! Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots ( #18870 ) ( #19213 )
2021-08-12 17:29:58 -05:00
5fb6b341c4
Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots ( #18870 )
...
* Handle cleaning zero-lamport accounts
Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot. This is part of the Incremental Snapshot work.
Fixes #18825
2021-08-12 15:56:08 -05:00
414d904959
Reject blocks for costs above the max block cost ( #18994 )
...
* added realtime cost checking logic to reject block that would exceed max limit:
- defines max limits at block_cost_limits.rs
- right after each bath's execution, accumulate its cost and check again
limit, return error if limit is exceeded
* update abi that changed due to adding additional TransactionError
* To avoid counting stats mltiple times, only accumulate execute-timing when a bank is completed
* gate it by a feature
* move cost const def into block_cost_limits.rs
* redefine the cost for signature and account access, removed signer part as it is not well defined for now
* check if per_program_timings of execute_timings before sending
2021-08-12 10:48:47 -05:00
446816de52
Remove redundant is_stake check ( #19185 )
2021-08-11 18:38:03 +00:00
5970083b4d
Use last_valid_block_height in services and client apps ( #19163 )
...
* Add deprecated tag to Bank::get_blockhash_last_valid_slot
* Update SendTransactionService to use last_valid_block_height
* Update solana-tokens to use last_valid_block_height
* Remove dangling file
* Update solana program to use last_valid_block_height
* Update Banks crates to use last_valid_block_height
2021-08-11 01:04:00 -06:00
00e5e12906
renames solana_runtime::vote_account::VoteAccount
...
Rename:
VoteAccount -> VoteAccountInner # the private type
ArcVoteAccount -> VoteAccount # the public type
2021-08-10 22:54:17 +00:00
1403eaeefc
makes solana_runtime::vote_account::VoteAccount private
...
VoteAccount is an implementation detail, and should not be public.
Only ArcVoteAccount is the public type.
2021-08-10 22:54:17 +00:00
faf99f4760
Move Bank::get_incremental_snapshot_storages() into snapshot_utils ( #19155 )
...
Filtering out storages for incremental snapshots will be needed by the
background services for incremental snapshot support, but there is not a
Bank at that point. Since the filtering doesn't apply only to Bank, and
more to snapshots, move the functionality into snapshot_utils.
2021-08-10 14:04:11 -05:00
536b763751
chore: bump libc from 0.2.98 to 0.2.99 ( #19142 )
...
* chore: bump libc from 0.2.98 to 0.2.99
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.98 to 0.2.99.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.98...0.2.99 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-10 10:54:54 -06:00
651343688d
Use option for account index bucket parameter ( #19150 )
2021-08-10 11:32:25 -05:00
05d92402f1
Simplify AccountsPackagePre::new() and friends ( #19134 )
2021-08-10 11:10:15 -05:00
f302774cf7
implements copy-on-write for staked-nodes ( #19090 )
...
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77
This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
2021-08-10 12:59:12 +00:00
c18bd08021
calculate store info while generating index ( #19107 )
...
* calculate store info while generating index
* update store info during index generation pass
2021-08-10 07:39:59 -05:00
47e0d9aa95
plumb more accounts_index bins ( #19123 )
2021-08-10 05:45:46 -05:00
f506da5e19
introduce StorageSizeAndCount ( #19137 )
2021-08-09 22:20:03 -05:00
5874f75a26
restore name: Bank::new_with_paths ( #19122 )
2021-08-09 17:30:37 -05:00
c56bccdf32
clean up types on account index generation ( #19105 )
2021-08-09 17:29:36 -05:00
c7b986220e
Accounts::new_with_config_for_tests ( #19104 )
2021-08-09 09:01:12 -05:00
1951fe4e80
accounts default stragglers ( #19101 )
2021-08-09 09:00:30 -05:00
db14bcc43b
AccountsIndex RefCount() returns u64 instead of atomic ( #19088 )
2021-08-09 08:59:56 -05:00
9616ae0f2c
signal to upsert whether reclaims are important ( #18900 )
...
* signal to upsert whether reclaims are important
* reclaims_must_be_empty -> previous_slot_entry_was_cached
* UPSERT_RECLAIMS_MUST_BE_EMPTY_FALSE -> UPSERT_PREVIOUS_SLOT_ENTRY_WAS_CACHED_FALSE
2021-08-09 08:58:59 -05:00
fd937548a0
Move SnapshotArchiveInfo and friends into its own module ( #19114 )
2021-08-08 07:57:06 -05:00
00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() ( #18983 )
...
While reviewing PR #18565 , as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots. A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify. Additionally, new tests have been
written and existing tests have been updated to use this new function.
Fixes #18973
While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled. Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`. And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats. This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
7923c26939
AccountsDb::new_with_config_for_tests ( #19103 )
2021-08-06 17:18:16 -05:00
f771063275
get rid of Accounts new and default ( #19100 )
2021-08-06 15:52:27 -05:00
216a1b3d74
remove Bank::default() ( #19099 )
2021-08-06 13:10:13 -05:00
c12289fd1b
remove AccountsDb::new() from public api ( #19098 )
2021-08-06 13:07:50 -05:00
41f4973f0d
accounts default refactoring stragglers ( #19097 )
2021-08-06 12:36:42 -05:00
ca37873e16
rework bank::new_with_paths ( #19087 )
...
* rework bank::new_with_paths
* missing 1 bench
2021-08-06 09:30:40 -05:00
8878f526ce
rework AccountsIndex traits ( #19089 )
2021-08-06 08:39:34 -05:00
592013eaf4
Clean within shrink_all_slots ( #19042 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-08-05 23:26:38 -07:00
0028442e14
rework defaults and construction of accountsdb, accounts, bank ( #19083 )
...
* rework defaults and construction of accountsdb, accounts, bank
* merge issues
2021-08-05 17:27:13 -05:00
e368f10973
add _for_tests to new_no_wallclock_throttle ( #19086 )
2021-08-05 14:50:25 -05:00
a9014ceceb
Bank::default_for_tests() ( #19084 )
2021-08-05 11:53:29 -05:00
24207a09ac
remove AccountsIndex::default ( #19082 )
...
* accounts_db calls AccountsDb::new(bins)
* remove AccountsIndex::default
2021-08-05 11:38:53 -05:00
5cf28689e6
accounts_db calls AccountsDb::new(bins) ( #19068 )
2021-08-05 11:15:26 -05:00
67788ad206
move AccountsIndex upsert into static WriteAccountMapEntry ( #18899 )
...
* rework accounts index to push upsert deeper
* clean up return value of upsert_existing_key
* upsert_existing_key -> update_key_if_exists
* upsert_new_key -> upsert
* upsert_item -> lock_and_update_slot_list
* update_static -> update_slot_list
2021-08-05 08:45:08 -05:00
bf16b0517c
add _for_tests to setup_bank_and_vote_pubkeys ( #19060 )
2021-08-05 08:43:35 -05:00
087db70df6
add traits required by IsCached ( #19066 )
2021-08-05 08:43:00 -05:00
14361906ca
for all tests, bank::new -> bank::new_for_tests ( #19064 )
2021-08-05 08:42:38 -05:00
dfe99efa7c
introduce AccountsIndex::default_for_tests() ( #19067 )
2021-08-04 21:58:53 -05:00
5a4979f25f
Handle 0-lamport account in index generation ( #19041 )
...
* Handle 0-lamport account in index generation
* rename duplicate to dirty keys
Co-authored-by: Carl Lin <carl@solana.com>
2021-08-04 23:33:47 +00:00
bde9b4de94
Bank::new -> Bank::new_for_benches ( #19063 )
2021-08-04 17:30:43 -05:00
3280ae3e9f
add validator option --accounts-db-skip-shrink ( #19028 )
...
* add validator option --accounts-db-skip-shrink
* typo
2021-08-04 17:28:33 -05:00
68cc71409e
Do not shell out for tar ( #19043 )
...
When making a snapshot archive, we used to shell out and call `tar -S`
for sparse file support. The tar crate supports sparse files, so no
need to do this anymore.
Fixes #10860
2021-08-04 17:07:55 -05:00
a1112254a5
Fix wrong old snapshot archives getting purged ( #19061 )
...
I introduced a bug where old snapshot archives were incorrectly purged.
Instead of purged to oldest, I was purged the newest...
The fix is to add a `reverse()` in the purge logic, and I've added a
test to catch this bug in the future.
Fixes #19057
2021-08-04 16:42:42 -05:00
0b8d14b0fc
move towards account index being dynamically allocated ( #19034 )
2021-08-04 15:28:35 -05:00
1ed12a07ab
introduce Bank::new_for_tests ( #19062 )
2021-08-04 15:06:57 -05:00
ca14475085
Add incremental_snapshot_archive_interval_slots to SnapshotConfig ( #19026 )
...
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
2021-08-04 14:40:20 -05:00
6a995f5dfd
rename to AccountsDb::new_single_for_tests ( #19039 )
2021-08-04 11:47:11 -05:00
31a620c42b
move towards accounts index being dynamic ( #19032 )
2021-08-04 09:18:05 -05:00
06e08c4840
move package_snapshots to AccountsPackagePre ctors ( #18997 )
...
This PR solves #18815 . Note that I had to make the snapshot prefix
constants inside `snapshot_utils.rs` public at the crate level in order
to make this work. I'm not sure whether or not introducing this
dependency is entirely good, either way the `snapshot_utils.rs` file
needs a lot of rework so things will move around, I believe this does
the work in the meantime. Any feedback will be greatly appreciated.
2021-08-04 09:03:03 -05:00
06a7a9e544
remove superfluous `collect()`s
2021-08-04 07:21:55 +00:00
79b4b83d3c
runtime: remove unused `allow` attrs
2021-08-04 07:21:55 +00:00
c16bf02448
chore: bump serde from 1.0.126 to 1.0.127 ( #19010 )
...
* chore: bump serde from 1.0.126 to 1.0.127
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-02 21:16:34 +00:00
77861e2d40
Feature disable fees sysvar ( #18981 )
...
* Feature disable fees sysvar
* nudge
2021-08-02 00:31:11 +00:00
21bc43ed58
nonce: Unify `NonceError` with `SystemError`
2021-07-30 14:47:14 -06:00
b05fb87f22
Add test_bank_forks_incremental_snapshot() ( #18565 )
...
This commit builds on PR #18504 by adding a test to core/tests/snapshot.rs for Incremental Snapshots. The test adds banks to bank forks in a loop and takes both full snapshots and incremental snapshots at intervals, and validates they are rebuild-able.
For background info about Incremental Snapshots, see #17088 .
Fixes #18829 and #18972
2021-07-29 16:46:54 -05:00
dfbb0c559b
deprecate fees sysvar ( #18960 )
2021-07-29 10:48:14 -07:00
9529284194
Nonce gets blockhash from invoke_context ( #18950 )
2021-07-29 01:50:20 -07:00
84e78316b1
Write helper for multithread update ( #18808 )
2021-07-29 03:16:36 +02:00
0167daa116
cleanup feature consistent_recent_blockhashes_sysvar ( #18932 )
2021-07-28 10:25:33 -07:00
14f0ce850d
chore: bump blake3 from 0.3.8 to 1.0.0 ( #18908 )
...
* chore: bump blake3 from 0.3.8 to 1.0.0
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 0.3.8 to 1.0.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.8...1.0.0 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* Make versions consistent
* Enable new feature
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-27 17:55:21 -06:00
72e374d0f3
Deprecate SysvarRecentBlockhashes ( #18875 )
2021-07-27 16:34:21 -07:00
6dadc75a6f
during range, stop scanning earlier than currently scanning ( #18896 )
...
* during range, stop scanning earlier than currently scanning
* refactor for better testing
* clean up a different edge case
2021-07-27 13:40:45 -05:00
53d8cad206
remove unused return value from account index upsert ( #18895 )
2021-07-27 08:46:27 -05:00
5cabb5bb11
Fix race in remove_unrooted_race and flush_slot_cache ( #18785 )
2021-07-22 13:49:08 -07:00
d1debcd971
Add incremental snapshot utils ( #18504 )
...
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks. This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.
Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.
Also of note, some renaming has happened:
1. Snapshots are now either `full_` or `incremental_` throughout the
codebase. If not specified, the code applies to both.
2. Bank snapshots now are called "bank snapshots"
(before they were called "slot snapshots", "bank snapshots", or
just "snapshots"). The one exception is within `Bank`, where they
are still just "snapshots", because they are already "bank
snapshots".
3. Snapshot archives now have `_archive` in the code. This
should clear up an ambiguity between bank snapshots and snapshot
archives.
2021-07-22 14:40:37 -05:00
7fc4cfebc8
Rename BpfComputeBudget ( #18768 )
2021-07-22 10:18:51 -07:00
d6f5945653
token: Swap new token program id for consistency on all networks ( #18823 )
2021-07-22 10:31:55 +02:00
51f3b9aa7c
feature: add new token program feature ( #18780 )
...
* feature: add new token program feature
* Fixup test
* Update to spl-token v3.2.0
* Update Cargo.lock + fmt
* Update token program version in fetch-spl.sh
* Bump associated token program to 1.0.3
* Add aToken so
2021-07-21 14:34:25 +02:00
a4c3db51fc
Disambiguate `archive_snapshot_package` IO error sources
2021-07-21 08:35:28 +00:00
207c90bd8b
Shorten long SerializeWith type paths in abi digest ( #18734 )
2021-07-20 08:59:50 -05:00
7ed8792647
Use saturating math for timings ( #18735 )
2021-07-19 11:05:10 -07:00
181f21529d
add Debug trait to T in AccountsIndex ( #18702 )
2021-07-16 17:05:23 -05:00
74539020b4
Transaction simulation includes cost ( #18715 )
2021-07-16 14:58:15 -07:00
ca71ca3d6d
Accumulate consumed units ( #18714 )
2021-07-16 12:40:12 -07:00
090fbeca24
lazy allocate buffers for bg reader in untar ( #18640 )
2021-07-16 13:17:03 -05:00
6cf3c1ab8f
tx wide compute budget ( #18631 )
2021-07-16 00:31:22 -07:00
d166b9856a
Move transaction sanitization earlier in the pipeline ( #18655 )
...
* Move transaction sanitization earlier in the pipeline
* Renamed HashedTransaction to SanitizedTransaction
* Implement deref for sanitized transaction
* bring back process_transactions test method
* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
37ee0b5599
Eliminate doc warnings and fix some markdown ( #18566 )
...
* Fix link target in doc comment
* Fix formatting of log examples in process_instruction
* Fix doc markdown in solana-gossip
* Fix doc markdown in solana-runtime
* Escape square braces in doc comments to avoid warnings
* Surround 'account references' doc items in code spans to avoid warnings
* Fix code block in loader_upgradeable_instruction
* Fix doctest for loader_upgradable_instruction
2021-07-16 00:40:07 +00:00
3a85b77bb5
hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade
2021-07-15 18:43:55 +00:00
568660b402
Revert "Remove feature switch for secp256k1 program ( #18467 )"
...
This reverts commit fd574dcb3b
2021-07-15 18:43:55 +00:00
958d27bb0e
accounts index scan starts looking in bin where bound would exist ( #18613 )
2021-07-15 10:26:50 -05:00
da9fdd785c
measure and speed up 2nd half of generate_index ( #18503 )
...
* add metrics and speedup storage calcs in generate_index
* add tests, refactor
* rename map variable
* rename another index variable
2021-07-15 10:26:00 -05:00
ad3f18f031
fix race condition in bg file reader file error ( #18682 )
2021-07-14 19:04:55 -05:00
8bb9ddf298
chore: bump ouroboros from 0.10.0 to 0.10.1 ( #18662 )
...
* chore: bump ouroboros from 0.10.0 to 0.10.1
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-07-14 13:37:52 -06:00
7f2254225e
Move entry/poh to own crate to speed up poh bench build ( #18225 )
2021-07-14 14:16:29 +02:00
bb41cf3461
remove unnecessary generic args to make refactoring easier ( #18619 )
2021-07-13 12:05:23 -05:00
0bd8710d34
prevent excess allocation with AccountsIndexIterator ( #18605 )
2021-07-13 11:11:17 -05:00
d092fa1f03
add ledger-tool verify verify-accounts-index option ( #18375 )
...
* add ledger-tool verify verify-accounts-index option
* comment, merge, respond to feedback, cleanup
2021-07-13 11:06:18 -05:00
3e11468a04
refactor SharedBuffer to separate Arc refcount shutdown issues ( #18563 )
2021-07-13 11:05:41 -05:00
0d3e8ada94
Cleanup secp256k1 deps ( #18618 )
2021-07-13 00:36:19 +00:00
4098af3b5b
Record vote account commission with voting/staking rewards and surface in RPC
2021-07-12 15:09:44 -07:00
bc47ed6c80
Remove outdated program verifification ( #18592 )
2021-07-12 14:40:49 -05:00
35123b71c2
fix test ( #18609 )
2021-07-12 11:45:25 -07:00
00f7e514b8
Update neon program id ( #18607 )
2021-07-12 10:07:22 -07:00
f5ff4b2058
serial insertion of bins into accounts index ( #18469 )
2021-07-12 12:00:45 -05:00
e806d31224
Optimize Message::is_non_loader_key method ( #18579 )
2021-07-12 10:42:22 -05:00
Jeff Washington (jwash)
Lijun Wang
Tyera Eulberg
Brooks Prumo
Jack May
dependabot[bot]
behzad nouri
Justin Starry
jon-chuang
Jon Cinque
steviez
Tao Zhu
sakridge
Oliver
Trent Nelson
carllin
Brian Anderson
Michael Vines