22f45dca80
Make bigtable delete-slots actually usable ( #20037 )
2021-09-20 20:00:32 +00:00
c1d181add5
add --accounts-index-memory-limit-mb ( #19269 )
2021-09-19 18:00:15 -05:00
4e038e94fd
add 'drives' to AccountsIndexConfig ( #19989 )
2021-09-18 09:54:57 -05:00
99f5684dc4
AcctIdx: handle future config defaults ( #19967 )
2021-09-17 13:12:06 -05:00
c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` ( #19931 )
...
* Add `delete` subcommand to `ledger-tool bigtable` command
* feedback
2021-09-16 23:37:45 +00:00
68da1909de
chore: bump serde_json from 1.0.67 to 1.0.68 ( #19942 )
...
* chore: bump serde_json from 1.0.67 to 1.0.68
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.67 to 1.0.68.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.67...v1.0.68 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-16 20:23:25 +00:00
4c69229a47
Add incremental snapshot CLI args to ledger-tool ( #19889 )
...
This commit only adds args to specify the maximum full/incremental
snapshot archives to retain when purging old snapshot archives. It
purposely does *not* add a way to generate new incremental snapshots.
Fixes #19857
2021-09-16 10:56:18 -05:00
b57e86abf2
cache account hash info ( #19426 )
...
* cache account hash info
* ledger_path -> accounts_hash_cache_path
2021-09-13 20:39:26 -05:00
36f870a7e9
chore: bump serde_yaml from 0.8.20 to 0.8.21 ( #19820 )
...
* chore: bump serde_yaml from 0.8.20 to 0.8.21
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml ) from 0.8.20 to 0.8.21.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases )
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.20...0.8.21 )
---
updated-dependencies:
- dependency-name: serde_yaml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-13 15:50:33 +00:00
62c8bcf565
Add default() to SnapshotConfig ( #19776 )
2021-09-12 13:44:27 -05:00
0f76077969
Add `--destake-vote-account <VOTE_ADDRESS>...` argument to `create-snapshot` command ( #19749 )
2021-09-11 14:44:37 -07:00
c3a610374a
unused deps
2021-09-11 14:33:53 -07:00
456bf15012
AccountsIndexConfig -> AccountsDbConfig ( #19687 )
2021-09-08 04:30:38 +00:00
a0552e5b46
Make startup aware of Incremental Snapshots ( #19600 )
2021-09-07 20:43:43 +00:00
9d9482b9d8
Plumb `maximum_incremental_snapshot_archives_to_retain` ( #19640 )
2021-09-06 18:01:56 -05:00
decec3cd8b
Demote write locks on transaction program ids ( #19593 )
...
* Add feature
* Demote write lock on program ids
* Fixup bpf tests
* Update MappedMessage::is_writable
* Comma nit
* Review comments
2021-09-04 03:05:30 +00:00
5e25ee5ebe
Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig ( #19612 )
2021-09-03 20:21:32 +00:00
7ab0aec61f
Rename maximum_full_snapshot_archives_to_retain ( #19610 )
...
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
2021-09-03 11:28:10 -05:00
e9374d32a3
Revert "Make startup aware of Incremental Snapshots ( #19550 )" ( #19599 )
...
This reverts commit d45ced0a5d
2021-09-02 19:14:41 -05:00
d45ced0a5d
Make startup aware of Incremental Snapshots ( #19550 )
2021-09-02 19:05:15 -05:00
1d5a8ebc6a
Revert "Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )" ( #19529 )
...
This reverts commit 4d361af976
2021-08-31 22:03:19 -05:00
fe9ee9134a
Make background services aware of incremental snapshots ( #19401 )
...
AccountsBackgroundService now knows about incremental snapshots. It is
now also in charge of deciding if an AccountsPackage is destined to be a
SnapshotPackage or not (or just used by AccountsHashVerifier).
!!! New behavior changes !!!
Taking snapshots (both bank and archive) **MUST** succeed.
This is required because of how the last full snapshot slot is
calculated, which is used by AccountsBackgroundService when calling
`clean_accounts()`.
File system calls are now unwrapped and will result in a crash. As Trent told me:
>Well I think if a snapshot fails due to some IO error, it's very likely that the operator is going to have to intervene before it works. We should exit error in this case, otherwise the validator might happily spin for several more hours, never successfully writing a complete snapshot, before something else brings it down. This would leave the validator's last local snapshot many more slots behind than it would be had we exited outright and potentially force the operator to abandon ledger continuity in favor of a quick catchup
Other errors will set the `exit` flag to `true`, and the node will gracefully shutdown.
Fixes #19167
Fixes #19168
2021-08-31 18:33:27 -05:00
c21bf29ce7
chore: bump serde_json from 1.0.66 to 1.0.67 ( #19501 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.66 to 1.0.67.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.66...v1.0.67 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-31 06:05:25 +00:00
8e4360cf9c
chore: bump futures from 0.3.16 to 0.3.17 ( #19513 )
...
* chore: bump futures from 0.3.16 to 0.3.17
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-30 22:12:38 -06:00
fbffdf9f3d
chore: bump futures-util from 0.3.16 to 0.3.17 ( #19498 )
...
* chore: bump futures-util from 0.3.16 to 0.3.17
Bumps [futures-util](https://github.com/rust-lang/futures-rs ) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17 )
---
updated-dependencies:
- dependency-name: futures-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 17:45:47 +00:00
8ad52fa095
implements copy-on-write for vote-accounts ( #19362 )
...
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186
This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
2021-08-30 15:54:01 +00:00
db2be00289
chore: bump serde_yaml from 0.8.19 to 0.8.20 ( #19464 )
...
* chore: bump serde_yaml from 0.8.19 to 0.8.20
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml ) from 0.8.19 to 0.8.20.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases )
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.19...0.8.20 )
---
updated-dependencies:
- dependency-name: serde_yaml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-27 09:33:08 -06:00
c721300ab5
chore: bump serde_yaml from 0.8.18 to 0.8.19 ( #19372 )
...
* chore: bump serde_yaml from 0.8.18 to 0.8.19
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml ) from 0.8.18 to 0.8.19.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases )
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.18...0.8.19 )
---
updated-dependencies:
- dependency-name: serde_yaml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-23 10:23:16 -06:00
6d939811e9
Name snapshots consistently ( #19346 )
...
#### Problem
Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:
```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```
#### Summary of Changes
For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
4d361af976
Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )
2021-08-20 17:06:53 +00:00
73aa004c59
stake: Remove v2 program references ( #19308 )
...
* stake: Remove v2 program references
* Remove stake v2 feature, along with stake rewrite
2021-08-20 01:08:44 -04:00
39459085ef
chore: bump serde_yaml from 0.8.17 to 0.8.18 ( #19280 )
...
* chore: bump serde_yaml from 0.8.17 to 0.8.18
Bumps [serde_yaml](https://github.com/dtolnay/serde-yaml ) from 0.8.17 to 0.8.18.
- [Release notes](https://github.com/dtolnay/serde-yaml/releases )
- [Commits](https://github.com/dtolnay/serde-yaml/compare/0.8.17...0.8.18 )
---
updated-dependencies:
- dependency-name: serde_yaml
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-18 08:23:20 -06:00
c50b01cb60
Store versioned transactions in the ledger, disabled by default ( #19139 )
...
* Add support for versioned transactions, but disable by default
* merge conflicts
* trent's feedback
* bump Cargo.lock
* Fix transaction error encoding
* Rename legacy_transaction method
* cargo clippy
* Clean up casts, int arithmetic, and unused methods
* Check for duplicates in sanitized message conversion
* fix clippy
* fix new test
* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
7c70f2158b
accounts_index_bins to AccountsIndexConfig ( #19257 )
...
* accounts_index_bins to AccountsIndexConfig
* rename param bins -> config
* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
e91988c977
cli for num account index bins ( #19085 )
2021-08-11 11:45:25 -05:00
db40cb41b2
Handle new security advisories ( #19126 )
...
* Update hyper 0.14
* Add audit ignores
* Bump assert_cmd and remove audit ignore
* Remove unnecessary unmaintained dependency
* Remove pin-project-lite warning
2021-08-09 19:36:03 +00:00
fd937548a0
Move SnapshotArchiveInfo and friends into its own module ( #19114 )
2021-08-08 07:57:06 -05:00
00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() ( #18983 )
...
While reviewing PR #18565 , as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots. A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify. Additionally, new tests have been
written and existing tests have been updated to use this new function.
Fixes #18973
While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled. Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`. And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats. This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
ca14475085
Add incremental_snapshot_archive_interval_slots to SnapshotConfig ( #19026 )
...
This commit also renames `snapshot_interval_slots` to
`full_snapshot_archive_interval_slots`, updates the comments on the
fields, and make appropriate updates where SnapshotConfig is used.
2021-08-04 14:40:20 -05:00
fe1ee49807
chore: bump serde_json from 1.0.65 to 1.0.66 ( #18990 )
...
* chore: bump serde_json from 1.0.65 to 1.0.66
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.65 to 1.0.66.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.65...v1.0.66 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-07-30 10:46:47 -06:00
0cf52e2060
chore: bump serde_json from 1.0.64 to 1.0.65 ( #18968 )
...
* chore: bump serde_json from 1.0.64 to 1.0.65
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.64 to 1.0.65.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.64...v1.0.65 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-07-29 16:52:02 +00:00
467c18e0d1
Add cli-output helpers ( #18933 )
...
* Add OutputFormat helper to reduce copy-pasta
* Add CliSignOnlyData constructor
2021-07-28 04:21:23 +00:00
4e41c05acd
chore: bump futures from 0.3.15 to 0.3.16 ( #18914 )
...
* chore: bump futures from 0.3.15 to 0.3.16
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-07-27 07:59:34 -06:00
b97113408b
chore: bump futures-util from 0.3.15 to 0.3.16 ( #18878 )
...
* chore: bump futures-util from 0.3.15 to 0.3.16
Bumps [futures-util](https://github.com/rust-lang/futures-rs ) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16 )
---
updated-dependencies:
- dependency-name: futures-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Nudge Cargo.lock
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-26 18:19:53 +00:00
d1debcd971
Add incremental snapshot utils ( #18504 )
...
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks. This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.
Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.
Also of note, some renaming has happened:
1. Snapshots are now either `full_` or `incremental_` throughout the
codebase. If not specified, the code applies to both.
2. Bank snapshots now are called "bank snapshots"
(before they were called "slot snapshots", "bank snapshots", or
just "snapshots"). The one exception is within `Bank`, where they
are still just "snapshots", because they are already "bank
snapshots".
3. Snapshot archives now have `_archive` in the code. This
should clear up an ambiguity between bank snapshots and snapshot
archives.
2021-07-22 14:40:37 -05:00
d166b9856a
Move transaction sanitization earlier in the pipeline ( #18655 )
...
* Move transaction sanitization earlier in the pipeline
* Renamed HashedTransaction to SanitizedTransaction
* Implement deref for sanitized transaction
* bring back process_transactions test method
* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
3a85b77bb5
hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade
2021-07-15 18:43:55 +00:00
568660b402
Revert "Remove feature switch for secp256k1 program ( #18467 )"
...
This reverts commit fd574dcb3b
2021-07-15 18:43:55 +00:00
7f2254225e
Move entry/poh to own crate to speed up poh bench build ( #18225 )
2021-07-14 14:16:29 +02:00
350baece21
Explicitly sanitize program id indexes before usage
...
1. check transaction has valid program_id before using it to avoid possible panic;
2. change calculate_cost function signature to return Result;
3. add CostModelError enum, update return type from Result<_, str> to Result<_, CostModelError>
2021-07-13 17:29:22 -05:00
Ryo Onodera
Jeff Washington (jwash)
Justin Starry
dependabot[bot]
Brooks Prumo
Michael Vines
Giles Cope
Tyera Eulberg
Brooks Prumo
behzad nouri
Jon Cinque
Trent Nelson
sakridge
Tao Zhu