Commit Graph

637 Commits

Author SHA1 Message Date
dependabot[bot] e492638b7d
chore: bump libc from 0.2.100 to 0.2.101 (#19442)
* chore: bump libc from 0.2.100 to 0.2.101

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-26 12:38:34 -06:00
Jeff Washington (jwash) f1d9c014f8
create test function test_process_blockstore (#19432) 2021-08-26 12:48:52 -05:00
dependabot[bot] 46890ac197
chore: bump serde from 1.0.128 to 1.0.129 (#19395)
* chore: bump serde from 1.0.128 to 1.0.129

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-24 12:32:30 -06:00
Brooks Prumo f3ce5fa5e8 clippy: branches-sharing-code
```text
error: all if blocks contain the same code at the end
    --> ledger/src/blockstore.rs:3473:5
     |
3473 | /         Ok(insert_map.get(&slot).unwrap().clone())
3474 | |     }
     | |_____^
     |
     = note: `-D clippy::branches-sharing-code` implied by `-D warnings`
     = note: The end suggestion probably needs some adjustments to use the expression result correctly
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#branches_sharing_code
help: consider moving the end statements out like this
     |
3473 ~     }
3474 +     Ok(insert_map.get(&slot).unwrap().clone())
     |

error: could not compile `solana-ledger` due to previous error
```
2021-08-23 11:17:21 -05:00
dependabot[bot] c0c95e88d8
chore: bump libc from 0.2.99 to 0.2.100 (#19371)
* chore: bump libc from 0.2.99 to 0.2.100

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-23 09:34:32 -06:00
dependabot[bot] aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 (#19370)
* chore: bump serde from 1.0.127 to 1.0.128

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-23 09:33:34 -06:00
Brooks Prumo 6d939811e9
Name snapshots consistently (#19346)
#### Problem

Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:

```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```

#### Summary of Changes

For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.


Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
Tao Zhu 4982dc20f9
replace function with const var for better readability (#19285) 2021-08-19 14:59:53 -05:00
Tao Zhu d7ba15cde8
Update block max cost limit to fix performance regession (#19276)
* increase block_cost_max by 10 times to accommodate updated account read and write costs; Otherwise during performace test, banking_stage will start reject and retry transactions in next block, result regressed performance.

* fix typo
2021-08-18 16:38:44 +00:00
Justin Starry c50b01cb60
Store versioned transactions in the ledger, disabled by default (#19139)
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
Jeff Washington (jwash) 7c70f2158b
accounts_index_bins to AccountsIndexConfig (#19257)
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
behzad nouri 7a8807b8bb retransmits shreds recovered from erasure codes
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.

This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
2021-08-17 13:44:10 +00:00
behzad nouri 3c71670bd9 returns completed-data-set-info from insert_data_shred
instead of opaque (u32, u32) which are then converted to
CompletedDataSetInfo at the call-site.
2021-08-17 13:44:10 +00:00
behzad nouri b64eeb7729 removes erroneous uses of &Arc<...> from window-service 2021-08-13 17:26:31 +00:00
dependabot[bot] 5b9671d01a
chore: bump matches from 0.1.8 to 0.1.9 (#19220)
* chore: bump matches from 0.1.8 to 0.1.9

Bumps [matches](https://github.com/SimonSapin/rust-std-candidates) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/SimonSapin/rust-std-candidates/releases)
- [Commits](https://github.com/SimonSapin/rust-std-candidates/commits)

---
updated-dependencies:
- dependency-name: matches
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-13 16:37:45 +00:00
behzad nouri d192d78ac2
removes unused code from staking_utils (#19189) 2021-08-13 02:42:41 +00:00
Brooks Prumo 5fb6b341c4
Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots (#18870)
* Handle cleaning zero-lamport accounts

Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot.  This is part of the Incremental Snapshot work.

Fixes #18825
2021-08-12 15:56:08 -05:00
Tao Zhu 414d904959
Reject blocks for costs above the max block cost (#18994)
* added realtime cost checking logic to reject block that would exceed max limit:
- defines max limits at block_cost_limits.rs
- right after each bath's execution, accumulate its cost and check again
  limit, return error if limit is exceeded

* update abi that changed due to adding additional TransactionError

* To avoid counting stats mltiple times, only accumulate execute-timing when a bank is completed

* gate it by a feature

* move cost const def into block_cost_limits.rs

* redefine the cost for signature and account access, removed signer part as it is not well defined for now

* check if per_program_timings of execute_timings before sending
2021-08-12 10:48:47 -05:00
Jeff Washington (jwash) e91988c977
cli for num account index bins (#19085) 2021-08-11 11:45:25 -05:00
behzad nouri 00e5e12906 renames solana_runtime::vote_account::VoteAccount
Rename:
  VoteAccount    -> VoteAccountInner  # the private type
  ArcVoteAccount -> VoteAccount       # the public type
2021-08-10 22:54:17 +00:00
Alexander Meißner 85befbc8cd
Removes bpf_loader_upgradeable_program feature gate. (#19102) 2021-08-11 00:08:57 +02:00
dependabot[bot] 536b763751
chore: bump libc from 0.2.98 to 0.2.99 (#19142)
* chore: bump libc from 0.2.98 to 0.2.99

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.98 to 0.2.99.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.98...0.2.99)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-10 10:54:54 -06:00
Jeff Washington (jwash) 651343688d
Use option for account index bucket parameter (#19150) 2021-08-10 11:32:25 -05:00
behzad nouri f302774cf7
implements copy-on-write for staked-nodes (#19090)
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77

This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
2021-08-10 12:59:12 +00:00
Jeff Washington (jwash) 47e0d9aa95
plumb more accounts_index bins (#19123) 2021-08-10 05:45:46 -05:00
Jeff Washington (jwash) 5874f75a26
restore name: Bank::new_with_paths (#19122) 2021-08-09 17:30:37 -05:00
Brooks Prumo 00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983)
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
Jeff Washington (jwash) ca37873e16
rework bank::new_with_paths (#19087)
* rework bank::new_with_paths

* missing 1 bench
2021-08-06 09:30:40 -05:00
Jeff Washington (jwash) 14361906ca
for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
Jeff Washington (jwash) 3280ae3e9f
add validator option --accounts-db-skip-shrink (#19028)
* add validator option --accounts-db-skip-shrink

* typo
2021-08-04 17:28:33 -05:00
carllin 03353d500f
Actively manage dead slots in AncestorHashesService (#18912) 2021-08-02 14:33:28 -07:00
dependabot[bot] c16bf02448
chore: bump serde from 1.0.126 to 1.0.127 (#19010)
* chore: bump serde from 1.0.126 to 1.0.127

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-02 21:16:34 +00:00
dependabot[bot] 4e41c05acd
chore: bump futures from 0.3.15 to 0.3.16 (#18914)
* chore: bump futures from 0.3.15 to 0.3.16

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-07-27 07:59:34 -06:00
dependabot[bot] a4024168de
chore: bump rocksdb from 0.16.0 to 0.17.0 (#18904)
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-07-27 12:36:42 +09:00
dependabot[bot] b97113408b
chore: bump futures-util from 0.3.15 to 0.3.16 (#18878)
* chore: bump futures-util from 0.3.15 to 0.3.16

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Nudge Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-07-26 18:19:53 +00:00
Brooks Prumo d1debcd971
Add incremental snapshot utils (#18504)
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
2021-07-22 14:40:37 -05:00
carllin 588c0464b8
Add sampling logic and DuplicateSlotRepairStatus module (#18721) 2021-07-21 11:15:08 -07:00
Justin Starry d166b9856a
Move transaction sanitization earlier in the pipeline (#18655)
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
2021-07-15 22:51:27 -05:00
Brian Anderson 37ee0b5599
Eliminate doc warnings and fix some markdown (#18566)
* Fix link target in doc comment

* Fix formatting of log examples in process_instruction

* Fix doc markdown in solana-gossip

* Fix doc markdown in solana-runtime

* Escape square braces in doc comments to avoid warnings

* Surround 'account references' doc items in code spans to avoid warnings

* Fix code block in loader_upgradeable_instruction

* Fix doctest for loader_upgradable_instruction
2021-07-16 00:40:07 +00:00
Trent Nelson 3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson 568660b402 Revert "Remove feature switch for secp256k1 program (#18467)"
This reverts commit fd574dcb3b.
2021-07-15 18:43:55 +00:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
Jeff Washington (jwash) d092fa1f03
add ledger-tool verify verify-accounts-index option (#18375)
* add ledger-tool verify verify-accounts-index option

* comment, merge, respond to feedback, cleanup
2021-07-13 11:06:18 -05:00
carllin a1c0f144f4
Add blockstore column for frozen hashes and duplicate confirmed (#18533) 2021-07-12 20:59:16 -07:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Tyera Eulberg 761de8b1a3
Bump prost, prost-types, and tonic (#18537)
* Bump prost+tonic and accommodate generated service changes

* Unignore advisory

* Fixup .proto error list
2021-07-09 17:31:26 +00:00
Justin Starry fd574dcb3b
Remove feature switch for secp256k1 program (#18467)
* Remove feature switch for secp256k1 program

* fix tests
2021-07-09 10:08:03 -05:00
dependabot[bot] c853da7424
chore: bump libc from 0.2.97 to 0.2.98 (#18516)
* chore: bump libc from 0.2.97 to 0.2.98

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.97 to 0.2.98.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.97...0.2.98)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-07-08 17:31:16 +00:00
jbiseda a86ced0bac
generate deterministic seeds for shreds (#17950)
* generate shred seed from leader pubkey

* clippy

* clippy

* review

* review 2

* fmt

* review

* check

* review

* cleanup

* fmt
2021-07-07 08:21:12 -07:00
Justin Starry 5dd399dafa
Fix transaction logs and inner ixs for leader nodes (#18395)
* Fix transaction logs and inner ixs for leader nodes

* Fix cpi log storage flag
2021-07-06 19:35:24 +00:00