blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
15,777 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

1901 Commits

Author SHA1 Message Date
Jeff Washington (jwash) 6a995f5dfd
rename to AccountsDb::new_single_for_tests (#19039) 2021-08-04 11:47:11 -05:00
Jeff Washington (jwash) 31a620c42b
move towards accounts index being dynamic (#19032) 2021-08-04 09:18:05 -05:00
Oliver 06e08c4840
move package_snapshots to AccountsPackagePre ctors (#18997) 
This PR solves #18815. Note that I had to make the snapshot prefix
constants inside `snapshot_utils.rs` public at the crate level in order
to make this work. I'm not sure whether or not introducing this
dependency is entirely good, either way the `snapshot_utils.rs` file
needs a lot of rework so things will move around, I believe this does
the work in the meantime. Any feedback will be greatly appreciated.
 2021-08-04 09:03:03 -05:00
Trent Nelson 06a7a9e544 remove superfluous `collect()`s 2021-08-04 07:21:55 +00:00
Trent Nelson 79b4b83d3c runtime: remove unused `allow` attrs 2021-08-04 07:21:55 +00:00
dependabot[bot] c16bf02448
chore: bump serde from 1.0.126 to 1.0.127 (#19010) 
* chore: bump serde from 1.0.126 to 1.0.127

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-02 21:16:34 +00:00
Jack May 77861e2d40
Feature disable fees sysvar (#18981) 
* Feature disable fees sysvar

* nudge
 2021-08-02 00:31:11 +00:00
Trent Nelson 21bc43ed58
nonce: Unify `NonceError` with `SystemError` 2021-07-30 14:47:14 -06:00
Brooks Prumo b05fb87f22
Add test_bank_forks_incremental_snapshot() (#18565) 
This commit builds on PR #18504 by adding a test to core/tests/snapshot.rs for Incremental Snapshots. The test adds banks to bank forks in a loop and takes both full snapshots and incremental snapshots at intervals, and validates they are rebuild-able.

For background info about Incremental Snapshots, see #17088.

Fixes #18829 and #18972
 2021-07-29 16:46:54 -05:00
Jack May dfbb0c559b
deprecate fees sysvar (#18960) 2021-07-29 10:48:14 -07:00
Jack May 9529284194
Nonce gets blockhash from invoke_context (#18950) 2021-07-29 01:50:20 -07:00
sakridge 84e78316b1
Write helper for multithread update (#18808) 2021-07-29 03:16:36 +02:00
Jack May 0167daa116
cleanup feature consistent_recent_blockhashes_sysvar (#18932) 2021-07-28 10:25:33 -07:00
dependabot[bot] 14f0ce850d
chore: bump blake3 from 0.3.8 to 1.0.0 (#18908) 
* chore: bump blake3 from 0.3.8 to 1.0.0

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 0.3.8 to 1.0.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.8...1.0.0)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make versions consistent

* Enable new feature

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-07-27 17:55:21 -06:00
Jack May 72e374d0f3
Deprecate SysvarRecentBlockhashes (#18875) 2021-07-27 16:34:21 -07:00
Jeff Washington (jwash) 6dadc75a6f
during range, stop scanning earlier than currently scanning (#18896) 
* during range, stop scanning earlier than currently scanning

* refactor for better testing

* clean up a different edge case
 2021-07-27 13:40:45 -05:00
Jeff Washington (jwash) 53d8cad206
remove unused return value from account index upsert (#18895) 2021-07-27 08:46:27 -05:00
carllin 5cabb5bb11
Fix race in remove_unrooted_race and flush_slot_cache (#18785) 2021-07-22 13:49:08 -07:00
Brooks Prumo d1debcd971
Add incremental snapshot utils (#18504) 
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
 2021-07-22 14:40:37 -05:00
Jack May 7fc4cfebc8
Rename BpfComputeBudget (#18768) 2021-07-22 10:18:51 -07:00
Jon Cinque d6f5945653
token: Swap new token program id for consistency on all networks (#18823) 2021-07-22 10:31:55 +02:00
Jon Cinque 51f3b9aa7c
feature: add new token program feature (#18780) 
* feature: add new token program feature

* Fixup test

* Update to spl-token v3.2.0

* Update Cargo.lock + fmt

* Update token program version in fetch-spl.sh

* Bump associated token program to 1.0.3

* Add aToken so
 2021-07-21 14:34:25 +02:00
Trent Nelson a4c3db51fc Disambiguate `archive_snapshot_package` IO error sources 2021-07-21 08:35:28 +00:00
Justin Starry 207c90bd8b
Shorten long SerializeWith type paths in abi digest (#18734) 2021-07-20 08:59:50 -05:00
Jack May 7ed8792647
Use saturating math for timings (#18735) 2021-07-19 11:05:10 -07:00
Jeff Washington (jwash) 181f21529d
add Debug trait to T in AccountsIndex (#18702) 2021-07-16 17:05:23 -05:00
Jack May 74539020b4
Transaction simulation includes cost (#18715) 2021-07-16 14:58:15 -07:00
Jack May ca71ca3d6d
Accumulate consumed units (#18714) 2021-07-16 12:40:12 -07:00
Jeff Washington (jwash) 090fbeca24
lazy allocate buffers for bg reader in untar (#18640) 2021-07-16 13:17:03 -05:00
Jack May 6cf3c1ab8f
tx wide compute budget (#18631) 2021-07-16 00:31:22 -07:00
Justin Starry d166b9856a
Move transaction sanitization earlier in the pipeline (#18655) 
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
 2021-07-15 22:51:27 -05:00
Brian Anderson 37ee0b5599
Eliminate doc warnings and fix some markdown (#18566) 
* Fix link target in doc comment

* Fix formatting of log examples in process_instruction

* Fix doc markdown in solana-gossip

* Fix doc markdown in solana-runtime

* Escape square braces in doc comments to avoid warnings

* Surround 'account references' doc items in code spans to avoid warnings

* Fix code block in loader_upgradeable_instruction

* Fix doctest for loader_upgradable_instruction
 2021-07-16 00:40:07 +00:00
Trent Nelson 3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson 568660b402 Revert "Remove feature switch for secp256k1 program (#18467)" 
This reverts commit fd574dcb3b.
 2021-07-15 18:43:55 +00:00
Jeff Washington (jwash) 958d27bb0e
accounts index scan starts looking in bin where bound would exist (#18613) 2021-07-15 10:26:50 -05:00
Jeff Washington (jwash) da9fdd785c
measure and speed up 2nd half of generate_index (#18503) 
* add metrics and speedup storage calcs in generate_index

* add tests, refactor

* rename map variable

* rename another index variable
 2021-07-15 10:26:00 -05:00
Jeff Washington (jwash) ad3f18f031
fix race condition in bg file reader file error (#18682) 2021-07-14 19:04:55 -05:00
dependabot[bot] 8bb9ddf298
chore: bump ouroboros from 0.10.0 to 0.10.1 (#18662) 
* chore: bump ouroboros from 0.10.0 to 0.10.1

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.10.0 to 0.10.1.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-07-14 13:37:52 -06:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
Jeff Washington (jwash) bb41cf3461
remove unnecessary generic args to make refactoring easier (#18619) 2021-07-13 12:05:23 -05:00
Jeff Washington (jwash) 0bd8710d34
prevent excess allocation with AccountsIndexIterator (#18605) 2021-07-13 11:11:17 -05:00
Jeff Washington (jwash) d092fa1f03
add ledger-tool verify verify-accounts-index option (#18375) 
* add ledger-tool verify verify-accounts-index option

* comment, merge, respond to feedback, cleanup
 2021-07-13 11:06:18 -05:00
Jeff Washington (jwash) 3e11468a04
refactor SharedBuffer to separate Arc refcount shutdown issues (#18563) 2021-07-13 11:05:41 -05:00
Jack May 0d3e8ada94
Cleanup secp256k1 deps (#18618) 2021-07-13 00:36:19 +00:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Justin Starry bc47ed6c80
Remove outdated program verifification (#18592) 2021-07-12 14:40:49 -05:00
Jack May 35123b71c2
fix test (#18609) 2021-07-12 11:45:25 -07:00
Jack May 00f7e514b8
Update neon program id (#18607) 2021-07-12 10:07:22 -07:00
Jeff Washington (jwash) f5ff4b2058
serial insertion of bins into accounts index (#18469) 2021-07-12 12:00:45 -05:00
Justin Starry e806d31224
Optimize Message::is_non_loader_key method (#18579) 2021-07-12 10:42:22 -05:00
joshua-maros c4827732c5 Update ouroboros 2021-07-10 12:16:09 -06:00
Jack May e9ace3a0d5
cost model nits (#18528) 2021-07-09 12:55:31 -07:00
Ryo Onodera f96de208e2 Implement MockInvokeContext::is_feature_active properly 2021-07-09 09:08:38 -07:00
Ryo Onodera b0734fabf7 Improve test 2021-07-09 09:08:38 -07:00
Ryo Onodera bd3c93f086 Add expected test removal remainder 2021-07-09 09:08:38 -07:00
Ryo Onodera 30a8fb86d0 Fix comment 2021-07-09 09:08:38 -07:00
Ryo Onodera b158353ca9 kill single letter variables 2021-07-09 09:08:38 -07:00
Ryo Onodera 0f1c0c2796 Improve comment 2021-07-09 09:08:38 -07:00
Ryo Onodera 4fa902c84f Fix typo 2021-07-09 09:08:38 -07:00
Ryo Onodera 3a7e5e65e4 Fix rebase... 2021-07-09 09:08:38 -07:00
Ryo Onodera f029af0fca Remove sysvar special cases for rent and assign 2021-07-09 09:08:38 -07:00
Justin Starry fd574dcb3b
Remove feature switch for secp256k1 program (#18467) 
* Remove feature switch for secp256k1 program

* fix tests
 2021-07-09 10:08:03 -05:00
Trent Nelson ca1d8f2adc runtime: ignore flakey `test_waitable_condvar` 2021-07-09 01:26:28 -06:00
carllin 4d3e301ee4
Introduce slot dumping to ReplayStage (#18160) 2021-07-08 19:07:32 -07:00
Jeff Washington (jwash) 334e1112a7
during untar, read ahead, causing decompression to occur in the bg (#18205) 
* shared buffered decompressor for untar

* spell check in comment
 2021-07-08 13:44:47 -05:00
Jack May 2867584985
Bump compute budget for neon evm (#17700) 2021-07-08 10:43:34 -07:00
dependabot[bot] c853da7424
chore: bump libc from 0.2.97 to 0.2.98 (#18516) 
* chore: bump libc from 0.2.97 to 0.2.98

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.97 to 0.2.98.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.97...0.2.98)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-07-08 17:31:16 +00:00
Dan Albert b8be09c09c
Add addresses 2021-07-07 21:02:22 -06:00
Jeff Washington (jwash) f04340b125
move allocation out of critical section and estimate capacity better (#18450) 2021-07-07 15:36:40 -05:00
Jeff Washington (jwash) eca0ceb04c
eliminate unnecessary copies in accounts index generation (#18466) 2021-07-07 15:36:05 -05:00
Jeff Washington (jwash) 49c4e54b28
accounts index insert uses pubkey by value (#18460) 2021-07-07 15:35:35 -05:00
Jack May ccdf93e2b8
featurize_policy_update (#18492) 2021-07-07 13:19:17 -07:00
Jack May 44289e6728
Update verify policy (#18459) 2021-07-07 09:09:52 -07:00
Ryo Onodera d69f469b83
Record parent slot to reconstruct fork tree from influxdb (#18482) 2021-07-07 15:56:38 +00:00
Justin Starry 72da25e9d2
Refactor verify_and_update write privileges check (#18468) 2021-07-07 09:14:00 -05:00
Justin Starry 100fabf469
Remove feature switch for demoting sysvar write locks (#18373) 2021-07-06 21:22:22 +00:00
Justin Starry 5dd399dafa
Fix transaction logs and inner ixs for leader nodes (#18395) 
* Fix transaction logs and inner ixs for leader nodes

* Fix cpi log storage flag
 2021-07-06 19:35:24 +00:00
Justin Starry 6319e8811a
Improve account unlock performance (#18442) 
* Improve account unlock performance

* fix clippy
 2021-07-06 14:11:54 -05:00
Alexander Meißner 7462c27d07
Refactoring: Unify account_deps and accounts (#17898) 
* Changes ThisInvokeContext::get_account() to use accounts instead of pre_accounts.

* Adds explicit keys to accounts to make them symmetric to account_deps.

* Appends account_deps to accounts in transaction loading and removes account_deps everywhere else.
 2021-07-05 13:49:37 +02:00
dependabot[bot] 4faa93418f
chore: bump memmap2 from 0.2.3 to 0.3.0 (#18387) 
* chore: bump memmap2 from 0.2.3 to 0.3.0

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.2.3 to 0.3.0.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.2.3...v.0.3.0)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-07-02 22:44:05 +00:00
Ruud van Asseldonk d951916d40 Make test_hash_stored_account pass in release mode 
The hash differs between debug and release mode, which makes the test
fail when running under "cargo test --release". Use cfg! to include a
different expected hash in release mode.
 2021-07-01 18:27:24 -07:00
dependabot[bot] 0426c2d28e
chore: bump bzip2 from 0.3.3 to 0.4.3 (#18361) 
* chore: bump bzip2 from 0.3.3 to 0.4.3

Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs) from 0.3.3 to 0.4.3.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases)
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits)

---
updated-dependencies:
- dependency-name: bzip2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update to new api

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-07-01 13:10:52 -06:00
Justin Starry d5961e9d9f
Reject transactions with extra signatures (#18306) 
* Reject transactions with extra signatures

* fix tests

* fix check

* fix check

* tx method

* fix checks
 2021-07-01 18:06:59 +00:00
Brooks Prumo 45d54b1fc6
Add SnapshotArchiveInfo and refactor functions in snapshot_utils (#18232) 2021-07-01 12:20:56 -05:00
dependabot[bot] 644632fc8a
chore: bump assert_matches from 1.3.0 to 1.5.0 (#18358) 
* chore: bump assert_matches from 1.3.0 to 1.5.0

Bumps [assert_matches](https://github.com/murarth/assert_matches) from 1.3.0 to 1.5.0.
- [Release notes](https://github.com/murarth/assert_matches/releases)
- [Commits](https://github.com/murarth/assert_matches/commits)

---
updated-dependencies:
- dependency-name: assert_matches
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-07-01 09:59:01 -06:00
Justin Starry 5ca975383c
Remove feature switch for using message hash for already processed check (#18340) 2021-07-01 09:33:55 -05:00
Brooks Prumo 89a3e4f91e
Move SnapshotConfig into its own module (#18331) 
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
 2021-07-01 08:55:26 -05:00
Trent Nelson d3a7e22112 sdk: refactor pda generation 2021-06-30 19:01:51 -06:00
Jeff Washington (jwash) 531f1bce78
report on min/max bin stats for accounts index (#18338) 2021-07-01 00:46:59 +00:00
Justin Starry b08f8bd1b0
Fail simulation if transaction contains duplicate accounts (#18304) 2021-06-30 13:50:55 -05:00
dependabot[bot] 78968d132f
chore: bump log from 0.4.11 to 0.4.14 (#18323) 
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-30 18:32:01 +00:00
dependabot[bot] 00d1125e98
chore: bump ouroboros from 0.9.3 to 0.9.5 (#18324) 
* chore: bump ouroboros from 0.9.3 to 0.9.5

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.9.3 to 0.9.5.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-30 09:37:55 -06:00
Jeff Washington (jwash) ce53b84cdc
refactor untar_snapshot_in to push parallelism deeper for further refactoring (#18310) 2021-06-29 18:26:15 -05:00
dependabot[bot] 52fd10ce03
chore: bump byteorder from 1.3.4 to 1.4.3 (#18300) 
* chore: bump byteorder from 1.3.4 to 1.4.3

Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.3.4 to 1.4.3.
- [Release notes](https://github.com/BurntSushi/byteorder/releases)
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.4...1.4.3)

---
updated-dependencies:
- dependency-name: byteorder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Bump expected instruction count

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-29 15:08:04 -06:00
Brooks Prumo c1b9d40a64
Add snapshot_runtime_info module (#18199) 
The `snapshot_runtime_info` module adds the `SnapshotRuntimeInfo` and
`SyncSnapshotRuntimeInfo` types to handle sharing runtime snapshot information
between threads; in particular: AccountsDb, Bank, SnapshotPackagerService, and
snapshot_utils.
 2021-06-29 15:13:34 -05:00
dependabot[bot] 9bd02c9570
chore: bump zstd from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0 (#18299) 
* chore: bump zstd from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0

Bumps [zstd](https://github.com/gyscos/zstd-rs) from 0.5.3+zstd.1.4.5 to 0.9.0+zstd.1.5.0.
- [Release notes](https://github.com/gyscos/zstd-rs/releases)
- [Commits](https://github.com/gyscos/zstd-rs/commits/0.9.0)

---
updated-dependencies:
- dependency-name: zstd
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-29 11:12:57 -06:00
carllin 47cafb70da
test (#18159) 2021-06-28 18:27:26 -07:00
Jeff Washington (jwash) afd64d27c9
bin accounts index (#18111) 2021-06-28 15:03:57 -05:00
Jeff Washington (jwash) 1f1e54b9d8
reduce pubkey copies on insert path (#18240) 2021-06-28 12:04:29 -05:00
Jack May 2fbedd834f Don't update if already an executable 2021-06-26 19:19:26 -06:00
Jeff Washington (jwash) e06376664b
replace account index bulk insert with iterator (#18198) 2021-06-25 14:31:55 -05:00
dependabot[bot] 91576fdd9b
chore: bump blake3 from 0.3.7 to 0.3.8 (#18223) 
* chore: bump blake3 from 0.3.7 to 0.3.8

Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3) from 0.3.7 to 0.3.8.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases)
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/0.3.7...0.3.8)

---
updated-dependencies:
- dependency-name: blake3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-06-25 11:44:52 -06:00
Jeff Washington (jwash) f2a2581259
metric for accounts index insertion time (#18202) 2021-06-25 08:36:55 -05:00
dependabot[bot] 9429d0463c
chore: bump ouroboros from 0.5.1 to 0.9.3 (#18189) 
* chore: bump ouroboros from 0.5.1 to 0.9.3

Bumps [ouroboros](https://github.com/joshua-maros/ouroboros) from 0.5.1 to 0.9.3.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases)
- [Commits](https://github.com/joshua-maros/ouroboros/commits)

---
updated-dependencies:
- dependency-name: ouroboros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Api changes

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-25 00:53:54 -06:00
Jeff Washington (jwash) f9fccdee85
remove unnecessary copies from accounts index code paths (#18196) 2021-06-24 17:56:25 -05:00
Jeff Washington (jwash) 31ec986ea1
untar in parallel (#18184) 
* untar in parallel

* make enum for 'ignore' return value
 2021-06-24 17:29:49 -05:00
Jeff Washington (jwash) 4b314be5bd
rework dirty_pubkeys from insert_new_if_missing_into_primary_index (#18200) 2021-06-24 14:52:11 -05:00
Jeff Washington (jwash) 77c3ffe4cf
add min/max bin size metrics to hash calculation (#18155) 2021-06-23 12:57:05 -05:00
sakridge 3b1738c000
Keep track of dirty stores on remove accounts to clean (#17601) 
* Keep track of dirty stores on remove accounts to clean

and not zero_lamport key set

* Only dirty when count==0?

* Add another clean
 2021-06-23 10:28:35 +02:00
Lijun Wang ccf6b21bf8
Removed unused purged_slots in purge_older_root_entries (#18102) 
Removed unused purged_slots in purge_older_root_entries
 2021-06-22 15:27:37 -07:00
Jeff Washington (jwash) d3ee73e151
capture sort time in hash calculation (#18118) 2021-06-22 16:02:21 -05:00
Jeff Washington (jwash) fbc94d84c8
apply 'debug_do_not_add_builtins' to feature activations (#18110) 2021-06-22 14:21:49 -05:00
Jeff Washington (jwash) d5f9f3b7ce
correctly calculate hash_total for hash/lamport calculation (#18144) 2021-06-22 11:36:43 -05:00
sakridge e808f34b0b
Add batch stats (#18096) 2021-06-22 15:23:26 +02:00
Tyera Eulberg 0ab361b828
Use newly stabilized Duration methods/consts (#18101) 2021-06-22 00:11:16 +00:00
Tyera Eulberg 19fe1dd463
Move stake_weighted_timestamp module (#18114) 
* Move timestamp module into runtime

* Less public

* Remove unused enum
 2021-06-21 16:14:54 -06:00
Jeff Washington (jwash) 2087f5da94
refactor generate_index_for_slot (#17984) 2021-06-21 15:32:50 -05:00
Jeff Washington (jwash) bf97627021
eliminate flatten and sort in hash calculation (#17802) 
* eliminate flatten and sort in hash calculation

* reduce critical section time

* remove now no-longer necessary test code

* conflict with reset bins to 0 pr
 2021-06-21 15:32:03 -05:00
Tyera Eulberg 4d46cb686a Fix build: clippy lint 2021-06-21 10:18:21 -07:00
dependabot[bot] d458fac2ff
chore: bump bincode from 1.3.1 to 1.3.3 (#18087) 
* chore: bump bincode from 1.3.1 to 1.3.3

Bumps [bincode](https://github.com/servo/bincode) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.3.1...v1.3.3)

---
updated-dependencies:
- dependency-name: bincode
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-21 16:06:31 +00:00
Jeff Washington (jwash) ae82e4e25b
refactor reconstruct_accountsdb_from_fields (#17987) 2021-06-21 10:47:58 -05:00
dependabot[bot] 491e4450cc
chore: bump libc from 0.2.96 to 0.2.97 (#18060) 
* chore: bump libc from 0.2.96 to 0.2.97

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-18 11:51:06 -06:00
Alexander Meißner 789f33e8db chore: cargo fmt 2021-06-18 10:42:46 -07:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
dependabot[bot] abc9839d86
chore: bump flate2 from 1.0.14 to 1.0.20 (#18062) 
* chore: bump flate2 from 1.0.14 to 1.0.20

Bumps [flate2](https://github.com/rust-lang/flate2-rs) from 1.0.14 to 1.0.20.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.14...1.0.20)

---
updated-dependencies:
- dependency-name: flate2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-18 15:55:48 +00:00
Jeff Washington (jwash) 0a81c37fce
introduce LockMapType for accounts_index (#18021) 2021-06-17 14:41:52 -05:00
Jeff Washington (jwash) 1feaaf009d
generate_index timings (#17983) 2021-06-17 13:35:25 -05:00
Jeff Washington (jwash) 20e714b3d0
remove unused code (#17981) 2021-06-17 13:30:46 -05:00
Jeff Washington (jwash) 71796f4951
break out generate index reporting (#17980) 2021-06-16 12:09:17 -05:00
Jeff Washington (jwash) 7ca04d6a86
refactor generate_index process_storage_slot (#17982) 2021-06-16 11:02:34 -05:00
Jeff Washington (jwash) bb7413c9f9
refactor rebuild_bank_from_snapshots (#17988) 2021-06-16 11:01:52 -05:00
Jeff Washington (jwash) eee5414c64
avoid unnecessary empty arrays when binning (#17944) 2021-06-16 09:24:59 -05:00
Jeff Washington (jwash) adc683956f
do not use index for verify_bank_hash_and_lamports (#17812) 2021-06-16 09:10:38 -05:00
Jeff Washington (jwash) 55ee3b5f2f
set hash bins to 65k (#17912) 2021-06-16 09:09:55 -05:00
Jeff Washington (jwash) 7de79425ce
parallel storage -> accounts folder (#17955) 2021-06-16 08:48:24 -05:00
Jeff Washington (jwash) dbd4dc04b0
ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974) 2021-06-15 15:39:22 -05:00
Jeff Washington (jwash) f558b9b6bf
verify bank hash on startup with ledger tool option (#17939) 2021-06-15 11:52:12 -05:00
Jon Cinque 1b1d34da59
Refactor stake program into solana_program (#17906) 
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
 2021-06-15 18:04:00 +02:00
Jeff Washington (jwash) 5ecb30ff58
nit: rename measure (#17946) 2021-06-15 09:01:11 -05:00
Jeff Washington (jwash) 4d8ffdcc11
remove unused parameters from CalculateHashIntermediate (#17949) 2021-06-15 09:00:43 -05:00
carllin ccc013e134
Handle removing slots during account scans (#17471) 2021-06-14 21:04:01 -07:00
Jeff Washington (jwash) 471b34132e
add metrics for startup (#17913) 
* add metrics for startup

* roll timings up higher

* fix test

* fix duplicate
 2021-06-14 17:46:49 -05:00
Jeff Washington (jwash) 7fde9b6ff0
name arguments to help with confusion (#17942) 2021-06-14 16:21:48 -05:00
Jeff Washington (jwash) 62e1b20e56
refactor so hash verify can be done by more callers (#17941) 2021-06-14 16:20:03 -05:00
Jeff Washington (jwash) e6bbd4b3f0
add metrics to handle_snapshot_requests (#17937) 2021-06-14 15:46:19 -05:00
dependabot[bot] a0872232d3
chore: bump itertools from 0.9.0 to 0.10.1 (#17929) 
* chore: bump itertools from 0.9.0 to 0.10.1

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.9.0...v0.10.1)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix versions

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-14 18:32:20 +00:00
sakridge c2191d885d
Fix accounts index panic in purge_exact (#17757) 2021-06-14 16:10:26 +02:00
Jeff Washington (jwash) d4cc975fe9
calculate_capitalization uses hash calculation (#17443) 
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
 2021-06-14 08:53:07 -05:00
Jeff Washington (jwash) 2dc6969858
use trait to simplify and consolidate cumulative code (#17852) 2021-06-14 08:43:59 -05:00
dependabot[bot] 2e75023840
chore: bump regex from 1.4.5 to 1.5.4 (#17885) 
* chore: bump regex from 1.4.5 to 1.5.4

Bumps [regex](https://github.com/rust-lang/regex) from 1.4.5 to 1.5.4.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.4.5...1.5.4)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-11 09:50:54 -06:00
dependabot[bot] f08ed1eb2d
chore: bump rayon from 1.5.0 to 1.5.1 (#17869) 
* chore: bump rayon from 1.5.0 to 1.5.1

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-06-10 13:18:47 -06:00
Lijun Wang 269d995832
Make account shrink configurable #17544 (#17778) 
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544
 2021-06-09 21:21:32 -07:00
Jack May a1fab0c5ca
Log more info on runtime account verification errors (#17861) 2021-06-09 16:27:00 -07:00
Justin Starry 050bb5446d
Add local cluster tests that broadcast duplicate slots (#13995) 
* Add duplicate node local cluster test

* fix clippy

* remove dupe test
 2021-06-09 15:01:48 -07:00
dependabot[bot] 9a2ca8dd2f
chore: bump rustc_version from 0.2.3 to 0.4.0 (#17854) 
* chore: bump rustc_version from 0.2.3 to 0.4.0

Bumps [rustc_version](https://github.com/Kimundi/rustc-version-rs) from 0.2.3 to 0.4.0.
- [Release notes](https://github.com/Kimundi/rustc-version-rs/releases)
- [Commits](https://github.com/Kimundi/rustc-version-rs/compare/v0.2.3...v0.4.0)

---
updated-dependencies:
- dependency-name: rustc_version
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make versions consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-09 16:57:39 +00:00
Jeff Washington (jwash) 576e3d95f7
refactor: create type (#17818) 2021-06-08 14:49:08 -06:00
Jeff Washington (jwash) 2a9b127029
Pass iterator for SortedStorages::new_with_slots (#17811) 
Co-authored-by: Carl Lin <carl@solana.com>
 2021-06-08 13:59:50 -06:00
Jeff Washington (jwash) 01ef2a5c4a
test: simple CalculateHashIntermediate refactoring (#17813) 2021-06-08 13:54:10 -06:00
Jeff Washington (jwash) 58fe1d0764
add min to roots_tracker (#17806) 2021-06-08 08:32:16 -06:00
Jeff Washington (jwash) 817d48be21
binary search in slices for hashing (#17755) 2021-06-07 12:09:17 -05:00
Jeff Washington (jwash) b5bb91b50f
rework hash calculation to not keep slot and write version (#17685) 
* rework hash calculation to not keep slot and write version

* refactor functions and add tests

* always use multiple slot code path
 2021-06-07 09:01:16 -05:00
Jon Cinque 8f5e773caf
system-program: Remove zero lamport check on transfers (#17726) 
* system-program: Move lamports == 0 check on transfers

* Address feedback

* Update stake split to explicitly allocate + assign

* Update stake tests referring to split instruction

* Revert whitespace

* Update split instruction index in test

* Remove unnecessary `assign_with_seed` from `split_with_seed`

* Fix stake instruction parser

* Update test to allow splitting into account with lamports
 2021-06-06 01:45:45 +02:00
Jeff Washington (jwash) 00ee84af37
break out hash time in metric (#17721) 2021-06-04 14:54:06 -05:00
Jeff Washington (jwash) f6fb8906c7
add data point for cap mismatch (#17746) 2021-06-04 13:54:39 -05:00
Jeff Washington (jwash) 5197454fea
increase bin count for hash scan (#17562) 2021-06-04 10:53:13 -05:00
sakridge f97ce2cd7e
Per-program id timings (#17554) 2021-06-04 16:04:31 +02:00
Jeff Washington (jwash) 81bafd9daf
add info for vm.max_map_count incorrectly set. (#17727) 2021-06-04 03:22:12 +00:00
dependabot[bot] 3dcc8e0046
chore: bump libloading from 0.6.2 to 0.7.0 (#17704) 
* chore: bump libloading from 0.6.2 to 0.7.0

Bumps [libloading](https://github.com/nagisa/rust_libloading) from 0.6.2 to 0.7.0.
- [Release notes](https://github.com/nagisa/rust_libloading/releases)
- [Commits](https://github.com/nagisa/rust_libloading/compare/0.6.2...0.7.0)

---
updated-dependencies:
- dependency-name: libloading
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* New libloading methods are marked unsafe

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-03 18:08:31 -06:00
Jeff Washington (jwash) 9388aaca15
use slots returned from get_snapshot_storages to sort (#17638) 
* use slots returned from get_snapshot_storages to sort

* add tests
 2021-06-02 18:24:55 -05:00
Jeff Washington (jwash) 4bd32d891f
remove slot lookup (#17691) 2021-06-02 17:56:19 -05:00
Jeff Washington (jwash) 83ceedc091
simplify test construction (#17686) 2021-06-02 17:32:31 -05:00
Jeff Washington (jwash) db8811eacd
support bin divisions up to 65536 (#17563) 
* support bin divisions up to 65536

* add tests
 2021-06-02 17:31:15 -05:00
dependabot[bot] 8e1e57a181
chore: bump tar from 0.4.28 to 0.4.35 (#17666) 
* chore: bump tar from 0.4.28 to 0.4.35

Bumps [tar](https://github.com/alexcrichton/tar-rs) from 0.4.28 to 0.4.35.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases)
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.28...0.4.35)

---
updated-dependencies:
- dependency-name: tar
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-06-02 14:17:58 -06:00
dependabot[bot] 19c49e2e25
chore: bump crossbeam-channel from 0.4.4 to 0.5.0 (#17665) 
* chore: bump crossbeam-channel from 0.4.4 to 0.5.0

Bumps [crossbeam-channel](https://github.com/crossbeam-rs/crossbeam) from 0.4.4 to 0.5.0.
- [Release notes](https://github.com/crossbeam-rs/crossbeam/releases)
- [Changelog](https://github.com/crossbeam-rs/crossbeam/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossbeam-rs/crossbeam/compare/crossbeam-channel-0.4.4...crossbeam-channel-0.5.0)

---
updated-dependencies:
- dependency-name: crossbeam-channel
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-06-02 19:36:28 +00:00
Jeff Washington (jwash) d802eb303c
reduce copy (#17672) 2021-06-02 14:03:02 -05:00
Jeff Washington (jwash) 07bac27ac7
remove antiquated assert (#17643) 2021-06-02 09:47:26 -05:00
Jeff Washington (jwash) eec996ba41
implement ancestors as rolling bit field (#17482) 2021-06-02 09:32:32 -05:00
Jeff Washington (jwash) 107af52deb
minor test code cleanup (#17645) 2021-06-02 08:35:10 -05:00
carllin bbcdf073ba
Support out of band dumping of unrooted slots in AccountsDb (#17269) 
* Accounts dumping logic

* Add test for interaction between cache flush and remove_unrooted_slot()

* Update comments

* Rename

* renaming

* Add more comments

* Renaming

* Fixup test and bad check
 2021-06-02 09:51:10 +00:00
Jack May 80e5b24b38
Fix CPI recursion depth (#17659) 
* Fix CPI recursion depth
 2021-06-02 11:15:19 +02:00
sakridge 14c52ab018
Skip shrink when it doesn't save anything (#17405) 2021-06-02 09:51:46 +02:00
Jeff Washington (jwash) 738cc9549f
parallel get_snapshot_storages (#17589) 2021-06-01 19:33:12 -05:00
Jeff Washington (jwash) c8fdf0762c
add Ancestors to get_snapshot_storages to prepare for the future (#17591) 2021-06-01 15:14:35 -05:00
Jeff Washington (jwash) ef5169ff24
sort storages by slot before scan (#17411) 
* sort storages by slot before scan

* fix return value
 2021-06-01 15:07:46 -05:00
Jeff Washington (jwash) 72bb271a94
add metric for collecting storages (#17527) 2021-06-01 13:17:49 -05:00
Jeff Washington (jwash) 886898eabf
rework test for flexibility (#17592) 
* rework test for flexibility

* respond to pr feedback
 2021-05-31 22:49:17 -05:00
dependabot[bot] ab0f4ff835
Bump serde from 1.0.122 to 1.0.126 (#17618) 
* Bump serde from 1.0.122 to 1.0.126

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.122 to 1.0.126.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.122...v1.0.126)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-05-31 22:41:25 +00:00
dependabot[bot] ac452f9bb8
Bump tempfile from 3.1.0 to 3.2.0 (#17587) 
* Bump tempfile from 3.1.0 to 3.2.0

Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases)
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.1.0...v3.2.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-05-31 18:05:07 +00:00
dependabot[bot] a35024123c
chore: bump memmap2 from 0.1.0 to 0.2.3 (#17548) 
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.1.0 to 0.2.3.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.1.0...v0.2.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-05-28 13:11:58 -06:00
Jeff Washington (jwash) 654918ab27
refactor SnapshotStorage helpers to prepare for later changes (#17560) 2021-05-28 13:54:53 -05:00
Jack May a3240aebde
Always bail if program modifies a ro account (#17569) 2021-05-28 09:50:25 -07:00
Jeff Washington (jwash) 8924fbf6a0
RollingBitField: bug fixes and add tests (#17525) 
* RollingBitField: bug fixes and add tests

* respond to feedback
 2021-05-28 16:33:07 +00:00
Jeff Washington (jwash) 1d02dba06f
remove unnecessary clone (#17559) 2021-05-28 10:30:01 -05:00
Jeff Washington (jwash) 55c22d3b76
add check_hash to non-index hash calculation (#17558) 2021-05-28 10:24:40 -05:00
Brooks Prumo 1953543274
Refactor bank_forks_utils::load() and some snapshot logic (#17492) 
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.

Additionally, add some tests and doc comments.
 2021-05-27 12:00:27 -05:00
Michael Vines 8eab0e8602
Bump version to v1.8.0 (#17541) 2021-05-27 08:51:53 -07:00
sakridge 49402b7d82
Only add keys with multiple index entries to the clean set (#17473) 2021-05-27 16:31:24 +02:00
Jeff Washington (jwash) 3973bf08f1
move local out of call (#17519) 2021-05-27 08:53:11 -05:00
Michael Vines 9541411c15 Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method 2021-05-27 03:05:05 +00:00
Michael Vines cbce440af4 simulateTransaction can now return accounts modified by the simulation 2021-05-26 14:20:23 -07:00
sakridge 3f9e3c7375
Remove copied shrink code (#17385) 2021-05-26 19:27:18 +02:00
Jeff Washington (jwash) 6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool (#17417) 2021-05-26 10:36:12 -05:00
Trent Nelson 2a6c5ed0ac add tests from `RentDebits` 2021-05-26 08:19:10 +00:00
Trent Nelson 97eab7edf9 runtime: add rent debit charges to block metadata 2021-05-26 08:19:10 +00:00
Tyera Eulberg e9bc1c6b07
Add last valid block height to rpc Fees (#17506) 
* Add last_valid_block_height to fees rpc

* Add getBlockHeight rpc

* Update docs
 2021-05-26 07:26:19 +00:00
steviez a5c2067273
fixup! Ancestors to HashSet: From(Vec) (#17447) (#17481) 2021-05-25 12:24:34 -05:00
Jeff Washington (jwash) 98f1b11edd
Ancestors to HashSet: From(Vec) (#17447) 
* ancestors.large_range_slots becomes HashSet

* Ancestors to HashSet: From(Vec)
 2021-05-25 11:01:11 -05:00
Jeff Washington (jwash) d39a327138
ancestors.large_range_slots becomes HashSet (#17446) 2021-05-25 11:00:50 -05:00
Jeff Washington (jwash) 40bb6577fc
initialize purge stats (#17345) 2021-05-25 09:24:38 -05:00
Jeff Washington (jwash) d4fc9e7ed4
ad metric for collect (#17418) 2021-05-24 18:14:52 -05:00
Jeff Washington (jwash) dd22ae047b
ancestors is set instead of map (#17363) 2021-05-24 17:01:02 -05:00
carllin d8bc56fa51
Refactor purge_slots_from_cache_and_store() and handle_reclaims() (#17319) 2021-05-24 13:51:17 -07:00
jon-chuang ad1f24d487
runtime: Executor usage counts retain only single-epoch memory (#17162) 2021-05-24 12:01:56 -07:00
sakridge a8dca3976b
Refactor genesis download/load/check functions (#17276) 
* Refactor genesis ingest functions

* Consolidate genesis.bin/genesis.tar.bz2 references
 2021-05-24 16:45:36 +02:00
Alexander Meißner 855ae79598
Fix InvokeContext::push() account_deps (#17350) 
* Reverts aliasing of account_deps with the previous invocation stack frame in InvokeContext::push().
* Adds explicit assert of programdata account content in test_program_bpf_upgrade_via_cpi.
 2021-05-21 22:34:07 +02:00
Jeff Washington (jwash) 3f3324231d
Hold read lock during startup shrink (#17309) 
* hold read lock during account scan of shrink

* rename and improve rusty
 2021-05-21 13:59:32 -05:00
Jeff Washington (jwash) 44831c18d2
reuse work in is_non_loader_key (#16521) 2021-05-21 10:07:04 -05:00
Jeff Washington (jwash) 8143ee5502
add AccountsIndex AccountMapsReadLock type (#17365) 2021-05-21 10:02:14 -05:00
Jeff Washington (jwash) 2f55547d37
remove flatten in storage scan (#17354) 
* remove flatten in storage scan

* fix tests
 2021-05-21 09:26:46 -05:00
Michael Vines 2c99b23ad7 Add get_sysvar() helper to sdk 2021-05-21 01:30:12 +00:00
Jeff Washington (jwash) 45552d271a
take mutex once in process_dead_slots (#17357) 2021-05-20 19:50:58 -05:00
Jeff Washington (jwash) 0486df02ba
don't limit to thread pool when cleaning on startup (#17317) 2021-05-20 14:36:35 -05:00
Jeff Washington (jwash) 33ab9c4e8d
batch insert account_index items in generate_index (#17290) 2021-05-20 10:29:13 -05:00
Jeff Washington (jwash) 9b74988fc6
calculate_accounts_hash does not throttle threads on startup (#17338) 2021-05-20 10:25:54 -05:00
Jeff Washington (jwash) a9aa533684
replace Ancestors HashMap for performance (#16477) 
* replace Ancestors HashMap for performance

* add ancestors module
 2021-05-20 10:11:56 -05:00
Jeff Washington (jwash) a544010b03
don't log shrink metrics on first call (#17328) 
* don't log shrink metrics on first call

* simplify logic
 2021-05-20 00:15:35 +00:00
Jeff Washington (jwash) 32ec8341f9
generate_index inserts ideal initial data (#17247) 
* improve insert into map initially

* rework towards single code path

* rename

* update test
 2021-05-19 16:21:24 -05:00
Jeff Washington (jwash) b5302e7619
add calc_stored_meta_us metric (#17318) 2021-05-19 12:29:41 -05:00
Jeff Washington (jwash) c20b27bc8f
shink all in parallel on startup (#17308) 2021-05-19 12:15:24 -05:00
Jeff Washington (jwash) ed9cbd50f0
move Ancestors to its own module (#17316) 2021-05-19 11:50:34 -05:00
Tao Zhu 0781fe1b4f
Upgrade Rust to 1.52.0 (#17096) 
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script

* upgrade to Rust 1.52.1

* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
 2021-05-19 09:31:47 -05:00
Jeff Washington (jwash) 4788976517
rework slot list update (#17232) 2021-05-19 08:48:46 -05:00
Jeff Washington (jwash) 7fe24c455c
simplify generate_index (#17292) 2021-05-19 08:11:41 -05:00
Tyera Eulberg 827355a6b1
Create solana-rpc crate and move subscriptions (#17320) 
* Move non_circulating_supply to runtime

* Add solana-rpc crate and move max_slots

* Move subscriptions to solana-rpc

* Single use statements
 2021-05-19 00:54:28 -06:00
Jeff Washington (jwash) a3c0833a1c
tests: cleanup test (#17305) 
* tests: cleanup test

* 2 more replacements
 2021-05-18 18:08:37 -05:00
Jeff Washington (jwash) 75335b4f58
datapoint for verify_snapshot_bank (#17306) 2021-05-18 12:14:43 -05:00
Jeff Washington (jwash) 2175fc0625
calculate account infos before lock (#17245) 2021-05-18 08:06:19 -05:00
Jeff Washington (jwash) 3a56a56d69
StoredMetaWriteVersion (#17293) 2021-05-17 23:58:36 +00:00
Ryo Onodera bcbe155575
Minor test cleanup and comments (#17283) 2021-05-18 07:22:43 +09:00
Jeff Washington (jwash) d461aba6a4
don't try to build secondary indexes if we don't need them (#17246) 2021-05-17 17:21:15 -05:00
Jeff Washington (jwash) 9d6837c904
fix comment (#17286) 2021-05-17 14:06:25 -05:00
Jeff Washington (jwash) d1db5448b9
hold lock to speed up insert (#17194) 
* hold lock to speed up insert

* add tests
 2021-05-17 11:58:33 -05:00
Jeff Washington (jwash) 3745e0babc
minor refactoring of AccountsIndex in preparation of bigger things (#17237) 2021-05-14 15:27:10 -05:00
Jeff Washington (jwash) fd88db7339
ancestors::new -> default (#17195) 2021-05-14 15:20:59 -05:00
Jeff Washington (jwash) c6f7867aa1
log mismatch (#17209) 2021-05-14 09:50:11 -05:00
Brooks Prumo 76dfacca61
Add documentation for BankForks::prune_non_rooted() (#17213) 2021-05-14 07:47:48 -05:00
Tyera Eulberg 27004f1b76
Return error for excluded secondary-index keys (#17193) 
* Add runtime helpers to check secondary indexes for key

* Add custom rpc error

* Check secondary-index key inclusion in rpc

* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
 2021-05-13 21:04:21 +00:00
Jeff Washington (jwash) 3dbc7744ab
metrics for generating index time (#17192) 
* metrics for generating index time

* update metrics to include scan time
 2021-05-13 14:32:19 -05:00
carllin 239ab8799c
Remove bloat from secondary indexes (#17048) 2021-05-12 15:29:30 -07:00
Lijun Wang 9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. (#17158) 
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
 2021-05-12 10:32:27 -07:00