blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
17,908 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

805 Commits

Author SHA1 Message Date
ryleung-solana 8cf36e5cb0
Add GPU sigverify for verify path (#20851) 
Allows the use of GPU acceleration in verifying the signatures in Entry's after deserialization in the replay stage

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
Co-authored-by: Ryan Leung <ryan.leung@solana.com>
 2021-11-30 21:16:13 -05:00
Yueh-Hsuan Chiang 38fcfb7542
Persist coding shreds (#21214) 2021-11-30 09:18:36 -08:00
Michael Vines e31b469f56 Quash M1 build warning 2021-11-29 22:42:09 -08:00
Michael Vines ba9dfa0d22 Remove frozen account support 2021-11-29 08:38:11 -08:00
dependabot[bot] 3c2276fd09
chore: bump futures from 0.3.17 to 0.3.18 (#21412) 
* chore: bump futures from 0.3.17 to 0.3.18

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-23 21:55:17 -07:00
Tyera Eulberg 9e043d2055
Use max_complete_transaction_status_slot in BigTableUploadService (#21401) 
* Use max_complete_transaction_status_slot in BigTableUploadService

* Use node root to limit BigTableUploadService
 2021-11-23 17:36:13 -07:00
behzad nouri dd338b6c9f
changes Shred::parent return type to Option<Slot> (#21370) 
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453

The commit adds more sanity checks and changes the return type to Option<Slot>.
 2021-11-23 14:45:26 +00:00
dependabot[bot] 191cb62c37
chore: bump libc from 0.2.107 to 0.2.108 (#21377) 
* chore: bump libc from 0.2.107 to 0.2.108

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.107 to 0.2.108.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.107...0.2.108)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-22 17:02:34 -07:00
Jeff Biseda 2ed7e3af89
prioritize slot repairs for unknown last index and close to completion (#21070) 2021-11-19 19:17:30 -08:00
behzad nouri 7da2df7d8c
adds methods to obtain shreds' erasure coded block and index (#21325) 2021-11-19 20:01:15 +00:00
behzad nouri 48dfdfb4d5 changes Blockstore::is_shred_duplicate arg type to ShredType 2021-11-19 14:16:39 +00:00
behzad nouri 57057f8d39 uses enum for shred type 
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167

Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348

The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
 2021-11-19 14:16:39 +00:00
Justin Starry 66fa062f13
rename process_entries to indicate it's only for tests (#21321) 2021-11-17 20:53:40 +01:00
Yueh-Hsuan Chiang 3aa49e2c69
Keep all erasure coding shreds even on successful recovery (#20968) (#21052) 
Problem
Blockstore currently removes erasure shreds if the data shreds are
successfully recovered on insert, which is an issue if we want to
serve coding shreds over repair.

Summary of Changes
This diff keeps all coding shreds even on successful recovery and
changes change the signature of prev_inserted_codes to immutable
reference to ensure its immunity.

Fixes #20968
 2021-11-16 11:26:34 -08:00
behzad nouri 3fc858eb60 adds methods to obtain data/coding shreds indices from ErasureMeta 2021-11-13 17:08:05 +00:00
behzad nouri d25d9be555 removes Default implementaion of ErasureConfig 
The Default implementation of ErasureConfig is off and never useful:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/erasure.rs#L49-L68
 2021-11-13 17:08:05 +00:00
behzad nouri 7e98b77b33 marks ErasureMeta.size as unused 
ErasureMeta.size is unused and redundant:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/blockstore_meta.rs#L57-L58
 2021-11-13 17:08:05 +00:00
behzad nouri 5fb0ab9d00
removes redundant args from Shredder::try_recovery (#21226) 
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
 2021-11-10 21:19:03 +00:00
Kirill Fomichev b4d09388aa
Disable JIT compiler on unsupported platforms (#21215) 2021-11-09 11:31:21 +00:00
dependabot[bot] c2754b8a51
chore: bump libc from 0.2.106 to 0.2.107 (#21202) 
* chore: bump libc from 0.2.106 to 0.2.107

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.106 to 0.2.107.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.107)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-08 12:14:12 -07:00
behzad nouri 11a53de0e3
rewrites Shredder::try_recovery (#21082) 
For every missing data-shred, Shredder::try_recovery calls into
new_empty_data_shred which does a redundant serialization into payload
buffer which is then immediately overwritten by the erasure recovery:
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L372-L417

Additionally, the implementation is unnecessary complex hindering
upcoming changes to erasure coding generation
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L814-L938

The commit simplifies the Shredder::try_recovery implementation.
 2021-11-04 16:15:51 +00:00
dependabot[bot] ca51ea3e04
chore: bump libc from 0.2.105 to 0.2.106 (#21093) 
* chore: bump libc from 0.2.105 to 0.2.106

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.105 to 0.2.106.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.105...0.2.106)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-01 14:17:45 -06:00
behzad nouri 484ead01ed
optimizes and simplifies SlotMeta::completed_data_indexes ops (#21059) 
SlotMeta::completed_data_indexes is defined as a Vec<u32>:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore_meta.rs#L31-L32

which results in inefficient updates:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore.rs#L3245-L3326

This commit changes the type to BTreeSet<u32> for efficient and simpler
updates and lookups.

The change should be backward compatible because Vec<T> and BTreeSet<T>
are both serialized as seq:
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L207-L208
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L216-L217
 2021-10-31 12:56:25 +00:00
sakridge 4642a2c856
Optimize test_recovery_with_expanded_coding_shreds (#20849) 2021-10-28 11:38:08 +02:00
Tyera Eulberg f0de3e9bf0
chore: bump tonic from 0.5.2 to 0.6.1 (#21035) 
* Bump tonic and prost crates

* connect_lazy no longer infallible
 2021-10-28 06:59:36 +00:00
Justin Starry 036d7fcc81
Clean up sanitized tx creation for tests (#21006) 2021-10-27 18:09:16 +01:00
dependabot[bot] cb1947bea2
chore: bump libc from 0.2.104 to 0.2.105 (#20972) 
* chore: bump libc from 0.2.104 to 0.2.105

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.104 to 0.2.105.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.104...0.2.105)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-26 00:14:58 -06:00
Michael Vines 350bb561eb Clippy 2021-10-23 08:21:20 +00:00
dependabot[bot] 66e431dad5
chore: bump libc from 0.2.103 to 0.2.104 (#20764) 
* chore: bump libc from 0.2.103 to 0.2.104

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-19 22:38:02 -06:00
carllin b5f21d5e34
Add ledger tool to remove dead slot (#20810) 2021-10-19 20:23:16 -07:00
behzad nouri 231b58b5f1
adds more metrics to blockstore insert shreds stats (#20701) 2021-10-17 11:02:34 -04:00
Jeff Washington (jwash) 70b2e5fef2
hash calc uses all threads during startup (#20735) 2021-10-15 17:41:23 -05:00
Tyera Eulberg e806fa6904
Include token owners in TransactionTokenBalances (#20642) 
* Cache owners in TransactionTokenBalances

* Light cleanup

* Use return struct, and remove pub

* Single-use statements

* Why not, just do the whole crate

* Add metrics

* Make datapoint_debug to prevent spam unless troubleshooting
 2021-10-13 21:46:52 -06:00
sakridge 09e7782d76
Refactor code to get block signatures in get_confirmed_signatures_for_address2 (#20575) 
* Refactor get_confirmed_signatures_for_address2

* Move blockstore benches to ledger where they belong
 2021-10-13 09:55:19 +02:00
Tao Zhu 005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; (#20527) 
- move related modules to runtime
 2021-10-12 08:51:33 -05:00
Jeff Washington (jwash) a8e000a2a6
add filler accounts to bloat validator and predict failure (#20491) 
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
 2021-10-11 12:46:27 -05:00
Brooks Prumo 5440c1d2e1
SnapshotPackagerService pushes incremental snapshot hashes to CRDS (#20442) 
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!

This commit does two main things:

1. SnapshotPackagerService now knows about incremental snapshot hashes,
   and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
   hashes need to be passed all the way to SnapshotPackagerService so it
   can push these starting hashes to CRDS.  Those values have been piped
   through.

Fixes #20441 and #20423
 2021-10-08 15:14:56 -05:00
Tao Zhu 675fa6993b
- update const cost values with data collected by #19627 (#20314) 
- update cost calculation to closely proposed fee schedule #16984
 2021-10-08 14:48:50 -05:00
Trent Nelson 767f740305 Bump version to 1.9.0 2021-10-06 17:57:41 -07:00
Michael Vines 7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
steviez d8e4840b49
Follow up to blockstore test cleanup (#19656) (#20143) 2021-10-04 10:35:03 -05:00
Lijun Wang fe97cb2ddf
AccountsDb plugin framework (#20047) 
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
 2021-09-30 14:26:17 -07:00
dependabot[bot] 9393654af0
chore: bump libc from 0.2.102 to 0.2.103 (#20280) 
* chore: bump libc from 0.2.102 to 0.2.103

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-09-28 07:50:46 +00:00
Justin Starry c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` (#19931) 
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback
 2021-09-16 23:37:45 +00:00
dependabot[bot] ea15c0b40b
chore: bump libc from 0.2.101 to 0.2.102 (#19924) 
* chore: bump libc from 0.2.101 to 0.2.102

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.101 to 0.2.102.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.101...0.2.102)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-09-16 09:45:01 -06:00
steviez 4fdd70c931
Properly clean temp files in Blockstore::create_new_ledger() (#19589) 
Use TempDir::path() instead of TempDir::into_path(); into_path() returns
the underlying PathBuf and voids the promise to delete the directory
when TempDir goes out of scope.
 2021-09-13 23:13:56 -04:00
Brooks Prumo 97fc64dd0c
Call exhaustively_free_unused_resource() instead (#19847) 2021-09-13 18:08:32 -05:00
Brooks Prumo ef749a2506
Flush accounts cache before calling clean_accounts() (#19839) 2021-09-13 14:56:13 -05:00
Brooks Prumo 3f6eb96d6e
Call `Bank::update_accounts_hash()` before taking bank snapshot (#19765) 
This is a bugfix.

When `load_frozen_forks()` called `snapshot_bank()`, the accounts hash
was not computed.  So then when a snapshot archive was eventually
created, its hash would be all 1s, which is clearly wrong.  The fix is
to update the bank's accounts hash before taking the bank snapshot.
 2021-09-13 08:27:50 -05:00
Brooks Prumo 62c8bcf565
Add default() to SnapshotConfig (#19776) 2021-09-12 13:44:27 -05:00
dependabot[bot] 857fce13f9
chore: bump sha2 from 0.9.6 to 0.9.8 (#19760) 
* chore: bump sha2 from 0.9.6 to 0.9.8

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.6 to 0.9.8.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.6...sha2-v0.9.8)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-09-11 16:22:40 -06:00
Giles Cope c3a610374a unused deps 2021-09-11 14:33:53 -07:00
steviez 2dee098b91
Blockstore test cleanup (#19656) 
* Use auto-deleting macro for ledger path
 2021-09-10 07:33:08 -05:00
Jeff Washington (jwash) 456bf15012
AccountsIndexConfig -> AccountsDbConfig (#19687) 2021-09-08 04:30:38 +00:00
Jeff Washington (jwash) d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it (#19686) 2021-09-07 20:09:40 -05:00
Brooks Prumo 4a5f83d3a7
Add fn to check when to take snapshots (#19682) 2021-09-07 23:26:35 +00:00
Brooks Prumo a0552e5b46
Make startup aware of Incremental Snapshots (#19600) 2021-09-07 20:43:43 +00:00
Tyera Eulberg 5fa3e5744c
Populate memo in blockstore signatures-for-address (#19515) 
* Add TransactionMemos column family

* Traitify extract_memos

* Write TransactionMemos in TransactionStatusService

* Populate memos from column

* Dedupe and add unit test
 2021-09-03 00:28:52 -06:00
Jack May 170927b7c4
nit: Move builtins types to builtins file (#19597) 2021-09-02 21:29:11 -07:00
Brooks Prumo e9374d32a3
Revert "Make startup aware of Incremental Snapshots (#19550)" (#19599) 
This reverts commit d45ced0a5d.
 2021-09-02 19:14:41 -05:00
Brooks Prumo d45ced0a5d
Make startup aware of Incremental Snapshots (#19550) 2021-09-02 19:05:15 -05:00
Brooks Prumo 1d2f0f6641
Simplify process_blockstore tests (#19553) 
Some of the blockstore_processor tests weren't using
`test_process_blockstore()`, but could.  I updated those tests, and also
changed to using `..` for ignoring, instead of `_`, since I'll be adding
another return value here shortly (thus reducing the need to change
these again).
 2021-09-01 20:13:52 +00:00
behzad nouri 6d9818b8e4
skips retransmit for shreds with unknown slot leader (#19472) 
Shreds' signatures should be verified before they reach retransmit
stage, and if the leader is unknown they should fail signature check.
Therefore retransmit-stage can as well expect to know who the slot
leader is and otherwise just skip the shred.

Blockstore checking signature of recovered shreds before sending them to
retransmit stage:
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/blockstore.rs#L884-L930

Shred signature verifier:
https://github.com/solana-labs/solana/blob/4305d4b7b/core/src/sigverify_shreds.rs#L41-L57
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/sigverify_shreds.rs#L105
 2021-09-01 15:44:26 +00:00
dependabot[bot] 8e4360cf9c
chore: bump futures from 0.3.16 to 0.3.17 (#19513) 
* chore: bump futures from 0.3.16 to 0.3.17

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-30 22:12:38 -06:00
dependabot[bot] dc3d7ad2bd
chore: bump sha2 from 0.9.5 to 0.9.6 (#19511) 
* chore: bump sha2 from 0.9.5 to 0.9.6

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.5 to 0.9.6.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.5...sha2-v0.9.6)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 19:58:26 -06:00
dependabot[bot] 6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 (#19497) 
* chore: bump serde from 1.0.129 to 1.0.130

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 18:19:34 +00:00
dependabot[bot] fbffdf9f3d
chore: bump futures-util from 0.3.16 to 0.3.17 (#19498) 
* chore: bump futures-util from 0.3.16 to 0.3.17

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 17:45:47 +00:00
behzad nouri 8ad52fa095
implements copy-on-write for vote-accounts (#19362) 
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186

This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
 2021-08-30 15:54:01 +00:00
dependabot[bot] e492638b7d
chore: bump libc from 0.2.100 to 0.2.101 (#19442) 
* chore: bump libc from 0.2.100 to 0.2.101

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-08-26 12:38:34 -06:00
Jeff Washington (jwash) f1d9c014f8
create test function test_process_blockstore (#19432) 2021-08-26 12:48:52 -05:00
dependabot[bot] 46890ac197
chore: bump serde from 1.0.128 to 1.0.129 (#19395) 
* chore: bump serde from 1.0.128 to 1.0.129

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-24 12:32:30 -06:00
Brooks Prumo f3ce5fa5e8 clippy: branches-sharing-code 
```text
error: all if blocks contain the same code at the end
    --> ledger/src/blockstore.rs:3473:5
     |
3473 | /         Ok(insert_map.get(&slot).unwrap().clone())
3474 | |     }
     | |_____^
     |
     = note: `-D clippy::branches-sharing-code` implied by `-D warnings`
     = note: The end suggestion probably needs some adjustments to use the expression result correctly
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#branches_sharing_code
help: consider moving the end statements out like this
     |
3473 ~     }
3474 +     Ok(insert_map.get(&slot).unwrap().clone())
     |

error: could not compile `solana-ledger` due to previous error
```
 2021-08-23 11:17:21 -05:00
dependabot[bot] c0c95e88d8
chore: bump libc from 0.2.99 to 0.2.100 (#19371) 
* chore: bump libc from 0.2.99 to 0.2.100

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-23 09:34:32 -06:00
dependabot[bot] aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 (#19370) 
* chore: bump serde from 1.0.127 to 1.0.128

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-08-23 09:33:34 -06:00
Brooks Prumo 6d939811e9
Name snapshots consistently (#19346) 
#### Problem

Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:

```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```

#### Summary of Changes

For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.


Co-authored-by: Michael Vines <mvines@gmail.com>
 2021-08-21 15:41:03 -05:00
Tao Zhu 4982dc20f9
replace function with const var for better readability (#19285) 2021-08-19 14:59:53 -05:00
Tao Zhu d7ba15cde8
Update block max cost limit to fix performance regession (#19276) 
* increase block_cost_max by 10 times to accommodate updated account read and write costs; Otherwise during performace test, banking_stage will start reject and retry transactions in next block, result regressed performance.

* fix typo
 2021-08-18 16:38:44 +00:00
Justin Starry c50b01cb60
Store versioned transactions in the ledger, disabled by default (#19139) 
* Add support for versioned transactions, but disable by default

* merge conflicts

* trent's feedback

* bump Cargo.lock

* Fix transaction error encoding

* Rename legacy_transaction method

* cargo clippy

* Clean up casts, int arithmetic, and unused methods

* Check for duplicates in sanitized message conversion

* fix clippy

* fix new test

* Fix bpf conditional compilation for message module
 2021-08-17 15:17:56 -07:00
Jeff Washington (jwash) 7c70f2158b
accounts_index_bins to AccountsIndexConfig (#19257) 
* accounts_index_bins to AccountsIndexConfig

* rename param bins -> config

* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
 2021-08-17 14:50:01 -05:00
behzad nouri 7a8807b8bb retransmits shreds recovered from erasure codes 
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.

This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
 2021-08-17 13:44:10 +00:00
behzad nouri 3c71670bd9 returns completed-data-set-info from insert_data_shred 
instead of opaque (u32, u32) which are then converted to
CompletedDataSetInfo at the call-site.
 2021-08-17 13:44:10 +00:00
behzad nouri b64eeb7729 removes erroneous uses of &Arc<...> from window-service 2021-08-13 17:26:31 +00:00
dependabot[bot] 5b9671d01a
chore: bump matches from 0.1.8 to 0.1.9 (#19220) 
* chore: bump matches from 0.1.8 to 0.1.9

Bumps [matches](https://github.com/SimonSapin/rust-std-candidates) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/SimonSapin/rust-std-candidates/releases)
- [Commits](https://github.com/SimonSapin/rust-std-candidates/commits)

---
updated-dependencies:
- dependency-name: matches
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-13 16:37:45 +00:00
behzad nouri d192d78ac2
removes unused code from staking_utils (#19189) 2021-08-13 02:42:41 +00:00
Brooks Prumo 5fb6b341c4
Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots (#18870) 
* Handle cleaning zero-lamport accounts

Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot.  This is part of the Incremental Snapshot work.

Fixes #18825
 2021-08-12 15:56:08 -05:00
Tao Zhu 414d904959
Reject blocks for costs above the max block cost (#18994) 
* added realtime cost checking logic to reject block that would exceed max limit:
- defines max limits at block_cost_limits.rs
- right after each bath's execution, accumulate its cost and check again
  limit, return error if limit is exceeded

* update abi that changed due to adding additional TransactionError

* To avoid counting stats mltiple times, only accumulate execute-timing when a bank is completed

* gate it by a feature

* move cost const def into block_cost_limits.rs

* redefine the cost for signature and account access, removed signer part as it is not well defined for now

* check if per_program_timings of execute_timings before sending
 2021-08-12 10:48:47 -05:00
Jeff Washington (jwash) e91988c977
cli for num account index bins (#19085) 2021-08-11 11:45:25 -05:00
behzad nouri 00e5e12906 renames solana_runtime::vote_account::VoteAccount 
Rename:
  VoteAccount    -> VoteAccountInner  # the private type
  ArcVoteAccount -> VoteAccount       # the public type
 2021-08-10 22:54:17 +00:00
Alexander Meißner 85befbc8cd
Removes bpf_loader_upgradeable_program feature gate. (#19102) 2021-08-11 00:08:57 +02:00
dependabot[bot] 536b763751
chore: bump libc from 0.2.98 to 0.2.99 (#19142) 
* chore: bump libc from 0.2.98 to 0.2.99

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.98 to 0.2.99.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.98...0.2.99)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-10 10:54:54 -06:00
Jeff Washington (jwash) 651343688d
Use option for account index bucket parameter (#19150) 2021-08-10 11:32:25 -05:00
behzad nouri f302774cf7
implements copy-on-write for staked-nodes (#19090) 
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77

This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
 2021-08-10 12:59:12 +00:00
Jeff Washington (jwash) 47e0d9aa95
plumb more accounts_index bins (#19123) 2021-08-10 05:45:46 -05:00
Jeff Washington (jwash) 5874f75a26
restore name: Bank::new_with_paths (#19122) 2021-08-09 17:30:37 -05:00
Brooks Prumo 00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() (#18983) 
While reviewing PR #18565, as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots.  A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify.  Additionally, new tests have been
written and existing tests have been updated to use this new function.

Fixes #18973

While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled.  Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`.  And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats.  This bundling simplifies bank
rebuilding.
 2021-08-06 20:16:06 -05:00
Jeff Washington (jwash) ca37873e16
rework bank::new_with_paths (#19087) 
* rework bank::new_with_paths

* missing 1 bench
 2021-08-06 09:30:40 -05:00
Jeff Washington (jwash) 14361906ca
for all tests, bank::new -> bank::new_for_tests (#19064) 2021-08-05 08:42:38 -05:00
Jeff Washington (jwash) 3280ae3e9f
add validator option --accounts-db-skip-shrink (#19028) 
* add validator option --accounts-db-skip-shrink

* typo
 2021-08-04 17:28:33 -05:00
carllin 03353d500f
Actively manage dead slots in AncestorHashesService (#18912) 2021-08-02 14:33:28 -07:00
dependabot[bot] c16bf02448
chore: bump serde from 1.0.126 to 1.0.127 (#19010) 
* chore: bump serde from 1.0.126 to 1.0.127

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.126 to 1.0.127.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.126...v1.0.127)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-02 21:16:34 +00:00
dependabot[bot] 4e41c05acd
chore: bump futures from 0.3.15 to 0.3.16 (#18914) 
* chore: bump futures from 0.3.15 to 0.3.16

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-07-27 07:59:34 -06:00
dependabot[bot] a4024168de
chore: bump rocksdb from 0.16.0 to 0.17.0 (#18904) 
Bumps [rocksdb](https://github.com/rust-rocksdb/rust-rocksdb) from 0.16.0 to 0.17.0.
- [Release notes](https://github.com/rust-rocksdb/rust-rocksdb/releases)
- [Changelog](https://github.com/rust-rocksdb/rust-rocksdb/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-rocksdb/rust-rocksdb/compare/v0.16.0...v0.17.0)

---
updated-dependencies:
- dependency-name: rocksdb
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-07-27 12:36:42 +09:00
dependabot[bot] b97113408b
chore: bump futures-util from 0.3.15 to 0.3.16 (#18878) 
* chore: bump futures-util from 0.3.15 to 0.3.16

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.15 to 0.3.16.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.15...0.3.16)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Nudge Cargo.lock

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-07-26 18:19:53 +00:00
Brooks Prumo d1debcd971
Add incremental snapshot utils (#18504) 
This commit adds high-level functions for creating and loading-from
incremental snapshots, plus all low-level functions required to perform
those tasks.  This commit **does not** add taking incremental snapshots
as part of a running validator, nor starting up a node with an
incremental snapshot; just laying ground work.

Additionally, `snapshot_utils` and `serde_snapshot` have been
refactored to use a common code paths for the different snapshots.

Also of note, some renaming has happened:
  1. Snapshots are now either `full_` or `incremental_` throughout the
     codebase.  If not specified, the code applies to both.
  2. Bank snapshots now are called "bank snapshots"
     (before they were called "slot snapshots", "bank snapshots", or
      just "snapshots").  The one exception is within `Bank`, where they
     are still just "snapshots", because they are already "bank
     snapshots".
  3. Snapshot archives now have `_archive` in the code.  This
     should clear up an ambiguity between bank snapshots and snapshot
     archives.
 2021-07-22 14:40:37 -05:00
carllin 588c0464b8
Add sampling logic and DuplicateSlotRepairStatus module (#18721) 2021-07-21 11:15:08 -07:00
Justin Starry d166b9856a
Move transaction sanitization earlier in the pipeline (#18655) 
* Move transaction sanitization earlier in the pipeline

* Renamed HashedTransaction to SanitizedTransaction

* Implement deref for sanitized transaction

* bring back process_transactions test method

* Use sanitized transactions for cost model calculation
 2021-07-15 22:51:27 -05:00
Brian Anderson 37ee0b5599
Eliminate doc warnings and fix some markdown (#18566) 
* Fix link target in doc comment

* Fix formatting of log examples in process_instruction

* Fix doc markdown in solana-gossip

* Fix doc markdown in solana-runtime

* Escape square braces in doc comments to avoid warnings

* Surround 'account references' doc items in code spans to avoid warnings

* Fix code block in loader_upgradeable_instruction

* Fix doctest for loader_upgradable_instruction
 2021-07-16 00:40:07 +00:00
Trent Nelson 3a85b77bb5 hijack secp256k1 enablement feature plumbing for libsecp256k1 upgrade 2021-07-15 18:43:55 +00:00
Trent Nelson 568660b402 Revert "Remove feature switch for secp256k1 program (#18467)" 
This reverts commit fd574dcb3b.
 2021-07-15 18:43:55 +00:00
sakridge 7f2254225e
Move entry/poh to own crate to speed up poh bench build (#18225) 2021-07-14 14:16:29 +02:00
Jeff Washington (jwash) d092fa1f03
add ledger-tool verify verify-accounts-index option (#18375) 
* add ledger-tool verify verify-accounts-index option

* comment, merge, respond to feedback, cleanup
 2021-07-13 11:06:18 -05:00
carllin a1c0f144f4
Add blockstore column for frozen hashes and duplicate confirmed (#18533) 2021-07-12 20:59:16 -07:00
Michael Vines 4098af3b5b Record vote account commission with voting/staking rewards and surface in RPC 2021-07-12 15:09:44 -07:00
Tyera Eulberg 761de8b1a3
Bump prost, prost-types, and tonic (#18537) 
* Bump prost+tonic and accommodate generated service changes

* Unignore advisory

* Fixup .proto error list
 2021-07-09 17:31:26 +00:00
Justin Starry fd574dcb3b
Remove feature switch for secp256k1 program (#18467) 
* Remove feature switch for secp256k1 program

* fix tests
 2021-07-09 10:08:03 -05:00
dependabot[bot] c853da7424
chore: bump libc from 0.2.97 to 0.2.98 (#18516) 
* chore: bump libc from 0.2.97 to 0.2.98

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.97 to 0.2.98.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.97...0.2.98)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-07-08 17:31:16 +00:00
jbiseda a86ced0bac
generate deterministic seeds for shreds (#17950) 
* generate shred seed from leader pubkey

* clippy

* clippy

* review

* review 2

* fmt

* review

* check

* review

* cleanup

* fmt
 2021-07-07 08:21:12 -07:00
Justin Starry 5dd399dafa
Fix transaction logs and inner ixs for leader nodes (#18395) 
* Fix transaction logs and inner ixs for leader nodes

* Fix cpi log storage flag
 2021-07-06 19:35:24 +00:00
carllin 0eca92de18
Make set roots an iterator (#18357) 2021-07-01 20:02:40 -07:00
Justin Starry d5961e9d9f
Reject transactions with extra signatures (#18306) 
* Reject transactions with extra signatures

* fix tests

* fix check

* fix check

* tx method

* fix checks
 2021-07-01 18:06:59 +00:00
Brooks Prumo 45d54b1fc6
Add SnapshotArchiveInfo and refactor functions in snapshot_utils (#18232) 2021-07-01 12:20:56 -05:00
Tao Zhu 5e424826ba
Persist cost table to blockstore (#18123) 
* Add `ProgramCosts` Column Family to blockstore, implement LedgerColumn; add `delete_cf` to Rocks
* Add ProgramCosts to compaction excluding list alone side with TransactionStatusIndex in one place: `excludes_from_compaction()`

* Write cost table to blockstore after `replay_stage` replayed active banks; add stats to measure persist time
* Deletes program from `ProgramCosts` in blockstore when they are removed from cost_table in memory
* Only try to persist to blockstore when cost_table is changed.
* Restore cost table during validator startup

* Offload `cost_model` related operations from replay main thread to dedicated service thread, add channel to send execute_timings between these threads;
* Move `cost_update_service` to its own module; replay_stage is now decoupled from cost_model.
 2021-07-01 11:32:41 -05:00
dependabot[bot] 644632fc8a
chore: bump assert_matches from 1.3.0 to 1.5.0 (#18358) 
* chore: bump assert_matches from 1.3.0 to 1.5.0

Bumps [assert_matches](https://github.com/murarth/assert_matches) from 1.3.0 to 1.5.0.
- [Release notes](https://github.com/murarth/assert_matches/releases)
- [Commits](https://github.com/murarth/assert_matches/commits)

---
updated-dependencies:
- dependency-name: assert_matches
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-07-01 09:59:01 -06:00
Brooks Prumo 89a3e4f91e
Move SnapshotConfig into its own module (#18331) 
Also move ArchiveFormat to snapshot_utils, and do not
reexport SnapshotVersion.
 2021-07-01 08:55:26 -05:00
Lijun Wang a67d26a1e8
Fixed an issue doing the set_roots repeatedly for the same set. Instead doing the per chunks. (#18314) 2021-06-30 13:29:16 -07:00
dependabot[bot] 78968d132f
chore: bump log from 0.4.11 to 0.4.14 (#18323) 
* chore: bump log from 0.4.11 to 0.4.14

Bumps [log](https://github.com/rust-lang/log) from 0.4.11 to 0.4.14.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.11...0.4.14)

---
updated-dependencies:
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make version consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-30 18:32:01 +00:00
sakridge 8d9a6deda4
Add repair number per slot (#18082) 2021-06-30 18:20:07 +02:00
dependabot[bot] 52fd10ce03
chore: bump byteorder from 1.3.4 to 1.4.3 (#18300) 
* chore: bump byteorder from 1.3.4 to 1.4.3

Bumps [byteorder](https://github.com/BurntSushi/byteorder) from 1.3.4 to 1.4.3.
- [Release notes](https://github.com/BurntSushi/byteorder/releases)
- [Changelog](https://github.com/BurntSushi/byteorder/blob/master/CHANGELOG.md)
- [Commits](https://github.com/BurntSushi/byteorder/compare/1.3.4...1.4.3)

---
updated-dependencies:
- dependency-name: byteorder
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Bump expected instruction count

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-29 15:08:04 -06:00
sakridge e808f34b0b
Add batch stats (#18096) 2021-06-22 15:23:26 +02:00
Michael Vines 84b9de8c18 Shredder no longer holds a keypair 2021-06-21 21:29:52 -07:00
dependabot[bot] d458fac2ff
chore: bump bincode from 1.3.1 to 1.3.3 (#18087) 
* chore: bump bincode from 1.3.1 to 1.3.3

Bumps [bincode](https://github.com/servo/bincode) from 1.3.1 to 1.3.3.
- [Release notes](https://github.com/servo/bincode/releases)
- [Commits](https://github.com/servo/bincode/compare/v1.3.1...v1.3.3)

---
updated-dependencies:
- dependency-name: bincode
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-21 16:06:31 +00:00
Jeff Washington (jwash) ec2f930475
user process.accounts_db_test_hash_calculation for debug_verify hash (#18053) 2021-06-21 10:20:27 -05:00
dependabot[bot] 491e4450cc
chore: bump libc from 0.2.96 to 0.2.97 (#18060) 
* chore: bump libc from 0.2.96 to 0.2.97

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.96 to 0.2.97.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.96...0.2.97)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-06-18 11:51:06 -06:00
Alexander Meißner 6514096a67 chore: cargo +nightly clippy --fix -Z unstable-options 2021-06-18 10:42:46 -07:00
Tyera Eulberg d0511de9a6
chore: bump trees from 0.2.1 to 0.4.2 (#18052) 
* chore: bump trees from 0.2.1 to 0.4.2 (#18041)

Bumps [trees](https://github.com/oooutlk/trees) from 0.2.1 to 0.4.2.
- [Release notes](https://github.com/oooutlk/trees/releases)
- [Commits](https://github.com/oooutlk/trees/commits)

---
updated-dependencies:
- dependency-name: trees
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Accommodate field & type changes

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-17 22:45:09 +00:00
Jeff Washington (jwash) bab96e878e
Revert "chore: bump trees from 0.2.1 to 0.4.2 (#18041)" (#18045) 
This reverts commit f2a7152c0c.
 2021-06-17 14:04:59 -05:00
dependabot[bot] f2a7152c0c
chore: bump trees from 0.2.1 to 0.4.2 (#18041) 
Bumps [trees](https://github.com/oooutlk/trees) from 0.2.1 to 0.4.2.
- [Release notes](https://github.com/oooutlk/trees/releases)
- [Commits](https://github.com/oooutlk/trees/commits)

---
updated-dependencies:
- dependency-name: trees
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-06-17 08:07:16 -06:00
Jeff Washington (jwash) dbd4dc04b0
ledger tool limit_load_slot_count_from_snapshot avoids assert failures (#17974) 2021-06-15 15:39:22 -05:00
Jeff Washington (jwash) f558b9b6bf
verify bank hash on startup with ledger tool option (#17939) 2021-06-15 11:52:12 -05:00
Jon Cinque 1b1d34da59
Refactor stake program into solana_program (#17906) 
* Move stake state / instructions into solana_program

* Update account-decoder

* Update cli and runtime

* Update all other parts

* Commit Cargo.lock changes in programs/bpf

* Update cli stake instruction import

* Allow integer arithmetic

* Update ABI digest

* Bump rust mem instruction count

* Remove useless structs

* Move stake::id() -> stake::program::id()

* Re-export from solana_sdk and mark deprecated

* Address feedback

* Run cargo fmt
 2021-06-15 18:04:00 +02:00
Jeff Washington (jwash) 471b34132e
add metrics for startup (#17913) 
* add metrics for startup

* roll timings up higher

* fix test

* fix duplicate
 2021-06-14 17:46:49 -05:00
dependabot[bot] a0872232d3
chore: bump itertools from 0.9.0 to 0.10.1 (#17929) 
* chore: bump itertools from 0.9.0 to 0.10.1

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.9.0 to 0.10.1.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.9.0...v0.10.1)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix versions

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-14 18:32:20 +00:00
steviez d52632f895
Add extra checks to verify_index_integrity (#17851) 2021-06-14 13:23:31 -05:00
Jeff Washington (jwash) d4cc975fe9
calculate_capitalization uses hash calculation (#17443) 
* calculate_capitalization uses hash calculation

* feedback

* remove debugging code, clean up slot math
 2021-06-14 08:53:07 -05:00
dependabot[bot] f08ed1eb2d
chore: bump rayon from 1.5.0 to 1.5.1 (#17869) 
* chore: bump rayon from 1.5.0 to 1.5.1

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.0 to 1.5.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-06-10 13:18:47 -06:00
Lijun Wang 269d995832
Make account shrink configurable #17544 (#17778) 
1. Added both options for measuring space usage using total accounts usage and for individual store shrink ratio using an enum. Validator CLI options: --accounts-shrink-optimize-total-space and --accounts-shrink-ratio
2. Added code for selecting candidates based on total usage in a separate function select_candidates_by_total_usage
3. Added unit tests for the new functions added
4. The default implementations is kept at 0.8 shrink ratio with --accounts-shrink-optimize-total-space set to true

Fixes #17544
 2021-06-09 21:21:32 -07:00
dependabot[bot] 9a2ca8dd2f
chore: bump rustc_version from 0.2.3 to 0.4.0 (#17854) 
* chore: bump rustc_version from 0.2.3 to 0.4.0

Bumps [rustc_version](https://github.com/Kimundi/rustc-version-rs) from 0.2.3 to 0.4.0.
- [Release notes](https://github.com/Kimundi/rustc-version-rs/releases)
- [Commits](https://github.com/Kimundi/rustc-version-rs/compare/v0.2.3...v0.4.0)

---
updated-dependencies:
- dependency-name: rustc_version
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make versions consistent

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
 2021-06-09 16:57:39 +00:00
Alfonso Subiotto Marques 8a9b7f5ef2 Disable reed-solomon-erasure/simd-accel feature on aarch64 (Apple M1) 
This feature requires clang to support -march=native on M1. Before this change,
cargo build would output a build error when building for aarch64.
 2021-06-08 12:09:34 -07:00
Tyera Eulberg a66566e75b
Remove budget program (#17816) 2021-06-08 01:20:17 +00:00
steviez 4677e6c132
Combine repeated logic for checking if slot has been cleaned up (#17713) 2021-06-04 17:40:27 -05:00