350bb561eb
Clippy
2021-10-23 08:21:20 +00:00
66e431dad5
chore: bump libc from 0.2.103 to 0.2.104 ( #20764 )
...
* chore: bump libc from 0.2.103 to 0.2.104
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-19 22:38:02 -06:00
b5f21d5e34
Add ledger tool to remove dead slot ( #20810 )
2021-10-19 20:23:16 -07:00
231b58b5f1
adds more metrics to blockstore insert shreds stats ( #20701 )
2021-10-17 11:02:34 -04:00
70b2e5fef2
hash calc uses all threads during startup ( #20735 )
2021-10-15 17:41:23 -05:00
e806fa6904
Include token owners in TransactionTokenBalances ( #20642 )
...
* Cache owners in TransactionTokenBalances
* Light cleanup
* Use return struct, and remove pub
* Single-use statements
* Why not, just do the whole crate
* Add metrics
* Make datapoint_debug to prevent spam unless troubleshooting
2021-10-13 21:46:52 -06:00
09e7782d76
Refactor code to get block signatures in get_confirmed_signatures_for_address2 ( #20575 )
...
* Refactor get_confirmed_signatures_for_address2
* Move blockstore benches to ledger where they belong
2021-10-13 09:55:19 +02:00
005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; ( #20527 )
...
- move related modules to runtime
2021-10-12 08:51:33 -05:00
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
5440c1d2e1
SnapshotPackagerService pushes incremental snapshot hashes to CRDS ( #20442 )
...
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!
This commit does two main things:
1. SnapshotPackagerService now knows about incremental snapshot hashes,
and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
hashes need to be passed all the way to SnapshotPackagerService so it
can push these starting hashes to CRDS. Those values have been piped
through.
Fixes #20441 and #20423
2021-10-08 15:14:56 -05:00
675fa6993b
- update const cost values with data collected by #19627 ( #20314 )
...
- update cost calculation to closely proposed fee schedule #16984
2021-10-08 14:48:50 -05:00
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
7027d56064
Resolve nightly-2021-10-05 clippy complaints
2021-10-06 10:37:58 -07:00
d8e4840b49
Follow up to blockstore test cleanup ( #19656 ) ( #20143 )
2021-10-04 10:35:03 -05:00
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
9393654af0
chore: bump libc from 0.2.102 to 0.2.103 ( #20280 )
...
* chore: bump libc from 0.2.102 to 0.2.103
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-09-28 07:50:46 +00:00
c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` ( #19931 )
...
* Add `delete` subcommand to `ledger-tool bigtable` command
* feedback
2021-09-16 23:37:45 +00:00
ea15c0b40b
chore: bump libc from 0.2.101 to 0.2.102 ( #19924 )
...
* chore: bump libc from 0.2.101 to 0.2.102
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.101 to 0.2.102.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.101...0.2.102 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-16 09:45:01 -06:00
4fdd70c931
Properly clean temp files in Blockstore::create_new_ledger() ( #19589 )
...
Use TempDir::path() instead of TempDir::into_path(); into_path() returns
the underlying PathBuf and voids the promise to delete the directory
when TempDir goes out of scope.
2021-09-13 23:13:56 -04:00
97fc64dd0c
Call exhaustively_free_unused_resource() instead ( #19847 )
2021-09-13 18:08:32 -05:00
ef749a2506
Flush accounts cache before calling clean_accounts() ( #19839 )
2021-09-13 14:56:13 -05:00
3f6eb96d6e
Call `Bank::update_accounts_hash()` before taking bank snapshot ( #19765 )
...
This is a bugfix.
When `load_frozen_forks()` called `snapshot_bank()`, the accounts hash
was not computed. So then when a snapshot archive was eventually
created, its hash would be all 1s, which is clearly wrong. The fix is
to update the bank's accounts hash before taking the bank snapshot.
2021-09-13 08:27:50 -05:00
62c8bcf565
Add default() to SnapshotConfig ( #19776 )
2021-09-12 13:44:27 -05:00
857fce13f9
chore: bump sha2 from 0.9.6 to 0.9.8 ( #19760 )
...
* chore: bump sha2 from 0.9.6 to 0.9.8
Bumps [sha2](https://github.com/RustCrypto/hashes ) from 0.9.6 to 0.9.8.
- [Release notes](https://github.com/RustCrypto/hashes/releases )
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.6...sha2-v0.9.8 )
---
updated-dependencies:
- dependency-name: sha2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-11 16:22:40 -06:00
c3a610374a
unused deps
2021-09-11 14:33:53 -07:00
2dee098b91
Blockstore test cleanup ( #19656 )
...
* Use auto-deleting macro for ledger path
2021-09-10 07:33:08 -05:00
456bf15012
AccountsIndexConfig -> AccountsDbConfig ( #19687 )
2021-09-08 04:30:38 +00:00
d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it ( #19686 )
2021-09-07 20:09:40 -05:00
4a5f83d3a7
Add fn to check when to take snapshots ( #19682 )
2021-09-07 23:26:35 +00:00
a0552e5b46
Make startup aware of Incremental Snapshots ( #19600 )
2021-09-07 20:43:43 +00:00
5fa3e5744c
Populate memo in blockstore signatures-for-address ( #19515 )
...
* Add TransactionMemos column family
* Traitify extract_memos
* Write TransactionMemos in TransactionStatusService
* Populate memos from column
* Dedupe and add unit test
2021-09-03 00:28:52 -06:00
170927b7c4
nit: Move builtins types to builtins file ( #19597 )
2021-09-02 21:29:11 -07:00
e9374d32a3
Revert "Make startup aware of Incremental Snapshots ( #19550 )" ( #19599 )
...
This reverts commit d45ced0a5d
2021-09-02 19:14:41 -05:00
d45ced0a5d
Make startup aware of Incremental Snapshots ( #19550 )
2021-09-02 19:05:15 -05:00
1d2f0f6641
Simplify process_blockstore tests ( #19553 )
...
Some of the blockstore_processor tests weren't using
`test_process_blockstore()`, but could. I updated those tests, and also
changed to using `..` for ignoring, instead of `_`, since I'll be adding
another return value here shortly (thus reducing the need to change
these again).
2021-09-01 20:13:52 +00:00
6d9818b8e4
skips retransmit for shreds with unknown slot leader ( #19472 )
...
Shreds' signatures should be verified before they reach retransmit
stage, and if the leader is unknown they should fail signature check.
Therefore retransmit-stage can as well expect to know who the slot
leader is and otherwise just skip the shred.
Blockstore checking signature of recovered shreds before sending them to
retransmit stage:
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/blockstore.rs#L884-L930
Shred signature verifier:
https://github.com/solana-labs/solana/blob/4305d4b7b/core/src/sigverify_shreds.rs#L41-L57
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/sigverify_shreds.rs#L105
2021-09-01 15:44:26 +00:00
8e4360cf9c
chore: bump futures from 0.3.16 to 0.3.17 ( #19513 )
...
* chore: bump futures from 0.3.16 to 0.3.17
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-30 22:12:38 -06:00
dc3d7ad2bd
chore: bump sha2 from 0.9.5 to 0.9.6 ( #19511 )
...
* chore: bump sha2 from 0.9.5 to 0.9.6
Bumps [sha2](https://github.com/RustCrypto/hashes ) from 0.9.5 to 0.9.6.
- [Release notes](https://github.com/RustCrypto/hashes/releases )
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.5...sha2-v0.9.6 )
---
updated-dependencies:
- dependency-name: sha2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 19:58:26 -06:00
6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 ( #19497 )
...
* chore: bump serde from 1.0.129 to 1.0.130
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 18:19:34 +00:00
fbffdf9f3d
chore: bump futures-util from 0.3.16 to 0.3.17 ( #19498 )
...
* chore: bump futures-util from 0.3.16 to 0.3.17
Bumps [futures-util](https://github.com/rust-lang/futures-rs ) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17 )
---
updated-dependencies:
- dependency-name: futures-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 17:45:47 +00:00
8ad52fa095
implements copy-on-write for vote-accounts ( #19362 )
...
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186
This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
2021-08-30 15:54:01 +00:00
e492638b7d
chore: bump libc from 0.2.100 to 0.2.101 ( #19442 )
...
* chore: bump libc from 0.2.100 to 0.2.101
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-26 12:38:34 -06:00
f1d9c014f8
create test function test_process_blockstore ( #19432 )
2021-08-26 12:48:52 -05:00
46890ac197
chore: bump serde from 1.0.128 to 1.0.129 ( #19395 )
...
* chore: bump serde from 1.0.128 to 1.0.129
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-24 12:32:30 -06:00
f3ce5fa5e8
clippy: branches-sharing-code
...
```text
error: all if blocks contain the same code at the end
--> ledger/src/blockstore.rs:3473:5
|
3473 | / Ok(insert_map.get(&slot).unwrap().clone())
3474 | | }
| |_____^
|
= note: `-D clippy::branches-sharing-code` implied by `-D warnings`
= note: The end suggestion probably needs some adjustments to use the expression result correctly
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#branches_sharing_code
help: consider moving the end statements out like this
|
3473 ~ }
3474 + Ok(insert_map.get(&slot).unwrap().clone())
|
error: could not compile `solana-ledger` due to previous error
```
2021-08-23 11:17:21 -05:00
c0c95e88d8
chore: bump libc from 0.2.99 to 0.2.100 ( #19371 )
...
* chore: bump libc from 0.2.99 to 0.2.100
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-23 09:34:32 -06:00
aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 ( #19370 )
...
* chore: bump serde from 1.0.127 to 1.0.128
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-23 09:33:34 -06:00
6d939811e9
Name snapshots consistently ( #19346 )
...
#### Problem
Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:
```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```
#### Summary of Changes
For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
4982dc20f9
replace function with const var for better readability ( #19285 )
2021-08-19 14:59:53 -05:00
d7ba15cde8
Update block max cost limit to fix performance regession ( #19276 )
...
* increase block_cost_max by 10 times to accommodate updated account read and write costs; Otherwise during performace test, banking_stage will start reject and retry transactions in next block, result regressed performance.
* fix typo
2021-08-18 16:38:44 +00:00
Michael Vines
dependabot[bot]
carllin
behzad nouri
Jeff Washington (jwash)
Tyera Eulberg
sakridge
Tao Zhu
Brooks Prumo
Trent Nelson
steviez
Lijun Wang
Justin Starry
Giles Cope
Jack May
Brooks Prumo