behzad nouri
49ba09b333
adds back ErasureMeta::first_coding_index field ( #21623 )
...
https://github.com/solana-labs/solana/pull/16646
removed first_coding_index since the field is currently redundant and
always equal to fec_set_index.
However, with upcoming changes to erasure coding schema, this will no
longer be the same as fec_set_index and so requires a separate field to
represent.
2021-12-10 20:08:04 +00:00
dependabot[bot]
d870f566ef
chore: bump serde from 1.0.130 to 1.0.131 ( #21758 )
...
* chore: bump serde from 1.0.130 to 1.0.131
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.130 to 1.0.131.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.130...v1.0.131 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-10 08:47:46 +00:00
behzad nouri
8063273d09
adds more sanity checks to shreds ( #21675 )
2021-12-09 16:43:57 +00:00
dependabot[bot]
b61b7189a5
chore: bump hmac from 0.11.0 to 0.12.0 ( #21681 )
...
* chore: bump hmac from 0.11.0 to 0.12.0
Bumps [hmac](https://github.com/RustCrypto/MACs ) from 0.11.0 to 0.12.0.
- [Release notes](https://github.com/RustCrypto/MACs/releases )
- [Commits](https://github.com/RustCrypto/MACs/compare/hmac-v0.11.0...hmac-v0.12.0 )
---
updated-dependencies:
- dependency-name: hmac
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Update dependabot-pr.sh
* Bump sha2 and pbkdf2 as well
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: axleiro <83293196+axleiro@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-12-08 18:12:35 -07:00
dependabot[bot]
f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 ( #21643 )
...
* chore: bump itertools from 0.10.1 to 0.10.3
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-06 23:23:26 -07:00
dependabot[bot]
b5353e2130
chore: bump libc from 0.2.108 to 0.2.109 ( #21627 )
...
* chore: bump libc from 0.2.108 to 0.2.109
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.108 to 0.2.109.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.108...0.2.109 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-12-06 09:37:26 -07:00
behzad nouri
cd17f63d81
adds back position field to coding-shred-header ( #21600 )
...
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.
2021-12-05 14:42:09 +00:00
Trent Nelson
d6f22433d0
Bump version to v1.10.0
2021-12-04 20:17:54 +00:00
Michael Vines
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
Michael Vines
dd12d90eac
Upgrade to Rust 2021
2021-11-30 20:43:46 -08:00
ryleung-solana
8cf36e5cb0
Add GPU sigverify for verify path ( #20851 )
...
Allows the use of GPU acceleration in verifying the signatures in Entry's after deserialization in the replay stage
Co-authored-by: Stephen Akridge <sakridge@gmail.com>
Co-authored-by: Ryan Leung <ryan.leung@solana.com>
2021-11-30 21:16:13 -05:00
Yueh-Hsuan Chiang
38fcfb7542
Persist coding shreds ( #21214 )
2021-11-30 09:18:36 -08:00
Michael Vines
e31b469f56
Quash M1 build warning
2021-11-29 22:42:09 -08:00
Michael Vines
ba9dfa0d22
Remove frozen account support
2021-11-29 08:38:11 -08:00
dependabot[bot]
3c2276fd09
chore: bump futures from 0.3.17 to 0.3.18 ( #21412 )
...
* chore: bump futures from 0.3.17 to 0.3.18
Bumps [futures](https://github.com/rust-lang/futures-rs ) from 0.3.17 to 0.3.18.
- [Release notes](https://github.com/rust-lang/futures-rs/releases )
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.17...0.3.18 )
---
updated-dependencies:
- dependency-name: futures
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-23 21:55:17 -07:00
Tyera Eulberg
9e043d2055
Use max_complete_transaction_status_slot in BigTableUploadService ( #21401 )
...
* Use max_complete_transaction_status_slot in BigTableUploadService
* Use node root to limit BigTableUploadService
2021-11-23 17:36:13 -07:00
behzad nouri
dd338b6c9f
changes Shred::parent return type to Option<Slot> ( #21370 )
...
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453
The commit adds more sanity checks and changes the return type to Option<Slot>.
2021-11-23 14:45:26 +00:00
dependabot[bot]
191cb62c37
chore: bump libc from 0.2.107 to 0.2.108 ( #21377 )
...
* chore: bump libc from 0.2.107 to 0.2.108
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.107 to 0.2.108.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.107...0.2.108 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-22 17:02:34 -07:00
Jeff Biseda
2ed7e3af89
prioritize slot repairs for unknown last index and close to completion ( #21070 )
2021-11-19 19:17:30 -08:00
behzad nouri
7da2df7d8c
adds methods to obtain shreds' erasure coded block and index ( #21325 )
2021-11-19 20:01:15 +00:00
behzad nouri
48dfdfb4d5
changes Blockstore::is_shred_duplicate arg type to ShredType
2021-11-19 14:16:39 +00:00
behzad nouri
57057f8d39
uses enum for shred type
...
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167
Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348
The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00
Justin Starry
66fa062f13
rename process_entries to indicate it's only for tests ( #21321 )
2021-11-17 20:53:40 +01:00
Yueh-Hsuan Chiang
3aa49e2c69
Keep all erasure coding shreds even on successful recovery ( #20968 ) ( #21052 )
...
Problem
Blockstore currently removes erasure shreds if the data shreds are
successfully recovered on insert, which is an issue if we want to
serve coding shreds over repair.
Summary of Changes
This diff keeps all coding shreds even on successful recovery and
changes change the signature of prev_inserted_codes to immutable
reference to ensure its immunity.
Fixes #20968
2021-11-16 11:26:34 -08:00
behzad nouri
3fc858eb60
adds methods to obtain data/coding shreds indices from ErasureMeta
2021-11-13 17:08:05 +00:00
behzad nouri
d25d9be555
removes Default implementaion of ErasureConfig
...
The Default implementation of ErasureConfig is off and never useful:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/erasure.rs#L49-L68
2021-11-13 17:08:05 +00:00
behzad nouri
7e98b77b33
marks ErasureMeta.size as unused
...
ErasureMeta.size is unused and redundant:
https://github.com/solana-labs/solana/blob/150951333/ledger/src/blockstore_meta.rs#L57-L58
2021-11-13 17:08:05 +00:00
behzad nouri
5fb0ab9d00
removes redundant args from Shredder::try_recovery ( #21226 )
...
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
2021-11-10 21:19:03 +00:00
Kirill Fomichev
b4d09388aa
Disable JIT compiler on unsupported platforms ( #21215 )
2021-11-09 11:31:21 +00:00
dependabot[bot]
c2754b8a51
chore: bump libc from 0.2.106 to 0.2.107 ( #21202 )
...
* chore: bump libc from 0.2.106 to 0.2.107
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.106 to 0.2.107.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.107 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-08 12:14:12 -07:00
behzad nouri
11a53de0e3
rewrites Shredder::try_recovery ( #21082 )
...
For every missing data-shred, Shredder::try_recovery calls into
new_empty_data_shred which does a redundant serialization into payload
buffer which is then immediately overwritten by the erasure recovery:
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L372-L417
Additionally, the implementation is unnecessary complex hindering
upcoming changes to erasure coding generation
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L814-L938
The commit simplifies the Shredder::try_recovery implementation.
2021-11-04 16:15:51 +00:00
dependabot[bot]
ca51ea3e04
chore: bump libc from 0.2.105 to 0.2.106 ( #21093 )
...
* chore: bump libc from 0.2.105 to 0.2.106
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.105 to 0.2.106.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.105...0.2.106 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-01 14:17:45 -06:00
behzad nouri
484ead01ed
optimizes and simplifies SlotMeta::completed_data_indexes ops ( #21059 )
...
SlotMeta::completed_data_indexes is defined as a Vec<u32>:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore_meta.rs#L31-L32
which results in inefficient updates:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore.rs#L3245-L3326
This commit changes the type to BTreeSet<u32> for efficient and simpler
updates and lookups.
The change should be backward compatible because Vec<T> and BTreeSet<T>
are both serialized as seq:
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L207-L208
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L216-L217
2021-10-31 12:56:25 +00:00
sakridge
4642a2c856
Optimize test_recovery_with_expanded_coding_shreds ( #20849 )
2021-10-28 11:38:08 +02:00
Tyera Eulberg
f0de3e9bf0
chore: bump tonic from 0.5.2 to 0.6.1 ( #21035 )
...
* Bump tonic and prost crates
* connect_lazy no longer infallible
2021-10-28 06:59:36 +00:00
Justin Starry
036d7fcc81
Clean up sanitized tx creation for tests ( #21006 )
2021-10-27 18:09:16 +01:00
dependabot[bot]
cb1947bea2
chore: bump libc from 0.2.104 to 0.2.105 ( #20972 )
...
* chore: bump libc from 0.2.104 to 0.2.105
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.104 to 0.2.105.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.104...0.2.105 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-26 00:14:58 -06:00
Michael Vines
350bb561eb
Clippy
2021-10-23 08:21:20 +00:00
dependabot[bot]
66e431dad5
chore: bump libc from 0.2.103 to 0.2.104 ( #20764 )
...
* chore: bump libc from 0.2.103 to 0.2.104
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-19 22:38:02 -06:00
carllin
b5f21d5e34
Add ledger tool to remove dead slot ( #20810 )
2021-10-19 20:23:16 -07:00
behzad nouri
231b58b5f1
adds more metrics to blockstore insert shreds stats ( #20701 )
2021-10-17 11:02:34 -04:00
Jeff Washington (jwash)
70b2e5fef2
hash calc uses all threads during startup ( #20735 )
2021-10-15 17:41:23 -05:00
Tyera Eulberg
e806fa6904
Include token owners in TransactionTokenBalances ( #20642 )
...
* Cache owners in TransactionTokenBalances
* Light cleanup
* Use return struct, and remove pub
* Single-use statements
* Why not, just do the whole crate
* Add metrics
* Make datapoint_debug to prevent spam unless troubleshooting
2021-10-13 21:46:52 -06:00
sakridge
09e7782d76
Refactor code to get block signatures in get_confirmed_signatures_for_address2 ( #20575 )
...
* Refactor get_confirmed_signatures_for_address2
* Move blockstore benches to ledger where they belong
2021-10-13 09:55:19 +02:00
Tao Zhu
005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; ( #20527 )
...
- move related modules to runtime
2021-10-12 08:51:33 -05:00
Jeff Washington (jwash)
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
Brooks Prumo
5440c1d2e1
SnapshotPackagerService pushes incremental snapshot hashes to CRDS ( #20442 )
...
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!
This commit does two main things:
1. SnapshotPackagerService now knows about incremental snapshot hashes,
and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
hashes need to be passed all the way to SnapshotPackagerService so it
can push these starting hashes to CRDS. Those values have been piped
through.
Fixes #20441 and #20423
2021-10-08 15:14:56 -05:00
Tao Zhu
675fa6993b
- update const cost values with data collected by #19627 ( #20314 )
...
- update cost calculation to closely proposed fee schedule #16984
2021-10-08 14:48:50 -05:00
Trent Nelson
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
Michael Vines
7027d56064
Resolve nightly-2021-10-05 clippy complaints
2021-10-06 10:37:58 -07:00