Tao Zhu
8590911b0a
Replace type alias with newtype for UnprocesedPacketBatches
2022-03-14 13:14:27 -05:00
Will Hickey
63bf0f66af
Bump version to 1.10.3 ( #23648 )
2022-03-14 11:18:45 -05:00
Brooks Prumo
7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit ( #23537 )
2022-03-13 15:58:57 +00:00
Yueh-Hsuan Chiang
1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting ( #23523 )
...
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.
To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type.
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.
Dependency: #23580
2022-03-11 15:17:34 -08:00
dependabot[bot]
b1da7cff66
chore: bump base64 from 0.12.3 to 0.13.0 ( #23616 )
...
* chore: bump base64 from 0.12.3 to 0.13.0
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.12.3 to 0.13.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases )
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.12.3...v0.13.0 )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update Cargo.lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-03-11 16:01:15 -07:00
Brooks Prumo
d20dd21600
Sort tables in Cargo.toml files ( #23602 )
2022-03-11 11:05:57 -06:00
sakridge
9b591286d7
Revert "chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )" ( #23592 )
...
This reverts commit 3a0271c113
.
2022-03-11 09:12:27 +01:00
Will Hickey
b444836a97
Bump version to 1.10.2 ( #23597 )
2022-03-10 16:41:06 -06:00
Tao Zhu
35d1235ed0
- move `unprocessed_packet_batches` from `BankingStage` to its own ( #23508 )
...
module
- deserialize packets during receving and buffering
2022-03-10 18:47:46 +00:00
carllin
588414a776
Report even if slot begins and ends in process_buffered_packets() ( #23549 )
2022-03-09 23:42:35 -05:00
Tao Zhu
f68c5a274d
remove persist_cost_table code
2022-03-09 21:05:47 -07:00
Tao Zhu
9f71958d7d
Patch validator from loading persisted program costs
2022-03-09 21:05:47 -07:00
Brooks Prumo
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
HaoranYi
a1c45d5acb
typo ( #23564 )
2022-03-09 15:11:21 -06:00
sakridge
7a9884c831
Quic limit connections ( #23283 )
...
* quic server limit connections
* bump per_ip
* Review comments
* Make the connections per port
2022-03-09 10:52:31 +01:00
dependabot[bot]
5c722519cf
chore: bump etcd-client from 0.8.3 to 0.8.4
...
Bumps [etcd-client](https://github.com/etcdv3/etcd-client ) from 0.8.3 to 0.8.4.
- [Release notes](https://github.com/etcdv3/etcd-client/releases )
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.8.3...v0.8.4 )
---
updated-dependencies:
- dependency-name: etcd-client
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-03-08 17:33:24 -08:00
Carl Lin
5a0cd05866
Revert "- estimate a program cost as 2 standard deviation above mean"
...
This reverts commit a25ac1c988
.
2022-03-08 17:18:44 -08:00
Carl Lin
9acbfa5eb1
Revert "use EMA in place of Welford"
...
This reverts commit 6587dbfa47
.
2022-03-08 17:18:44 -08:00
Carl Lin
c878c9e2cb
Revert "1. Persist to blockstore less frequently;"
...
This reverts commit 7aa1fb4e24
.
2022-03-08 17:18:44 -08:00
Carl Lin
0a17edcc1f
Revert "fix tests after merge"
...
This reverts commit ba2d83f580
.
2022-03-08 17:18:44 -08:00
Michael Vines
b719d6a2ad
`solana-validator set-identity` no longer writes a tower file unnecessarily
2022-03-08 15:34:23 -08:00
Justin Starry
3114c199bd
Add RPC support for versioned transactions ( #22530 )
...
* Add RPC support for versioned transactions
* fix doc tests
* Add rpc test for versioned txs
* Switch to preflight bank
2022-03-08 15:20:34 +08:00
dependabot[bot]
38d8bbb19c
chore: bump sysctl from 0.4.3 to 0.4.4 ( #23505 )
...
Bumps [sysctl](https://github.com/johalun/sysctl-rs ) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/johalun/sysctl-rs/releases )
- [Changelog](https://github.com/johalun/sysctl-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/johalun/sysctl-rs/commits )
---
updated-dependencies:
- dependency-name: sysctl
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 14:24:19 -07:00
HaoranYi
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
dependabot[bot]
3a0271c113
chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )
...
* chore: bump dashmap from 4.0.2 to 5.1.0
Bumps [dashmap](https://github.com/xacrimon/dashmap ) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases )
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0 )
---
updated-dependencies:
- dependency-name: dashmap
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-07 00:00:26 -05:00
Yueh-Hsuan Chiang
b8b7163b66
(Ledger Store) Report RocksDB Column Family Metrics ( #22503 )
...
This PR enables blockstore to periodically report RocksDB column family properties.
The reported properties are under blockstore_rocksdb_cfs, and the properties also
support group by operation on cf_name.
2022-03-05 16:13:03 -08:00
Yueh-Hsuan Chiang
62d2a4cd88
Make ShredStorageType::RocksLevel public ( #23272 )
...
#### Summary of Changes
This PR adds two hidden arguments to the validator that allow users to use RocksDB's FIFO compaction for storing shreds.
--shred-storage <SHRED_STORAGE>
EXPERIMENTAL: Controls how RocksDB compacts shreds. *WARNING*: You will lose your ledger data
when you switch between options. Possible values are: 'level': stores shreds using RocksDB's default (level)
compaction. 'fifo': stores shreds under RocksDB's FIFO compaction. This option is more efficient on
disk-write-bytes of the ledger store. [default: level] [possible values: level, fifo]
--shred-storage-size <SHRED_STORAGE_SIZE_BYTES>
The shred storage size in bytes. The suggested value is 50% of your ledger storage size in bytes. [default:
268435456000]
2022-03-03 12:43:58 -08:00
dependabot[bot]
f2fa49a771
chore: bump lru from 0.7.2 to 0.7.3 ( #23462 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.2...0.7.3 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-03 12:01:49 -07:00
dependabot[bot]
61d7bdd66f
chore: bump serde_json from 1.0.78 to 1.0.79 ( #23461 )
...
* chore: bump serde_json from 1.0.78 to 1.0.79
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-02 23:38:06 -07:00
dependabot[bot]
fe7604589d
chore: bump retain_mut from 0.1.5 to 0.1.7 ( #23450 )
...
Bumps [retain_mut](https://github.com/upsuper/retain_mut ) from 0.1.5 to 0.1.7.
- [Release notes](https://github.com/upsuper/retain_mut/releases )
- [Commits](https://github.com/upsuper/retain_mut/commits )
---
updated-dependencies:
- dependency-name: retain_mut
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-02 23:15:33 +00:00
Will Hickey
1a99251498
Bump version to 1.10.1 ( #23453 )
2022-03-02 13:47:01 -06:00
Jeff Washington (jwash)
26aa18b3f3
fmt ( #23448 )
2022-03-02 11:54:58 -06:00
HaoranYi
41f78b9925
small optimization. use shift for pow of 2. ( #22975 )
2022-03-02 09:11:12 -06:00
HaoranYi
4f0070a5c6
unittest for bind two consecutive ports ( #23008 )
...
* minor fix of comments in fork-selection tests
* fix doc link
* add unittest for bind_two_consecutive_in_range
2022-03-02 09:10:29 -06:00
HaoranYi
8de88d0a55
Refactor packet_threshold adjustment code into its own struct ( #23216 )
...
* refactor packet_threshold adjustment code into own struct and add unittest for it
* fix a typo in error message
* code review feedbacks
* another code review feedback
* Update core/src/ancestor_hashes_service.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* share packet threshold with repair service (credit to carl)
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-03-02 09:09:06 -06:00
HaoranYi
86e2f728c3
Fix a batch limits bug in banking ( #23327 )
...
* add thread index in thread name for debugging
* fix batch_limit
* use NUM_VOTE_THREAD instead of hardcoded number (credit to carllin)
2022-03-02 09:08:08 -06:00
Tyera Eulberg
d3ebe8d8f5
Remove unneeded jsonrpc dependencies/features; update do-audit ( #23436 )
...
* Update generic-array note
* Remove unneeded jsonrpc deps
* Remove unneeded jsonrpc features
* Rewrite slot-update test without websocket crate
* Rewrite rpc-subscription test without websocket crate, and remove jsonrpc deps
* Update expected balance to accommodate rent-exempt minimum transfer amount
* Remove obsolete audit ignores
2022-03-02 01:42:01 -07:00
Jeff Biseda
c69e3b73ff
bench get_retransmit_peers ( #23292 )
2022-03-01 19:10:29 -08:00
dependabot[bot]
ce4d579499
chore: bump serial_test from 0.5.1 to 0.6.0 ( #23414 )
...
Bumps [serial_test](https://github.com/palfrey/serial_test ) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/palfrey/serial_test/releases )
- [Commits](https://github.com/palfrey/serial_test/compare/v0.5.1...v0.6.0 )
---
updated-dependencies:
- dependency-name: serial_test
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-01 12:13:40 -07:00
Brooks Prumo
533eca3b4c
Simplify replay_blockstore_into_bank() ( #23282 )
2022-02-25 06:57:04 -06:00
Trent Nelson
d4292774c5
checks
2022-02-25 08:05:28 +00:00
dependabot[bot]
98f059e89c
chore: bump reqwest from 0.11.6 to 0.11.9 ( #23337 )
...
* chore: bump reqwest from 0.11.6 to 0.11.9
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.6 to 0.11.9.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.6...v0.11.9 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-02-24 22:40:01 -07:00
Justin Starry
d0e85c293f
Fix rustfmt check ( #23296 )
2022-02-23 16:38:53 +08:00
Gavin Chan
20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array ( #23085 )
2022-02-22 14:46:56 -08:00
Tyera Eulberg
7e08ae1d0c
Revert "Add simulation detection countermeasure ( #22880 )" ( #23261 )
...
This reverts commit c42b80f099
.
2022-02-21 21:15:37 +00:00
Yueh-Hsuan Chiang
8c872e9ce0
(LedgerStore/FIFO) Refactor FIFO options and sanity check. ( #23131 )
2022-02-19 00:58:38 -08:00
buffalu
70ebab2c82
Add rustfmt.toml and `cargo fmt` ( #23238 )
...
* fmt
* formatted
Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
Yueh-Hsuan Chiang
1add82aa9e
(Ledger Store Benchmark) Add flags for checking shred storage size. ( #22451 )
2022-02-18 19:35:28 -08:00
carllin
619335df1a
Add execute timings ( #23097 )
2022-02-17 01:14:32 -05:00
anatoly yakovenko
83d31c9e65
shrink batches when over 80% of the space is wasted ( #23066 )
...
* shrink batches when over 80% of the space is wasted
2022-02-16 08:18:17 -08:00
Jeff Biseda
115d71536b
forward_buffered_packets return packet count in error path ( #23167 )
2022-02-16 07:46:32 -08:00
Michael Vines
a6d736572c
`solana-validator set-identity` now supports the `--require-tower` flag
2022-02-15 19:45:00 -08:00
Tao Zhu
03bf66a51b
flag end-of-slot when poh bank is gone
2022-02-15 15:01:27 -06:00
Ashwin Sekar
ab92578b02
Fix the flaky test test_restart_tower_rollback ( #23129 )
...
* Add flag to disable voting until a slot to avoid duplicate voting
* Fix the tower rollback test and remove it from flaky.
2022-02-15 13:19:34 -07:00
Michael Vines
c42b80f099
Add simulation detection countermeasure ( #22880 )
...
* Add simulation detection countermeasures
* Add program and test using TestValidator
* Remove incinerator deposit
* Remove incinerator
* Update Cargo.lock
* Add more features to simulation bank
* Update Cargo.lock per rebase
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-15 13:09:59 +01:00
Yueh-Hsuan Chiang
824446710b
(Ledger Store Benchmark) Enable FIFO compmaction in the benchmark. ( #22162 )
2022-02-12 00:43:25 -08:00
Lijun Wang
c04438be4b
Retaining transaction logs when transaction plugin is loaded. ( #22874 )
...
Transaction logs are not being saved to the database through the plugin interface.
Summary of Changes
Retain the transaction logs when transaction notification plugin is loaded.
Fixes #
lijunwangs/solana-accountsdb-plugin-postgres#6
2022-02-11 20:29:07 -08:00
carllin
2f9e30a1f7
Introduce slot-specific packet metrics ( #22906 )
2022-02-11 03:07:45 -05:00
dependabot[bot]
59e64d5e99
chore: bump raptorq from 1.6.4 to 1.6.5 ( #23042 )
...
Bumps [raptorq](https://github.com/cberner/raptorq ) from 1.6.4 to 1.6.5.
- [Release notes](https://github.com/cberner/raptorq/releases )
- [Commits](https://github.com/cberner/raptorq/compare/v1.6.4...v1.6.5 )
---
updated-dependencies:
- dependency-name: raptorq
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-10 19:07:07 -07:00
Justin Starry
d5dec989b9
Enforce tx metadata upload with static types ( #23028 )
2022-02-10 13:28:18 +08:00
Yueh-Hsuan Chiang
1b287f1b59
(Ledger Cleanup) Add code comments for ledger_cleanup. ( #22807 )
2022-02-08 22:48:56 -08:00
Tao Zhu
ba2d83f580
fix tests after merge
2022-02-08 16:18:23 -06:00
Tao Zhu
7aa1fb4e24
1. Persist to blockstore less frequently;
...
2. reduce alpha for EMA to 1 percent to have roughly 200 data points for estimatio
2022-02-08 16:18:23 -06:00
Tao Zhu
6587dbfa47
use EMA in place of Welford
2022-02-08 16:18:23 -06:00
Tao Zhu
a25ac1c988
- estimate a program cost as 2 standard deviation above mean
...
- replaced get_average / get_mode with get_default to assign max units to unknown program
2022-02-08 16:18:23 -06:00
Tao Zhu
e52e48076e
bench should update leader schedule cache ( #22991 )
2022-02-08 02:28:28 +00:00
Ashwin Sekar
5acf0f6331
Add feature gate for new vote instruction and plumb through replay ( #21683 )
...
* Add feature gate for new vote instruction and plumb through replay
Add tower versions
* Add check for slot hashes history
* Update is_recent check to exclude voting on hard fork root slot
* Move tower rollback test to flaky and ignore it until #22551 lands
2022-02-07 14:06:19 -08:00
behzad nouri
27aaf9df85
removes VoteTracker::new in favor of VoteTracker::default ( #22941 )
...
VoteTracker::new does not need a bank and is so redundant:
https://github.com/solana-labs/solana/blob/5a230f418/core/src/cluster_info_vote_listener.rs#L103-L107
2022-02-04 19:01:59 +00:00
sakridge
5a230f418d
Add quic port for accepting transactions ( #22753 )
...
using quinn library
streamer: Sign TLS cert with validator identity key
Handle multiple incoming chunks
2022-02-04 15:27:09 +01:00
Tao Zhu
4bec182b32
Allow buffered packets be consumed if bank is active, regardless leader schedule ( #22913 )
2022-02-03 21:29:41 +00:00
Justin Starry
60af1a4cce
Refactor: Add trait for loading addresses ( #22903 )
2022-02-03 11:00:12 +00:00
carllin
bd1850df25
Return actual committed transactions from process_transactions() ( #22802 )
2022-02-03 03:56:36 -05:00
Trent Nelson
c62f9839a2
test-validator-bin: reinstate full rpc method set
2022-02-03 02:43:03 +00:00
Ikko Ashimine
58a70d76a3
fix typo in broadcast_duplicates_run.rs ( #22888 )
...
Creat -> Create
2022-02-02 12:29:14 -07:00
behzad nouri
dccbddad80
adds reverse lookup index to cluster-nodes ( #22892 )
...
retransmit has to exclude slot leader from set of nodes for each shred;
which currently requires a linear scan:
https://github.com/solana-labs/solana/blob/e3b137066/core/src/cluster_nodes.rs#L238-L242
This commit adds a reverse lookup index to avoid linear scan.
2022-02-02 19:27:50 +00:00
behzad nouri
e3b137066d
caches WeightedShuffle struct in ClusterNodes ( #22877 )
...
Instead of reconstructing WeightedShuffle struct for each shred
broadcast or retransmit, we can use the same struct with minimal
mutations.
2022-02-02 15:12:26 +00:00
Trent Nelson
eac4a6df68
rpc: use minimal mode by default
2022-02-01 19:00:06 -07:00
behzad nouri
45e09664b8
removes Rng field from WeightedShuffle struct ( #22850 )
2022-02-01 15:27:23 +00:00
behzad nouri
604ca9316c
includes zero weighted entries in WeightedShuffle ( #22829 )
...
Current WeightedShuffle implementation excludes zero weighted entries
from the shuffle:
https://github.com/solana-labs/solana/blob/13e631dcf/gossip/src/weighted_shuffle.rs#L29-L30
Though mathematically this might make more sense, for our use-cases
(turbine specifically), this results in less efficient code:
https://github.com/solana-labs/solana/blob/13e631dcf/core/src/cluster_nodes.rs#L409-L430
This commit changes the implementation so that zero weighted indices are
also included in the shuffle but appear only at the end after non-zero
weighted indices.
2022-01-31 16:23:50 +00:00
dependabot[bot]
cc74693176
chore: bump serde from 1.0.134 to 1.0.136 ( #22817 )
...
* chore: bump serde from 1.0.134 to 1.0.136
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.134 to 1.0.136.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.134...v1.0.136 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-28 12:20:37 -07:00
Justin Starry
220aa6ada0
Fix poh recorder initialization on startup ( #22755 )
2022-01-28 14:21:15 +08:00
Michael Vines
331b953551
Add vote account address to vote subscription
2022-01-27 08:22:29 -08:00
dependabot[bot]
db481e1799
chore: bump serde_json from 1.0.75 to 1.0.78 ( #22748 )
...
* chore: bump serde_json from 1.0.75 to 1.0.78
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.75 to 1.0.78.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.75...v1.0.78 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-26 18:01:02 -07:00
Justin Starry
d9c259a231
Set the correct root in block commitment cache initialization ( #22750 )
...
* Set the correct root in block commitment cache initialization
* clean up test
* bump
2022-01-27 00:48:00 +08:00
sakridge
2e56c59bcb
Handle already discarded packets in gpu sigverify path ( #22680 )
2022-01-24 14:35:47 +01:00
Justin Starry
1240217a73
Refactor: Rename variables and helper method to `PohRecorder` ( #22676 )
...
* Refactor: Rename leader_first_tick_height field
* Refactor: add `PohRecorder::slot_for_tick_height` helper
* Refactor: Add type for poh leader status
2022-01-23 10:28:50 +08:00
anatoly yakovenko
d6011ba14d
Dedup bloom filter is too slow ( #22607 )
...
* Faster dedup
* use ahash
* fixup
* single threaded
* use duration type
* remove the count
* fixup
2022-01-21 20:23:48 -07:00
Michael Vines
6d5bbca630
Pacify clippy
2022-01-21 19:12:57 -08:00
Michael Vines
ce4f7601af
Avoid unstable_name_collisions warning
2022-01-21 19:12:57 -08:00
dependabot[bot]
0cf886302d
chore: bump serde from 1.0.133 to 1.0.134 ( #22650 )
...
* chore: bump serde from 1.0.133 to 1.0.134
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.133 to 1.0.134.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.133...v1.0.134 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-21 17:14:02 -07:00
dependabot[bot]
95bbb70c91
chore: bump serde_json from 1.0.74 to 1.0.75 ( #22541 )
...
* chore: bump serde_json from 1.0.74 to 1.0.75
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.74 to 1.0.75.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.74...v1.0.75 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-21 12:26:51 -07:00
sakridge
38b02bbcc0
Handle already discarded packets in discard_excess_packets ( #22594 )
2022-01-21 17:22:50 +01:00
Jeff Biseda
e7777281d6
regularly report network limits ( #22563 )
2022-01-20 12:38:42 -08:00
Trent Nelson
cca3dbc76d
system-monitor-service: support percentages from bigger numbers
2022-01-20 09:51:23 +00:00
Yueh-Hsuan Chiang
f8db314134
(Ledger Store Benchmark) Display storage size of all data shreds ( #22445 )
...
* (Ledger Store) APIs for obtaining physical size of all data and coding shreds
* (Ledger Store Benchmark) Display total data shred storage size.
2022-01-19 19:33:08 -08:00
Justin Starry
7f20c6149e
Refactor: move simple vote parsing to runtime ( #22537 )
2022-01-20 10:39:21 +08:00
anatoly yakovenko
d343713f61
Optimize packet dedup ( #22571 )
...
* Use bloom filter to dedup packets
* dedup first
* Update bloom/src/bloom.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/sigverify_stage.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* fixup
* fixup
* fixup
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-01-19 13:58:20 -08:00
behzad nouri
dcf44d2523
improves sigverify discard_excess_packets performance ( #22577 )
...
As shown by the added benchmark, current code does worse if there is a
spam address plus a lot of unique addresses.
on current master:
test bench_packet_discard_many_senders ... bench: 1,997,960 ns/iter (+/- 103,715)
test bench_packet_discard_mixed_senders ... bench: 14,256,116 ns/iter (+/- 534,865)
test bench_packet_discard_single_sender ... bench: 1,306,809 ns/iter (+/- 61,992)
with this commit:
test bench_packet_discard_many_senders ... bench: 1,644,025 ns/iter (+/- 83,715)
test bench_packet_discard_mixed_senders ... bench: 1,089,789 ns/iter (+/- 86,324)
test bench_packet_discard_single_sender ... bench: 955,234 ns/iter (+/- 55,953)
2022-01-19 18:10:02 +00:00
buffalu
650882217c
Add PacketBatch packet_indexes stat ( #22564 )
...
* collect stats on packet batch indicies
* cleanup
* cleanup
* cleanup
* change name
2022-01-19 08:13:07 +00:00
anatoly yakovenko
e616a7ebfc
Track discard time of excess packets in sigverify ( #22554 )
...
* discard time histogram
* closer to the if
* update
2022-01-18 15:09:39 -07:00
Michael Vines
8dc6f9f589
Remove unused mut
2022-01-18 12:10:31 -08:00
sakridge
49443406fd
Use VecDeque instead of Vec in sigverify stage ( #22538 )
...
avoid bad performance of remove(0) for a single sender
2022-01-17 18:37:05 +01:00
anatoly yakovenko
2d94e6e5d3
metrics for generate new bank forks ( #22492 )
...
* metrics for generate new bank forks
* fixed
* Apply suggestions from code review
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* --fixup
* fixup!
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-01-17 09:59:47 -07:00
Tao Zhu
a724fa2347
Add hidden cli option to allow validator reports replayed transaction cost metrics ( #22369 )
...
* add hidden cli option to allow validator reports replayed transaction cost detail metrics
* Update validator/src/main.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* - rebase master, using unbounded instead of channel; dowgrade to datapoint_trace
* removed cli arg, prefer log at trace
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-01-15 00:31:21 +00:00
Tao Zhu
1309a9cea0
Add estimated and actual block cost units metrics ( #22326 )
...
* - report cost details for transactions selected to be packed into block;
- report estimated execution units packed into block, and actual units and time after execution
* revert reporting per-transaction details
* rollup transaction cost details (eg signature cost, wirte lock, data cost and execution costs) into block stats
* change naming from units to cu, use struct to replace tuple
2022-01-14 23:44:18 +00:00
Tao Zhu
9c9f2dd5bd
port counting vote CUs to block cost ( #22477 )
2022-01-14 10:50:29 -06:00
Justin Starry
f804ccdece
Store address table lookups in blockstore and bigtable ( #22402 )
2022-01-14 15:24:41 +08:00
carllin
4ab7d6c23e
Filter out outdated slots ( #22450 )
...
* Filter out outdated slots
* Fixup error
2022-01-13 19:51:00 -05:00
Jeff Washington (jwash)
e291342c4a
Revert "chore: bump dashmap from 4.0.2 to 5.0.0 ( #21824 )" ( #22488 )
...
This reverts commit 8aa3d690b5
.
2022-01-13 13:06:39 -06:00
Tao Zhu
6614727be8
downgrade individual per-program-timing to trace to reduce writes to influx
2022-01-12 18:52:13 -06:00
Eric Warehime
8161cee70f
Remove unnecessary var in banking_stage bench ( #22408 )
2022-01-11 22:25:21 -06:00
Tyera Eulberg
637e366b18
Prevent rent-paying account creation ( #22292 )
...
* Fixup typo
* Add new feature
* Add new TransactionError
* Add framework for checking account state before and after transaction processing
* Fail transactions that leave new rent-paying accounts
* Only check rent-state of writable tx accounts
* Review comments: combine process_result success behavior; log and metrics before feature activation
* Fix tests that assume rent-exempt accounts are okay
* Remove test no longer relevant
* Remove native/sysvar special case
* Move metrics submission to report legacy->legacy rent paying transitions as well
2022-01-11 11:32:25 -07:00
Jeff Biseda
8b66625c95
convert std::sync::mpsc to crossbeam_channel ( #22264 )
2022-01-11 02:44:46 -08:00
dependabot[bot]
2cc6f863bf
Bump etcd-client from 0.8.2 to 0.8.3 ( #22415 )
...
Bumps [etcd-client](https://github.com/etcdv3/etcd-client ) from 0.8.2 to 0.8.3.
- [Release notes](https://github.com/etcdv3/etcd-client/releases )
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.8.2...0.8.3 )
---
updated-dependencies:
- dependency-name: etcd-client
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-01-10 13:25:15 -07:00
steviez
5f1f4dcbdd
Use struct to pass all Tpu sockets as one argument to Tpu::new() ( #21965 )
...
Tpu::new() now matches Tvu::new() in having struct to reduce argument
list. Additionally, Rust supports partial moves, so there is no need to
clone the Tvu sockets out of Node object.
2022-01-10 11:29:48 -06:00
Ashwin Sekar
eeec1ce2ad
Add local cluster test to repro slot hash expiry bug ( #21873 )
2022-01-10 00:58:21 -05:00
dependabot[bot]
28275a33d6
Bump tempfile from 3.2.0 to 3.3.0 ( #22401 )
...
* Bump tempfile from 3.2.0 to 3.3.0
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.2.0 to 3.3.0.
- [Release notes](https://github.com/Stebalien/tempfile/releases )
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/NEWS )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.2.0...v3.3.0 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-09 12:26:08 -07:00
Yueh-Hsuan Chiang
5771c36d3f
Rename open_with_access_type() to open_with_options() ( #22123 )
2022-01-07 12:11:43 -08:00
Justin Starry
52d12cc802
Add runtime support for address table lookups ( #22223 )
...
* Add support for address table lookups in runtime
* feedback
* feedback
2022-01-07 11:59:09 +08:00
Trent Nelson
390ef0fbcd
Consolidate process instruction execution timings to own struct
2022-01-06 03:56:46 -07:00
Trent Nelson
848b6dfbdd
Add metrics for executor creation
2022-01-06 03:56:46 -07:00
Carl Lin
b25e4a200b
Add execute metrics
2022-01-06 03:56:46 -07:00
Trent Nelson
7d32909e17
move `ExecuteTimings` from `runtime::bank` to `program_runtime::timings`
2022-01-06 03:56:46 -07:00
Yueh-Hsuan Chiang
47b74e28ec
Add CLEANUP_SERVICE flag to ledger cleanup benchmark ( #22108 )
2022-01-05 23:46:02 -08:00
Yueh-Hsuan Chiang
44d61465f1
(Ledger store benchmark - 3/N) Add comments about the benchmark and its arguments ( #22160 )
...
* Avoid shred generation in ledger_cleanup.rs
* Update comment for test_ledger_cleanup_compaction to include benchmark information.
2022-01-04 23:35:55 -10:00
Justin Starry
45458e7139
Refactor: Improve type safety and readability of transaction execution ( #22215 )
...
* Refactor Bank::load_and_execute_transactions
* Refactor: improve type safety of TransactionExecutionResult
* Add enum for extra type safety in execution results
* feedback
2022-01-05 10:15:15 +08:00
Yueh-Hsuan Chiang
e201b41341
Avoid shred generation in ledger_cleanup.rs ( #22091 )
2022-01-04 15:29:43 -10:00
behzad nouri
4b24499916
removes total-size from return value of recv_mmsg
2022-01-04 21:06:59 +00:00
behzad nouri
01a096adc8
adds bitflags to Packet.Meta
...
Instead of a separate bool type for each flag, all the flags can be
encoded in a type-safe bitflags encoded in a single u8:
https://github.com/solana-labs/solana/blob/d6ec103be/sdk/src/packet.rs#L19-L31
2022-01-04 13:53:40 +00:00
behzad nouri
73a7741c49
uses std::net::IpAddr type for Packet.Meta.addr
2022-01-04 13:53:40 +00:00
sakridge
2486e21ffe
Lower vote-only-mode to 400 ( #22210 )
2022-01-04 12:49:14 +01:00
Jeff Biseda
ca8fef5855
retransmit consecutive leader blocks ( #22157 )
2022-01-04 00:24:16 -08:00
Yueh-Hsuan Chiang
e8b7f96a89
Add struct BlockstoreOptions ( #22121 )
2022-01-03 18:30:45 -10:00
dependabot[bot]
672fed04cb
Bump serde from 1.0.132 to 1.0.133 ( #22233 )
...
* Bump serde from 1.0.132 to 1.0.133
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.132 to 1.0.133.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.132...v1.0.133 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 22:19:39 +00:00
carllin
005592998d
Fix bug, add error specific timings ( #22225 )
2022-01-03 16:33:54 -05:00
behzad nouri
69d71f8f86
removes epoch_authorized_voters from VoteTracker ( #22192 )
...
https://github.com/solana-labs/solana/pull/22169
verifies authorized-voter early on in vote-listener pipeline; and so
VoteTracker no longer needs to maintain and check for epoch authorized
voters.
2022-01-03 21:07:47 +00:00
Jeff Biseda
0e4ede46d1
work around rust 39364 for stats_reporter_sender ( #22227 )
2022-01-03 11:46:02 -08:00
dependabot[bot]
ecbfc70bfa
Bump serde_json from 1.0.73 to 1.0.74 ( #22231 )
...
* Bump serde_json from 1.0.73 to 1.0.74
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.73 to 1.0.74.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.73...v1.0.74 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 10:36:59 -07:00
Yueh-Hsuan Chiang
557d35ec79
Remove duplicate code in ledger_cleanup_compaction_test ( #22204 )
2021-12-31 11:19:33 -05:00
Yueh-Hsuan Chiang
f479ab7af2
ledger_cleanup test improvement (1/N) -- make the test lockless and simplify the logic ( #22090 )
2021-12-30 20:18:47 -10:00
carllin
d06e6c7425
Count compute units even when transaction errors ( #22182 )
2021-12-30 21:21:42 -05:00
Jeff Biseda
95dfcc546a
bypass retransmission for slots without propagated stats ( #22176 )
2021-12-30 16:07:34 -08:00
behzad nouri
c0c6038654
checks for authorized voter early on in the vote-listener pipeline ( #22169 )
...
Before votes are verified that they are signed by the authorized voter,
they might be dropped in verified-vote-packets code. If there are
enough many spam votes from unauthorized voters, this may potentially
drop valid votes but keep the false ones.
https://github.com/solana-labs/solana/blob/57986f982/core/src/verified_vote_packets.rs#L165-L168
2021-12-30 15:03:14 +00:00
carllin
33d0b5e011
Revert "Count compute units even when transaction errors ( #22059 )" ( #22174 )
...
This reverts commit eaa8c67bde
.
2021-12-30 02:42:32 -05:00
Lijun Wang
f14928a970
Stream additional block metadata via plugin ( #22023 )
...
* Stream additional block metadata through plugin
blockhash, block_height, block_time, rewards are streamed
2021-12-29 15:12:01 -08:00
Justin Starry
b1d9a2e60e
Don't forward packets received from TPU forwards port ( #22078 )
...
* Don't forward packets received from TPU forwards port
* Add banking stage test
2021-12-29 19:34:31 +01:00
dependabot[bot]
d20a3774db
Bump lru from 0.7.1 to 0.7.2 ( #22161 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.1...0.7.2 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-29 10:47:37 -07:00
carllin
eaa8c67bde
Count compute units even when transaction errors ( #22059 )
2021-12-28 17:05:11 -05:00
carllin
f061059e45
Prevent log spam ( #22148 )
2021-12-28 17:04:48 -05:00
Tao Zhu
3d6ab96587
push live packets straight to buffer, leader only process packets from buffer
2021-12-28 15:21:24 -06:00
Yueh-Hsuan Chiang
b89cd8cd1a
Avoid cloning Vec<Entry> when calling entries_to_test_shreds() ( #22093 )
2021-12-24 12:32:43 -08:00
Justin Starry
93c776ce19
Refactor packet deduplication and harden bench test ( #22080 )
2021-12-22 23:05:10 -06:00
Tao Zhu
dd80a525ef
Leader QoS service metrics ( #21708 )
...
* - qos_service metrics tagged with leader thread ids to separate gossip/tpu votes and transactions;
- qos_service metrics is reported with bank slot;
- replaced timer-based reporting with signal via channel; removed async report test as qos_service now lives within a thread
* - add tpu live packets (eg, not buffered packets) states to qos metrics reporting
2021-12-22 21:39:59 +00:00
behzad nouri
f67ecd5c18
removes unused Cargo dependencies ( #22022 )
...
Also moving some to [dev-dependencies] so that they are not propagated
to other packages which depend on the package.
2021-12-22 17:01:33 +00:00
behzad nouri
4d62f03297
uses enum instead of trait for VoteTransaction ( #22019 )
...
Box<dyn Trait> involves runtime dispatch, has significant overhead and
is slow. It also requires hacky boilerplate code for implementing Clone
or other basic traits:
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L70-L102
Only limited known types can be VoteTransaction and they are all defined
in the same crate. So using a trait here only adds overhead.
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L125-L165
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L221-L264
2021-12-22 14:25:46 +00:00
Tao Zhu
9c5d82557a
skip reporting all-zero stats
2021-12-21 16:20:36 -06:00
dependabot[bot]
69d0b08dd8
chore: bump lru from 0.7.0 to 0.7.1 ( #22018 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.0...0.7.1 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 21:56:56 -07:00
behzad nouri
65d59f4ef0
tracks erasure coding shreds' indices explicitly ( #21822 )
...
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921
However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.
The commit adds constructs to track coding shreds indices explicitly.
2021-12-19 22:37:55 +00:00
behzad nouri
7476dfeec0
removes Select in favor of recv_timeout/try_iter ( #21981 )
...
crossbeam_channel::Select::ready_timeout might return with success spuriously.
2021-12-18 17:39:07 +00:00
Jeff Biseda
3fe942ab30
new net-stats require a new table ( #21996 )
2021-12-18 00:13:16 -08:00
dependabot[bot]
8f547a6c98
chore: bump serde from 1.0.131 to 1.0.132 ( #21989 )
...
* chore: bump serde from 1.0.131 to 1.0.132
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.131 to 1.0.132.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.131...v1.0.132 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-18 07:45:20 +00:00
carllin
7f6fb6937a
Ensure AncestorHashesSerice selects an open port ( #21919 )
2021-12-18 00:44:01 -05:00
Jeff Biseda
97a1fa10a6
streamer send destination metrics for repair, gossip ( #21564 )
2021-12-17 15:21:05 -08:00
segfaultdoctor
76098dd42a
RPC Block Subscription ( #21787 )
...
* add stuff
* compiling
* add notify block
* wip
* feat: add blockSubscribe pubsub method
* address PR comments
Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
behzad nouri
89d66c3210
removes next_shred_index from return value of entries to shreds api ( #21961 )
...
next-shred-index is already readily available from returned data shreds.
The commit simplifies the api for upcoming changes to erasure coding
schema which will require explicit tracking of indices for coding shreds
as well as data shreds.
2021-12-17 15:01:55 +00:00
Jeff Biseda
7ec39f5a1e
time based retransmit in replay_stage ( #21498 )
2021-12-17 05:44:40 -08:00
carllin
385efae4b3
Remove need to send bank in retransmit request from ReplayStage ( #21943 )
...
* Remove need to send bank in retransmitter
2021-12-16 21:11:01 -05:00
Justin Starry
6ff0be6a82
Clean up demote program write lock feature ( #21949 )
...
* Clean up demote program write lock feature
* fix test
2021-12-16 17:27:22 -05:00
steviez
e83ca4bb28
Clean up test_ledger_cleanup_compaction prints ( #21875 )
...
- Use info!()/warn!() over println!()/eprintln!()
- Make status prints consistent
- Add default RUST_LOG filter to see test printouts
- Adjust reported data to show shreds and rates we care about
2021-12-16 11:24:29 -06:00
carllin
cb395abff7
Fix subtraction overflow ( #21871 )
2021-12-14 14:24:22 -05:00
behzad nouri
8d980f07ba
uses Option<Slot> for SlotMeta.parent_slot ( #21808 )
...
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205
The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
behzad nouri
4ceb2689f5
adds ShredId uniquely identifying each shred ( #21820 )
2021-12-14 17:34:02 +00:00
steviez
4a9d7318d1
Rework test parameters to be shreds instead of entries ( #21780 )
...
The number of shreds that result from a given number of entries is
variable and in our test case, somewhat unintuitive to think about when
trying to determine how much data we're pushing into the blockstore. So,
this change converts the unit of test parameters from entries to shreds.
This change also cleans up some variable naming for clarity and prints.
2021-12-13 23:34:43 -06:00
dependabot[bot]
018b54dbd7
chore: bump serde_json from 1.0.72 to 1.0.73 ( #21856 )
...
* chore: bump serde_json from 1.0.72 to 1.0.73
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.72...v1.0.73 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 04:08:22 +00:00
dependabot[bot]
b81124deec
chore: bump etcd-client from 0.8.1 to 0.8.2 ( #21825 )
...
Bumps [etcd-client](https://github.com/etcdv3/etcd-client ) from 0.8.1 to 0.8.2.
- [Release notes](https://github.com/etcdv3/etcd-client/releases )
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.8.1...0.8.2 )
---
updated-dependencies:
- dependency-name: etcd-client
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 16:20:22 -07:00
dependabot[bot]
8aa3d690b5
chore: bump dashmap from 4.0.2 to 5.0.0 ( #21824 )
...
* chore: bump dashmap from 4.0.2 to 5.0.0
Bumps [dashmap](https://github.com/xacrimon/dashmap ) from 4.0.2 to 5.0.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases )
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.0.0 )
---
updated-dependencies:
- dependency-name: dashmap
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-13 13:23:58 -07:00
Jeff Washington (jwash)
90f41fd9b7
use cost model to limit new account creation ( #21369 )
...
* use cost model to limit new account creation
* handle every system instruction
* remove &
* simplify match
* simplify match
* add datapoint for account data size
* add postgres error handling
* handle accounts:unlock_accounts
2021-12-12 14:57:18 -06:00
behzad nouri
e08139f949
uses Option<u64> for SlotMeta.last_index ( #21775 )
...
SlotMeta.last_index may be unknown and current code is using u64::MAX to
indicate that:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L169-L174
This lacks type-safety and can introduce bugs if not always checked for
Several instances of slot_meta.last_index + 1 are also subject to
overflow.
This commit updates the type to Option<u64>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-11 14:47:20 +00:00
Justin Starry
254ef3e7b6
Rename Packets to PacketBatch ( #21794 )
2021-12-11 09:44:15 -05:00
Yueh-Hsuan Chiang
65194c7ae8
Add NUM_WRITERS to ledger_cleanup to enable multiple writers. ( #21729 )
...
Summary:
* Add NUM_WRITERS to ledger_cleanup to enable multiple writers.
(Note that our insert_shreds() is still single threaded because
it has a lock that limits only one writer at a time.)
* Make pre-generated slots more performent by directly inserting
into the shared queue. Otherwise, the main-thread which
prepares the slots will be slower than the writers.
* Correct the shred insertion time -- before this diff it did not
wait for joining all writer threads.
2021-12-10 09:42:51 -08:00
dependabot[bot]
d870f566ef
chore: bump serde from 1.0.130 to 1.0.131 ( #21758 )
...
* chore: bump serde from 1.0.130 to 1.0.131
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.130 to 1.0.131.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.130...v1.0.131 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-10 08:47:46 +00:00
behzad nouri
8063273d09
adds more sanity checks to shreds ( #21675 )
2021-12-09 16:43:57 +00:00
Ashwin Sekar
f0acf7681e
Add vote instructions that directly update on chain vote state ( #21531 )
...
* Add vote state instructions
UpdateVoteState and UpdateVoteStateSwitch
* cargo tree
* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
dependabot[bot]
f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 ( #21643 )
...
* chore: bump itertools from 0.10.1 to 0.10.3
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-06 23:23:26 -07:00
behzad nouri
cd17f63d81
adds back position field to coding-shred-header ( #21600 )
...
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.
2021-12-05 14:42:09 +00:00
Trent Nelson
d6f22433d0
Bump version to v1.10.0
2021-12-04 20:17:54 +00:00
Jeff Biseda
9c6b95e1e1
fix distance calculation in get_closest_completion ( #21601 )
2021-12-03 22:36:46 -08:00
dependabot[bot]
446089edb4
chore: bump retain_mut from 0.1.4 to 0.1.5 ( #21599 )
...
Bumps [retain_mut](https://github.com/upsuper/retain_mut ) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/upsuper/retain_mut/releases )
- [Commits](https://github.com/upsuper/retain_mut/compare/v0.1.4...v0.1.5 )
---
updated-dependencies:
- dependency-name: retain_mut
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-03 22:11:42 -07:00
Justin Starry
1430b58a6d
Remove deprecated slow epoch boundary methods ( #21568 )
2021-12-03 17:59:10 +00:00
Michael Vines
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
Alexander Meißner
b78f5b6032
Refactor: Cleanup InstructionProcessor ( #21404 )
...
* Moves create_message(), native_invoke() and process_cross_program_instruction()
from the InstructionProcessor to the InvokeContext so that they can have a useful "self" parameter.
* Moves InstructionProcessor into InvokeContext and Bank.
* Moves ExecuteDetailsTimings into its own file.
* Moves Executor into invoke_context.rs
* Moves PreAccount into its own file.
* impl AbiExample for BuiltinPrograms
2021-12-01 08:54:42 +01:00
Michael Vines
dd12d90eac
Upgrade to Rust 2021
2021-11-30 20:43:46 -08:00
dependabot[bot]
de8edad30b
chore: bump etcd-client from 0.7.2 to 0.8.1 ( #21513 )
...
Bumps [etcd-client](https://github.com/etcdv3/etcd-client ) from 0.7.2 to 0.8.1.
- [Release notes](https://github.com/etcdv3/etcd-client/releases )
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.7.2...0.8.1 )
---
updated-dependencies:
- dependency-name: etcd-client
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-30 18:16:32 +00:00
dependabot[bot]
a125e7b8c2
chore: bump serde_json from 1.0.71 to 1.0.72 ( #21426 )
...
* chore: bump serde_json from 1.0.71 to 1.0.72
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00
Michael Vines
ba9dfa0d22
Remove frozen account support
2021-11-29 08:38:11 -08:00
Tao Zhu
9edfc5936d
Refactor accounts.rs with Justin's comments to improve lock accounts ( #21406 )
...
with results code path.
- fix a bug that could unlock accounts that weren't locked
- add test to the refactored function
- skip enumerating transaction accounts if qos results is an error
- add #[must_use] annotation
- avoid clone error in results
- add qos error code to unlock_accounts match statement
- remove unnecessary AbiExample
2021-11-23 21:17:55 +00:00
Lijun Wang
c29838fce1
Accountsdb plugin transaction part 3: Transaction Notifier ( #21374 )
...
The TransactionNotifierInterface interface for notifying transactions.
Changes to transaction_status_service to notify the notifier of the transaction data.
Interface to query the plugin's interest in transaction data
2021-11-23 09:55:53 -08:00
Tao Zhu
2602e7c3bc
Fix flaky test ( #21402 )
...
* the async test is flaky on ci
* fix unstable test by increasing stats repoting time
2021-11-23 09:47:17 -06:00
behzad nouri
dd338b6c9f
changes Shred::parent return type to Option<Slot> ( #21370 )
...
Shred::parent can return garbage if the struct fields are invalid:
https://github.com/solana-labs/solana/blob/8a50b6302/ledger/src/shred.rs#L446-L453
The commit adds more sanity checks and changes the return type to Option<Slot>.
2021-11-23 14:45:26 +00:00
Tao Zhu
cd5a39ee43
the async test is flaky on ci ( #21365 )
2021-11-22 18:16:20 -06:00
Jeff Washington (jwash)
87831e7f8d
start system monitor earlier in validator so we get memory stats at startup ( #21372 )
2021-11-22 14:37:17 -06:00
sakridge
f31ca8ba8c
Report cluster slots size ( #21380 )
2021-11-22 17:47:58 +01:00
Jeff Biseda
2ed7e3af89
prioritize slot repairs for unknown last index and close to completion ( #21070 )
2021-11-19 19:17:30 -08:00
sakridge
0bda0c3e0c
Add bank drop service ( #21322 )
2021-11-19 17:20:18 +01:00
behzad nouri
48dfdfb4d5
changes Blockstore::is_shred_duplicate arg type to ShredType
2021-11-19 14:16:39 +00:00
behzad nouri
57057f8d39
uses enum for shred type
...
Current code is using u8 which does not have any type-safety and can
contain invalid values:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L167
Checks for invalid shred-types are scattered through the code:
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/blockstore.rs#L849-L851
https://github.com/solana-labs/solana/blob/66fa062f1/ledger/src/shred.rs#L346-L348
The commit uses enum for shred type with #[repr(u8)]. Backward
compatibility is maintained by implementing Serialize and Deserialize
compatible with u8, and adding a test to assert that.
2021-11-19 14:16:39 +00:00
carllin
b30c94ce55
ClusterInfoVoteListener send only missing votes to BankingStage ( #20873 )
2021-11-18 15:20:41 -08:00
Tao Zhu
0ca255220e
- Encapsulate QoS Service metrics reporting within QosServioce, so client ( #21191 )
...
code (eg banking_stage) doesn't need to worry about it.
- Remove dead cost_* stats from banking_stage, clean up call path.
2021-11-18 15:35:30 -06:00
dependabot[bot]
89102540b1
chore: bump serde_json from 1.0.70 to 1.0.71 ( #21338 )
...
* chore: bump serde_json from 1.0.70 to 1.0.71
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-18 02:58:30 +00:00
Lijun Wang
89c45a57f8
Refactor slot status notification to decouple from accounts notifications ( #21308 )
...
Problem
Slot status can be used of in other scenarios in addition to account information such as transactions, blocks. The current implementation is too tightly coupled.
Summary of Changes
Decouple the slot status notification from accounts notification. Created a new slot status notification module.
2021-11-17 17:11:38 -08:00
Justin Starry
66fa062f13
rename process_entries to indicate it's only for tests ( #21321 )
2021-11-17 20:53:40 +01:00
Jeff Biseda
d5de0c8e12
add --no-os-network-stats-reporting option ( #21296 )
2021-11-16 10:26:03 -08:00
sakridge
398af132a5
More set_root metrics ( #21286 )
2021-11-15 16:28:18 -07:00
dependabot[bot]
55ea672041
chore: bump serde_json from 1.0.69 to 1.0.70 ( #21285 )
...
* chore: bump serde_json from 1.0.69 to 1.0.70
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-11-15 15:27:47 -07:00
Jeff Washington (jwash)
f2bd9947cc
mem stats: rescale from kb to bytes ( #21282 )
2021-11-15 14:42:41 -06:00
Jeff Washington (jwash)
f8dcb2f38b
report mem stats ( #21258 )
2021-11-13 00:59:41 +00:00
Michael Keleti
b0ca335463
Rename "trusted" to "known" in `validators/` ( #21197 )
...
* Replaced trusted with known validator
* Format Convention
2021-11-12 11:57:55 -07:00
Tao Zhu
11153e1f87
refactor cost calculation ( #21062 )
...
* - cache calculated transaction cost to allow sharing;
- atomic cost tracking op;
- only lock accounts for transactions eligible for current block;
- moved qos service and stats reporting to its own model;
- add cost_weight default to neutral (as 1), vote has zero weight;
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update core/src/qos_service.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update core/src/qos_service.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-12 01:04:53 -06:00
behzad nouri
5fb0ab9d00
removes redundant args from Shredder::try_recovery ( #21226 )
...
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
2021-11-10 21:19:03 +00:00
dependabot[bot]
3c5f505d3e
chore: bump serde_json from 1.0.68 to 1.0.69 ( #21211 )
...
* chore: bump serde_json from 1.0.68 to 1.0.69
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-09 11:04:40 -07:00
Ivan Mironov
c78f474373
Add validator option to change niceness of snapshot packager thread
2021-11-04 17:16:46 -06:00
Alexander Meißner
7200c5106e
Replaces MockInvokeContext by ThisInvokeContext in tests ( #20881 )
...
* Replaces MockInvokeContext by ThisInvokeContext in BpfLoader, SystemInstructionProcessor, CLIs, ConfigProcessor, StakeProcessor and VoteProcessor.
* Finally, removes MockInvokeContext, MockComputeMeter and MockLogger.
* Adjusts assert_instruction_count test.
* Moves ThisInvokeContext to the program-runtime crate.
2021-11-04 21:47:32 +01:00
Justin Starry
140a5f633d
Simplify replay vote tracking by using packet metadata ( #21112 )
2021-11-03 09:02:48 +00:00
steviez
e6280fc1fa
Add additional checks for should_retransmit_and_persist() ( #20672 )
...
Add additional checks to should_retransmit_and_persist()
- Check invalid shred index
- Update cases that check if node was leader
- Some comments and variable rename for clarity
2021-11-03 02:01:07 -05:00
sakridge
a8d78e89d3
Move test-validator to own module to reduce core dependencies ( #20658 )
...
* Move test-validator to own module to reduce core dependencies
* Fix a few TestValidator paths
* Use solana_test_validator crate for solana_test_validator bin
* Move client int tests to separate crate
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-29 01:27:07 +00:00
Justin Starry
036d7fcc81
Clean up sanitized tx creation for tests ( #21006 )
2021-10-27 18:09:16 +01:00
sakridge
261dd96ae3
Swap banking stage vote channels ( #20987 )
2021-10-26 21:20:31 +02:00
behzad nouri
1297a13586
adds metrics tracking crds writes and votes ( #20953 )
2021-10-26 13:02:30 +00:00
dependabot[bot]
8efba88dde
chore: bump systemstat from 0.1.8 to 0.1.10 ( #20929 )
...
Bumps [systemstat](https://github.com/unrelentingtech/systemstat ) from 0.1.8 to 0.1.10.
- [Release notes](https://github.com/unrelentingtech/systemstat/releases )
- [Commits](https://github.com/unrelentingtech/systemstat/compare/v0.1.8...v0.1.10 )
---
updated-dependencies:
- dependency-name: systemstat
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-25 20:32:15 +00:00
Jeff Washington (jwash)
43ea579f63
add cli for --accounts-hash-num-passes ( #20827 )
2021-10-25 09:45:46 -05:00
Tao Zhu
c2bfce90b3
- cost_tracker is data member of a bank, it can report metrics when bank is frozen ( #20802 )
...
- removed cost_tracker_stats and histogram
- move stats reporting outside of bank freeze
2021-10-24 22:19:23 -05:00
behzad nouri
5e1cf39c74
adds metrics for number of outgoing shreds in retransmit stage ( #20882 )
2021-10-24 13:12:27 +00:00
Michael Vines
350bb561eb
Clippy
2021-10-23 08:21:20 +00:00
Jack May
bfbbc53dac
Divorce the runtime from FeeCalculator ( #20737 )
2021-10-22 14:32:40 -07:00
Justin Starry
735016661b
Report timing info for stakes cache updates from txs ( #20856 )
2021-10-22 12:49:02 -04:00
dependabot[bot]
0ac89841bf
chore: bump reqwest from 0.11.5 to 0.11.6 ( #20844 )
...
* chore: bump reqwest from 0.11.5 to 0.11.6
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.5...v0.11.6 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-10-21 12:54:18 -06:00
dependabot[bot]
73a7abdadf
chore: bump ahash from 0.7.5 to 0.7.6 ( #20721 )
...
Bumps [ahash](https://github.com/tkaitchuck/ahash ) from 0.7.5 to 0.7.6.
- [Release notes](https://github.com/tkaitchuck/ahash/releases )
- [Commits](https://github.com/tkaitchuck/ahash/commits )
---
updated-dependencies:
- dependency-name: ahash
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-21 17:52:21 +00:00
Tao Zhu
71d0bd4605
Add counter for dropped duplicated packets, fix dropped_packets_count ( #20834 )
2021-10-21 02:56:48 +00:00
Trent Nelson
fe098b5ddc
rpc-send-tx-svc: add with_config constructor
2021-10-20 13:43:27 -06:00
Jeff Washington (jwash)
95e91a4863
disable gossip publish of snapshots when using filler accts ( #20824 )
2021-10-20 18:07:29 +00:00
Tao Zhu
7496b5784b
- make cost_tracker a member of bank, remove shared instance from TPU; ( #20627 )
...
- decouple cost_model from cost_tracker; allowing one cost_model
instance being shared within a validator;
- update cost_model api to calculate_cost(&self...)->transaction_cost
2021-10-19 14:37:33 -05:00
Jeff Biseda
4cac66244d
report udp stats from validator ( #20587 )
2021-10-15 15:11:11 -07:00
carllin
44ff30b65b
Retry `SampleNotDuplicateConfirmed` decisions in AncestorHashesService ( #20240 )
2021-10-15 11:40:03 -07:00
behzad nouri
0f03971c3c
adds counters for errors in window-service run_insert ( #20670 )
2021-10-15 14:13:26 +00:00
behzad nouri
0c0384ec32
revises turbine peers shuffling order ( #20480 )
...
Turbine randomly shuffles cluster nodes on a broadcast tree for each
shred. This requires knowing the stakes and nodes' contact-infos (from
gossip).
However gossip is subject to partitioning and propogation delays.
Additionally unstaked nodes may join and leave the cluster at any
moment, changing the cluster view from one node to another.
This commit:
* Always arranges the unstaked nodes at the bottom of turbine broadcast
tree.
* Staked nodes are always included regardless of if their contact-info
is available in gossip or not.
* Uses the unbiased WeightedShuffle construct for shuffling nodes.
2021-10-14 15:09:36 +00:00
sakridge
588168b99d
Add check for shred data header size ( #20668 )
2021-10-14 05:56:14 +02:00
Jack May
da45be366a
Remove blockhash from fee calculation ( #20641 )
2021-10-13 13:10:58 -07:00
sakridge
09e7782d76
Refactor code to get block signatures in get_confirmed_signatures_for_address2 ( #20575 )
...
* Refactor get_confirmed_signatures_for_address2
* Move blockstore benches to ledger where they belong
2021-10-13 09:55:19 +02:00
Tao Zhu
005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; ( #20527 )
...
- move related modules to runtime
2021-10-12 08:51:33 -05:00
dependabot[bot]
a4c9b0f358
chore: bump ahash from 0.7.4 to 0.7.5 ( #20592 )
...
* chore: bump ahash from 0.7.4 to 0.7.5
Bumps [ahash](https://github.com/tkaitchuck/ahash ) from 0.7.4 to 0.7.5.
- [Release notes](https://github.com/tkaitchuck/ahash/releases )
- [Commits](https://github.com/tkaitchuck/ahash/commits )
---
updated-dependencies:
- dependency-name: ahash
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* Bump ahash patch
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-10-11 14:41:35 -06:00
Jeff Washington (jwash)
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
Michael Vines
c16510152e
Rework AVX/AVX2 detection again
2021-10-10 12:22:10 -07:00
carllin
838ff3b871
Separate out interrupted slots broadcast metrics ( #20537 )
2021-10-09 01:46:06 -07:00
Lijun Wang
d621994fee
Accountsdb stream plugin improvement ( #20419 )
...
Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.
Support multiple plugins at the same time.
2021-10-08 20:06:58 -07:00
Brooks Prumo
5440c1d2e1
SnapshotPackagerService pushes incremental snapshot hashes to CRDS ( #20442 )
...
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!
This commit does two main things:
1. SnapshotPackagerService now knows about incremental snapshot hashes,
and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
hashes need to be passed all the way to SnapshotPackagerService so it
can push these starting hashes to CRDS. Those values have been piped
through.
Fixes #20441 and #20423
2021-10-08 15:14:56 -05:00
Tao Zhu
675fa6993b
- update const cost values with data collected by #19627 ( #20314 )
...
- update cost calculation to closely proposed fee schedule #16984
2021-10-08 14:48:50 -05:00
dependabot[bot]
8422a12ac6
chore: bump reqwest from 0.11.4 to 0.11.5 ( #20536 )
...
* chore: bump reqwest from 0.11.4 to 0.11.5
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-10-08 13:25:31 -06:00
Tao Zhu
0ebd8c53ee
cost model to ignore vote transactions ( #20510 )
2021-10-07 12:49:07 -05:00
Tao Zhu
177a375479
Tpu vote 1.7 ( #20187 ) ( #20494 )
...
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
Co-authored-by: sakridge <sakridge@gmail.com>
2021-10-07 09:38:23 +00:00
Trent Nelson
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
dependabot[bot]
5ceb5ef328
chore: bump lru from 0.6.6 to 0.7.0 ( #20437 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.6.6 to 0.7.0.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.6.6...0.7.0 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-06 17:32:06 -06:00
Michael Vines
7027d56064
Resolve nightly-2021-10-05 clippy complaints
2021-10-06 10:37:58 -07:00
Tao Zhu
03913f6661
add tx count and thread id to stats, each stat reports and resets when slot changes ( #20451 )
2021-10-06 00:09:19 -05:00
Justin Starry
129716f3f0
Optimize stakes cache and rewards at epoch boundaries ( #20432 )
...
* Optimize stakes cache and rewards at epoch boundaries
* Fetch from accounts db
* Add cli flag for disabling epoch boundary optimization
2021-10-06 00:53:26 -04:00
Tao Zhu
6ff508c643
add transaction cost histogram metrics ( #20350 )
2021-10-05 08:57:39 -05:00
Brooks Prumo
4cd50f5d45
Don't gossip more snapshot hashes than what we retain ( #20379 )
2021-10-01 15:59:45 -05:00
Lijun Wang
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
Jeff Biseda
3854cfaa00
Use batch_send in forward_buffered_packets ( #20330 )
2021-09-29 20:49:43 -07:00
sakridge
94668c95c2
Prune sigverify queue ( #20331 )
2021-09-30 05:41:05 +02:00
dependabot[bot]
15aea3acdf
chore: bump retain_mut from 0.1.3 to 0.1.4 ( #20277 )
...
Bumps [retain_mut](https://github.com/upsuper/retain_mut ) from 0.1.3 to 0.1.4.
- [Release notes](https://github.com/upsuper/retain_mut/releases )
- [Commits](https://github.com/upsuper/retain_mut/commits )
---
updated-dependencies:
- dependency-name: retain_mut
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-28 00:00:25 -06:00
Brooks Prumo
3ea6a01254
Only gossip snapshot hashes for full snapshots ( #20271 )
2021-09-27 19:29:08 -05:00
Jon Cinque
567f30aa1a
windows: Make solana-test-validator work ( #20099 )
...
* windows: Make solana-test-validator work
The important changes to get this going on Windows:
* ledger lock needs to be done on a file instead of the directory
* IPC service needs to use the Windows pipe naming scheme
* always disable the JIT
* file logging not possible yet because we can't redirect stderr,
but this will change once env_logger fixes the pipe output target!
* Integrate review feedback
2021-09-22 23:10:35 +02:00
Jeff Biseda
640e93187c
periodically report sigverify_stage stats ( #19674 )
2021-09-21 10:37:58 -07:00
sakridge
013e1d9d49
Limit transaction forwarding from banking_stage ( #19940 )
2021-09-21 08:49:41 -07:00
carllin
e6b4dd3866
Add bank to banking stage regardless of if there is a working bank ( #19855 )
2021-09-17 16:55:53 -07:00
Tyera Eulberg
c4fe352965
Tiny toml fmt fix
2021-09-17 14:16:14 -06:00
Pavel Strakhov
65227f44dc
Optimize RPC pubsub for multiple clients with the same subscription ( #18943 )
...
* reimplement rpc pubsub with a broadcast queue
* update tests for new pubsub implementation
* fix: fix review suggestions
* chore(rpc): add additional pubsub metrics
* integrate max subscriptions check into SubscriptionTracker to reduce locking
* separate subscription control from tracker
* limit memory usage of items in pubsub broadcast queue, improve error handling
* add more pubsub metrics
* add final count metrics to pubsub
* add metric for total number of subscriptions
* fix small review suggestions
* remove by_params from SubscriptionTracker and add node_progress_watchers map instead
* add subscription tracker tests
* add metrics for number of pubsub notifications as a counter
* ignore clippy lint in TokenCounter
* fix underflow in token counter
* reduce queue capacity in pubsub tests
* fix(rpc): fix test timeouts
* fix race in account subscription test
* Add RpcSubscriptions::new_for_tests
Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-17 13:40:14 -06:00
dependabot[bot]
68da1909de
chore: bump serde_json from 1.0.67 to 1.0.68 ( #19942 )
...
* chore: bump serde_json from 1.0.67 to 1.0.68
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.67 to 1.0.68.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.67...v1.0.68 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-16 20:23:25 +00:00
sakridge
dc69cc1ae4
Only allow votes when root distance gets too high ( #19917 )
2021-09-16 15:12:26 +02:00
Justin Starry
ca3f147670
Add banking metrics for buffered and dropped packets ( #19902 )
2021-09-15 15:53:55 -05:00
Tao Zhu
67fa9945e1
Add few more metrics data points ( #19624 )
...
* Add slot, count and accumulated-units to per-program-timings for determining transaction cost elements
* correct the stats naming; fixes the dirty bit resetting
2021-09-15 09:49:49 -05:00
Justin Starry
34c1a9ac85
Report consumed_buffered_packets_count stat to metrics ( #19900 )
2021-09-15 14:19:39 +00:00
Tyera Eulberg
c91519961c
Use f64 for stake math in get_stake_percent_in_gossip ( #19895 )
2021-09-14 23:36:30 -06:00
Michael
4ff50519ff
Add an info log to indicate the node has reached supermajority and print the active stake percentage ( #19893 )
2021-09-14 21:48:15 -06:00
Jeff Washington (jwash)
b57e86abf2
cache account hash info ( #19426 )
...
* cache account hash info
* ledger_path -> accounts_hash_cache_path
2021-09-13 20:39:26 -05:00
carllin
87a7f00926
Track reset bank in PohRecorder ( #19810 )
2021-09-13 16:55:35 -07:00
Brooks Prumo
62c8bcf565
Add default() to SnapshotConfig ( #19776 )
2021-09-12 13:44:27 -05:00
Giles Cope
c9261e034e
remove dev deps
2021-09-11 14:33:53 -07:00
Giles Cope
c3a610374a
unused deps
2021-09-11 14:33:53 -07:00
Brooks Prumo
7aa5f6b833
Add CLI args for incremental snapshots ( #19694 )
...
Add `--incremental-snapshots` flag to enable incremental snapshots.
This will allow setting `--full-snapshot-interval-slots` and
`--incremental-snapshot-interval-slots`.
Also added `--maximum-incremental-snapshots-to-retain`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-10 15:59:26 -05:00
Michael Vines
4386e09710
Reduce wait for supermajority threshold back to 80%
2021-09-09 21:17:35 -07:00
sakridge
3a8c678f62
Remove some copying ( #19691 )
2021-09-08 18:32:38 +02:00
Jeff Washington (jwash)
456bf15012
AccountsIndexConfig -> AccountsDbConfig ( #19687 )
2021-09-08 04:30:38 +00:00
Jeff Washington (jwash)
d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it ( #19686 )
2021-09-07 20:09:40 -05:00
Brooks Prumo
4a5f83d3a7
Add fn to check when to take snapshots ( #19682 )
2021-09-07 23:26:35 +00:00
Brooks Prumo
a0552e5b46
Make startup aware of Incremental Snapshots ( #19600 )
2021-09-07 20:43:43 +00:00
behzad nouri
01a7ec8198
uses rayon thread-pool for retransmit-stage parallelization ( #19486 )
2021-09-07 15:15:01 +00:00
Brooks Prumo
fe8ba81ce6
Rename to is_valid instead of is_invalid ( #19670 )
2021-09-07 09:31:54 -05:00
dependabot[bot]
fdbde2df52
chore: bump etcd-client from 0.7.1 to 0.7.2 ( #19661 )
...
Bumps [etcd-client](https://github.com/etcdv3/etcd-client ) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/etcdv3/etcd-client/releases )
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.7.1...0.7.2 )
---
updated-dependencies:
- dependency-name: etcd-client
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-06 19:53:28 -06:00
Brooks Prumo
9d9482b9d8
Plumb `maximum_incremental_snapshot_archives_to_retain` ( #19640 )
2021-09-06 18:01:56 -05:00
Sean Young
d461a9ac10
verify_precompiles needs FeatureSet
...
Rather than pass in individual features, pass in the entire feature set
so that we can add the ed25519 program feature in a later commit.
2021-09-05 18:59:37 +01:00
Tyera Eulberg
decec3cd8b
Demote write locks on transaction program ids ( #19593 )
...
* Add feature
* Demote write lock on program ids
* Fixup bpf tests
* Update MappedMessage::is_writable
* Comma nit
* Review comments
2021-09-04 03:05:30 +00:00
Brooks Prumo
1828579580
Pass SnapshotConfig to SnapshotPackagerService ( #19616 )
2021-09-03 21:42:32 +00:00
Brooks Prumo
5e25ee5ebe
Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig ( #19612 )
2021-09-03 20:21:32 +00:00
Brooks Prumo
7ab0aec61f
Rename maximum_full_snapshot_archives_to_retain ( #19610 )
...
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
2021-09-03 11:28:10 -05:00
Brooks Prumo
e9374d32a3
Revert "Make startup aware of Incremental Snapshots ( #19550 )" ( #19599 )
...
This reverts commit d45ced0a5d
.
2021-09-02 19:14:41 -05:00
Brooks Prumo
d45ced0a5d
Make startup aware of Incremental Snapshots ( #19550 )
2021-09-02 19:05:15 -05:00
Jeff Biseda
7a8eba10b2
add synchronization comment to handle_new_root ( #19571 )
2021-09-02 13:52:14 -07:00
Lijun Wang
8378e8790f
Accountsdb replication installment 2 ( #19325 )
...
This is the 2nd installment for the AccountsDb replication.
Summary of Changes
The basic google protocol buffer protocol for replicating updated slots and accounts. tonic/tokio is used for transporting the messages.
The basic framework of the client and server for replicating slots and accounts -- the persisting of accounts in the replica-side will be done at the next PR -- right now -- the accounts are streamed to the replica-node and dumped. Replication for information about Bank is also not done in this PR -- to be addressed in the next PR to limit the change size.
Functionality used by both the client and server side are encapsulated in the replica-lib crate.
There is no impact to the existing validator by default.
Tests:
Observe the confirmed slots replicated to the replica-node.
Observe the accounts for the confirmed slot are received at the replica-node side.
2021-09-01 14:10:16 -07:00
behzad nouri
6d9818b8e4
skips retransmit for shreds with unknown slot leader ( #19472 )
...
Shreds' signatures should be verified before they reach retransmit
stage, and if the leader is unknown they should fail signature check.
Therefore retransmit-stage can as well expect to know who the slot
leader is and otherwise just skip the shred.
Blockstore checking signature of recovered shreds before sending them to
retransmit stage:
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/blockstore.rs#L884-L930
Shred signature verifier:
https://github.com/solana-labs/solana/blob/4305d4b7b/core/src/sigverify_shreds.rs#L41-L57
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/sigverify_shreds.rs#L105
2021-09-01 15:44:26 +00:00
Brooks Prumo
1d5a8ebc6a
Revert "Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )" ( #19529 )
...
This reverts commit 4d361af976
.
2021-08-31 22:03:19 -05:00
Brooks Prumo
fe9ee9134a
Make background services aware of incremental snapshots ( #19401 )
...
AccountsBackgroundService now knows about incremental snapshots. It is
now also in charge of deciding if an AccountsPackage is destined to be a
SnapshotPackage or not (or just used by AccountsHashVerifier).
!!! New behavior changes !!!
Taking snapshots (both bank and archive) **MUST** succeed.
This is required because of how the last full snapshot slot is
calculated, which is used by AccountsBackgroundService when calling
`clean_accounts()`.
File system calls are now unwrapped and will result in a crash. As Trent told me:
>Well I think if a snapshot fails due to some IO error, it's very likely that the operator is going to have to intervene before it works. We should exit error in this case, otherwise the validator might happily spin for several more hours, never successfully writing a complete snapshot, before something else brings it down. This would leave the validator's last local snapshot many more slots behind than it would be had we exited outright and potentially force the operator to abandon ledger continuity in favor of a quick catchup
Other errors will set the `exit` flag to `true`, and the node will gracefully shutdown.
Fixes #19167
Fixes #19168
2021-08-31 18:33:27 -05:00
dependabot[bot]
c21bf29ce7
chore: bump serde_json from 1.0.66 to 1.0.67 ( #19501 )
...
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.66 to 1.0.67.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.66...v1.0.67 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-31 06:05:25 +00:00
Tyera Eulberg
a3bef2e537
Fix shreds-to-hours/days estimations ( #19477 )
2021-08-30 13:16:06 -06:00
dependabot[bot]
6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 ( #19497 )
...
* chore: bump serde from 1.0.129 to 1.0.130
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 18:19:34 +00:00
behzad nouri
8ad52fa095
implements copy-on-write for vote-accounts ( #19362 )
...
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186
This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
2021-08-30 15:54:01 +00:00
carllin
84db04ce6c
Fix duplicate broadcast test ( #19365 )
2021-08-27 17:53:24 -07:00
Justin Starry
2d7f036afd
Add solana-program-runtime crate ( #19438 )
2021-08-27 00:30:36 +00:00
dependabot[bot]
e492638b7d
chore: bump libc from 0.2.100 to 0.2.101 ( #19442 )
...
* chore: bump libc from 0.2.100 to 0.2.101
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-26 12:38:34 -06:00
dependabot[bot]
46890ac197
chore: bump serde from 1.0.128 to 1.0.129 ( #19395 )
...
* chore: bump serde from 1.0.128 to 1.0.129
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-24 12:32:30 -06:00
dependabot[bot]
c0c95e88d8
chore: bump libc from 0.2.99 to 0.2.100 ( #19371 )
...
* chore: bump libc from 0.2.99 to 0.2.100
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-23 09:34:32 -06:00
dependabot[bot]
aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 ( #19370 )
...
* chore: bump serde from 1.0.127 to 1.0.128
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-23 09:33:34 -06:00
Brooks Prumo
1a4bede918
Remove filter_snapshot_storages_for_incremental_snapshot() ( #19349 )
...
Now that we can get just the subset of snapshot storages we want, the extra
filtering afterwards is redundant.
2021-08-23 09:45:29 -05:00
Brooks Prumo
6d939811e9
Name snapshots consistently ( #19346 )
...
#### Problem
Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:
```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```
#### Summary of Changes
For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
Brooks Prumo
234461f779
Add base_slot to get_snapshot_storages() ( #19348 )
2021-08-20 16:23:43 -05:00
Brooks Prumo
4d361af976
Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )
2021-08-20 17:06:53 +00:00
behzad nouri
1deb4add81
removes Slot from TransmitShreds ( #19327 )
...
An earlier version of the code was funneling through stakes along with
shreds to broadcast:
https://github.com/solana-labs/solana/blob/b67ffab37/core/src/broadcast_stage.rs#L127
This was changed to only slots as stakes computation was pushed further
down the pipeline in:
https://github.com/solana-labs/solana/pull/18971
However shreds themselves embody which slot they belong to. So pairing
them with slot is redundant and adds rooms for bugs should they become
inconsistent.
2021-08-20 13:48:33 +00:00
Trent Nelson
e0bc5fa690
validator: Trusted validators are now called known validators
2021-08-19 22:43:49 -06:00
Jack May
3ec33e7d02
Fail secp256k1 if the instruction data looks incorrect ( #19300 )
2021-08-19 13:13:54 -07:00
Tao Zhu
4982dc20f9
replace function with const var for better readability ( #19285 )
2021-08-19 14:59:53 -05:00
Justin Starry
c50b01cb60
Store versioned transactions in the ledger, disabled by default ( #19139 )
...
* Add support for versioned transactions, but disable by default
* merge conflicts
* trent's feedback
* bump Cargo.lock
* Fix transaction error encoding
* Rename legacy_transaction method
* cargo clippy
* Clean up casts, int arithmetic, and unused methods
* Check for duplicates in sanitized message conversion
* fix clippy
* fix new test
* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
Jeff Washington (jwash)
7c70f2158b
accounts_index_bins to AccountsIndexConfig ( #19257 )
...
* accounts_index_bins to AccountsIndexConfig
* rename param bins -> config
* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
Brooks Prumo
f9986c66b8
Make SnapshotPackagerService aware of Incremental Snapshots ( #19254 )
...
Add a field to SnapshotPackage that is an enum for SnapshotType, so archive_snapshot_package() will do the right thing.
Fixes #19166
2021-08-17 13:01:59 -05:00
behzad nouri
7a8807b8bb
retransmits shreds recovered from erasure codes
...
Shreds recovered from erasure codes have not been received from turbine
and have not been retransmitted to other nodes downstream. This results
in more repairs across the cluster which is slower.
This commit channels through recovered shreds to retransmit stage in
order to further broadcast the shreds to downstream nodes in the tree.
2021-08-17 13:44:10 +00:00
behzad nouri
3efccbffab
sends shreds (instead of packets) to retransmit stage
...
Working towards channelling through shreds recovered from erasure codes
to retransmit stage.
2021-08-17 13:44:10 +00:00
behzad nouri
6e413331b5
removes erroneous uses of Arc<...> from retransmit stage
2021-08-17 13:44:10 +00:00
behzad nouri
8198a7eae1
adds packet/shred count stats to window-service
...
Adding back these metrics from the earlier commit which removed them
from retransmit stage.
2021-08-17 13:44:10 +00:00
behzad nouri
bf437b0336
removes packet-count metrics from retransmit stage
...
Working towards sending shreds (instead of packets) to retransmit stage
so that shreds recovered from erasure codes are as well retransmitted.
Following commit will add these metrics back to window-service, earlier
in the pipeline.
2021-08-17 13:44:10 +00:00
behzad nouri
563aec0b4d
discards epoch-slots epochs ahead of the current root ( #19256 )
...
Cross cluster gossip contamination is causing cluster-slots hash map to
contain a lot of bogus values and consume too much memory:
https://github.com/solana-labs/solana/issues/17789
If a node is using the same identity key across clusters, then these
erroneous values might not be filtered out by shred-versions check,
because one of the variants of the contact-info will have matching
shred-version:
https://github.com/solana-labs/solana/issues/17789#issuecomment-896304969
The cluster-slots hash-map is bounded and trimmed at the lower end by
the current root. This commit also discards slots epochs ahead of the
root.
2021-08-17 13:13:28 +00:00
behzad nouri
f33b7abffb
adds back cluster partitions to broadcast-duplicates ( #19253 )
...
An earlier version of this code was aiming to create a partition by
manipulating stakes, and setting some of them to zero:
https://github.com/solana-labs/solana/blob/cde146155/core/src/broadcast_stage/broadcast_duplicates_run.rs#L65-L116
https://github.com/solana-labs/solana/pull/18971
moved stakes computation further down the stream, and so that logic
could no longer live there. This commit adds back cluster partitions
by intercepting packets before send.
2021-08-16 22:24:30 +00:00
Michael Vines
3e5ba594e0
Revert `TestValidatorGenesis::start()` to v1.7.8 signature; add `TestValidatorGenesis::start_with_socket_addr_space()`
2021-08-16 06:37:23 +00:00
Michael Vines
b15fa9fbd2
Add EtcdTowerStorage
2021-08-14 09:46:36 -07:00
carllin
22674000bd
Add EpochSlots frozen state transition ( #19112 )
2021-08-13 14:21:52 -07:00
Brooks Prumo
176036aa58
Rename AccountsPacakge to SnapshotPackage and AccountsPackagePre to AccountsPackage ( #19231 )
...
Renaming these types to better communicate their usages, which will
further diverge as incremental snapshot support is added.
With the new names, AccountsPacakge now refers to the type between
AccountsBackgroundProcess and AccountsHashVerifier, and SnapshotPackage
refers to the type between AccountsHashVerifier and
SnapshotPackagerService.
2021-08-13 16:08:09 -05:00
behzad nouri
b64eeb7729
removes erroneous uses of &Arc<...> from window-service
2021-08-13 17:26:31 +00:00
behzad nouri
d57398a959
removes repeated bank-forks locking in window-service
...
Window service is repeatedly locking bank-forks to look-up working-bank
for every single shred:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L597-L606
This commit updates shred_filter signature in recv_window so that where
we already obtain the lock on bank-forks, we can also look-up
working-bank once for all packets:
https://github.com/solana-labs/solana/blob/5fde4ee3a/core/src/window_service.rs#L256-L277
2021-08-13 17:26:31 +00:00
dependabot[bot]
5b9671d01a
chore: bump matches from 0.1.8 to 0.1.9 ( #19220 )
...
* chore: bump matches from 0.1.8 to 0.1.9
Bumps [matches](https://github.com/SimonSapin/rust-std-candidates ) from 0.1.8 to 0.1.9.
- [Release notes](https://github.com/SimonSapin/rust-std-candidates/releases )
- [Commits](https://github.com/SimonSapin/rust-std-candidates/commits )
---
updated-dependencies:
- dependency-name: matches
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-13 16:37:45 +00:00
Jack May
0b50bb2b20
Deprecate FeeCalculator returning APIs ( #19120 )
2021-08-13 09:08:20 -07:00
behzad nouri
7a789e0763
filters for recent contact-infos when checking for live stake ( #19204 )
...
Contact-infos are saved to disk:
https://github.com/solana-labs/solana/blob/9dfeee299/gossip/src/cluster_info.rs#L1678-L1683
and restored on validator start-up:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L450
Staked nodes entries will not expire until an epoch after. So when the
validator checks for online stake it is erroneously picking up
contact-infos restored from disk, which breaks the entire
wait-for-supermajority logic:
https://github.com/solana-labs/solana/blob/9dfeee299/core/src/validator.rs#L1515-L1561
This commit adds an extra check for the age of contact-info entries and
filters out old ones.
2021-08-13 12:12:40 +00:00
Tao Zhu
414d904959
Reject blocks for costs above the max block cost ( #18994 )
...
* added realtime cost checking logic to reject block that would exceed max limit:
- defines max limits at block_cost_limits.rs
- right after each bath's execution, accumulate its cost and check again
limit, return error if limit is exceeded
* update abi that changed due to adding additional TransactionError
* To avoid counting stats mltiple times, only accumulate execute-timing when a bank is completed
* gate it by a feature
* move cost const def into block_cost_limits.rs
* redefine the cost for signature and account access, removed signer part as it is not well defined for now
* check if per_program_timings of execute_timings before sending
2021-08-12 10:48:47 -05:00