HaoranYi
e105547c14
tvu and tpu timeout on joining its microservices ( #24111 )
...
* panic when test timeout
* nonblocking send when when droping banks
* debug log
* timeout for tvu
* unused varaible
* timeout for tpu
* Revert "debug log"
This reverts commit da780a3301a51d7c496141a85fcd35014fe6dff5.
* add timeout const
* fix typo
* Revert "nonblocking send when when droping banks".
I will create another pull request for this.
This reverts commit 088c98ec0facf825b5eca058fb860deba6d28888.
* Update core/src/tpu.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/tpu.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/tvu.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/tvu.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/validator.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-04-07 20:20:13 -05:00
Jeff Washington (jwash)
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
Jeff Washington (jwash)
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
Jeff Washington (jwash)
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
Tyera Eulberg
fb67ff14de
Remove replica-node crates ( #24152 )
2022-04-06 16:52:19 -06:00
Tyera Eulberg
afeb1d3cca
Bump lru crate ( #24150 )
2022-04-06 16:18:42 -06:00
Brooks Prumo
c322842257
Replace channel with Mutex<Option> for AccountsPackage ( #24013 )
2022-04-06 05:47:19 -05:00
HaoranYi
302142bb25
fix typo ( #24123 )
2022-04-05 15:55:47 -05:00
behzad nouri
db23295e1c
removes legacy weighted_shuffle and weighted_best methods ( #24125 )
...
Older weighted_shuffle is based on a heuristic which results in biased
samples as shown in:
https://github.com/solana-labs/solana/pull/18343
and can be replaced with WeightedShuffle.
Also, as described in:
https://github.com/solana-labs/solana/pull/13919
weighted_best can be replaced with rand::distributions::WeightedIndex,
or WeightdShuffle::first.
2022-04-05 19:19:22 +00:00
carllin
4ea59d8cb4
Set drop callback on first root bank ( #23999 )
2022-04-05 13:02:33 -05:00
behzad nouri
2282571493
removes outdated and flaky test_skip_repair from retransmit-stage ( #24121 )
...
test_skip_repair in retransmit-stage is no longer relevant because
following: https://github.com/solana-labs/solana/pull/19233
repair packets are filtered out earlier in window-service and so
retransmit stage does not know if a shred is repaired or not.
Also, following turbine peer shuffle changes:
https://github.com/solana-labs/solana/pull/24080
the test has become flaky since it does not take into account how peers
are shuffled for each shred.
2022-04-05 16:02:53 +00:00
behzad nouri
2b718d00b0
removes legacy compatibility turbine peers shuffle code
2022-04-05 12:04:12 +00:00
behzad nouri
d0b850cdd9
removes turbine peers shuffle patch feature
2022-04-05 12:04:12 +00:00
behzad nouri
855801cc95
removes deterministic-shred-seed feature
2022-04-05 12:04:12 +00:00
Jeff Biseda
ee6bb0d5d3
track fec set turbine stats ( #23989 )
2022-04-04 14:44:21 -07:00
HaoranYi
6ba4e870c4
Blockstore should drop signals before validator exit ( #24025 )
...
* timeout for validator exits
* clippy
* print backtrace when panic
* add backtrace package
* increase time out to 30s
* debug logging
* make rpc complete service non blocking
* reduce log level
* remove logging
* recv_timeout
* remove backtrace
* remove sleep
* wip
* remove unused variable
* add comments
* Update core/src/validator.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/validator.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* whitespace
* more whitespace
* fix build
* clean up import
* add mutex for signal senders in blockstore
* remove mut
* refactor: extract add signal functions
* make blockstore signal private
* let compiler infer mutex type
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-04-04 11:38:05 -05:00
behzad nouri
7cb3b6cbe2
demotes WeightedShuffle failures to error metrics ( #24079 )
...
Since call-sites are calling unwrap anyways, panicking seems too punitive
for our use cases.
2022-04-03 16:20:06 +00:00
HaoranYi
ffa4cafe1c
Revert sequential execution of validator_exit and validator_parallel_exit tests ( #24048 )
...
* handle channel disconnect
* revert sequential execution of validator_exit and parallel_validator_exit tests
2022-04-02 10:22:47 -05:00
Yueh-Hsuan Chiang
0b5ed87220
(LedgerStore) Enable performance sampling in column family get() ( #23834 )
...
#### Summary of Changes
This PR enables RocksDB read side performance metrics to report to blockstore_rocksdb_read_perf.
The sampling rate is controlled by an env arg `SOLANA_METRICS_ROCKSDB_PERF_SAMPLES_IN_1K`,
specifies the number of perf samples for every 1000 operations. The default value is set to 10, meaning
we will report 10 out of 1000 (or 1/100) reads.
The metrics are based on the RocksDB [PerfContext](https://github.com/facebook/rocksdb/blob/main/include/rocksdb/perf_context.h ).
It includes many useful metrics including block read time, cache hit rate, and time spent on decompressing the block.
2022-04-01 13:13:32 -07:00
Pankaj Garg
df4d92f9cf
Revert voting service to use UDP instead of QUIC ( #24032 )
2022-04-01 09:34:18 -07:00
HaoranYi
51b37f0184
Modify rpc_completed_slot_service to be non-blocking ( #24007 )
...
* timeout for validator exits
* clippy
* print backtrace when panic
* add backtrace package
* increase time out to 30s
* debug logging
* make rpc complete service non blocking
* reduce log level
* remove logging
* recv_timeout
* remove backtrace
* remove sleep
* remove unused variable
* add comments
* Update core/src/validator.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* Update core/src/validator.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* whitespace
* more whitespace
* fix build
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-03-31 16:44:23 -05:00
Jeff Washington (jwash)
9c8dad33c7
add epoch_schedule and rent_collector to hash calc ( #24012 )
2022-03-31 10:51:18 -05:00
Jeff Washington (jwash)
da001d54e5
calculate_accounts_hash_helper uses config ( #24003 )
2022-03-31 09:29:45 -05:00
Jeff Washington (jwash)
125f9634fd
add hash calc config.use_write_cache ( #24005 )
2022-03-30 17:19:34 -05:00
HaoranYi
ba770832d0
Poh timing service ( #23736 )
...
* initial work for poh timing report service
* add poh_timing_report_service to validator
* fix comments
* clippy
* imrove test coverage
* delete record when complete
* rename shred full to slot full.
* debug logging
* fix slot full
* remove debug comments
* adding fmt trait
* derive default
* default for poh timing reporter
* better comments
* remove commented code
* fix test
* more test fixes
* delete timestamps for slot that are older than root_slot
* debug log
* record poh start end in bank reset
* report full to start time instead
* fix poh slot offset
* report poh start for normal ticks
* fix typo
* refactor out poh point report fn
* rename
* optimize delete - delete only when last_root changed
* change log level to trace
* convert if to match
* remove redudant check
* fix SlotPohTiming comments
* review feedback on poh timing reporter
* review feedback on poh_recorder
* add test case for out-of-order arrival of timing points and incomplete timing points
* refactor poh_timing_points into its own mod
* remove option for poh_timing_report service
* move poh_timing_point_sender to constructor
* clippy
* better comments
* more clippy
* more clippy
* add slot poh timing point macro
* clippy
* assert in test
* comments and display fmt
* fix check
* assert format
* revise comments
* refactor
* extrac send fn
* revert reporting_poh_timing_point
* align loggin
* small refactor
* move type declaration to the top of the module
* replace macro with constructor
* clippy: remove redundant closure
* review comments
* simplify poh timing point creation
Co-authored-by: Haoran Yi <hyi@Haorans-MacBook-Air.local>
2022-03-30 09:04:49 -05:00
Jeff Washington (jwash)
c24de17278
remove index hash calculation as an option ( #23928 )
2022-03-25 15:32:53 -05:00
HaoranYi
01af40d6b6
Fix intermittent validator_exit test failure ( #23594 )
...
* run validator_exit_test sequentially
* limit validator exit run to its own serial run subset
add 10ms delay in the validator exit tests
* fix intermittent validator exit failure
* no sleep
* undo the code move
2022-03-25 14:38:19 -05:00
ryleung-solana
6b85c2104c
Implement forwarding via TpuConnection ( #23817 )
2022-03-25 11:31:40 -04:00
Steven Luscher
f44c8f296f
fix: thread `enforce_ulimit_nofile` config down when opening blockstore ( #23925 )
2022-03-25 03:13:33 -05:00
Jeff Washington (jwash)
51f5524e2f
make verify_accounts_package_hash like other hash calc ( #23906 )
2022-03-24 17:49:48 -05:00
Jeff Washington (jwash)
55d61023f7
document 'accounts' hash ( #23907 )
2022-03-24 15:58:52 -05:00
HaoranYi
fedf4e984f
typo ( #23910 )
2022-03-24 15:21:59 -05:00
Jeff Washington (jwash)
37c36ce3fa
pass stats separately from CalcAccountsHashConfig ( #23892 )
2022-03-24 12:48:47 -05:00
steviez
c31db81ac4
Use VoteAccountsHashMap type alias in all applicable spots ( #23904 )
2022-03-24 12:09:48 -05:00
ryleung-solana
82945ba973
Optimize TpuConnection and its implementations and refactor connection-cache to not use dyn in order to enable those changes ( #23877 )
2022-03-24 11:40:26 -04:00
Jeff Washington (jwash)
5b916961b5
HashCalc uses self.accounts_cache ( #23890 )
2022-03-24 10:34:28 -05:00
Jeff Washington (jwash)
b22165ad69
hash calc uses self.filler_account_suffix ( #23887 )
2022-03-24 09:58:06 -05:00
Jeff Washington (jwash)
9022931689
calc hash uses self.num_hash_scan_passes ( #23883 )
2022-03-24 09:44:42 -05:00
Jeff Washington (jwash)
db5d68f01f
HashCalc uses self.accounts_hash_cache_path ( #23882 )
2022-03-24 09:31:55 -05:00
Jeff Washington (jwash)
3e22d4b286
calc hash uses self.thread_pool_clean ( #23881 )
2022-03-23 20:52:38 -05:00
Jeff Washington (jwash)
9e61fe7583
add AccountsHashConfig to manage parameters ( #23850 )
2022-03-23 13:44:23 -05:00
HaoranYi
db49b826f0
seperate blockstore metrics from window service metrics ( #23871 )
2022-03-23 13:38:17 -05:00
HaoranYi
7ff8ed869c
typos ( #23870 )
2022-03-23 13:36:55 -05:00
Jeff Washington (jwash)
b1280b670a
calculate_accounts_hash_without_index takes &self ( #23846 )
...
* calculate_accounts_hash_without_index takes &self
* Update runtime/src/snapshot_package.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-03-23 11:57:32 -05:00
Justin Starry
92462ae031
Manually serialize and use `send_wire_transaction` for votes ( #23826 )
...
* Revert "core: partial versioned transaction support for voting service"
This reverts commit eb3df4c20e
.
* Manually serialize vote tx before sending to TPU
2022-03-23 09:47:55 +08:00
Jon Cinque
7af48465fa
transaction-status: Add return data to meta ( #23688 )
...
* transaction-status: Add return data to meta
* Add return data to simulation results
* Use pretty-hex for printing return data
* Update arg name, make TransactionRecord struct
* Rename TransactionRecord -> ExecutionRecord
2022-03-22 23:17:05 +01:00
Yueh-Hsuan Chiang
ae75b1a25f
(LedgerStore) Add compression type ( #23578 )
...
This PR adds `--rocksdb-ledger-compression` as a hidden argument to the validator
for specifying the compression algorithm for TransactionStatus. Available compression
algorithms include `lz4`, `snappy`, `zlib`. The default value is `none`.
Experimental results show that with lz4 compression, we can achieve ~37% size-reduction
on the TransactionStatus column family, or ~8% size-reduction of the ledger store size.
2022-03-22 02:27:09 -07:00
Trent Nelson
eb3df4c20e
core: partial versioned transaction support for voting service
2022-03-21 22:59:05 -06:00
HaoranYi
45a7c6edfb
Fix typos and a small refactor ( #23805 )
...
* fix typo
* remove packet_has_more_unprocessed_transactions function
2022-03-21 18:35:31 -05:00
Will Hickey
c4ecfa5716
Bump version to v1.11 ( #23807 )
...
* Revert crossbeam_epoch to stable. 0.9.8 only works with nightly
* Remove unneeded unit expression
2022-03-21 17:40:50 -05:00
Pankaj Garg
5d03b188c8
Use QUIC client in voting service ( #23713 )
...
* Use QUIC client in voting service
* guard quic-client usage with a flag
* add measure to time the quic client
* move time measure outside if block
* remove quic vs UDP flag from voting service
2022-03-21 09:10:16 -07:00
Tao Zhu
71ea05c176
replace nested for_each with flat_map
2022-03-18 16:37:41 -05:00
Tao Zhu
1c369fb55f
Scan entire UnprocessedPacketBatches buffer to produce stake and locator of each packet
2022-03-18 16:37:41 -05:00
Yueh-Hsuan Chiang
f999eef452
(LedgerStore) Rename BlockstoreAdvancedOptions to LedgerColumnOptions ( #23764 )
...
This PR renames BlockstoreAdvancedOptions to LedgerColumnOptions, as we will
pass-down this struct to LedgerColumn to allow it to perform metric reporting.
2022-03-18 11:13:35 -07:00
Tao Zhu
56428be629
Not exposing inner cost_table to encapsulating implementation details,
...
making future change easier.
2022-03-18 12:58:43 -05:00
Tao Zhu
0ed23899e7
directly use compute_budget MAX_UNITS and DEFAULT_UNITS
2022-03-18 08:53:11 -05:00
Tao Zhu
a4cacf3389
add deterministic default cost
2022-03-18 08:53:11 -05:00
Tao Zhu
c478fe2047
add timing metrics, some renaming
2022-03-17 19:31:28 -05:00
Tao Zhu
fd515097d8
leader qos part 2: add stage to find sender stake, set to packet meta
2022-03-17 19:31:28 -05:00
Stephen Akridge
976b138e76
Add tx weighting stage
2022-03-17 19:31:28 -05:00
Will Hickey
2f58c9e501
Bump version to 1.10.4 ( #23743 )
2022-03-17 14:02:13 -05:00
Michael Vines
3773b753d1
Configure shrink paths during blockstore load
2022-03-15 23:08:07 -07:00
Michael Vines
ab373bb1a9
Refactor new_banks_from_ledger() into load and process steps
2022-03-15 23:08:07 -07:00
Michael Vines
2da4e3eb6c
Add --no-os-memory-stats-reporting
2022-03-15 17:07:40 -07:00
Michael Vines
dbc62f2e28
Use consistent variable naming for DropBankService
2022-03-15 17:07:13 -07:00
Michael Vines
d44f3d7216
Remove unhelpful log message
2022-03-15 17:07:13 -07:00
Tao Zhu
2d3501dff9
make upsert infallible op
2022-03-15 17:05:41 -05:00
dependabot[bot]
62b26f012e
chore: bump reqwest from 0.11.9 to 0.11.10 ( #23671 )
...
* chore: bump reqwest from 0.11.9 to 0.11.10
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.9 to 0.11.10.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.9...v0.11.10 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-15 12:54:23 -06:00
Tao Zhu
61cead9b9b
Remove injection of exit signal into cost_update_service
2022-03-15 09:58:56 -05:00
Tao Zhu
eb73dacd58
harden banking tests
2022-03-15 09:58:08 -05:00
Justin Starry
8c8f9694e0
Refactor: Sanitized transaction creation ( #23558 )
...
* Refactor: SanitizedTransaction::try_create optionally computes hash
* Refactor: Add SimpleAddressLoader
2022-03-15 12:02:22 +08:00
Tyera Eulberg
102dd68a03
Rename AccountsDb plugins to Geyser plugins ( #23604 )
2022-03-14 19:18:46 -06:00
Michael Vines
17cc095d28
Slot warping doesn't need to be in new_banks_from_ledger
2022-03-14 15:29:58 -07:00
Michael Vines
2e7ee0f177
Tower loading doesn't need to be in new_banks_from_ledger
2022-03-14 15:29:58 -07:00
Michael Vines
390dc24608
Create leader schedule before processing blockstore
2022-03-14 15:29:58 -07:00
Michael Vines
543d5d4a5d
Reduce new_banks_from_ledger arguments
2022-03-14 15:29:58 -07:00
Michael Vines
115f376465
Factor out bank_forks_utils::load_bank_forks()
2022-03-14 15:29:58 -07:00
Michael Vines
c2ce152be8
Inline do_process_blockstore_from_root
2022-03-14 15:29:58 -07:00
Tao Zhu
5ea6a1e500
code review
2022-03-14 13:14:27 -05:00
Tao Zhu
8590911b0a
Replace type alias with newtype for UnprocesedPacketBatches
2022-03-14 13:14:27 -05:00
Will Hickey
63bf0f66af
Bump version to 1.10.3 ( #23648 )
2022-03-14 11:18:45 -05:00
Brooks Prumo
7758c32035
Banking Stage drops transactions that'll exceed the total account data size limit ( #23537 )
2022-03-13 15:58:57 +00:00
Yueh-Hsuan Chiang
1e20bd8f9a
(LedgerStore) Include storage type as a tag in RocksDB metric reporting ( #23523 )
...
#### Summary of Changes
This PR further enables group by operation on storage type in blockstore_rocksdb_cfs metrics.
Such group-by allows us to further compare the performance metrics between rocks-level and
rocks-fifo.
To make things extensible, this PR introduces BlockstoreAdvancedOptions and move shred_storage_type.
All fields in BlockstoreAdvancedOptions will support group-by operation in blockstore_rocksdb_cfs.
Dependency: #23580
2022-03-11 15:17:34 -08:00
dependabot[bot]
b1da7cff66
chore: bump base64 from 0.12.3 to 0.13.0 ( #23616 )
...
* chore: bump base64 from 0.12.3 to 0.13.0
Bumps [base64](https://github.com/marshallpierce/rust-base64 ) from 0.12.3 to 0.13.0.
- [Release notes](https://github.com/marshallpierce/rust-base64/releases )
- [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md )
- [Commits](https://github.com/marshallpierce/rust-base64/compare/v0.12.3...v0.13.0 )
---
updated-dependencies:
- dependency-name: base64
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update Cargo.lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2022-03-11 16:01:15 -07:00
Brooks Prumo
d20dd21600
Sort tables in Cargo.toml files ( #23602 )
2022-03-11 11:05:57 -06:00
sakridge
9b591286d7
Revert "chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )" ( #23592 )
...
This reverts commit 3a0271c113
.
2022-03-11 09:12:27 +01:00
Will Hickey
b444836a97
Bump version to 1.10.2 ( #23597 )
2022-03-10 16:41:06 -06:00
Tao Zhu
35d1235ed0
- move `unprocessed_packet_batches` from `BankingStage` to its own ( #23508 )
...
module
- deserialize packets during receving and buffering
2022-03-10 18:47:46 +00:00
carllin
588414a776
Report even if slot begins and ends in process_buffered_packets() ( #23549 )
2022-03-09 23:42:35 -05:00
Tao Zhu
f68c5a274d
remove persist_cost_table code
2022-03-09 21:05:47 -07:00
Tao Zhu
9f71958d7d
Patch validator from loading persisted program costs
2022-03-09 21:05:47 -07:00
Brooks Prumo
9bbccbe27c
Use `AsRef<Path>` instead of `PathBuf` for parameters ( #23560 )
2022-03-09 16:08:33 -06:00
HaoranYi
a1c45d5acb
typo ( #23564 )
2022-03-09 15:11:21 -06:00
sakridge
7a9884c831
Quic limit connections ( #23283 )
...
* quic server limit connections
* bump per_ip
* Review comments
* Make the connections per port
2022-03-09 10:52:31 +01:00
dependabot[bot]
5c722519cf
chore: bump etcd-client from 0.8.3 to 0.8.4
...
Bumps [etcd-client](https://github.com/etcdv3/etcd-client ) from 0.8.3 to 0.8.4.
- [Release notes](https://github.com/etcdv3/etcd-client/releases )
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.8.3...v0.8.4 )
---
updated-dependencies:
- dependency-name: etcd-client
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-03-08 17:33:24 -08:00
Carl Lin
5a0cd05866
Revert "- estimate a program cost as 2 standard deviation above mean"
...
This reverts commit a25ac1c988
.
2022-03-08 17:18:44 -08:00
Carl Lin
9acbfa5eb1
Revert "use EMA in place of Welford"
...
This reverts commit 6587dbfa47
.
2022-03-08 17:18:44 -08:00
Carl Lin
c878c9e2cb
Revert "1. Persist to blockstore less frequently;"
...
This reverts commit 7aa1fb4e24
.
2022-03-08 17:18:44 -08:00
Carl Lin
0a17edcc1f
Revert "fix tests after merge"
...
This reverts commit ba2d83f580
.
2022-03-08 17:18:44 -08:00
Michael Vines
b719d6a2ad
`solana-validator set-identity` no longer writes a tower file unnecessarily
2022-03-08 15:34:23 -08:00
Justin Starry
3114c199bd
Add RPC support for versioned transactions ( #22530 )
...
* Add RPC support for versioned transactions
* fix doc tests
* Add rpc test for versioned txs
* Switch to preflight bank
2022-03-08 15:20:34 +08:00
dependabot[bot]
38d8bbb19c
chore: bump sysctl from 0.4.3 to 0.4.4 ( #23505 )
...
Bumps [sysctl](https://github.com/johalun/sysctl-rs ) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/johalun/sysctl-rs/releases )
- [Changelog](https://github.com/johalun/sysctl-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/johalun/sysctl-rs/commits )
---
updated-dependencies:
- dependency-name: sysctl
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-07 14:24:19 -07:00
HaoranYi
181fffb916
rename status filename to be consistent ( #23501 )
2022-03-07 17:34:35 +00:00
dependabot[bot]
3a0271c113
chore: bump dashmap from 4.0.2 to 5.1.0 ( #23372 )
...
* chore: bump dashmap from 4.0.2 to 5.1.0
Bumps [dashmap](https://github.com/xacrimon/dashmap ) from 4.0.2 to 5.1.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases )
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.1.0 )
---
updated-dependencies:
- dependency-name: dashmap
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-03-07 00:00:26 -05:00
Yueh-Hsuan Chiang
b8b7163b66
(Ledger Store) Report RocksDB Column Family Metrics ( #22503 )
...
This PR enables blockstore to periodically report RocksDB column family properties.
The reported properties are under blockstore_rocksdb_cfs, and the properties also
support group by operation on cf_name.
2022-03-05 16:13:03 -08:00
Yueh-Hsuan Chiang
62d2a4cd88
Make ShredStorageType::RocksLevel public ( #23272 )
...
#### Summary of Changes
This PR adds two hidden arguments to the validator that allow users to use RocksDB's FIFO compaction for storing shreds.
--shred-storage <SHRED_STORAGE>
EXPERIMENTAL: Controls how RocksDB compacts shreds. *WARNING*: You will lose your ledger data
when you switch between options. Possible values are: 'level': stores shreds using RocksDB's default (level)
compaction. 'fifo': stores shreds under RocksDB's FIFO compaction. This option is more efficient on
disk-write-bytes of the ledger store. [default: level] [possible values: level, fifo]
--shred-storage-size <SHRED_STORAGE_SIZE_BYTES>
The shred storage size in bytes. The suggested value is 50% of your ledger storage size in bytes. [default:
268435456000]
2022-03-03 12:43:58 -08:00
dependabot[bot]
f2fa49a771
chore: bump lru from 0.7.2 to 0.7.3 ( #23462 )
...
Bumps [lru](https://github.com/jeromefroe/lru-rs ) from 0.7.2 to 0.7.3.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases )
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.2...0.7.3 )
---
updated-dependencies:
- dependency-name: lru
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-03 12:01:49 -07:00
dependabot[bot]
61d7bdd66f
chore: bump serde_json from 1.0.78 to 1.0.79 ( #23461 )
...
* chore: bump serde_json from 1.0.78 to 1.0.79
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.78 to 1.0.79.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.78...v1.0.79 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2022-03-02 23:38:06 -07:00
dependabot[bot]
fe7604589d
chore: bump retain_mut from 0.1.5 to 0.1.7 ( #23450 )
...
Bumps [retain_mut](https://github.com/upsuper/retain_mut ) from 0.1.5 to 0.1.7.
- [Release notes](https://github.com/upsuper/retain_mut/releases )
- [Commits](https://github.com/upsuper/retain_mut/commits )
---
updated-dependencies:
- dependency-name: retain_mut
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-02 23:15:33 +00:00
Will Hickey
1a99251498
Bump version to 1.10.1 ( #23453 )
2022-03-02 13:47:01 -06:00
Jeff Washington (jwash)
26aa18b3f3
fmt ( #23448 )
2022-03-02 11:54:58 -06:00
HaoranYi
41f78b9925
small optimization. use shift for pow of 2. ( #22975 )
2022-03-02 09:11:12 -06:00
HaoranYi
4f0070a5c6
unittest for bind two consecutive ports ( #23008 )
...
* minor fix of comments in fork-selection tests
* fix doc link
* add unittest for bind_two_consecutive_in_range
2022-03-02 09:10:29 -06:00
HaoranYi
8de88d0a55
Refactor packet_threshold adjustment code into its own struct ( #23216 )
...
* refactor packet_threshold adjustment code into own struct and add unittest for it
* fix a typo in error message
* code review feedbacks
* another code review feedback
* Update core/src/ancestor_hashes_service.rs
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
* share packet threshold with repair service (credit to carl)
Co-authored-by: Trent Nelson <trent.a.b.nelson@gmail.com>
2022-03-02 09:09:06 -06:00
HaoranYi
86e2f728c3
Fix a batch limits bug in banking ( #23327 )
...
* add thread index in thread name for debugging
* fix batch_limit
* use NUM_VOTE_THREAD instead of hardcoded number (credit to carllin)
2022-03-02 09:08:08 -06:00
Tyera Eulberg
d3ebe8d8f5
Remove unneeded jsonrpc dependencies/features; update do-audit ( #23436 )
...
* Update generic-array note
* Remove unneeded jsonrpc deps
* Remove unneeded jsonrpc features
* Rewrite slot-update test without websocket crate
* Rewrite rpc-subscription test without websocket crate, and remove jsonrpc deps
* Update expected balance to accommodate rent-exempt minimum transfer amount
* Remove obsolete audit ignores
2022-03-02 01:42:01 -07:00
Jeff Biseda
c69e3b73ff
bench get_retransmit_peers ( #23292 )
2022-03-01 19:10:29 -08:00
dependabot[bot]
ce4d579499
chore: bump serial_test from 0.5.1 to 0.6.0 ( #23414 )
...
Bumps [serial_test](https://github.com/palfrey/serial_test ) from 0.5.1 to 0.6.0.
- [Release notes](https://github.com/palfrey/serial_test/releases )
- [Commits](https://github.com/palfrey/serial_test/compare/v0.5.1...v0.6.0 )
---
updated-dependencies:
- dependency-name: serial_test
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-01 12:13:40 -07:00
Brooks Prumo
533eca3b4c
Simplify replay_blockstore_into_bank() ( #23282 )
2022-02-25 06:57:04 -06:00
Trent Nelson
d4292774c5
checks
2022-02-25 08:05:28 +00:00
dependabot[bot]
98f059e89c
chore: bump reqwest from 0.11.6 to 0.11.9 ( #23337 )
...
* chore: bump reqwest from 0.11.6 to 0.11.9
Bumps [reqwest](https://github.com/seanmonstar/reqwest ) from 0.11.6 to 0.11.9.
- [Release notes](https://github.com/seanmonstar/reqwest/releases )
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md )
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.6...v0.11.9 )
---
updated-dependencies:
- dependency-name: reqwest
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-02-24 22:40:01 -07:00
Justin Starry
d0e85c293f
Fix rustfmt check ( #23296 )
2022-02-23 16:38:53 +08:00
Gavin Chan
20d031e2b8
Refactor ExecuteTimings w/ enum-indexed array ( #23085 )
2022-02-22 14:46:56 -08:00
Tyera Eulberg
7e08ae1d0c
Revert "Add simulation detection countermeasure ( #22880 )" ( #23261 )
...
This reverts commit c42b80f099
.
2022-02-21 21:15:37 +00:00
Yueh-Hsuan Chiang
8c872e9ce0
(LedgerStore/FIFO) Refactor FIFO options and sanity check. ( #23131 )
2022-02-19 00:58:38 -08:00
buffalu
70ebab2c82
Add rustfmt.toml and `cargo fmt` ( #23238 )
...
* fmt
* formatted
Co-authored-by: Lucas B <buffalu@jito.network>
2022-02-19 13:32:29 +08:00
Yueh-Hsuan Chiang
1add82aa9e
(Ledger Store Benchmark) Add flags for checking shred storage size. ( #22451 )
2022-02-18 19:35:28 -08:00
carllin
619335df1a
Add execute timings ( #23097 )
2022-02-17 01:14:32 -05:00
anatoly yakovenko
83d31c9e65
shrink batches when over 80% of the space is wasted ( #23066 )
...
* shrink batches when over 80% of the space is wasted
2022-02-16 08:18:17 -08:00
Jeff Biseda
115d71536b
forward_buffered_packets return packet count in error path ( #23167 )
2022-02-16 07:46:32 -08:00
Michael Vines
a6d736572c
`solana-validator set-identity` now supports the `--require-tower` flag
2022-02-15 19:45:00 -08:00
Tao Zhu
03bf66a51b
flag end-of-slot when poh bank is gone
2022-02-15 15:01:27 -06:00
Ashwin Sekar
ab92578b02
Fix the flaky test test_restart_tower_rollback ( #23129 )
...
* Add flag to disable voting until a slot to avoid duplicate voting
* Fix the tower rollback test and remove it from flaky.
2022-02-15 13:19:34 -07:00
Michael Vines
c42b80f099
Add simulation detection countermeasure ( #22880 )
...
* Add simulation detection countermeasures
* Add program and test using TestValidator
* Remove incinerator deposit
* Remove incinerator
* Update Cargo.lock
* Add more features to simulation bank
* Update Cargo.lock per rebase
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2022-02-15 13:09:59 +01:00
Yueh-Hsuan Chiang
824446710b
(Ledger Store Benchmark) Enable FIFO compmaction in the benchmark. ( #22162 )
2022-02-12 00:43:25 -08:00
Lijun Wang
c04438be4b
Retaining transaction logs when transaction plugin is loaded. ( #22874 )
...
Transaction logs are not being saved to the database through the plugin interface.
Summary of Changes
Retain the transaction logs when transaction notification plugin is loaded.
Fixes #
lijunwangs/solana-accountsdb-plugin-postgres#6
2022-02-11 20:29:07 -08:00
carllin
2f9e30a1f7
Introduce slot-specific packet metrics ( #22906 )
2022-02-11 03:07:45 -05:00
dependabot[bot]
59e64d5e99
chore: bump raptorq from 1.6.4 to 1.6.5 ( #23042 )
...
Bumps [raptorq](https://github.com/cberner/raptorq ) from 1.6.4 to 1.6.5.
- [Release notes](https://github.com/cberner/raptorq/releases )
- [Commits](https://github.com/cberner/raptorq/compare/v1.6.4...v1.6.5 )
---
updated-dependencies:
- dependency-name: raptorq
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-10 19:07:07 -07:00
Justin Starry
d5dec989b9
Enforce tx metadata upload with static types ( #23028 )
2022-02-10 13:28:18 +08:00
Yueh-Hsuan Chiang
1b287f1b59
(Ledger Cleanup) Add code comments for ledger_cleanup. ( #22807 )
2022-02-08 22:48:56 -08:00
Tao Zhu
ba2d83f580
fix tests after merge
2022-02-08 16:18:23 -06:00
Tao Zhu
7aa1fb4e24
1. Persist to blockstore less frequently;
...
2. reduce alpha for EMA to 1 percent to have roughly 200 data points for estimatio
2022-02-08 16:18:23 -06:00
Tao Zhu
6587dbfa47
use EMA in place of Welford
2022-02-08 16:18:23 -06:00
Tao Zhu
a25ac1c988
- estimate a program cost as 2 standard deviation above mean
...
- replaced get_average / get_mode with get_default to assign max units to unknown program
2022-02-08 16:18:23 -06:00
Tao Zhu
e52e48076e
bench should update leader schedule cache ( #22991 )
2022-02-08 02:28:28 +00:00
Ashwin Sekar
5acf0f6331
Add feature gate for new vote instruction and plumb through replay ( #21683 )
...
* Add feature gate for new vote instruction and plumb through replay
Add tower versions
* Add check for slot hashes history
* Update is_recent check to exclude voting on hard fork root slot
* Move tower rollback test to flaky and ignore it until #22551 lands
2022-02-07 14:06:19 -08:00
behzad nouri
27aaf9df85
removes VoteTracker::new in favor of VoteTracker::default ( #22941 )
...
VoteTracker::new does not need a bank and is so redundant:
https://github.com/solana-labs/solana/blob/5a230f418/core/src/cluster_info_vote_listener.rs#L103-L107
2022-02-04 19:01:59 +00:00
sakridge
5a230f418d
Add quic port for accepting transactions ( #22753 )
...
using quinn library
streamer: Sign TLS cert with validator identity key
Handle multiple incoming chunks
2022-02-04 15:27:09 +01:00
Tao Zhu
4bec182b32
Allow buffered packets be consumed if bank is active, regardless leader schedule ( #22913 )
2022-02-03 21:29:41 +00:00
Justin Starry
60af1a4cce
Refactor: Add trait for loading addresses ( #22903 )
2022-02-03 11:00:12 +00:00