Ashwin Sekar
17b64005d3
Add more logging and documentation to flaky optimistic confirmation tests ( #29418 )
...
* Revert "add retry for flakey local cluster test (#29228 )"
This reverts commit 7a97121747
.
* Add logging for repair
2022-12-27 10:47:45 -07:00
behzad nouri
456d06785d
experiments different turbine fanouts for propagating shreds ( #29393 )
...
The commit allocates 2% of slots to running experiments with different
turbine fanouts based on the slot number.
The experiment is feature gated with an additional feature to disable
the experiment.
2022-12-26 14:18:56 +00:00
Ashwin Sekar
f2ba16ee87
Plumb dumps from replay_stage to repair ( #29058 )
...
* Plumb dumps from replay_stage to repair
When dumping a slot from replay_stage as a result of duplicate or
ancestor hashes, properly update repair subtrees to keep weighting and
forks view accurate.
* add test
* pr comments
2022-12-25 09:58:30 -07:00
behzad nouri
558292466b
rolls back merkle shreds on testnet ( #29340 )
...
https://github.com/solana-labs/solana/pull/29339
adds hash domain to merkle shreds. In order to merge that change, need
to temporarily disable merkle shreds on testnet.
2022-12-20 18:33:48 +00:00
Brooks
053775ad77
Elides unnecessary lifetimes ( #29299 )
2022-12-20 12:44:17 -05:00
Tao Zhu
c657f42d77
remove a wrapper function ( #29305 )
2022-12-19 16:10:16 +00:00
Brennan Watt
86b2e545e1
Prune redundant const SLOT_MS ( #29278 )
...
* Alias redundant const SLOT_MS to DEFAULT_MS_PER_SLOT
* Slate SLOT_MS for deprecation
* Add doc comments
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-12-16 08:05:09 -08:00
Jeff Biseda
a44ea779bd
add support for a repair protocol whitelist ( #29161 )
2022-12-15 19:24:23 -08:00
dependabot[bot]
dca5d7f9b4
chore: bump test-case from 2.1.0 to 2.2.2 ( #28184 )
...
Bumps [test-case](https://github.com/frondeus/test-case ) from 2.1.0 to 2.2.2.
- [Release notes](https://github.com/frondeus/test-case/releases )
- [Changelog](https://github.com/frondeus/test-case/blob/master/CHANGELOG.md )
- [Commits](https://github.com/frondeus/test-case/compare/v2.1.0...v2.2.2 )
---
updated-dependencies:
- dependency-name: test-case
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 16:07:50 +00:00
Jeff Washington (jwash)
b95835143e
remove AccountsBackgroundService::new(caching_enabled) ( #29234 )
2022-12-13 07:18:02 -08:00
Jeff Washington (jwash)
bb0bfc4214
remove bank_from_latest_snapshot_archives(caching_enabled) ( #29238 )
2022-12-13 07:16:24 -08:00
Jeff Washington (jwash)
fec8f61566
remove ProcessOptions::accounts_db_caching_enabled ( #29217 )
2022-12-12 20:25:00 -08:00
Jeff Washington (jwash)
2c2324f4ea
remove caching_enabled from Bank::new_with_paths_for_tests ( #29214 )
2022-12-12 15:30:46 -08:00
Brooks Prumo
1b0aaf1607
Makes a new PathBuf instead of moving the test's TempDir ( #29220 )
2022-12-12 18:29:36 -05:00
apfitzge
249607dbfe
Use a different tempdir for unpacking snapshots ( #29219 )
2022-12-12 17:26:52 -06:00
Brooks Prumo
391f68da61
Uses Storages to calculate accounts hash in EAH warp tests ( #29192 )
2022-12-12 13:30:23 -05:00
Jeff Biseda
88a8f40bd2
apply [limit repairs to top staked... #28673 ] to non-MainnetBeta clusters ( #29163 )
2022-12-11 15:52:41 -08:00
behzad nouri
4ee318b2b2
fixes rust code formatting in core/src/consensus.rs ( #29204 )
2022-12-11 23:20:52 +00:00
Jeff Washington (jwash)
631a98a3b6
warp_from_parents works with write_cache enabled ( #29185 )
2022-12-09 14:28:18 -08:00
apfitzge
cd9f1f1862
Typo/filter_and_forward_with_account_limits ( #29183 )
2022-12-09 16:22:25 -06:00
Jeff Washington (jwash)
560143a267
remove ValidatorConfig.caching_enabled ( #29172 )
2022-12-09 11:31:55 -08:00
Lijun Wang
ecea802fe6
Bidirectional quic communication support ( #29155 )
...
* Support bi-directional quic communication, use the same endpoint for the quic server and client
This is needed for supporting using quic for repair
* Added comments on the bi-directional communication tests
* Removed some debug logs
* clippy issue
2022-12-09 10:59:43 -08:00
Jeff Washington (jwash)
6a90abd056
remove handle_snapshot_requests.caching_enabled ( #29174 )
2022-12-09 10:51:44 -08:00
Jeff Washington (jwash)
45ba5ef6fd
remove bank_from_snapshot_archives caching_enabled ( #29171 )
2022-12-09 10:45:21 -08:00
Jeff Washington (jwash)
ec5098a723
remove bank_test_config_caching_enabled ( #29170 )
2022-12-09 08:28:02 -08:00
HaoranYi
ca5d8c4b4d
refactor sysmonitor config ( #29132 )
2022-12-09 07:43:03 -06:00
Jason Davis
049fb3d725
Remove an unnecessary clone of a PohConfig inside Validator::new
2022-12-07 13:03:14 -06:00
Jason Davis
8f24ceffbd
Removed Arcs from PohConfig parameters and pass the struct by reference only
2022-12-07 10:52:07 -06:00
HaoranYi
582397ad48
fix solRptLdgrStat thread hang ( #29118 )
2022-12-06 17:09:56 -06:00
HaoranYi
33b15240ac
Revert #28945 ( #29127 )
...
revert #28945
2022-12-06 17:08:56 -06:00
steviez
aeb6b53502
Remove unused Option<> around ValidatorConfig's SnapshotConfig ( #29090 )
...
Remove Option<> around ValidatorConfig's SnapshotConfig
The SnapshotConfig is required and is currently hard-coded to be a
Some().
2022-12-06 22:47:55 +00:00
behzad nouri
df7fd8ae5f
patches rust code formatting in core/src/replay_stage.rs ( #29123 )
2022-12-06 22:09:57 +00:00
behzad nouri
9524c9dbff
patches errors from clippy::uninlined_format_args
...
https://rust-lang.github.io/rust-clippy/master/index.html#uninlined_format_args
2022-12-06 19:32:15 +00:00
behzad nouri
9433c06745
patches errors from clippy::unchecked_duration_subtraction
...
https://rust-lang.github.io/rust-clippy/master/index.html#unchecked_duration_subtraction
2022-12-06 19:32:15 +00:00
Haoran Yi
bbd49acb2f
fix merge error
2022-12-06 13:31:50 -06:00
haoran
7a512d7f27
report number of open files
2022-12-06 13:31:50 -06:00
haoran
412cf3df27
sort deps
2022-12-06 13:31:50 -06:00
haoran
f716cad4af
don't use procfs as it is not supported on mac and windows.
...
make open_fd stats only on linux platform
2022-12-06 13:31:50 -06:00
haoran
fc97d818b6
share code
2022-12-06 13:31:50 -06:00
haoran
36dc3a457f
get mmap with wc-l
2022-12-06 13:31:50 -06:00
haoran
dd81af9fff
increase fd report interval to 3 minutes
2022-12-06 13:31:50 -06:00
haoran
0ce507c20d
refactor SystemMonitorReportConfig
2022-12-06 13:31:50 -06:00
Haoran Yi
914f7bd85d
fix
2022-12-06 13:31:50 -06:00
Haoran Yi
1635b99486
add mmmap file count
2022-12-06 13:31:50 -06:00
Haoran Yi
e1ba5a2a63
add monitoring for open file descriptors stat
2022-12-06 13:31:50 -06:00
Tao Zhu
7ed22f7b18
Remove gate from accepting packets for forwarding ( #29049 )
2022-12-06 12:13:01 -06:00
Jon Cinque
b1340d77a2
sdk: Make Packet::meta private, use accessor functions ( #29092 )
...
sdk: Make packet meta private
2022-12-06 12:54:49 +01:00
apfitzge
fd3b5d08d7
Refactor/banking_stage_make_decision_consume_bank ( #28946 )
2022-12-02 10:07:01 -06:00
Tao Zhu
5850af5316
Refactor to remove requested_cu from cost_trarcker ( #29015 )
...
* refactor cost tracker by removing requested_cu from it, call sites to use cost_model forr consistency
* review fix
2022-12-02 00:25:09 +00:00
steviez
3c42c87098
Remove obsoleted return value from Blockstore insert shred method ( #28992 )
2022-12-01 11:17:46 -06:00
steviez
b6dce6cf3b
Move BlockstoreInsertionMetrics field update to blockstore.rs ( #28991 )
...
The num_repair field is only blockstore insertion metric being updated
outside of Blockstore::insert() call chain; move the update to insert()
with the rest of the fields in BlockstoreInsertionMetrics struct.
2022-11-30 11:46:35 -06:00
Ashwin Sekar
edacd3c411
Add dump_node to update stake for heaviest subtrees ( #28827 )
...
* Add dump_node to update stake for heaviest subtrees
Additionally refactor subtrees to store children as a hashset
* Add a more complicated forks test
* chose -> choose
* remove is_dumped flag and reuse latest_invalid_ancestor instead
2022-11-30 09:26:13 -08:00
apfitzge
4d338ed882
Bugfix/mi_remove_never_entries ( #28978 )
2022-11-29 16:00:21 -06:00
Ashwin Sekar
0d0a491f27
More documentation + small refactor for RepairService ( #28933 )
2022-11-28 19:46:06 -08:00
Tao Zhu
9f370475d4
remove obsoleted comment ( #28960 )
2022-11-28 13:39:40 -06:00
behzad nouri
7d99cddb9f
dedups turbine retransmit peers by tvu socket addresses ( #28944 )
...
No need to send duplicate shreds if several nodes have the same tvu
socket address because they are behind a relayer or whatever.
2022-11-28 19:23:02 +00:00
HaoranYi
7e87998091
reduce memory usage report freq to 1 per 5s ( #28327 )
2022-11-28 19:08:06 +00:00
apfitzge
bdd162492c
Feature/multi-iterator-scanner-read-locks ( #28862 )
2022-11-28 11:23:04 -06:00
Brooks Prumo
9327658007
Promotes accounts hash to a strong type ( #28930 )
2022-11-28 10:09:47 -05:00
Brooks Prumo
638b26ea65
Renames EAH test fn ( #28939 )
2022-11-23 05:18:50 +00:00
apfitzge
38f7122605
separate make_decision in BankingStage ( #28884 )
2022-11-22 19:01:09 -06:00
Maximilian Schneider
c8b0c3ede9
Update cost model to use requested_cu instead of estimated cu #27608 ( #28281 )
...
* Update cost model to use requested_cu instead of estimated cu #27608
* remove CostUpdate and CostModel from replay/tvu
* revive cost update service to send cost tracker stats
* CostModel is now static
* remove unused package
Co-authored-by: Tao Zhu <tao@solana.com>
2022-11-22 11:55:56 -06:00
apfitzge
637e8a937b
clean up: remove my_pubkey arg from consume_buffered_packets ( #28888 )
2022-11-22 11:40:04 -06:00
Jeff Washington (jwash)
20d8b5e98b
default some tests to write cache = true ( #28917 )
2022-11-21 15:53:39 -08:00
apfitzge
dd723210ca
remove unnecessary clippy attributes ( #28891 )
2022-11-21 12:54:54 -06:00
behzad nouri
d43b001189
rolls out merkle shreds to ~20% of testnet ( #28905 )
2022-11-21 16:20:02 +00:00
Michael Vines
c6927151ef
Sort offline/wrong-shred nodes by stake weight while waiting for supermajority ( #28872 )
2022-11-18 15:26:21 -08:00
Jeff Washington (jwash)
f22104d46b
use write cache by default in some tests ( #28876 )
2022-11-18 14:35:52 -08:00
apfitzge
a636038fff
Clean up: banking_stage_prepare_sanitized_batch ( #28841 )
...
Use measure! for bank.prepare_sanitized_batch_with_results
2022-11-18 14:04:44 -06:00
Tyera
c32377b5af
Split out quic- and udp-client definitions ( #28762 )
...
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there
* Dedupe thin_client modules
* Dedupe tpu_client modules
* Move TpuClient to TpuConnectionCache
* Move ThinClient to TpuConnectionCache
* Move TpuConnection and quic/udp trait implementations back to solana-client
* Remove enum_dispatch from solana-tpu-client
* Move udp-client to its own crate
* Move quic-client to its own crate
2022-11-18 12:21:45 -07:00
apfitzge
88e6ea37d9
refactor: move more BankingStage cost_model stuff into qos_service ( #28840 )
2022-11-17 14:03:17 -06:00
Andrew Fitzgerald
ee2f760d3d
MultiIteratorScanner - improve banking stage performance with high contention
2022-11-17 10:54:12 -06:00
Brooks Prumo
2bafb0cb12
Requires EAH state cannot be Invalid ( #28817 )
2022-11-17 11:01:01 -05:00
Brooks Prumo
ae0bb44401
Fixes test_snapshots_with_background_services ( #28848 )
2022-11-17 00:49:28 -05:00
Jeff Biseda
17ee3349f8
limit repairs to top staked requests in batch ( #28673 )
2022-11-16 16:30:41 -08:00
Ashwin Sekar
ddf4ff2d26
Repair service documentation ( #28592 )
...
* repair doc update
* tree_root rename
* remove extra todo
2022-11-16 02:38:07 +00:00
Jeff Biseda
e10d958352
signed repair request test fixes/cleanup ( #28691 )
2022-11-15 16:46:17 -08:00
Brooks Prumo
d798e751a0
Disables EAH with short epochs ( #28803 )
2022-11-15 13:26:19 -05:00
Brooks Prumo
d4cf18421d
Use 400 slots-per-epoch in EAH tests ( #28801 )
2022-11-14 17:49:20 -05:00
Brooks Prumo
0bfea02056
Snapshots wait for EAH calculations to complete ( #28777 )
2022-11-14 11:34:44 -06:00
Tao Zhu
e5ae0b3371
check is_forwarded packet earlier ( #28159 )
...
* check and filter is_forwarded packet earlier
* review fix: renaming; and rebase
2022-11-11 23:32:03 +00:00
Brooks Prumo
4d6653598b
Upgrades to Rust 1.65.0 ( #28741 )
2022-11-09 17:15:03 -05:00
Brooks Prumo
d1ba42180d
clippy for rust 1.65.0 ( #28765 )
2022-11-09 19:39:38 +00:00
Brooks Prumo
9e1cdc7e60
Enables not taking a bank snapshot ( #28756 )
2022-11-09 12:43:33 -05:00
Brooks Prumo
d4c2900590
Removes `snapshot_bank()` wrapper fn ( #28753 )
2022-11-07 15:09:31 +00:00
Brooks Prumo
0b9426e734
Simplifies AHV's `test_max_hashes()` ( #28754 )
2022-11-07 02:32:33 +00:00
Brooks Prumo
064cfc70d2
Removes cluster_type from AccountsPackage ( #28725 )
2022-11-02 18:21:13 -04:00
Brooks Prumo
d0f639745a
Uses AccountsPackage::default_for_tests() in AHV tests ( #28723 )
2022-11-02 14:13:35 -04:00
Lijun Wang
f156bc12ca
Enforce stream receive timeout ( #28513 )
...
In the quic server handle_connection, when we timed out in receiving the chunks, we loop forever to wait for the chunk. If the client never provide another chunk, the server can hopelessly wait for that chunk and wasting server resources. Instead WAIT_FOR_CHUNK_TIMEOUT_MS is introduced to bound this to 10 seconds at maximum. The stream will be dropped if it times out.
2022-11-02 10:09:32 -07:00
Brooks Prumo
59bf1809fe
Uses SnapshotHash type in snapshot archive fields ( #28681 )
2022-10-31 14:28:35 -04:00
Dmitri Makarov
34865d032c
chore: update Solana docs and code comments that specify "BPF" to "SBF"
2022-10-31 14:14:25 -04:00
Brooks Prumo
37507a2de6
Removes EAH parameter from serde_snapshot::reserialize_bank() ( #28669 )
2022-10-31 09:43:17 -04:00
sakridge
340ad68223
Banking stage refactor commit transactions ( #28660 )
...
* Refactor commit transactions step
* Cleanup token pre-balances
* Collect prebalances together
* Collect pre/post balances in separate function
* Fix clippy
2022-10-29 21:36:57 +02:00
steviez
6b93d05c37
Add LedgerCleanupService::find_slots_to_clean() test ( #28656 )
...
Add a test to better exercise find_slots_to_clean(), as well as a minor
bug fix to this method that was found as a result of writing test.
2022-10-29 00:55:21 +02:00
apfitzge
22ce49ae7f
Maintain original queue capacity for unprocessed packet buffer ( #28661 )
2022-10-28 16:37:21 -05:00
apfitzge
0a148b2bf7
remove unused handle_retryable_packets_elapsed ( #28355 )
2022-10-28 16:36:41 -05:00
Brooks Prumo
5a3d252899
Renames fn to Bank::update_accounts_hash_for_tests() ( #28620 )
2022-10-28 14:33:05 -04:00
steviez
2272fd807e
Remove Blockstore manual compaction code ( #28409 )
...
The manual Blockstore compaction that was being initiated from
LedgerCleanupService has been disabled for quite some time in favor of
several optimizations.
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2022-10-28 10:39:00 +02:00
Ashwin Sekar
ae557a9eb5
Exit when stuck in an unrecoverable repair/purge loop ( #28596 )
...
* Exit when stuck in an unrecoverable repair/purge loop
* add tests
2022-10-27 20:06:06 -07:00
apfitzge
340d3b5468
rename and change capacity on unprocessed transaction storage - max_receive_size ( #28586 )
2022-10-26 10:03:47 -05:00