* Notify replay of pruned duplicate confirmed slots
* Ingest replay signal and run ancestor hashes for pruned
* Forward PDC to ancestor hashes and ingest pruned dumps from ancestor hashes service
* Add local-cluster test
* Move entry_notifier_interface
* Add EntryNotifierService
* Use descriptive struct in sender/receiver
* Optionally initialize EntryNotifierService in validator
* Plumb EntryNotfierSender into Tvu, blockstore_processor
* Plumb EntryNotfierSender into Tpu
* Only return one option when constructing EntryNotifierService
Working towards LegacyContactInfo => ContactInfo migration, the commit
hides some implementation details of LegacyContactInfo and expands API
parity with the new ContactInfo.
* Add a type_select param for purge_old_bank_snapshots
* Use flags to make the function calls more readable
* Remove the extra purge calls
* replace select_type with filter_by_type
* Add test
* Use matches
* Fix CI test on reference
* use match and call do_purge once
* Let bank_snapshots_dir be TempDir
* remove account_paths in the test
* replace bank with _bank
* Remove create_snapshot_dirs_for_tests, will take the lastest from master
* Fix merge errors
* refactor out fault hash inject output AccountsHashVerifier
* refactor faught injector out of AccountHashVerifier
* use type alias
* Apply suggestions from code review
Co-authored-by: Brooks <brooks@prumo.org>
* move type alias
* rename
---------
Co-authored-by: Brooks <brooks@prumo.org>
`poh_verify` actually disables transaction signature, tick count and
built in program argument verifications as well. It is somewhat
confusing to call it `poh_verify`.
* Add accounts hard-link files into the bank snapshot directory
* Small adjustments and fixes.
* Address some of the review issues
* Fix compilation issues
* Change the latest slot snapshot storage from VecDeque to Option
* IoWithSourceAndFile and expanded comments on accounts
* last_slot_snapshot_storages in return value
* Update comments following the review input
* rename dir_accounts_hard_links to hard_link_path
* Add dir_full_state flag for add_bank_snapshot
* Let appendvec files hardlinking work with multiple accounts paths across multiple partitions
* Fixes for rebasing
* fix tests which generates account_path without adding run/
* rebasing fixes
* fix account path test failures
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* update with more review comments
* handle error from create_accounts_run_and_snapshot_dirs
* fix rebasing duplicate
* minor accounts_dir path cleanup
* minor cleanup, remove commented code
* misc review comments
* build error fix
* Fix test_incremental_snapshot_download_with_crossing_full_snapshot_interval_at_startup
* fix build error on MAX_BANK_SNAPSHOTS_TO_RETAIN
* rebase fix, update hardlink filename
* minor comment spelling fix
* rebasing fixes
* fix rebase issues; with_extension
* comments changes for review
* misc minor review issues
* bank.fill_bank_with_ticks_for_tests
* error handling on appendvec path
* fix use_jit
* minor comments refining
* Remove type AccountStorages
* get_account_path_from_appendvec_path return changed to Option
* removed appendvec_path.to_path_buf in create_accounts_run_and_snapshot_dirs
* add test_get_snapshot_accounts_hardlink_dir
* update last_snapshot_storages comment
* update last_snapshot_storages comment
* symlink map_err
* simplify test_get_snapshot_accounts_hardlink_dir with fake paths
* log last_snapshot_storages at the end of the loop
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
* Clean up files in the old account_path before trasitioning to the new run path
* try_exist and accounts_dir removing extra
* sync rmdir, is_dir check
* handle the account_path not deletable case
* Add run parent directory for accounts files
* fix test test_concurrent_snapshot_packaging
* review comments. renamed the path setup function
* Addressed most of the review comments
* remove explict type def for map result
* handle create_accounts_run_and_snapshot_dirs error with expect
* update with more review comments
* minor fixes from review comments
* simplify account_filename option assignment
* handle error from create_accounts_run_and_snapshot_dirs
* use then instead of then_some for lazy evaluation
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there
* Dedupe thin_client modules
* Dedupe tpu_client modules
* Move TpuClient to TpuConnectionCache
* Move ThinClient to TpuConnectionCache
* Move TpuConnection and quic/udp trait implementations back to solana-client
* Remove enum_dispatch from solana-tpu-client
* Move udp-client to its own crate
* Move quic-client to its own crate
* Split out voting and banking threads in banking stage
Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.
* Update local cluster test to use new Vote ix
* Encapsulate transaction storage filtering better
* Address pr comments
* Commit cargo lock change
* clippy
* Remove unsafe impls
* pr comments
* compute_sanitized_transaction -> build_sanitized_transaction
* &Arc -> Arc
* Move test
* Refactor metrics enums
* clippy
* Fixed the local-cluster test case test_optimistic_confirmation_violation_detection, which is an existing test issue exposed when we disable the udp based tpu port. In the test, when we restart the node, we restarted the entry point validator. And when the gossip port of the restarted node changes, the two nodes are not able to talk to each other via gossip. I have changed the restart the logic to make the second validator the heavier node and restart it instead of the entypoint one (the first node).
* Addressed a review comment from Carl
* Fix test_bench_tps_local_cluster_solana
* Remove #[ignore] annotations from dos tests (which are also fixed by this change)
* Remove #[ignore] annotations from local cluster tests (which are also fixed by this change)
* Keypair: implement clone()
This was not implemented upstream in ed25519-dalek to force everyone to
think twice before creating another copy of a potentially sensitive
private key in memory.
See https://github.com/dalek-cryptography/ed25519-dalek/issues/76
However, there are now 9 instances of
Keypair::from_bytes(&keypair.to_bytes())
in the solana codebase and it would be preferable to have a function.
In particular since this also comes up when writing programs and can
cause users to either start messing with lifetimes or discover the
from_bytes() workaround themselves.
This patch opts to not implement the Clone trait. This avoids automatic
use in order to preserve some of the original "let developers think
twice about this" intention.
* Use Keypair::clone
Shred slot and parent are not verified until window-service where
resources are already wasted to sig-verify and deserialize shreds.
This commit moves above verification to earlier in the pipeline in fetch
stage.
* Make sure to root local slots even with hard fork
* Address review comments
* Cleanup a bit
* Further clean up
* Further clean up a bit
* Add comment
* Tweak hard fork reconciliation code placement
* client: Remove static connection cache, plumb it instead
* Add TpuClient::new_with_connection_cache to not break downstream
* Refactor get_connection and RwLock into ConnectionCache
* Fix merge conflicts from new async TpuClient
* Remove `ConnectionCache::set_use_quic`
* Move DEFAULT_TPU_USE_QUIC to client, use ConnectionCache::default()
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.
#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.
By doing this, we address the mutual dependency and also make the code cleaner.
Refactor the thin_client::create_client to take addresses separately instead of as a tuple
Co-authored-by: Bijie Zhu <bijiezhu@Bijies-MBP.cable.rcn.com>
* Add quic-client module to send transactions via quic, abstracted behind the TpuConnection trait (along with a legacy UDP implementation of TpuConnection) and change thin-client to use TpuConnection
* Add feature gate for new vote instruction and plumb through replay
Add tower versions
* Add check for slot hashes history
* Update is_recent check to exclude voting on hard fork root slot
* Move tower rollback test to flaky and ignore it until #22551 lands
#### Problem
Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:
```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```
#### Summary of Changes
For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.
Co-authored-by: Michael Vines <mvines@gmail.com>
Ashwin Sekar
Tyera
behzad nouri
Xiang Zhu
HaoranYi
steviez
Illia Bobyr
Brooks
Brennan
carllin
Jeff Washington (jwash)
Christian Kamm
Lijun Wang
Will Hickey
Ryo Onodera
Jon Cinque
Yueh-Hsuan Chiang
Michael Vines
DimAn
Justin Starry
BG Zhu
ryleung-solana
Trent Nelson
Michael Keleti
sakridge