Commit Graph

21 Commits

Author SHA1 Message Date
Pankaj Garg 9d42cd7efe
Initialize fork graph in program cache during bank_forks creation (#33810)
* Initialize fork graph in program cache during bank_forks creation

* rename BankForks::new to BankForks::new_rw_arc

* fix compilation

* no need to set fork_graph on insert()

* fix partition tests
2023-10-23 09:32:41 -07:00
steviez 56ccffdaa5
Replace get_tmp_ledger_path!() with self cleaning version (#33702)
This macro is used a lot for tests to create a ledger path in order to
open a Blockstore. Files will be left on disk unless the test remembers
to call Blockstore::destroy() on the directory. So, instead of requiring
this, use the get_tmp_ledger_path_auto_delete!() macro that creates a
TempDir (which automatically deletes itself when it goes out of scope).
2023-10-21 11:38:31 +02:00
behzad nouri dc3c827299
prunes repair QUIC connections (#33775)
The commit implements lazy eviction for repair QUIC connections.
The cache is allowed to grow to 2 x capacity at which point at least
half of the entries with lowest stake are evicted, resulting in an
amortized O(1) performance.
2023-10-20 17:50:54 +00:00
behzad nouri 7aa0faea96
separates out routing repair requests from establishing connections (#33742)
Currently each outgoing repair request will attempt to establish a
connection if one does not already exist. This is very wasteful and
consumes many tokio tasks if the remote node is down or unresponsive.

The commit decouples routing packets from establishing connections by
adding a buffering channel for each remote address. Outgoing packets are
always sent down this channel to be processed once the connection is
established. If connecting attempt fails, all packets already pushed to
the channel are dropped at once, reducing the number of attempts to make
a connection if the remote node is down or unresponsive.
2023-10-19 13:25:53 +00:00
behzad nouri e01269a9de
sends repair requests over QUIC protocol (#33016)
The commit implements client-side of serve-repair and
ancestor-hash-service over QUIC protocol.
2023-09-11 22:22:04 +00:00
behzad nouri 7fc6fea8d8
serves remote repair requests from QUIC endpoint (#33069)
The commit implements server-side of repair using QUIC protocol.

UDP repair requests are adapted as RemoteRequest and sent down the same
channel as remote requests arriving over QUIC, and the rest of the
server code is update to process over RemoteRequest type.
2023-09-11 16:57:10 +00:00
behzad nouri 9ff0b35f29
adds QUIC endpoint for repair service (#33057)
Working towards using QUIC protocol for repair, the commit adds a QUIC
endpoint for repair service.

Outgoing local requests are sent as

    struct LocalRequest {
        remote_address: SocketAddr,
        bytes: Vec<u8>,
        num_expected_responses: usize,
        response_sender: Sender<(SocketAddr, Vec<u8>)>,
    }

to the client-side of the endpoint. The client opens a bidirectional
stream with the LocalRequest.remote_address and once received the
response, sends it down the LocalRequest.response_sender channel.

Incoming requests from remote nodes are received from bidirectional
streams and sent as

    struct RemoteRequest {
        remote_pubkey: Option<Pubkey>,
        remote_address: SocketAddr,
        bytes: Vec<u8>,
        response_sender: Option<OneShotSender<Vec<Vec<u8>>>>,
    }

to the repair-service. The response is received from the receiver end of
RemoteRequest.response_sender channel and send back to the remote node
using the send side of the bidirectional stream.
2023-09-07 18:00:25 +00:00
Trent Nelson b8dc5daedb
preliminaries for bumping nightly to 2023-08-25 (#33047)
* remove unnecessary hashes around raw string literals

* remove unncessary literal `unwrap()`s

* remove panicking `unwrap()`

* remove unnecessary `unwrap()`

* use `[]` instead of `vec![]` where applicable

* remove (more) unnecessary explicit `into_iter()` calls

* remove redundant pattern matching

* don't cast to same type and constness

* do not `cfg(any(...` a single item

* remove needless pass by `&mut`

* prefer `or_default()` to `or_insert_with(T::default())`

* `filter_map()` better written as `filter()`

* incorrect `PartialOrd` impl on `Ord` type

* replace "slow zero-filled `Vec` initializations"

* remove redundant local bindings

* add required lifetime to associated constant
2023-08-29 23:05:35 +00:00
behzad nouri 1171002f46
adds repair-protocol identifying serve-repair socket (#33028)
Working towards migrating repair over QUIC protocol, the commit adds
repair-protocol argument for identifying right serve-repair socket.
2023-08-28 21:34:09 +00:00
Jon Cinque 0fe902ced7
Bump rand to 0.8, rand_chacha to 0.3, getrandom to 0.2 (#32871)
* sdk: Add concurrent support for rand 0.7 and 0.8

* Update rand, rand_chacha, and getrandom versions

* Run command to replace `gen_range`

Run `git grep -l gen_range | xargs sed -i'' -e 's/gen_range(\(\S*\), /gen_range(\1../'

* sdk: Fix users of older `gen_range`

* Replace `hash::new_rand` with `hash::new_with_thread_rng`

Run:
```
git grep -l hash::new_rand | xargs sed -i'' -e 's/hash::new_rand([^)]*/hash::new_with_thread_rng(/'
```

* perf: Use `Keypair::new()` instead of `generate`

* Use older rand version in zk-token-sdk

* program-runtime: Inline random key generation

* bloom: Fix clippy warnings in tests

* streamer: Scope rng usage correctly

* perf: Fix clippy warning

* accounts-db: Map to char to generate a random string

* Remove `from_secret_key_bytes`, it's just `keypair_from_seed`

* ledger: Generate keypairs by hand

* ed25519-tests: Use new rand

* runtime: Use new rand in all tests

* gossip: Clean up clippy and inline keypair generators

* core: Inline keypair generation for tests

* Push sbf lockfile change

* sdk: Sort dependencies correctly

* Remove `hash::new_with_thread_rng`, use `Hash::new_unique()`

* Use Keypair::new where chacha isn't used

* sdk: Fix build by marking rand 0.7 optional

* Hardcode secret key length, add static assertion

* Unify `getrandom` crate usage to fix linking errors

* bloom: Fix tests that require a random hash

* Remove some dependencies, try to unify others

* Remove unnecessary uses of rand and rand_core

* Update lockfiles

* Add back some dependencies to reduce rebuilds

* Increase max rebuilds from 14 to 15

* frozen-abi: Remove `getrandom`

* Bump rebuilds to 17

* Remove getrandom from zk-token-proof
2023-08-21 19:11:21 +02:00
behzad nouri 7bd7410592
uses typed Pubkey instead of opaque [u8; 32] (#32842) 2023-08-18 23:28:08 +00:00
steviez a4c8cc3ce0
Remove improper uses of &Arc<Bank> (#32802)
In most cases, either a &Bank or an Arc<Bank> is more proper.
- &Bank is used if the function only needs a momentary reference
- Arc<Bank> is used if the function needs its' own copy

This PR leaves several instances of &Arc<Bank> around; these instances
are situations where a clone may only happen conditionally.
2023-08-18 16:46:34 -05:00
Jeff Biseda 13b7ae1432
legacy repair type cleanup (#32849) 2023-08-17 11:11:35 -07:00
behzad nouri 0de8ccfda9
adds socket address for repair service over QUIC (#32834)
Working towards migrating repair to QUIC.
2023-08-15 17:09:09 +00:00
behzad nouri 52616cf7aa
removes repair socket from gossip ContactInfo (#32831)
Repair responses are sent back to the address the repair request came
from and there is no need to gossip repair socket address.
https://github.com/solana-labs/solana/blob/9212ac347/core/src/repair/serve_repair.rs#L519
2023-08-14 20:01:18 +00:00
Pankaj Garg f4287d70bb
Move accounts-db code to its own crate (#32766) 2023-08-09 13:03:36 -07:00
behzad nouri b7c2ad5b67
repurposes tvu_forwards socket for TVU over QUIC (#32737)
LegacyContactInfo.tvu_forwards is unused.
Repurposing the field for TVU over QUIC will avoid QUIC_PORT_OFFSET hack
in a backward compatible way.
2023-08-07 22:02:41 +00:00
behzad nouri d54b6204be
removes instances of clippy::manual_let_else (#32417) 2023-07-09 21:41:36 +00:00
Ashwin Sekar c172dfd268
Only send earliest mismatched ancestor to replay (#31842) 2023-07-06 21:31:12 -07:00
behzad nouri 0da01270ef
removes redundant recycler clones (#32401) 2023-07-06 18:25:20 +00:00
Jeff Biseda bad5197cb0
refactor core to create repair module (#32303) 2023-07-05 12:20:46 -07:00