9b866d79fb
filter_crds_values checks every crds filter against every hash value: https://github.com/solana-labs/solana/blob/ee646aa7/core/src/crds_gossip_pull.rs#L432 which can be inefficient if the filter's bit-mask only matches small portion of the entire crds table. This commit shards crds values into separate tables based on shard_bits first bits of their hash prefix. Given a (mask, mask_bits) filter, filtering crds can be done by inspecting only relevant shards. If CrdsFilter.mask_bits <= shard_bits, then precisely only the crds values which match (mask, mask_bits) bit pattern are traversed. If CrdsFilter.mask_bits > shard_bits, then approximately only 1/2^shard_bits of crds values are inspected. Benchmarking on a gce cluster of 20 nodes, I see ~10% improvement in generate_pull_responses metric, but with larger clusters, crds table and 2^mask_bits are both larger, so the impact should be more significant. |
||
|---|---|---|
| .. | ||
| banking_stage.rs | ||
| blockstore.rs | ||
| cluster_info.rs | ||
| crds_gossip_pull.rs | ||
| crds_shards.rs | ||
| gen_keys.rs | ||
| poh.rs | ||
| poh_verify.rs | ||
| retransmit_stage.rs | ||
| shredder.rs | ||
| sigverify_stage.rs | ||