#### Problem
Our existing datapoint macro syntax does directly not support group-by tags.
The existing workaround is to embed the group-by tags into the metric name which does not scale well.
#### Summary of Changes
This PR extends the existing syntax to support group-by tags as follows. The new syntax is also compatible with the existing syntax:
```
datapoint_debug!(
"metric_name",
"tag" => "tag-value",
"tag2" => "tag-value2",
....
("field1", 100, i64), // field syntax is the same as the current syntax.
("field2", "hello", String),
...
);
```
#### Problem
blockstore_db.rs has a mutual dependency between blockstore_metrics.rs.
#### Summary of Changes
This PR removes the mutual dependency by moving the option-related stuff
out from blockstore_db.rs to its new home --- blockstore_options.rs.
By doing this, we address the mutual dependency and also make the code cleaner.
Prior to this change, genesis.tar.bz2 and the snapshot would be copied
over to a new directory but not rocksdb. As such, the verify step would
only unpack the snapshot and verify that hash.
test_pull_from_entrypoint_if_not_present relies on a deterministic
ordering for the entries when generating gossip pull requests.
https://github.com/solana-labs/solana/pull/25460
changed an intermediate type for gossip pull-requests from Vec to
HashMap, and so the entries are no longer deterministically ordered.
This causes the test to be flaky.
The commit updates the test so that it no longer relies on the ordering.
* Removes is_duplicate().
* Replaces use sites of visit_each_account_once().
* Removes visit_each_account_once().
* cargo fmt
* Adds comments to clarify the order of pre_accounts and instruction_accounts.
* Simplify control flow.
Move all syscall declarations under solana_program::syscalls. If the target
supports static syscalls (sbfv2), then actually define them statically to avoid
dynamic relocations.
Each time a node generates gossip pull-requests, it sends out all the
requests to a single randomly selected peer:
https://github.com/solana-labs/solana/blob/fd7ad31ee/gossip/src/crds_gossip_pull.rs#L253-L266
This causes a burst of pull-requests at a single node at once. In order
to make gossip in-bound traffic less bursty, this commit fans out gossip
pull-requests to several randomly selected peers.
This should reduce spikes in inbound gossip traffic without changing the
average load which may help reduce number of times outbound data budget is
exhausted when responding to gossip pull-requests at the receiving node, and
reduce number of pull-requests dropped.
* FindPacketSenderStake: Remove parallelism to improve performance
The work unit sizes were so small that using the thread pool
slowed down this stage significantly.
* fix checks
Co-authored-by: Justin Starry <justin@solana.com>
Indices for code and data shreds of the same slot overlap; and so they
will have the same random number generator seed when shuffling cluster
nodes for turbine broadcast.
This results in the same propagation path for code and data shreds of
the same index and effectively smaller sample size for re-transmitter
nodes. For example a 32:32 batch (32 code + 32 data shreds), is
retransmitted through _at most_ 32 unique nodes, whereas ideally we want
~64 unique re-transmitters.
This commit adds shred-type to seed function so that code and data
sherds of the same (slot, index) will (most likely) have different
propagation paths.
Bytes past Packet.meta.size are not valid to read from.
The commit makes the buffer field private and instead provides two
methods:
* Packet::data() which returns an immutable reference to the underlying
buffer up to Packet.meta.size. The rest of the buffer is not valid to
read from.
* Packet::buffer_mut() which returns a mutable reference to the entirety
of the underlying buffer to write into. The caller is responsible to
update Packet.meta.size after writing to the buffer.
* Removes the offset InstructionContext::get_number_of_program_accounts() from InstructionAccount::index_in_caller.
* Removes unreachable SyscallError::InvalidLength in orig_data_lens.get().
Upcoming changes to PacketBatch to support variable sized packets will
modify the internals of PacketBatch. So, this change removes usage of
the internal packet struct and instead uses accessors (which are
currently just wrappers of Vector functions but will change down the
road).
Justin Starry
dependabot[bot]
Yueh-Hsuan Chiang
buffalu
steveluscher
steviez
behzad nouri
Alexander Meißner
ryleung-solana
HaoranYi
Alessandro Decina
Christian Kamm
Brooks Prumo
Artie Kushner
samkim-crypto
apfitzge
Tyera Eulberg
carllin
Jeff Biseda
Jack May
Yihau Chen