* Adds a module `address_lookup_table` to the SDK.
* Adds a module `address_lookup_table::instruction` to the SDK.
* Adds a module `address_lookup_table::error` to the SDK.
* Adds a module `address_lookup_table::state` to the SDK.
* Moves AddressLookupTable into SDK as well.
* Moves AddressLookupTableAccount into address_lookup_table.
* Adds deprecation messages.
* Disentangles dependencies across cargo files.
Working towards using QUIC protocol for repair, the commit adds a QUIC
endpoint for repair service.
Outgoing local requests are sent as
struct LocalRequest {
remote_address: SocketAddr,
bytes: Vec<u8>,
num_expected_responses: usize,
response_sender: Sender<(SocketAddr, Vec<u8>)>,
}
to the client-side of the endpoint. The client opens a bidirectional
stream with the LocalRequest.remote_address and once received the
response, sends it down the LocalRequest.response_sender channel.
Incoming requests from remote nodes are received from bidirectional
streams and sent as
struct RemoteRequest {
remote_pubkey: Option<Pubkey>,
remote_address: SocketAddr,
bytes: Vec<u8>,
response_sender: Option<OneShotSender<Vec<Vec<u8>>>>,
}
to the repair-service. The response is received from the receiver end of
RemoteRequest.response_sender channel and send back to the remote node
using the send side of the bidirectional stream.
removes outdated matches crate from the dependencies
std::matches has been stable since rust 1.42.0.
Other use-cases are covered by assert_matches crate.
* programs/sbf: add TEST_[FORBID|ALLOW]_WRITE_AFTER_OWNERSHIP_CHANGE*
* programs/sbf: add tests for the AccessViolation -> InstructionError mapping
* cpi: add more tests
* programs/sbf: add tests for immutable AccountInfo pointers
* programs/sbf: add tests for verification of SolAccountInfo pointers too
* programs/sbf: add tests for ref_to_len_in_vm handling in CPI
Add TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE_MOVING_DATA_POINTER
and TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE that exercise the new
logic.
* cpi: tweak tests
Remove some copy pasta and rename two tests to better describe what they're doing
* cpi: add tests that check that CPI updates all accounts at once
* direct mapping: test that writes to executable accounts trigger ExecutableDataModified
* programs/sbf: add explicit tests for when an account's data allocation changes
* allow pedantic invalid cast lint
* allow lint with false-positive triggered by `test-case` crate
* nightly `fmt` correction
* adapt to rust layout changes
* remove dubious test
* Use transmute instead of pointer cast and de/ref when check_aligned is false.
* Renames clippy::integer_arithmetic to clippy::arithmetic_side_effects.
* bump rust nightly to 2023-08-25
* Upgrades Rust to 1.72.0
---------
Co-authored-by: Trent Nelson <trent@solana.com>
* Moves initialization and growth of program accounts
from "write" to "truncate" instruction in loader-v4.
* Removes the parameter payer account from the truncate instruction.
Computing Poseidon[0] hashes is too expensive to be done in a Solana
program in one transaction. Poseidon is a zero-knowlege proof friendly
hash function, used by the majority of ZK-based projects, including the
ones built on top of Solana.
This change introduces the `sol_poseidon` syscall which takes 2D byte
slice as an input and then calculates a Poseidon hash using a BN254
curve and the following Poseidon parameters:
* x^5 S-boxes
* width - 2 ≤ t ≤ 13
* inputs - 1 ≤ n ≤ 12
* 8 full rounds and partial rounds depending on t: [56, 57, 56, 60, 60,
63, 64, 63, 60, 66, 60, 65]
Computation of Poseidon hashes is done with the light-poseidon[1]
crate, which is audited[2] and compatible with Circom[3] (BN254 curve,
the same parameters and constants).
Proposed compute costs depend on number of inputs and are based on
light-poseidon benchmarks[4].
[0] https://www.poseidon-hash.info/
[1] https://crates.io/crates/light-poseidon
[2] https://github.com/Lightprotocol/light-poseidon/blob/main/assets/audit.pdf
[3] https://docs.circom.io/
[4] https://github.com/Lightprotocol/light-poseidon/tree/main#performance
* transaction_context: update make_data_mut comment
* bpf_loader: cpi: pass SerializeAccountMetadata to CallerAccount::from*
We now have a way to provide CallerAccount with trusted values coming
from our internal serialization code and not from untrusted vm space
* bpf_loader: direct_mapping: enforce account info pointers to be immutable
When direct mapping is enabled, we might need to update account data
memory regions across CPI calls. Since the only way we have to retrieve
the regions is based on their vm addresses, we enforce vm addresses to
be stable. Accounts can still be mutated and resized of course, but it
must be done in place.
This also locks all other AccountInfo pointers, since there's no legitimate
reason to make them point to anything else.
* bpf_loader: cpi: access ref_to_len_in_vm through VmValue
Direct mapping needs to translate vm values at each access since
permissions of the underlying memory might have changed.
* direct mapping: improve memory permission tracking across CPI calls
Ensure that the data and realloc regions of an account always track the
account's permissions. In order to do this, we also need to split
realloc regions in their own self contained regions, where before we
had:
[account fields][account data][account realloc + more account fields + next account fields][next account data][...]
we now have:
[account fields][account data][account realloc][more account fields + next account fields][next account data][...]
Tested in TEST_[FORBID|ALLOW]_WRITE_AFTER_OWNERSHIP_CHANGE*
Additionally when direct mapping is on, we must update all perms at once before
doing account data updates. Otherwise, updating an account might write into
another account whose perms we haven't updated yet. Tested in
TEST_FORBID_LEN_UPDATE_AFTER_OWNERSHIP_CHANGE.
* bpf_loader: serialization: address review comment don't return vm_addr from push_account_region
* bpf_loader: rename push_account_region to push_account_data_region
* cpi: fix slow edge case zeroing extra account capacity after shrinking an account
When returning from CPI we need to zero all the account memory up to the
original length only if we know we're potentially dealing with uninitialized
memory.
When we know that the spare capacity has deterministic content, we only need to
zero new_len..prev_len.
This fixes a slow edge case that was triggerable by the following scenario:
- load a large account (say 10MB) into the vm
- shrink to 10 bytes - would memset 10..10MB
- shrink to 9 bytes - would memset 9..10MB
- shrink to 8 bytes - would memset 8..10MB
- ...
Now instead in the scenario above the following will happen:
- load a large account (say 10MB) into the vm
- shrink to 10 bytes - memsets 10..10MB
- shrink to 9 bytes - memsets 9..10
- shrink to 8 bytes - memset 8..9
- ...
* bpf_loader: add account_data_region_memory_state helper
Shared between serialization and CPI to figure out the MemoryState of an
account.
* cpi: direct_mapping: error out if ref_to_len_in_vm points to account memory
If ref_to_len_in_vm is allowed to be in account memory, calles could mutate it,
essentially letting callees directly mutate callers memory.
* bpf_loader: direct_mapping: map AccessViolation -> InstructionError
Return the proper ReadonlyDataModified / ExecutableDataModified /
ExternalAccountDataModified depending on where the violation occurs
* bpf_loader: cpi: remove unnecessary infallible slice::get call
* remove unnecessary hashes around raw string literals
* remove unncessary literal `unwrap()`s
* remove panicking `unwrap()`
* remove unnecessary `unwrap()`
* use `[]` instead of `vec![]` where applicable
* remove (more) unnecessary explicit `into_iter()` calls
* remove redundant pattern matching
* don't cast to same type and constness
* do not `cfg(any(...` a single item
* remove needless pass by `&mut`
* prefer `or_default()` to `or_insert_with(T::default())`
* `filter_map()` better written as `filter()`
* incorrect `PartialOrd` impl on `Ord` type
* replace "slow zero-filled `Vec` initializations"
* remove redundant local bindings
* add required lifetime to associated constant
* dereplicode address alignment check
* Uses `checked_div` and `checked_rem` in built-in loaders.
* Uses `checked_div` and `checked_rem`.
* sdk: replace sub() with saturating_sub()
* eliminate `String` "arithmetic"
* allow arithmetic side-effects in tests and benches and on types we don't control
---------
Co-authored-by: Trent Nelson <trent@solana.com>