Propose Solana ABI management (#7524)
* Propose Solana ABI management * Mention fuzz testing * Address minor review comments * Remove versioning and unit tests * Rename * Clean up a bit * Pass through Grammarly * Yet more tweaks...
This commit is contained in:
Ryo Onodera
GitHub
parent
6775e83420
commit
c33b54794c
|
|
@ -60,6 +60,7 @@
|
|||
* [Slashing](proposals/slashing.md)
|
||||
* [Tick Verification](proposals/tick-verification.md)
|
||||
* [Block Confirmation](proposals/block-confirmation.md)
|
||||
* [ABI Management](proposals/abi-management.md)
|
||||
* [Implemented Design Proposals](implemented-proposals/README.md)
|
||||
* [Blocktree](implemented-proposals/blocktree.md)
|
||||
* [Cluster Software Installation and Updates](implemented-proposals/installer.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,90 @@
|
|||
# Solana ABI management process
|
||||
|
||||
This document proposes the Solana ABI management process. The ABI management
|
||||
process is an engineering practice and a supporting technical framework to avoid
|
||||
introducing unintended incompatible ABI changes.
|
||||
|
||||
# Problem
|
||||
|
||||
The Solana ABI (binary interface to the cluster) is currently only defined
|
||||
implicitly by the implementation and requires a very careful eye to notice
|
||||
breaking changes. This makes it extremely difficult to upgrade the software
|
||||
on an existing cluster without rebooting the ledger.
|
||||
|
||||
# Requirements and objectives
|
||||
|
||||
- Unintended ABI changes can be detected as CI failures mechanically.
|
||||
- Newer implementation must be able to process the oldest data (since genesis)
|
||||
once we go mainnet.
|
||||
- The objective of this proposal is to protect the ABI while sustaining rather
|
||||
rapid development by opting for a mechanical process rather than a very long
|
||||
human-driven auditing process.
|
||||
- Once signed cryptographically, data blob must be identical, so no
|
||||
in-place data format update is possible regardless of inbound and outbound of
|
||||
the online system. Also, considering the sheer volume of transactions we're
|
||||
aiming to handle, retrospective in-place update is undesirable at best.
|
||||
|
||||
# Solution
|
||||
|
||||
Instead of natural human's eye due-diligence, which should be assumed to fail
|
||||
regularly, we need a systematic assurance of not breaking the cluster when
|
||||
changing the source code.
|
||||
|
||||
For that purpose, we introduce a mechanism of marking every ABI-related things
|
||||
in source code (`struct`s, `enum`s) with the new `#[frozen_abi]` attribute. This
|
||||
takes hard-coded digest value derived from types of its fields via
|
||||
`ser::Serialize`. And the attribute automatically generates a unit test to try
|
||||
to detect any unsanctioned changes to the marked ABI-related things.
|
||||
|
||||
However, the detection cannot be complete; no matter how hard we statically
|
||||
analyze the source code, it's still possible to break ABI. For example, this
|
||||
includes not-`derive`d hand-written `ser::Serialize`, underlying library's
|
||||
implementation changes (for example `bincode`), CPU architecture differences.
|
||||
The detection of these possible ABI incompatibilities is out-of-scope for this
|
||||
ABI management.
|
||||
|
||||
# Definitions
|
||||
|
||||
ABI item/type: various types to be used for serialization, which collectively
|
||||
comprises the whole ABI for any system components. For example, those types
|
||||
include `struct`s and `enum`s.
|
||||
|
||||
ABI item digest: Some fixed hash derived from type information of ABI item's
|
||||
fields.
|
||||
|
||||
# Example
|
||||
|
||||
```patch
|
||||
+#[frozen_abi(digest="1c6a53e9")]
|
||||
#[derive(Serialize, Default, Deserialize, Debug, PartialEq, Eq, Clone)]
|
||||
pub struct Vote {
|
||||
/// A stack of votes starting with the oldest vote
|
||||
pub slots: Vec<Slot>,
|
||||
/// signature of the bank's state at the last slot
|
||||
pub hash: Hash,
|
||||
}
|
||||
```
|
||||
|
||||
# Developer's workflow
|
||||
|
||||
To know the digest for new ABI items, developers can add `frozen_abi` with a
|
||||
random digest value and run the unit tests and replace it with the correct
|
||||
digest from the assertion test error message.
|
||||
|
||||
In general, once we add `frozen_abi` and its change is published in the stable
|
||||
release channel, its digest should never change. If such a change is needed, we
|
||||
should opt for defining a new struct like `FooV1`. And special release flow like
|
||||
hard forks should be approached.
|
||||
|
||||
# Implementation remarks
|
||||
|
||||
We use some degree of macro machinery to automatically generate unit tests
|
||||
and calculate a digest from ABI items. This is doable by clever use of
|
||||
`serde::Serialize` ([1]) and `any::typename` ([2]). For a precedent for similar
|
||||
implementation, `ink` from the Parity Technologies [3] could be informational.
|
||||
|
||||
# References
|
||||
|
||||
1. [(De)Serialization with type info · Issue #1095 · serde-rs/serde](https://github.com/serde-rs/serde/issues/1095#issuecomment-345483479)
|
||||
2. [`std::any::type_name` - Rust](https://doc.rust-lang.org/std/any/fn.type_name.html)
|
||||
3. [Parity's ink to write smart contracts](https://github.com/paritytech/ink)
|
||||
Loading…
Reference in New Issue