diff --git a/art/fullnode.bob b/art/fullnode.bob index 1a3b0586e1..addf79c804 100644 --- a/art/fullnode.bob +++ b/art/fullnode.bob @@ -1,21 +1,27 @@ - .------------------------. - | Fullnode | - | | - .--------. | .----------------. | - | |---->| | | - | Client | | | JsonRpcService | | - | |<----| | | - `----+---` | `----------------` | - | | ^ | - | | | | - | | .--+---. .-----. | - | | | Bank |<--| Tvu | | - | | `------` `-----` | - | | ^ ^ | - | | | | | .------------. - | | .--+--. .--+--. | | | - `-------->| Tpu +-->| Ncp |<-------->| Validators | - | `-----` `-----` | | | - | | `------------` - `------------------------` + .---------------------------. + | Fullnode | + | | + .--------. | .----------------. | + | |---->| | | + | Client | | | JsonRpcService | | + | |<----| | | + `----+---` | `----------------` | + | | ^ | .------------------. + | | | .-----. | | Validators | + | | | | Ncp |<---------->| | + | | | `---+-` | | .------------. | + | | | ^ | | | | | | + | | | | v | | | Upstream | | + | | .--+---. .-+---. | | | Validators | | + | | | Bank |<--| Tvu |<--------------+ | | + | | `------` `-----` | | `------------` | + | | ^ | | | + | | | | | .------------. | + | | .--+--. .-----------. | | | | | + `-------->| Tpu +-->| Broadcast +--------->| Downstream | | + | `-----` | Stage | | | | Validators | | + | `-----------` | | | | | + | | | `------------` | + `---------------------------` | | + `------------------` diff --git a/art/tpu.bob b/art/tpu.bob index cf046ecfb4..8350fae43b 100644 --- a/art/tpu.bob +++ b/art/tpu.bob @@ -1,19 +1,19 @@ - .----------------------------------------------------. - | Tpu .------------. | - | | PohService | | - | `-------+----` | - | ^ | | - | | v | - | .-------. .-----------. .-+-------. .-------. | - .---------. | | Fetch | | SigVerify | | Banking | | Write | | .------------. - | Clients |--->| Stage |->| Stage |->| Stage |-->| Stage +--->| Validators | - `---------` | | | | | | | | | | `------------` - | `-------` `-----------` `----+----` `---+---` | - | | | | - | | | | - | | | | - | | | | - `---------------------------------|------------|-----` + .------------------------------------------------------. + | Tpu .------------. | + | | PohService | | + | `-------+----` | + | ^ | | + | | v | + | .-------. .-----------. .-+-------. .--------. | .------------. + .---------. | | Fetch | | SigVerify | | Banking | | Ledger | | | Broadcast | + | Clients |--->| Stage |->| Stage |->| Stage |-->| Write +---->| Stage | + `---------` | | | | | | | | Stage | | | | + | `-------` `-----------` `----+----` `---+----` | `------------` + | | | | + | | | | + | | | | + | | | | + `---------------------------------|------------|-------` | | v v .------. .--------. diff --git a/art/tvu.bob b/art/tvu.bob index 55c569b123..7c82080907 100644 --- a/art/tvu.bob +++ b/art/tvu.bob @@ -1,18 +1,21 @@ - .--------------------------------------------. - | | - | .------------------------------------|---------------------------------------. - | | Tvu | | - | | | | - v | .-------. .------------. .----+---------. .--------. .---------. | - .--------. | | Blob | | Retransmit | | Replicate | | Ledger | | Storage | | - | Leader |----->| Fetch |-->| Stage |-->| Stage |-->| Write |-->| Stage | | - `--------` | | Stage | | | | | | Stage | | | | - | `-------` `----+-------` `----+---------` `--------` `---------` | - | ^ | | | - | | | | | - `--------|----------|----------------|---------------------------------------` - | | | - | V v - .+-----------. .------. - | Validators | | Bank | - `------------` `------` + .--------. + | Leader | + `--------` + ^ + | + .------------------------------------|---------------------------------------. + | Tvu | | + | | | + | .-------. .------------. .----+---------. .--------. .---------. | + .------------. | | Blob | | Retransmit | | Replicate | | Ledger | | Storage | | + | Upstream +----->| Fetch |-->| Stage |-->| Stage |-->| Write |-->| Stage | | + | Validators | | | Stage | | | | | | Stage | | | | + `------------` | `-------` `----+-------` `----+---------` `--------` `---------` | + | ^ | | | + | | | | | + `--------|----------|----------------|---------------------------------------` + | | | + | V v + .+-----------. .------. + | Ncp | | Bank | + `------------` `------` diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 23eb0b5ee5..baa574ec62 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -4,14 +4,7 @@ - [Terminology](terminology.md) -- [On-chain programs](programs.md) - - [The Runtime](runtime.md) - -- [Fullnode](fullnode.md) - - [Tpu](tpu.md) - - [Tvu](tvu.md) - - [Ncp](ncp.md) - - [JsonRpcService](jsonrpc-service.md) +- [Programming Model](programs.md) - [A Solana Cluster](cluster.md) - [Synchronization](synchronization.md) @@ -21,6 +14,13 @@ - [Ledger Replication](storage.md) - [Leader Rotation](leader-scheduler.md) +- [Anatomy of a Fullnode](fullnode.md) + - [Tpu](tpu.md) + - [Tvu](tvu.md) + - [Ncp](ncp.md) + - [JsonRpcService](jsonrpc-service.md) + - [The Runtime](runtime.md) + ## Appendix - [Appendix](appendix.md) diff --git a/src/img/fullnode.svg b/src/img/fullnode.svg index bed2672213..a2723fedcc 100644 --- a/src/img/fullnode.svg +++ b/src/img/fullnode.svg @@ -1,4 +1,4 @@ - + @@ -56,7 +56,7 @@ } - + @@ -68,13 +68,13 @@ - + - + - + @@ -94,19 +94,19 @@ - + - - + + - - + + - - + + @@ -114,14 +114,14 @@ - - - + + + - - - + + + @@ -132,108 +132,175 @@ - - - - + + + + - - + + - - - - + + + + - - + + - + - - - + + + - + - - + + - - - + + + - - - - + + - - + + - - - + + + - - + + + - - + + - + + + + - + + + + + + + + + + + - + - - + - + + - + + + + + + - - - + + + - - + - - + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + @@ -251,27 +318,57 @@ JsonRpcService - + Bank - + Tpu - + +Broadcast + + + + +Stage + + + + Ncp - + Tvu - + +Validators + + + + +Upstream + + + + +Validators + + + + +Downstream + + + + Validators diff --git a/src/img/tpu.svg b/src/img/tpu.svg index 794b8c9cd2..ca6be6740a 100644 --- a/src/img/tpu.svg +++ b/src/img/tpu.svg @@ -1,4 +1,4 @@ - + @@ -56,7 +56,7 @@ } - + @@ -85,14 +85,14 @@ - - + + - - + + @@ -203,14 +203,14 @@ - - + + - - + + @@ -221,34 +221,34 @@ - - - + + + - + - + - - - + + + - - + + - - + + - + @@ -276,7 +276,7 @@ SigVerify - + Stage @@ -291,7 +291,7 @@ Banking - + Stage @@ -302,11 +302,16 @@ Bank -Write +Ledger +Write + + + + Stage @@ -316,8 +321,13 @@ Ledger - -Validators + +Broadcast + + + + +Stage diff --git a/src/img/tvu.svg b/src/img/tvu.svg index 6080e033bc..582ca60841 100644 --- a/src/img/tvu.svg +++ b/src/img/tvu.svg @@ -1,4 +1,4 @@ - + @@ -56,267 +56,287 @@ } - + - - - + + + - - + + - - + + - - + + + - - + + - + + - + + + - - + + + + + - - + + + - - - + + - - - - - + + - - - + + + - - + + - - + + + + + - - - + - - + - - - - - + + + - + + - + + + + - - - + - - + - - - - + - + + + - + + + - + + - - - + + - - - - + + + + - - - - + + + + - - - + + + - - + + - - + + - + - + - + - - - + - - + - - + + + - + + - + + - - - + - - + - - + + + - + + - + + - -Leader + + + + + + + +Upstream - -Tvu - - - - -Blob - - - - -Fetch - - - - -Stage - - - - + Validators - + +Tvu + + + + +Blob + + + + +Fetch + + + + +Stage + + + + +Ncp + + + + Retransmit - + Stage - + +Leader + + + + Replicate - + Stage - + Bank - + Ledger - + Write - + Stage - + Storage - + Stage diff --git a/src/ncp.md b/src/ncp.md index 8c08052f3d..4238c5f705 100644 --- a/src/ncp.md +++ b/src/ncp.md @@ -1,3 +1,3 @@ -# Ncp +# The Network Control Plane -The Network Control Plane implements a gossip network between all nodes on in the cluster. +The Network Control Plane (Ncp) implements a gossip network between all nodes on in the cluster. diff --git a/src/programs.md b/src/programs.md index 97a6869506..09cdc5a287 100644 --- a/src/programs.md +++ b/src/programs.md @@ -1,6 +1,4 @@ -# The Solana SDK - -## Introduction +# Programming Model With the Solana runtime, we can execute on-chain programs concurrently, and written in the client’s choice of programming language. @@ -21,14 +19,14 @@ ELF and passes it the client's message for interpretation. Solana supports several kinds of persistent storage, called *accounts*: 1. Executable -2. Writable by a client -3. Writable by a program -4. Read-only +2. Owned by a client +3. Owned by a program +4. Credit-only All accounts are identified by public keys and may hold arbirary data. When the client sends messages to programs, it requests access to storage using those keys. The runtime loads the account data and passes it to the program. The runtime also ensures accounts aren't written to if not owned -by the client or program. Any writes to read-only accounts are discarded +by the client or program. Any writes to credit-only accounts are discarded unless the write was to credit tokens. Any user may credit other accounts tokens, regardless of account permission. diff --git a/src/runtime.md b/src/runtime.md index 4a715d4096..77f37155c2 100644 --- a/src/runtime.md +++ b/src/runtime.md @@ -1,39 +1,29 @@ # Runtime -The goal with the runtime is to have a general purpose execution environment -that is highly parallelizable. To achieve this goal the runtime forces each -Instruction to specify all of its memory dependencies up front, and therefore a -single Instruction cannot cause a dynamic memory allocation. An explicit -Instruction for memory allocation from the `SystemProgram::CreateAccount` is -the only way to allocate new memory in the engine. A Transaction may compose -multiple Instruction, including `SystemProgram::CreateAccount`, into a single -atomic sequence which allows for memory allocation to achieve a result that is -similar to dynamic allocation. +The runtime is a concurrent transaction processor. Transactions specify their +data dependencies upfront and dynamic memory allocation is explicit. By +separating program code from the state it operates on, the runtime is able to +choreograph concurrent access. Transactions accessing only credit-only +accounts are executed in parallel whereas transactions accessing writable +accounts are serialized. The runtime interacts with the program through an +entrypoint with a well-defined interface. The userdata stored in an account is +an opaque type, an array of bytes. The program has full control over its +contents. +The transaction structure specifies a list of public keys and signatures for +those keys and a sequential list of instructions that will operate over the +states associated with the account keys. For the transaction to be committed +all the instructions must execute successfully; if any abort the whole +transaction fails to commit. -### State +### Account structure -State is addressed by an Account which is at the moment simply the Pubkey. Our -goal is to eliminate memory allocation from within the program itself. Thus -the client of the program provides all the state that is necessary for the -program to execute in the transaction itself. The runtime interacts with the -program through an entry point with a well defined interface. The userdata -stored in an Account is an opaque type to the runtime, a `Vec`, the -contents of which the program code has full control over. - -The Transaction structure specifies a list of Pubkey's and signatures for those -keys and a sequential list of instructions that will operate over the state's -associated with the `account_keys`. For the transaction to be committed all -the instructions must execute successfully, if any abort the whole transaction -fails to commit. - -### Account structure Accounts maintain token state as well as program specific -memory. +Accounts maintain a token balance and program-specific memory. # Transaction Engine -At its core, the engine looks up all the Pubkeys maps them to accounts and -routs them to the `program_id` entry point. +The engine maps public keys to accounts and routes them to the program's +entrypoint. ## Execution @@ -41,48 +31,48 @@ Transactions are batched and processed in a pipeline Runtime pipeline -At the `execute` stage, the loaded pages have no data dependencies, so all the -programs can be executed in parallel. +At the *execute* stage, the loaded pages have no data dependencies, so all the +programs can be executed in parallel. The runtime enforces the following rules: -1. The `program_id` code is the only code that will modify the contents of - `Account::userdata` of Account's that have been assigned to it. This means -that upon assignment userdata vector is guaranteed to be `0`. +1. Only the *owner* program may modify the contents of an account. This means + that upon assignment userdata vector is guaranteed to be zero. 2. Total balances on all the accounts is equal before and after execution of a - Transaction. -3. Balances of each of the accounts not assigned to `program_id` must be equal - to or greater after the Transaction than before the transaction. -4. All Instructions in the Transaction executed without a failure. + transaction. +3. After the transaction is executed, balances of credit-only accounts must be + greater than or equal to the balances before the transaction. +4. All instructions in the transaction executed atomically. If one fails, all + account modifications are discarded. -## Entry Point Execution of the program involves mapping the Program's public -key to an entry point which takes a pointer to the transaction, and an array of -loaded pages. +Execution of the program involves mapping the program's public key to an +entrypoint which takes a pointer to the transaction, and an array of loaded +pages. -## System Interface +## SystemProgram Interface -The interface is best described by the `Instruction::userdata` that the -user encodes. -* `CreateAccount` - This allows the user to create and assign an Account to a +The interface is best described by the `Instruction::userdata` that the user +encodes. + +* `CreateAccount` - This allows the user to create and assign an account to a Program. -* `Assign` - allows the user to assign an existing account to a `Program`. -* `Move` - moves tokens between `Account`s that are associated with - `SystemProgram`. This cannot be used to move tokens of other `Account`s. -Programs need to implement their own version of Move. +* `Assign` - allows the user to assign an existing account to a program. +* `Move` - moves tokens between account's that are associated with +* `Spawn` - spawn a new program from an account ## Notes -1. There is no dynamic memory allocation. Client's need to call the -`SystemProgram` to create memory before passing it to another program. This -Instruction can be composed into a single Transaction with the call to the +1. There is no dynamic memory allocation. Client's need to use `CreateAccount` +instructions to create memory before passing it to another program. This +instruction can be composed into a single transaction with the call to the program itself. -2. Runtime guarantees that when memory is assigned to the `Program` it is zero +2. Runtime guarantees that when memory is assigned to the program it is zero initialized. -3. Runtime guarantees that `Program`'s code is the only thing that can modify +3. Runtime guarantees that a program's code is the only thing that can modify memory that its assigned to -4. Runtime guarantees that the `Program` can only spend tokens that are in -`Account`s that are assigned to it -5. Runtime guarantees the balances belonging to `Account`s are balanced before +4. Runtime guarantees that the program can only spend tokens that are in +accounts that are assigned to it +5. Runtime guarantees the balances belonging to accounts are balanced before and after the transaction 6. Runtime guarantees that multiple instructions all executed successfully when a transaction is committed. diff --git a/src/vdf.md b/src/vdf.md index 5188950d9d..414148d27d 100644 --- a/src/vdf.md +++ b/src/vdf.md @@ -1,4 +1,4 @@ -# Introduction to VDFs +# Verifiable Delay Functions A Verifiable Delay Function is conceptually a water clock where its water marks can be recorded and later verified that the water most certainly passed