diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 07ea6c5319..ff42336f1c 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -95,14 +95,14 @@ * [Validator Timestamp Oracle](implemented-proposals/validator-timestamp-oracle.md) * [Commitment](implemented-proposals/commitment.md) * [Snapshot Verification](implemented-proposals/snapshot-verification.md) + * [Cross-Program Invocation](implemented-proposals/cross-program-invocation.md) + * [Program Derived Addresses](implemented-proposals/program-derived-addresses.md) * [Accepted Design Proposals](proposals/README.md) * [Optimistic Confirmation and Slashing](proposals/optimistic-confirmation-and-slashing.md) * [Secure Vote Signing](proposals/vote-signing-to-implement.md) * [Cluster Test Framework](proposals/cluster-test-framework.md) * [Validator](proposals/validator-proposal.md) * [Simple Payment and State Verification](proposals/simple-payment-and-state-verification.md) - * [Cross-Program Invocation](proposals/cross-program-invocation.md) - * [Program Keys and Signatures](proposals/program-keys-and-signatures.md) * [Inter-chain Transaction Verification](proposals/interchain-transaction-verification.md) * [Snapshot Verification](proposals/snapshot-verification.md) * [Bankless Leader](proposals/bankless-leader.md) diff --git a/docs/src/implemented-proposals/cross-program-invocation.md b/docs/src/implemented-proposals/cross-program-invocation.md new file mode 100644 index 0000000000..a6071b8c5d --- /dev/null +++ b/docs/src/implemented-proposals/cross-program-invocation.md @@ -0,0 +1,95 @@ +# Cross-Program Invocation + +## Problem + +In today's implementation, a client can create a transaction that modifies two accounts, each owned by a separate on-chain program: + +```rust,ignore +let message = Message::new(vec![ + token_instruction::pay(&alice_pubkey), + acme_instruction::launch_missiles(&bob_pubkey), +]); +client.send_message(&[&alice_keypair, &bob_keypair], &message); +``` + +However, the current implementation does not allow the `acme` program to conveniently invoke `token` instructions on the client's behalf: + +```rust,ignore +let message = Message::new(vec![ + acme_instruction::pay_and_launch_missiles(&alice_pubkey, &bob_pubkey), +]); +client.send_message(&[&alice_keypair, &bob_keypair], &message); +``` + +Currently, there is no way to create instruction `pay_and_launch_missiles` that executes `token_instruction::pay` from the `acme` program. A possible workaround is to extend the `acme` program with the implementation of the `token` program and create `token` accounts with `ACME_PROGRAM_ID`, which the `acme` program is permitted to modify. With that workaround, `acme` can modify token-like accounts created by the `acme` program, but not token accounts created by the `token` program. + +## Proposed Solution + +The goal of this design is to modify Solana's runtime such that an on-chain program can invoke an instruction from another program. + +Given two on-chain programs `token` and `acme`, each implementing instructions `pay()` and `launch_missiles()` respectively, we would ideally like to implement the `acme` module with a call to a function defined in the `token` module: + +```rust,ignore +mod acme { + use token; + + fn launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { + ... + } + + fn pay_and_launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { + token::pay(&keyed_accounts[1..])?; + + launch_missiles(keyed_accounts)?; + } +``` + +The above code would require that the `token` crate be dynamically linked so that a custom linker could intercept calls and validate accesses to `keyed_accounts`. Even though the client intends to modify both `token` and `acme` accounts, only `token` program is permitted to modify the `token` account, and only the `acme` program is allowed to modify the `acme` account. + +Backing off from that ideal direct cross-program call, a slightly more verbose solution is to allow `acme` to invoke `token` by issuing a token instruction via the runtime. + +```rust,ignore +mod acme { + use token_instruction; + + fn launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { + ... + } + + fn pay_and_launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { + let alice_pubkey = keyed_accounts[1].key; + let instruction = token_instruction::pay(&alice_pubkey); + invoke(&instruction, accounts)?; + + launch_missiles(keyed_accounts)?; + } +``` + +`invoke()` is built into Solana's runtime and is responsible for routing the given instruction to the `token` program via the instruction's `program_id` field. + +Before invoking `pay()`, the runtime must ensure that `acme` didn't modify any accounts owned by `token`. It does this by applying the runtime's policy to the current state of the accounts at the time `acme` calls `invoke` vs. the initial state of the accounts at the beginning of the `acme`'s instruction. After `pay()` completes, the runtime must again ensure that `token` didn't modify any accounts owned by `acme` by again applying the runtime's policy, but this time with the `token` program ID. Lastly, after `pay_and_launch_missiles()` completes, the runtime must apply the runtime policy one more time, where it normally would, but using all updated `pre_*` variables. If executing `pay_and_launch_missiles()` up to `pay()` made no invalid account changes, `pay()` made no invalid changes, and executing from `pay()` until `pay_and_launch_missiles()` returns made no invalid changes, then the runtime can transitively assume `pay_and_launch_missiles()` as whole made no invalid account changes, and therefore commit all these account modifications. + +### Instructions that require privileges + +The runtime uses the privileges granted to the caller program to determine what privileges can be extended to the callee. Privileges in this context refer to signers and writable accounts. For example, if the instruction the caller is processing contains a signer or writable account, then the caller can invoke an instruction that also contains that signer and/or writable account. + +This privilege extension relies on the fact that programs are immutable. In the case of the `acme` program, the runtime can safely treat the transaction's signature as a signature of a `token` instruction. When the runtime sees the `token` instruction references `alice_pubkey`, it looks up the key in the `acme` instruction to see if that key corresponds to a signed account. In this case, it does and thereby authorizes the `token` program to modify Alice's account. + +### Program signed accounts + +Programs can issue instructions that contain signed accounts that were not signed in the original transaction by +using [Program derived addresses](program-derived-addresses.md). + +To sign an account with program derived addresses, a program may `invoke_signed()`. + +```rust,ignore + invoke_signed( + &instruction, + accounts, + &[&["First addresses seed"], + &["Second addresses first seed", "Second addresses second seed"]], + )?; + +### Reentrancy + +Reentrancy is currently limited to direct self recursion capped at a fixed depth. This restriction prevents situations where a program might invoke another from an intermediary state without the knowledge that it might later be called back into. Direct recursion gives the program full control of its state at the point that it gets called back. diff --git a/docs/src/implemented-proposals/program-derived-addresses.md b/docs/src/implemented-proposals/program-derived-addresses.md new file mode 100644 index 0000000000..4eed1a0ec4 --- /dev/null +++ b/docs/src/implemented-proposals/program-derived-addresses.md @@ -0,0 +1,154 @@ +# Program Derived Addresses + +## Problem + +Programs cannot generate signatures when issuing instructions to +other programs as defined in the [Cross-Program Invocations](cross-program-invocation.md) +design. + +The lack of programmatic signature generation limits the kinds of programs +that can be implemented in Solana. A program may be given the +authority over an account and later want to transfer that authority to another. +This is impossible today because the program cannot act as the signer in the transaction that gives authority. + +For example, if two users want +to make a wager on the outcome of a game in Solana, they must each +transfer their wager's assets to some intermediary that will honor +their agreement. Currently, there is no way to implement this intermediary +as a program in Solana because the intermediary program cannot transfer the +assets to the winner. + +This capability is necessary for many DeFi applications since they +require assets to be transferred to an escrow agent until some event +occurs that determines the new owner. + +* Decentralized Exchanges that transfer assets between matching bid and +ask orders. + +* Auctions that transfer assets to the winner. + +* Games or prediction markets that collect and redistribute prizes to +the winners. + +## Proposed Solution + +The key to the design is two-fold: + +1. Allow programs to control specific addresses, called Program-Addresses, in such a way that no external +user can generate valid transactions with signatures for those +addresses. + +2. Allow programs to programmatically sign for Program-Addresses that are +present in instructions invoked via [Cross-Program Invocations](cross-program-invocation.md). + +Given the two conditions, users can securely transfer or assign +the authority of on-chain assets to Program-Addresses and the program +can then assign that authority elsewhere at its discretion. + +### Private keys for Program Addresses + +A Program -Address has no private key associated with it, and generating +a signature for it is impossible. While it has no private key of +its own, it can issue an instruction that includes the Program-Address as a signer. + +### Hash-based generated Program Addresses + +All 256-bit values are valid ed25519 curve points and valid ed25519 public +keys. All are equally secure and equally as hard to break. +Based on this assumption, Program Addresses can be deterministically +derived from a base seed using a 256-bit preimage resistant hash function. + +Deterministic Program Addresses for programs follow a similar derivation +path as Accounts created with `SystemInstruction::CreateAccountWithSeed` +which is implemented with `system_instruction::create_address_with_seed`. + +For reference that implementation is as follows: + +```rust,ignore +pub fn create_address_with_seed( + base: &Pubkey, + seed: &str, + program_id: &Pubkey, +) -> Result { + if seed.len() > MAX_ADDRESS_SEED_LEN { + return Err(SystemError::MaxSeedLengthExceeded); + } + + Ok(Pubkey::new( + hashv(&[base.as_ref(), seed.as_ref(), program_id.as_ref()]).as_ref(), + )) +} +``` + +Programs can deterministically derive any number of addresses by +using keywords. These keywords can symbolically identify how the addresses are used. + +```rust,ignore +//! Generate a derived program address +//! * seeds, symbolic keywords used to derive the key +//! * owner, program that the key is derived for +pub fn create_program_address(seeds: &[&str], owner: &Pubkey) -> Result { + let mut hasher = Hasher::default(); + for seed in seeds.iter() { + if seed.len() > MAX_SEED_LEN { + return Err(PubkeyError::MaxSeedLengthExceeded); + } + hasher.hash(seed.as_ref()); + } + hasher.hashv(&[owner.as_ref(), "ProgramDerivedAddress".as_ref()]); + + Ok(Pubkey::new(hashv(&[hasher.result().as_ref()]).as_ref())) +} +``` + +### Using Program Addresses + +Clients can use the `create_program_address` function to generate +a destination address. + +```rust,ignore +//deterministically derive the escrow key +let escrow_pubkey = create_program_address(&[&["escrow"]], &escrow_program_id); +let message = Message::new(vec![ + token_instruction::transfer(&alice_pubkey, &escrow_pubkey, 1), +]); +//transfer 1 token to escrow +client.send_message(&[&alice_keypair], &message); +``` + +Programs can use the same function to generate the same address. +In the function below the program issues a `token_instruction::transfer` from +Program Address as if it had the private key to sign the transaction. + +```rust,ignore +fn transfer_one_token_from_escrow( + program_id: &Pubkey, + keyed_accounts: &[KeyedAccount] +) -> Result<()> { + + // User supplies the destination + let alice_pubkey = keyed_accounts[1].unsigned_key(); + + // Deterministically derive the escrow pubkey. + let escrow_pubkey = create_program_address(&[&["escrow"]], program_id); + + // Create the transfer instruction + let instruction = token_instruction::transfer(&escrow_pubkey, &alice_pubkey, 1); + + // The runtime deterministically derives the key from the currently + // executing program ID and the supplied keywords. + // If the derived key matches a key marked as signed in the instruction + // then that key is accepted as signed. + invoke_signed(&instruction, &[&["escrow"]])? +} +``` + +### Instructions that require signers + +The addresses generated with `create_program_address` are indistinguishable +from any other public key. The only way for the runtime to verify that the +address belongs to a program is for the program to supply the keywords used +to generate the address. + +The runtime will internally call `create_program_address`, and compare the +result against the addresses supplied in the instruction. \ No newline at end of file diff --git a/docs/src/proposals/cross-program-invocation.md b/docs/src/proposals/cross-program-invocation.md deleted file mode 100644 index 9dd1c2167d..0000000000 --- a/docs/src/proposals/cross-program-invocation.md +++ /dev/null @@ -1,71 +0,0 @@ -# Cross-Program Invocation - -## Problem - -In today's implementation a client can create a transaction that modifies two accounts, each owned by a separate on-chain program: - -```text -let message = Message::new(vec![ - token_instruction::pay(&alice_pubkey), - acme_instruction::launch_missiles(&bob_pubkey), -]); -client.send_message(&[&alice_keypair, &bob_keypair], &message); -``` - -The current implementation does not, however, allow the `acme` program to conveniently invoke `token` instructions on the client's behalf: - -```text -let message = Message::new(vec![ - acme_instruction::pay_and_launch_missiles(&alice_pubkey, &bob_pubkey), -]); -client.send_message(&[&alice_keypair, &bob_keypair], &message); -``` - -Currently, there is no way to create instruction `pay_and_launch_missiles` that executes `token_instruction::pay` from the `acme` program. The workaround is to extend the `acme` program with the implementation of the `token` program, and create `token` accounts with `ACME_PROGRAM_ID`, which the `acme` program is permitted to modify. With that workaround, `acme` can modify token-like accounts created by the `acme` program, but not token accounts created by the `token` program. - -## Proposed Solution - -The goal of this design is to modify Solana's runtime such that an on-chain program can invoke an instruction from another program. - -Given two on-chain programs `token` and `acme`, each implementing instructions `pay()` and `launch_missiles()` respectively, we would ideally like to implement the `acme` module with a call to a function defined in the `token` module: - -```text -use token; - -fn launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { - ... -} - -fn pay_and_launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { - token::pay(&keyed_accounts[1..])?; - - launch_missiles(keyed_accounts)?; -} -``` - -The above code would require that the `token` crate be dynamically linked, so that a custom linker could intercept calls and validate accesses to `keyed_accounts`. That is, even though the client intends to modify both `token` and `acme` accounts, only `token` program is permitted to modify the `token` account, and only the `acme` program is permitted to modify the `acme` account. - -Backing off from that ideal cross-program call, a slightly more verbose solution is to expose token's existing `process_instruction()` entrypoint to the acme program: - -```text -use token_instruction; - -fn launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { - ... -} - -fn pay_and_launch_missiles(keyed_accounts: &[KeyedAccount]) -> Result<()> { - let alice_pubkey = keyed_accounts[1].key; - let instruction = token_instruction::pay(&alice_pubkey); - process_instruction(&instruction)?; - - launch_missiles(keyed_accounts)?; -} -``` - -where `process_instruction()` is built into Solana's runtime and responsible for routing the given instruction to the `token` program via the instruction's `program_id` field. Before invoking `pay()`, the runtime must also ensure that `acme` didn't modify any accounts owned by `token`. It does this by calling `runtime::verify_account_changes()` and then afterward updating all the `pre_*` variables to tentatively commit `acme`'s account modifications. After `pay()` completes, the runtime must again ensure that `token` didn't modify any accounts owned by `acme`. It should call `verify_account_changes()` again, but this time with the `token` program ID. Lastly, after `pay_and_launch_missiles()` completes, the runtime must call `verify_account_changes()` one more time, where it normally would, but using all updated `pre_*` variables. If executing `pay_and_launch_missiles()` up to `pay()` made no invalid account changes, `pay()` made no invalid changes, and executing from `pay()` until `pay_and_launch_missiles()` returns made no invalid changes, then the runtime can transitively assume `pay_and_launch_missiles()` as whole made no invalid account changes, and therefore commit all account modifications. - -### Setting `KeyedAccount.is_signer` - -When `process_instruction()` is invoked, the runtime must create a new `KeyedAccounts` parameter using the signatures from the _original_ transaction data. Since the `token` program is immutable and existed on-chain prior to the `acme` program, the runtime can safely treat the transaction signature as a signature of a transaction with a `token` instruction. When the runtime sees the given instruction references `alice_pubkey`, it looks up the key in the transaction to see if that key corresponds to a transaction signature. In this case it does and so sets `KeyedAccount.is_signer`, thereby authorizing the `token` program to modify Alice's account. - diff --git a/docs/src/proposals/program-keys-and-signatures.md b/docs/src/proposals/program-keys-and-signatures.md deleted file mode 100644 index 6d32ef0254..0000000000 --- a/docs/src/proposals/program-keys-and-signatures.md +++ /dev/null @@ -1,169 +0,0 @@ -# Program Keys and Signatures - -## Problem - -Programs cannot generate their own signatures in `process_instruction` -as defined in the [Cross-Program Invocations](cross-program-invocation.md) -design. - -Lack of programmatic signature generation limits the kinds of programs -that can be implemented in Solana. For example, a program cannot take -ownership of a TokenAccount and later in a different transaction transfer -the ownership based on the state of another program. If two users want -to make a wager in tokens on the outcome of a game in Solana, they must -transfer tokens to some intermediary that will honor their agreement. -Currently there is no way to implement this intermediary as a program -in Solana. - -This capability is necessary for many DeFi applications, since they -require assets to be transferred to an escrow agent until some event -occurs that determines the new owner. - -* Decentralized Exchanges that transfer assets between matching bid and -ask orders. - -* Auctions that transfer assets to the winner. - -* Games or prediction markets that collect and redistribute prizes to -the winners. - -## Proposed Solution - -The key to the design is two fold: - -1. Allow programs to control specific addresses, called Program -Addresses, in such a way that it is impossible for any external -user to generate valid transactions with signatures for those -addresses. - -2. To allow programs to programatically control -`KeyedAccount::is_signer` value for Program Addresses that are -present in instructions that is invoked via `process_instruction()`. - -Given the two conditions, users can securely transfer or assign -ownershp of on chain assets to Program Addresses. Once assigned, -the program and only the program can execute instructions that -refences a Program Address with `KeyedAccount::is_signer` set to -true. - -### Private keys for Program Addresses - -This address has no private key associated with it, and generating -a signature for it is impossible. While it has no private key of -its own, the program can issue an instruction to set the -`KeyedAccount::is_signer` flag for this address. - -### Hash based generated Program Addresses - -All 256 bit values are valid ed25519 curve points, and valid ed25519 public -keys. All are equally secure and equally as hard to break. -Based on this assumption, Program Addresses can be deterministically -derived from a base seed using a 256 bit preimage resistant hash function. - -Deterministic Program Addresses for programs follow a similar derivation -path as Accounts created with `SystemInstruction::CreateAccountWithSeed` -which is implemented with `system_instruction::create_address_with_seed`. - -For reference the implementation is as follows: - -```rust,ignore -pub fn create_address_with_seed( - base: &Pubkey, - seed: &str, - program_id: &Pubkey, -) -> Result { - if seed.len() > MAX_ADDRESS_SEED_LEN { - return Err(SystemError::MaxSeedLengthExceeded); - } - - Ok(Pubkey::new( - hashv(&[base.as_ref(), seed.as_ref(), program_id.as_ref()]).as_ref(), - )) -} -``` - -Programs can deterministically derive any number of addresses by -using a keyword. The keyword can symbolically identify how this -address is used. - -```rust,ignore -//! Generate a derived program address -//! * program_id, the program's id -//! * key_base, can be any public key chosen by the program -//! * keyword, symbolic keyword to identify the key -//! -//! The tuple (`key_base`, `keyword`) is used by programs to create user specific -//! symbolic keys. For example for the staking contact, the program may need: -//! * /<"withdrawer"> -//! * /<"staker"> -//! * /<"custodian"> -//! As generated keys to control a single stake account for each user. -pub fn derive_program_address( - program_id: &Pubkey, - key_base, &Pubkey, - keyword, &str, -) -> Result { - - // Generate a deterministic base for all program addresses that - // are owned by `program_id`. - // Hashing twice is recommended to prevent lenght extension attacks. - Ok(Pubkey::new( - hashv(&[hashv(&[program_id.as_ref(), key_base.as_ref(), keyword.as_ref(), - &"ProgramAddress11111111111111111111111111111"]).as_ref()]) - )) -} -``` - -### Using Program Addresses - -Clients can use the `derive_program_address` function to generate -a destination address. - -```rust,ignore -//deterministically derive the escrow key -let escrow_pubkey = derive_program_address(&escrow_program_id, &alice_pubkey, &"escrow"); -let message = Message::new(vec![ - token_instruction::transfer(&alice_pubkey, &escrow_pubkey, 1), -]); -//transfer 1 token to escrow -client.send_message(&[&alice_keypair], &message); -``` - -Programs can use the same function to generate the same address. -Below the program issue a `token_instruction::transfer` from its -own address as if it had a private key to sign the transaction. - -```rust,ignore -fn transfer_one_token_from_escrow( - program_id: &Pubkey, - keyed_accounts: &[KeyedAccount] -) -> Result<()> { - - - //user supplies the destination - let alice_pubkey = keyed_accounts[1].key; - - // Deterministically derive the escrow pubkey. - let escrow_pubkey = derive_program_address(program_id, &alice_pubkey, &"escrow"); - - //create the transfer instruction - let instruction = token_instruction::transfer(&escrow_pubkey, &alice_pubkey, 1); - - // The runtime deterministically derives the key from the current - // program id and the supplied keywords. - // If the derived key matches a key in the instruction - // the `is_signed` flag is set. - process_signed_instruction(&instruction, &[(&alice_pubkey, &"escrow")])? -} -``` - -### Setting `KeyedAccount::is_signer` - -The addresses generated with `derive_program_address` are blinded -and are indistinguishable from any other pubkey. The only way for -the runtime to verify that the address belongs to a program is for -the program to supply the keyword used to generate the address. - -The runtime will internally run `derive_program_address(program_id, -&alice_pubkey, &"escrow")`, and compare the result against the addresses -supplied in the instruction.