Add return data implementation

This consists of:
 - syscalls
 - passing return data from invoked to invoker
 - printing to stable log
 - rust and C SDK changes

Cargo.lock

@@ -5520,6 +5520,7 @@ name = "solana-sdk"
 version = "1.8.0"
 dependencies = [
  "assert_matches",
+ "base64 0.13.0",
  "bincode",
  "borsh",
  "borsh-derive",

docs/src/proposals/return-data.md

@@ -0,0 +1,144 @@
+# Return data from BPF programs
+
+## Problem
+
+In the Solidity langauge it is permitted to return any number of values from a function,
+for example a variable length string can be returned:
+
+```
+function foo1() public returns (string) {
+    return "Hello, world!\n";
+}
+```
+
+Multiple values, arrays and structs are permitted too.
+
+```
+struct S {
+    int f1;
+    bool f2
+};
+
+function foo2() public returns (string, int[], S) {
+    return (a, b, c);
+}
+```
+
+All the return values are eth abi encoded to a variable-length byte array.
+
+On ethereum errors can be returned too:
+
+```
+function withdraw() public {
+    require(msg.sender == owner, "Permission denied");
+}
+
+function failure() public {
+    revert("I afraid I can't do that dave");
+}
+```
+These errors help the developer debug any issue they are having, and can
+also be caught in a Solidity `try` .. `catch` block. Outside of a `try` .. `catch`
+block, any of these would cause the transaction or rpc to fail.
+
+## Existing solution
+
+The existing solution that Solang uses, writes the return data to the callee account data.
+The caller's account cannot be used, since the callee may not be the same BPF program, so
+it will not have permission to write to the callee's account data.
+
+Another solution would be to have a single return data account which is passed
+around through CPI. Again this does not work for CPI as the callee may not have
+permission to write to it.
+
+The problem with this solution is:
+
+- It does not work for RPC calls
+- It is very racey; a client has to submit the Tx and then retrieve the account
+  data. This is not atomic so the return data can be overwritten by another transaction.
+
+## Requirements for Solution
+
+It must work for:
+
+- RPC: An RPC should be able to return any number of values without writing to account data
+- Transaction: An transaction should be able to return any number of values without needing to write them account data
+- CPI: The callee must "set" return value, and the caller must be able to retrieve it.
+
+## Review of other chains
+
+### Ethereum (EVM)
+
+The `RETURN` opcode allows a contract to set a buffer as a returndata. This opcode takes a pointer to memory and a size. The `REVERT` opcode works similarly but signals that the call failed, and all account data changes must be reverted.
+
+For CPI, the caller can retrieve the returned data of the callee using the `RETURNDATASIZE` opcode which returns the length, and the `RETURNDATACOPY` opcode, which takes a memory destination pointer, offset into the returndata, and a length argument.
+
+Ethereum stores the returndata in blocks.
+
+### Parity Substrate
+
+The return data can be set using the `seal_return(u32 flags, u32 pointer, u32 size)` syscall.
+- Flags can be 1 for revert, 0 for success (nothing else defined)
+- Function does not return
+
+CPI: The `seal_call()` syscall takes pointer to buffer and pointer to buffer size where return data goes
+ - There is a 32KB limit for return data.
+
+Parity Substrate does not write the return data to blocks.
+
+## Rejected Solution
+
+The concept of ephemeral accounts has been proposed a solution for this. This would
+certainly work for the CPI case, but this would not work RPC or Transaction case.
+
+## Proposed Solution
+
+The callee can set the return data using a new system call `sol_set_return_data(buf: *const u8, length: u64)`.
+There is a limit of 1024 bytes for the returndata. This function can be called multiple times, and
+will simply overwrite what was written in the last call.
+
+The return data can be retrieved with `sol_get_return_data(buf: *mut u8, length: u64, program_id: *mut Pubkey) -> u64`.
+This function copies the return buffer, and the program_id that set the return data, and
+returns the length of the return data, or `0` if no return data is set. In this case, program_id is not set.
+
+When an instruction calls `sol_invoke()`, the return data of the callee is copied into the return data
+of the current instruction. This means that any return data is automatically passed up the call stack,
+to the callee of the current instruction (or the RPC call).
+
+Note that `sol_invoke()` clears the returns data before invoking the callee, so that any return data from
+a previous invoke is not reused if the invoked fails to set a return data. For example:
+
+ - A invokes B
+ - Before entry to B, return data is cleared.0
+ - B sets some return data and returns
+ - A invokes C
+ - Before entry to C, return data is cleared.
+ - C does not set return data and returns
+ - A checks return data and finds it empty
+
+Another scenario to consider:
+
+ - A invokes B
+ - B invokes C
+ - C sets return data and returns
+ - B does not touch return data and returns
+ - A gets return data from C
+ - A does not touch return data
+ - Return data from transaction is what C set.
+
+The compute costs are calculated for getting and setting the return data using
+the syscalls.
+
+For a normal RPC or Transaction, the returndata is base64-encoded and stored along side the sol_log
+strings in the [stable log](https://github.com/solana-labs/solana/blob/95292841947763bdd47ef116b40fc34d0585bca8/sdk/src/process_instruction.rs#L275-L281).
+
+## Note on returning errors
+
+Solidity on Ethereum allows the contract to return an error in the return data. In this case, all
+the account data changes for the account should be reverted. On Solana, any non-zero exit code
+for a BPF prorgram means the entire transaction fails. We do not wish to support an error return
+by returning success and then returning an error in the return data. This would mean we would have
+to support reverting the account data changes; this too expensive both on the VM side and the BPF
+contract side.
+
+Errors will be reported via sol_log.

program-runtime/src/instruction_processor.rs

@@ -631,6 +631,9 @@ impl InstructionProcessor {
             // Verify the calling program hasn't misbehaved
             invoke_context.verify_and_update(instruction, accounts, caller_write_privileges)?;
 
+            // clear the return data
+            invoke_context.set_return_data(None);
+
             // Invoke callee
             invoke_context.push(program_id, message, instruction, program_indices, accounts)?;
 

programs/bpf/Cargo.lock

@@ -3260,6 +3260,7 @@ name = "solana-sdk"
 version = "1.8.0"
 dependencies = [
  "assert_matches",
+ "base64 0.13.0",
  "bincode",
  "borsh",
  "borsh-derive",

programs/bpf/c/src/invoke/invoke.c

@@ -8,6 +8,7 @@
 #include <sol/log.h>
 #include <sol/assert.h>
 #include <sol/deserialize.h>
+#include <sol/return_data.h>
 
 static const uint8_t TEST_SUCCESS = 1;
 static const uint8_t TEST_PRIVILEGE_ESCALATION_SIGNER = 2;
@@ -26,6 +27,7 @@ static const uint8_t TEST_WRITABLE_DEESCALATION_WRITABLE = 14;
 static const uint8_t TEST_NESTED_INVOKE_TOO_DEEP = 15;
 static const uint8_t TEST_EXECUTABLE_LAMPORTS = 16;
 static const uint8_t ADD_LAMPORTS = 17;
+static const uint8_t TEST_RETURN_DATA_TOO_LARGE = 18;
 
 static const int MINT_INDEX = 0;
 static const int ARGUMENT_INDEX = 1;
@@ -174,6 +176,32 @@ extern uint64_t entrypoint(const uint8_t *input) {
                  sol_invoke(&instruction, accounts, SOL_ARRAY_SIZE(accounts)));
     }
 
+    sol_log("Test return data");
+    {
+      SolAccountMeta arguments[] = {{accounts[ARGUMENT_INDEX].key, true, true}};
+      uint8_t data[] = { SET_RETURN_DATA };
+      uint8_t buf[100];
+
+      const SolInstruction instruction = {accounts[INVOKED_PROGRAM_INDEX].key,
+                                          arguments, SOL_ARRAY_SIZE(arguments),
+                                          data, SOL_ARRAY_SIZE(data)};
+
+      // set some return data, so that the callee can check it is cleared
+      sol_set_return_data((uint8_t[]){1, 2, 3, 4}, 4);
+
+      sol_assert(SUCCESS ==
+                 sol_invoke(&instruction, accounts, SOL_ARRAY_SIZE(accounts)));
+
+      SolPubkey setter;
+
+      uint64_t ret = sol_get_return_data(data, sizeof(data), &setter);
+
+      sol_assert(ret == sizeof(RETURN_DATA_VAL));
+
+      sol_assert(sol_memcmp(data, RETURN_DATA_VAL, sizeof(RETURN_DATA_VAL)));
+      sol_assert(SolPubkey_same(&setter, accounts[INVOKED_PROGRAM_INDEX].key));
+    }
+
     sol_log("Test create_program_address");
     {
       uint8_t seed1[] = {'Y', 'o', 'u', ' ', 'p', 'a', 's', 's',
@@ -564,6 +592,12 @@ extern uint64_t entrypoint(const uint8_t *input) {
     *accounts[0].lamports += 1;
      break;
   }
+  case TEST_RETURN_DATA_TOO_LARGE: {
+    sol_log("Test setting return data too long");
+    // The actual buffer doesn't matter, just pass null
+    sol_set_return_data(NULL, 1027);
+    break;
+  }
 
   default:
     sol_panic();

programs/bpf/c/src/invoked/instruction.h

@@ -16,3 +16,6 @@ const uint8_t VERIFY_PRIVILEGE_DEESCALATION = 8;
 const uint8_t VERIFY_PRIVILEGE_DEESCALATION_ESCALATION_SIGNER = 9;
 const uint8_t VERIFY_PRIVILEGE_DEESCALATION_ESCALATION_WRITABLE = 10;
 const uint8_t WRITE_ACCOUNT = 11;
+const uint8_t SET_RETURN_DATA = 12;
+
+#define RETURN_DATA_VAL "return data test"

programs/bpf/c/src/invoked/invoked.c

@@ -14,6 +14,9 @@ extern uint64_t entrypoint(const uint8_t *input) {
     return ERROR_INVALID_ARGUMENT;
   }
 
+  // on entry, return data must not be set
+  sol_assert(sol_get_return_data(NULL, 0, NULL) == 0);
+
   if (params.data_len == 0) {
     return SUCCESS;
   }
@@ -91,6 +94,12 @@ extern uint64_t entrypoint(const uint8_t *input) {
     sol_log("return Ok");
     return SUCCESS;
   }
+  case SET_RETURN_DATA: {
+    sol_set_return_data((const uint8_t*)RETURN_DATA_VAL, sizeof(RETURN_DATA_VAL));
+    sol_log("set return data");
+    sol_assert(sol_get_return_data(NULL, 0, NULL) == sizeof(RETURN_DATA_VAL));
+    return SUCCESS;
+  }
   case RETURN_ERROR: {
     sol_log("return error");
     return 42;

programs/bpf/c/src/return_data/return_data.c

@@ -0,0 +1,40 @@
+/**
+ * @brief return data Syscall test
+ */
+#include <solana_sdk.h>
+
+#define DATA "the quick brown fox jumps over the lazy dog"
+
+extern uint64_t entrypoint(const uint8_t *input) {
+  uint8_t buf[1024];
+  SolPubkey me;
+
+  // There should be no return data on entry
+  uint64_t ret = sol_get_return_data(NULL, 0, NULL);
+
+  sol_assert(ret == 0);
+
+  // set some return data
+  sol_set_return_data((const uint8_t*)DATA, sizeof(DATA));
+
+  // ensure the length is correct
+  ret = sol_get_return_data(NULL, 0, &me);
+  sol_assert(ret == sizeof(DATA));
+
+  // try getting a subset
+  ret = sol_get_return_data(buf, 4, &me);
+
+  sol_assert(ret == sizeof(DATA));
+
+  sol_assert(!sol_memcmp(buf, "the ", 4));
+
+  // try getting the whole thing
+  ret = sol_get_return_data(buf, sizeof(buf), &me);
+
+  sol_assert(ret == sizeof(DATA));
+
+  sol_assert(!sol_memcmp(buf, (const uint8_t*)DATA, sizeof(DATA)));
+
+  // done
+  return SUCCESS;
+}

programs/bpf/rust/invoke/src/lib.rs

@@ -10,7 +10,7 @@ use solana_program::{
     entrypoint,
     entrypoint::{ProgramResult, MAX_PERMITTED_DATA_INCREASE},
     msg,
-    program::{invoke, invoke_signed},
+    program::{get_return_data, invoke, invoke_signed, set_return_data},
     program_error::ProgramError,
     pubkey::{Pubkey, PubkeyError},
     system_instruction,
@@ -394,6 +394,27 @@ fn process_instruction(
                     assert_eq!(data[i], i as u8);
                 }
             }
+
+            msg!("Test return data via invoked");
+            {
+                // this should be cleared on entry, the invoked tests for this
+                set_return_data(b"x");
+
+                let instruction = create_instruction(
+                    *accounts[INVOKED_PROGRAM_INDEX].key,
+                    &[(accounts[ARGUMENT_INDEX].key, false, true)],
+                    vec![SET_RETURN_DATA],
+                );
+                let _ = invoke(&instruction, accounts);
+
+                assert_eq!(
+                    get_return_data(),
+                    Some((
+                        *accounts[INVOKED_PROGRAM_INDEX].key,
+                        b"Set by invoked".to_vec()
+                    ))
+                );
+            }
         }
         TEST_PRIVILEGE_ESCALATION_SIGNER => {
             msg!("Test privilege escalation signer");

programs/bpf/rust/invoked/src/instruction.rs

@@ -18,6 +18,7 @@ pub const VERIFY_PRIVILEGE_DEESCALATION_ESCALATION_SIGNER: u8 = 9;
 pub const VERIFY_PRIVILEGE_DEESCALATION_ESCALATION_WRITABLE: u8 = 10;
 pub const WRITE_ACCOUNT: u8 = 11;
 pub const CREATE_AND_INIT: u8 = 12;
+pub const SET_RETURN_DATA: u8 = 13;
 
 pub fn create_instruction(
     program_id: Pubkey,

programs/bpf/rust/invoked/src/processor.rs

@@ -8,7 +8,7 @@ use solana_program::{
     bpf_loader, entrypoint,
     entrypoint::{ProgramResult, MAX_PERMITTED_DATA_INCREASE},
     msg,
-    program::{invoke, invoke_signed},
+    program::{get_return_data, invoke, invoke_signed, set_return_data},
     program_error::ProgramError,
     pubkey::Pubkey,
     system_instruction,
@@ -27,6 +27,8 @@ fn process_instruction(
         return Ok(());
     }
 
+    assert_eq!(get_return_data(), None);
+
     match instruction_data[0] {
         VERIFY_TRANSLATIONS => {
             msg!("verify data translations");
@@ -286,6 +288,11 @@ fn process_instruction(
                 }
             }
         }
+        SET_RETURN_DATA => {
+            msg!("Set return data");
+
+            set_return_data(b"Set by invoked");
+        }
         _ => panic!(),
     }
 

programs/bpf/tests/programs.rs

@@ -227,6 +227,8 @@ fn run_program(
     for i in 0..2 {
         let mut parameter_bytes = parameter_bytes.clone();
         {
+            invoke_context.set_return_data(None);
+
             let mut vm = create_vm(
                 &loader_id,
                 executable.as_ref(),
@@ -432,6 +434,7 @@ fn test_program_bpf_sanity() {
             ("noop++", true),
             ("panic", false),
             ("relative_call", true),
+            ("return_data", true),
             ("sanity", true),
             ("sanity++", true),
             ("secp256k1_recover", true),
@@ -756,6 +759,7 @@ fn test_program_bpf_invoke_sanity() {
     const TEST_WRITABLE_DEESCALATION_WRITABLE: u8 = 14;
     const TEST_NESTED_INVOKE_TOO_DEEP: u8 = 15;
     const TEST_EXECUTABLE_LAMPORTS: u8 = 16;
+    const TEST_RETURN_DATA_TOO_LARGE: u8 = 18;
 
     #[allow(dead_code)]
     #[derive(Debug)]
@@ -878,6 +882,7 @@ fn test_program_bpf_invoke_sanity() {
                 invoked_program_id.clone(),
                 invoked_program_id.clone(),
                 invoked_program_id.clone(),
+                invoked_program_id.clone(),
             ],
             Languages::Rust => vec![
                 system_program::id(),
@@ -902,6 +907,7 @@ fn test_program_bpf_invoke_sanity() {
                 invoked_program_id.clone(),
                 invoked_program_id.clone(),
                 system_program::id(),
+                invoked_program_id.clone(),
             ],
         };
         assert_eq!(invoked_programs.len(), expected_invoked_programs.len());
@@ -1030,6 +1036,12 @@ fn test_program_bpf_invoke_sanity() {
             &[invoke_program_id.clone()],
         );
 
+        do_invoke_failure_test_local(
+            TEST_RETURN_DATA_TOO_LARGE,
+            TransactionError::InstructionError(0, InstructionError::ProgramFailedToComplete),
+            &[],
+        );
+
         // Check resulting state
 
         assert_eq!(43, bank.get_balance(&derived_key1));
@@ -1312,6 +1324,7 @@ fn assert_instruction_count() {
             ("noop", 5),
             ("noop++", 5),
             ("relative_call", 10),
+            ("return_data", 480),
             ("sanity", 169),
             ("sanity++", 168),
             ("secp256k1_recover", 359),

programs/bpf_loader/src/lib.rs

@@ -917,6 +917,11 @@ impl Executor for BpfExecutor {
                 let trace_string = String::from_utf8(trace_buffer).unwrap();
                 trace!("BPF Program Instruction Trace:\n{}", trace_string);
             }
+            drop(vm);
+            let return_data = invoke_context.get_return_data();
+            if let Some((program_id, return_data)) = return_data {
+                stable_log::program_return_data(&logger, program_id, return_data);
+            }
             match result {
                 Ok(status) => {
                     if status != SUCCESS {

programs/bpf_loader/src/syscalls.rs

@@ -24,7 +24,7 @@ use solana_sdk::{
         allow_native_ids, blake3_syscall_enabled, check_seed_length,
         close_upgradeable_program_accounts, demote_program_write_locks, disable_fees_sysvar,
         enforce_aligned_host_addrs, libsecp256k1_0_5_upgrade_enabled, mem_overlap_fix,
-        secp256k1_recover_syscall_enabled,
+        return_data_syscall_enabled, secp256k1_recover_syscall_enabled,
     },
     hash::{Hasher, HASH_BYTES},
     ic_msg,
@@ -33,6 +33,7 @@ use solana_sdk::{
     keyed_account::KeyedAccount,
     native_loader,
     process_instruction::{self, stable_log, ComputeMeter, InvokeContext, Logger},
+    program::MAX_RETURN_DATA,
     pubkey::{Pubkey, PubkeyError, MAX_SEEDS, MAX_SEED_LEN},
     rent::Rent,
     secp256k1_recover::{
@@ -43,6 +44,7 @@ use solana_sdk::{
 use std::{
     alloc::Layout,
     cell::{Ref, RefCell, RefMut},
+    cmp::min,
     mem::{align_of, size_of},
     rc::Rc,
     slice::from_raw_parts_mut,
@@ -62,9 +64,9 @@ pub enum SyscallError {
     Abort,
     #[error("BPF program Panicked in {0} at {1}:{2}")]
     Panic(String, u64, u64),
-    #[error("cannot borrow invoke context")]
+    #[error("Cannot borrow invoke context")]
     InvokeContextBorrowFailed,
-    #[error("malformed signer seed: {0}: {1:?}")]
+    #[error("Malformed signer seed: {0}: {1:?}")]
     MalformedSignerSeed(Utf8Error, Vec<u8>),
     #[error("Could not create program address with signer seeds: {0}")]
     BadSeeds(PubkeyError),
@@ -82,6 +84,8 @@ pub enum SyscallError {
     TooManyAccounts,
     #[error("Overlapping copy")]
     CopyOverlapping,
+    #[error("Return data too large ({0} > {1})")]
+    ReturnDataTooLarge(u64, u64),
 }
 impl From<SyscallError> for EbpfError<BpfError> {
     fn from(error: SyscallError) -> Self {
@@ -173,6 +177,14 @@ pub fn register_syscalls(
     // Memory allocator
     syscall_registry.register_syscall_by_name(b"sol_alloc_free_", SyscallAllocFree::call)?;
 
+    // Return data
+    if invoke_context.is_feature_active(&return_data_syscall_enabled::id()) {
+        syscall_registry
+            .register_syscall_by_name(b"sol_set_return_data", SyscallSetReturnData::call)?;
+        syscall_registry
+            .register_syscall_by_name(b"sol_get_return_data", SyscallGetReturnData::call)?;
+    }
+
     Ok(syscall_registry)
 }
 
@@ -353,6 +365,8 @@ pub fn bind_syscall_context_objects<'a>(
 
     let is_fee_sysvar_via_syscall_active =
         !invoke_context.is_feature_active(&disable_fees_sysvar::id());
+    let is_return_data_syscall_active =
+        invoke_context.is_feature_active(&return_data_syscall_enabled::id());
 
     let invoke_context = Rc::new(RefCell::new(invoke_context));
 
@@ -386,6 +400,25 @@ pub fn bind_syscall_context_objects<'a>(
         None,
     )?;
 
+    // Return data
+    bind_feature_gated_syscall_context_object!(
+        vm,
+        is_return_data_syscall_active,
+        Box::new(SyscallSetReturnData {
+            invoke_context: invoke_context.clone(),
+            loader_id,
+        }),
+    );
+
+    bind_feature_gated_syscall_context_object!(
+        vm,
+        is_return_data_syscall_active,
+        Box::new(SyscallGetReturnData {
+            invoke_context: invoke_context.clone(),
+            loader_id,
+        }),
+    );
+
     // Cross-program invocation syscalls
     vm.bind_syscall_context_object(
         Box::new(SyscallInvokeSignedC {
@@ -2548,6 +2581,142 @@ fn call<'a>(
     Ok(SUCCESS)
 }
 
+// Return data handling
+pub struct SyscallSetReturnData<'a> {
+    invoke_context: Rc<RefCell<&'a mut dyn InvokeContext>>,
+    loader_id: &'a Pubkey,
+}
+impl<'a> SyscallObject<BpfError> for SyscallSetReturnData<'a> {
+    fn call(
+        &mut self,
+        addr: u64,
+        len: u64,
+        _arg3: u64,
+        _arg4: u64,
+        _arg5: u64,
+        memory_mapping: &MemoryMapping,
+        result: &mut Result<u64, EbpfError<BpfError>>,
+    ) {
+        let mut invoke_context = question_mark!(
+            self.invoke_context
+                .try_borrow_mut()
+                .map_err(|_| SyscallError::InvokeContextBorrowFailed),
+            result
+        );
+
+        let budget = invoke_context.get_compute_budget();
+
+        question_mark!(
+            invoke_context
+                .get_compute_meter()
+                .consume(len / budget.cpi_bytes_per_unit + budget.syscall_base_cost),
+            result
+        );
+
+        if len > MAX_RETURN_DATA as u64 {
+            *result = Err(SyscallError::ReturnDataTooLarge(len, MAX_RETURN_DATA as u64).into());
+            return;
+        }
+
+        if len == 0 {
+            invoke_context.set_return_data(None);
+        } else {
+            let return_data = question_mark!(
+                translate_slice::<u8>(memory_mapping, addr, len, self.loader_id, true),
+                result
+            );
+
+            let program_id = *question_mark!(
+                invoke_context
+                    .get_caller()
+                    .map_err(SyscallError::InstructionError),
+                result
+            );
+
+            invoke_context.set_return_data(Some((program_id, return_data.to_vec())));
+        }
+
+        *result = Ok(0);
+    }
+}
+
+pub struct SyscallGetReturnData<'a> {
+    invoke_context: Rc<RefCell<&'a mut dyn InvokeContext>>,
+    loader_id: &'a Pubkey,
+}
+impl<'a> SyscallObject<BpfError> for SyscallGetReturnData<'a> {
+    fn call(
+        &mut self,
+        return_data_addr: u64,
+        len: u64,
+        program_id_addr: u64,
+        _arg4: u64,
+        _arg5: u64,
+        memory_mapping: &MemoryMapping,
+        result: &mut Result<u64, EbpfError<BpfError>>,
+    ) {
+        let invoke_context = question_mark!(
+            self.invoke_context
+                .try_borrow()
+                .map_err(|_| SyscallError::InvokeContextBorrowFailed),
+            result
+        );
+
+        let budget = invoke_context.get_compute_budget();
+
+        question_mark!(
+            invoke_context
+                .get_compute_meter()
+                .consume(budget.syscall_base_cost),
+            result
+        );
+
+        if let Some((program_id, return_data)) = invoke_context.get_return_data() {
+            if len != 0 {
+                let length = min(return_data.len() as u64, len);
+
+                question_mark!(
+                    invoke_context
+                        .get_compute_meter()
+                        .consume((length + size_of::<Pubkey>() as u64) / budget.cpi_bytes_per_unit),
+                    result
+                );
+
+                let return_data_result = question_mark!(
+                    translate_slice_mut::<u8>(
+                        memory_mapping,
+                        return_data_addr,
+                        length,
+                        self.loader_id,
+                        true,
+                    ),
+                    result
+                );
+
+                return_data_result.copy_from_slice(&return_data[..length as usize]);
+
+                let program_id_result = question_mark!(
+                    translate_slice_mut::<Pubkey>(
+                        memory_mapping,
+                        program_id_addr,
+                        1,
+                        self.loader_id,
+                        true,
+                    ),
+                    result
+                );
+
+                program_id_result[0] = *program_id;
+            }
+
+            // Return the actual length, rather the length returned
+            *result = Ok(return_data.len() as u64);
+        } else {
+            *result = Ok(0);
+        }
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;

runtime/src/message_processor.rs

@@ -67,6 +67,8 @@ pub struct ThisInvokeContext<'a> {
     sysvars: RefCell<Vec<(Pubkey, Option<Rc<Vec<u8>>>)>>,
     blockhash: &'a Hash,
     fee_calculator: &'a FeeCalculator,
+    // return data and program_id that set it
+    return_data: Option<(Pubkey, Vec<u8>)>,
 }
 impl<'a> ThisInvokeContext<'a> {
     #[allow(clippy::too_many_arguments)]
@@ -116,6 +118,7 @@ impl<'a> ThisInvokeContext<'a> {
             sysvars: RefCell::new(vec![]),
             blockhash,
             fee_calculator,
+            return_data: None,
         };
         invoke_context.push(program_id, message, instruction, program_indices, accounts)?;
         Ok(invoke_context)
@@ -329,6 +332,12 @@ impl<'a> InvokeContext for ThisInvokeContext<'a> {
     fn get_fee_calculator(&self) -> &FeeCalculator {
         self.fee_calculator
     }
+    fn set_return_data(&mut self, return_data: Option<(Pubkey, Vec<u8>)>) {
+        self.return_data = return_data;
+    }
+    fn get_return_data(&self) -> &Option<(Pubkey, Vec<u8>)> {
+        &self.return_data
+    }
 }
 pub struct ThisLogger {
     log_collector: Option<Rc<LogCollector>>,

sdk/Cargo.toml

@@ -42,6 +42,7 @@ assert_matches = { version = "1.5.0", optional = true }
 bincode = "1.3.3"
 bytemuck = { version = "1.7.2", features = ["derive"] }
 borsh = "0.9.0"
+base64 = "0.13"
 borsh-derive = "0.9.0"
 bs58 = "0.4.0"
 bv = { version = "0.11.1", features = ["serde"] }

sdk/bpf/c/inc/sol/return_data.h

@@ -0,0 +1,41 @@
+#pragma once
+/**
+ * @brief Solana return data system calls
+**/
+
+#include <sol/types.h>
+#include <sol/pubkey.h>
+
+#ifdef __cplusplus
+extern "C"
+{
+#endif
+
+/**
+ * Maximum size of return data
+ */
+#define MAX_RETURN_DATA 1024
+
+/**
+ * Set the return data
+ *
+ * @param bytes byte array to set
+ * @param bytes_len length of byte array. This may not exceed MAX_RETURN_DATA.
+ */
+void sol_set_return_data(const uint8_t *bytes, uint64_t bytes_len);
+
+/**
+ * Get the return data
+ *
+ * @param bytes byte buffer
+ * @param bytes_len maximum length of buffer
+ * @param program_id the program_id which set the return data. Only set if there was some return data (the function returns non-zero).
+ * @param result length of return data (may exceed bytes_len if the return data is longer)
+ */
+uint64_t sol_get_return_data(const uint8_t *bytes, uint64_t bytes_len, SolPubkey *program_id);
+
+#ifdef __cplusplus
+}
+#endif
+
+/**@}*/

sdk/bpf/c/inc/solana_sdk.h

@@ -12,6 +12,7 @@
 #include <sol/keccak.h>
 #include <sol/log.h>
 #include <sol/pubkey.h>
+#include <sol/return_data.h>
 #include <sol/secp256k1.h>
 #include <sol/sha.h>
 #include <sol/string.h>

sdk/program/src/program.rs

@@ -1,4 +1,6 @@
-use crate::{account_info::AccountInfo, entrypoint::ProgramResult, instruction::Instruction};
+use crate::{
+    account_info::AccountInfo, entrypoint::ProgramResult, instruction::Instruction, pubkey::Pubkey,
+};
 
 /// Invoke a cross-program instruction
 ///
@@ -35,6 +37,16 @@ pub fn invoke_signed(
 
     #[cfg(target_arch = "bpf")]
     {
+        extern "C" {
+            fn sol_invoke_signed_rust(
+                instruction_addr: *const u8,
+                account_infos_addr: *const u8,
+                account_infos_len: u64,
+                signers_seeds_addr: *const u8,
+                signers_seeds_len: u64,
+            ) -> u64;
+        }
+
         let result = unsafe {
             sol_invoke_signed_rust(
                 instruction as *const _ as *const u8,
@@ -54,13 +66,48 @@ pub fn invoke_signed(
     crate::program_stubs::sol_invoke_signed(instruction, account_infos, signers_seeds)
 }
 
+/// Maximum size that can be set using sol_set_return_data()
+pub const MAX_RETURN_DATA: usize = 1024;
+
+/// Set a program's return data
+pub fn set_return_data(data: &[u8]) {
     #[cfg(target_arch = "bpf")]
+    {
         extern "C" {
-    fn sol_invoke_signed_rust(
+            fn sol_set_return_data(data: *const u8, length: u64);
-        instruction_addr: *const u8,
+        }
-        account_infos_addr: *const u8,
+
-        account_infos_len: u64,
+        unsafe { sol_set_return_data(data.as_ptr(), data.len() as u64) };
-        signers_seeds_addr: *const u8,
+    }
-        signers_seeds_len: u64,
+
-    ) -> u64;
+    #[cfg(not(target_arch = "bpf"))]
+    crate::program_stubs::sol_set_return_data(data)
+}
+
+/// Get the return data from invoked program
+pub fn get_return_data() -> Option<(Pubkey, Vec<u8>)> {
+    #[cfg(target_arch = "bpf")]
+    {
+        use std::cmp::min;
+
+        extern "C" {
+            fn sol_get_return_data(data: *mut u8, length: u64, program_id: *mut Pubkey) -> u64;
+        }
+
+        let mut buf = [0u8; MAX_RETURN_DATA];
+        let mut program_id = Pubkey::default();
+
+        let size =
+            unsafe { sol_get_return_data(buf.as_mut_ptr(), buf.len() as u64, &mut program_id) };
+
+        if size == 0 {
+            None
+        } else {
+            let size = min(size as usize, MAX_RETURN_DATA);
+            Some((program_id, buf[..size as usize].to_vec()))
+        }
+    }
+
+    #[cfg(not(target_arch = "bpf"))]
+    crate::program_stubs::sol_get_return_data()
 }

sdk/program/src/program_stubs.rs

@@ -4,7 +4,7 @@
 
 use crate::{
     account_info::AccountInfo, entrypoint::ProgramResult, instruction::Instruction,
-    program_error::UNSUPPORTED_SYSVAR,
+    program_error::UNSUPPORTED_SYSVAR, pubkey::Pubkey,
 };
 use std::sync::{Arc, RwLock};
 
@@ -80,6 +80,10 @@ pub trait SyscallStubs: Sync + Send {
             *val = c;
         }
     }
+    fn sol_get_return_data(&self) -> Option<(Pubkey, Vec<u8>)> {
+        None
+    }
+    fn sol_set_return_data(&mut self, _data: &[u8]) {}
 }
 
 struct DefaultSyscallStubs {}
@@ -153,3 +157,11 @@ pub(crate) fn sol_memset(s: *mut u8, c: u8, n: usize) {
         SYSCALL_STUBS.read().unwrap().sol_memset(s, c, n);
     }
 }
+
+pub(crate) fn sol_get_return_data() -> Option<(Pubkey, Vec<u8>)> {
+    SYSCALL_STUBS.read().unwrap().sol_get_return_data()
+}
+
+pub(crate) fn sol_set_return_data(data: &[u8]) {
+    SYSCALL_STUBS.write().unwrap().sol_set_return_data(data)
+}

sdk/src/compute_budget.rs

@@ -70,6 +70,8 @@ pub struct ComputeBudget {
     pub sysvar_base_cost: u64,
     /// Number of compute units consumed to call secp256k1_recover
     pub secp256k1_recover_cost: u64,
+    /// Number of compute units consumed to do a syscall without any work
+    pub syscall_base_cost: u64,
     /// Optional program heap region size, if `None` then loader default
     pub heap_size: Option<usize>,
 }
@@ -95,6 +97,7 @@ impl ComputeBudget {
             cpi_bytes_per_unit: 250,        // ~50MB at 200,000 units
             sysvar_base_cost: 100,
             secp256k1_recover_cost: 25_000,
+            syscall_base_cost: 100,
             heap_size: None,
         }
     }

sdk/src/feature_set.rs

@@ -207,6 +207,10 @@ pub mod check_seed_length {
     solana_sdk::declare_id!("8HYXgkoKGreAMA3MfJkdjbKNVbfZRQP3jqFpa7iqN4v7");
 }
 
+pub mod return_data_syscall_enabled {
+    solana_sdk::declare_id!("BJVXq6NdLC7jCDGjfqJv7M1XHD4Y13VrpDqRF2U7UBcC");
+}
+
 lazy_static! {
     /// Map of feature identifiers to user-visible description
     pub static ref FEATURE_NAMES: HashMap<Pubkey, &'static str> = [
@@ -254,6 +258,7 @@ lazy_static! {
         (ed25519_program_enabled::id(), "enable builtin ed25519 signature verify program"),
         (allow_native_ids::id(), "allow native program ids in program derived addresses"),
         (check_seed_length::id(), "Check program address seed lengths"),
+        (return_data_syscall_enabled::id(), "enable sol_{set,get}_return_data syscall")
         /*************** ADD NEW FEATURES HERE ***************/
     ]
     .iter()

sdk/src/process_instruction.rs

@@ -117,6 +117,10 @@ pub trait InvokeContext {
     fn get_blockhash(&self) -> &Hash;
     /// Get this invocation's `FeeCalculator`
     fn get_fee_calculator(&self) -> &FeeCalculator;
+    /// Set the return data
+    fn set_return_data(&mut self, return_data: Option<(Pubkey, Vec<u8>)>);
+    /// Get the return data
+    fn get_return_data(&self) -> &Option<(Pubkey, Vec<u8>)>;
 }
 
 /// Convenience macro to log a message with an `Rc<RefCell<dyn Logger>>`
@@ -197,6 +201,8 @@ pub struct BpfComputeBudget {
     pub sysvar_base_cost: u64,
     /// Number of compute units consumed to call secp256k1_recover
     pub secp256k1_recover_cost: u64,
+    /// Number of compute units consumed to do a syscall without any work
+    pub syscall_base_cost: u64,
     /// Optional program heap region size, if `None` then loader default
     pub heap_size: Option<usize>,
 }
@@ -217,6 +223,7 @@ impl From<ComputeBudget> for BpfComputeBudget {
             max_cpi_instruction_size: item.max_cpi_instruction_size,
             cpi_bytes_per_unit: item.cpi_bytes_per_unit,
             sysvar_base_cost: item.sysvar_base_cost,
+            syscall_base_cost: item.syscall_base_cost,
             secp256k1_recover_cost: item.secp256k1_recover_cost,
             heap_size: item.heap_size,
         }
@@ -240,6 +247,7 @@ impl From<BpfComputeBudget> for ComputeBudget {
             cpi_bytes_per_unit: item.cpi_bytes_per_unit,
             sysvar_base_cost: item.sysvar_base_cost,
             secp256k1_recover_cost: item.secp256k1_recover_cost,
+            syscall_base_cost: item.syscall_base_cost,
             heap_size: item.heap_size,
         }
     }
@@ -313,6 +321,25 @@ pub mod stable_log {
         ic_logger_msg!(logger, "Program log: {}", message);
     }
 
+    /// Log return data as from the program itself. This line will not be present if no return
+    /// data was set, or if the return data was set to zero length.
+    ///
+    /// The general form is:
+    ///
+    /// ```notrust
+    /// "Program return data: <program-id> <program-generated-data-in-base64>"
+    /// ```
+    ///
+    /// That is, any program-generated output is guaranteed to be prefixed by "Program return data: "
+    pub fn program_return_data(logger: &Rc<RefCell<dyn Logger>>, program_id: &Pubkey, data: &[u8]) {
+        ic_logger_msg!(
+            logger,
+            "Program return data: {} {}",
+            program_id,
+            base64::encode(data)
+        );
+    }
+
     /// Log successful program execution.
     ///
     /// The general form is:
@@ -397,7 +424,9 @@ pub struct MockInvokeContext<'a> {
     pub disabled_features: HashSet<Pubkey>,
     pub blockhash: Hash,
     pub fee_calculator: FeeCalculator,
+    pub return_data: Option<(Pubkey, Vec<u8>)>,
 }
+
 impl<'a> MockInvokeContext<'a> {
     pub fn new(keyed_accounts: Vec<KeyedAccount<'a>>) -> Self {
         let compute_budget = ComputeBudget::default();
@@ -415,6 +444,7 @@ impl<'a> MockInvokeContext<'a> {
             disabled_features: HashSet::default(),
             blockhash: Hash::default(),
             fee_calculator: FeeCalculator::default(),
+            return_data: None,
         };
         invoke_context
             .invoke_stack
@@ -543,4 +573,10 @@ impl<'a> InvokeContext for MockInvokeContext<'a> {
     fn get_fee_calculator(&self) -> &FeeCalculator {
         &self.fee_calculator
     }
+    fn set_return_data(&mut self, return_data: Option<(Pubkey, Vec<u8>)>) {
+        self.return_data = return_data;
+    }
+    fn get_return_data(&self) -> &Option<(Pubkey, Vec<u8>)> {
+        &self.return_data
+    }
 }