solana/src/lib.rs

//! The `solana` library implements the Solana high-performance blockchain architecture.
//! It includes a full Rust implementation of the architecture (see
//! [Fullnode](server/struct.Fullnode.html)) as well as hooks to GPU implementations of its most
//! paralellizable components (i.e. [SigVerify](sigverify/index.html)).  It also includes
//! command-line tools to spin up fullnodes and a Rust library
//! (see [ThinClient](thin_client/struct.ThinClient.html)) to interact with them.
//!

#![cfg_attr(feature = "unstable", feature(test))]
#[macro_use]
pub mod counter;
pub mod bank;
pub mod banking_stage;
pub mod blob_fetch_stage;
pub mod broadcast_stage;
pub mod budget;
pub mod choose_gossip_peer_strategy;
pub mod client;
pub mod instruction;
#[macro_use]
pub mod crdt;
pub mod budget_program;
pub mod drone;
pub mod entry;
pub mod entry_writer;
#[cfg(feature = "erasure")]
pub mod erasure;
pub mod fetch_stage;
pub mod fullnode;
pub mod hash;
pub mod ledger;
pub mod logger;
pub mod metrics;
pub mod mint;
pub mod ncp;
pub mod netutil;
pub mod packet;
pub mod payment_plan;
pub mod poh;
pub mod poh_service;
pub mod record_stage;
pub mod recorder;
pub mod recvmmsg;
pub mod recycler;
pub mod replicate_stage;
pub mod replicator;
pub mod request;
pub mod request_processor;
pub mod request_stage;
pub mod result;
pub mod retransmit_stage;
pub mod rpc;
pub mod rpu;
pub mod service;
pub mod signature;
pub mod sigverify;
pub mod sigverify_stage;
pub mod storage_program;
pub mod store_ledger_stage;
pub mod streamer;
pub mod system_program;
pub mod thin_client;
pub mod timing;
pub mod tpu;
pub mod transaction;
pub mod tvu;
pub mod vote_stage;
pub mod wallet;
pub mod window;
pub mod window_service;
pub mod write_stage;
extern crate bincode;
extern crate bs58;
extern crate byteorder;
extern crate bytes;
extern crate chrono;
extern crate clap;
extern crate dirs;
extern crate generic_array;
extern crate ipnetwork;
extern crate itertools;
extern crate jsonrpc_core;
#[macro_use]
extern crate jsonrpc_macros;
extern crate jsonrpc_http_server;
extern crate libc;
#[macro_use]
extern crate log;
extern crate nix;
extern crate pnet_datalink;
extern crate rayon;
extern crate reqwest;
extern crate ring;
extern crate serde;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
extern crate sha2;
extern crate socket2;
extern crate sys_info;
extern crate tokio;
extern crate tokio_codec;
extern crate untrusted;

#[cfg(test)]
#[macro_use]
extern crate matches;

extern crate influx_db_client;
extern crate rand;
Better docs for the bank 2018-06-06 08:49:22 -07:00			//! The `solana` library implements the Solana high-performance blockchain architecture.
Cleanup top-level lib doc 2018-06-07 13:51:29 -07:00			`//! It includes a full Rust implementation of the architecture (see`
Fix broken doc 2018-08-22 09:42:28 -07:00			`//! [Fullnode](server/struct.Fullnode.html)) as well as hooks to GPU implementations of its most`
Cleanup top-level lib doc 2018-06-07 13:51:29 -07:00			`//! paralellizable components (i.e. [SigVerify](sigverify/index.html)). It also includes`
			`//! command-line tools to spin up fullnodes and a Rust library`
			`//! (see [ThinClient](thin_client/struct.ThinClient.html)) to interact with them.`
Better docs for the bank 2018-06-06 08:49:22 -07:00			`//!`

Parallelize verification 2018-02-14 13:19:30 -08:00			`#![cfg_attr(feature = "unstable", feature(test))]`
perf counters 2018-05-30 21:24:21 -07:00			`#[macro_use]`
			`pub mod counter;`
accountant -> bank 2018-05-14 14:33:11 -07:00			`pub mod bank;`
Extract event processing from request_stage 2018-05-14 16:31:13 -07:00			`pub mod banking_stage;`
TVU rework (#352) Refactored TVU, into stages * blob fetch stage for blobs * window stage for maintaining the blob window * pulled out NCP out of the TVU so they can be separate units TVU is now just the fetch -> window -> request and bank processing 2018-06-13 21:52:23 -07:00			`pub mod blob_fetch_stage;`
Rename broadcaster to broadcast_stage And move retransmitter code into retransmit_stage. TODO: Add a BroadcastStage service 2018-08-09 13:41:21 -07:00			`pub mod broadcast_stage;`
Move Budget EDSL into its own module 2018-05-29 12:28:07 -07:00			`pub mod budget;`
Fix compiler warnings 2018-06-26 14:43:20 -07:00			`pub mod choose_gossip_peer_strategy;`
Submit leader's vote after observing 2/3 validator votes (#780) * fixup! * fixups! * send the vote and count it * actually vote * test * Spelling fixes * Process the voting transaction in the leader's bank * Send tokens to the leader * Give leader tokens in more cases * Test for write_stage::leader_vote * Request airdrop inside fullnode and not the script * Change readme to indicate that drone should be up before leader And start drone before leader in snap scripts * Rename _kp => _keypair for keypairs and other review fixups * Remove empty else * tweak test_leader_vote numbers to be closer to testing 2/3 boundary * combine creating blob and transaction for leader/validator 2018-07-31 22:07:53 -07:00			`pub mod client;`
budget as separate contract and system call contract (#1189) * budget and system contracts and verification * contract check_id methods * system call contract * verify contract execution rules * move system into its own file * allocate before transfer for budget * store error in budget context * budget contract and tests without bank * moved budget of of bank 2018-09-17 13:36:31 -07:00			`pub mod instruction;`
Open multiple sockets for transaction UDP port (#1128) * Reuse UDP port and open multiple sockets for transaction address * Fixed failing crdt tests * Add tests for reusing UDP ports * Address review comments * Updated bench-streamer to use multiple receive sockets * Fix minimum number of recv sockets for bench-streamer * Address review comments Fixes #1132 * Moved bind_to function to nat.rs 2018-09-06 14:13:40 -07:00			`#[macro_use]`
initial crdt implementation 2018-04-21 11:02:49 -07:00			`pub mod crdt;`
s/contract/program 2018-09-19 17:25:57 -07:00			`pub mod budget_program;`
Update config and dependencies for solana-drone 2018-06-19 21:42:03 -07:00			`pub mod drone;`
Move entry into its own module Hmm, Logger doesn't depend on log. 2018-03-06 16:31:17 -08:00			`pub mod entry;`
Move the writer stage's utilities to its own module 2018-05-11 21:36:16 -07:00			`pub mod entry_writer;`
improve the error messages 2018-04-21 06:12:57 -07:00			`#[cfg(feature = "erasure")]`
			`pub mod erasure;`
Finally made fetch happen 2018-05-29 10:18:12 -07:00			`pub mod fetch_stage;`
fullnode lib 2018-07-02 11:20:35 -07:00			`pub mod fullnode;`
Nightly rustfmt Format code with the nightly version of rustfmt, which sorts imports. 2018-03-26 21:03:26 -07:00			`pub mod hash;`
log -> ledger Free up namespace for traditional runtime logs. 2018-03-19 09:09:17 -07:00			`pub mod ledger;`
Add broadcast impl 2018-04-28 00:31:20 -07:00			`pub mod logger;`
Metrics v0.1 2018-07-05 21:40:09 -07:00			`pub mod metrics;`
Rename Genesis to Mint Genesis is a story of creation. We should only use that term to for the event log that bootstraps the system. 2018-03-07 16:08:12 -08:00			`pub mod mint;`
data_replicator -> ncp Fixes #327 2018-06-07 15:06:32 -07:00			`pub mod ncp;`
improve localnet-sanity's robustness (#1160) * fix poll_gossip_for_leader() loop to actually wait for 30 seconds * reduce reuseaddr use to only when necessary, try to avoid already bound sockets * move nat.rs to netutil.rs * add gossip tracing to thin_client and bench-tps 2018-09-08 12:50:43 -07:00			`pub mod netutil;`
Change for cuda verify integration 2018-03-26 21:07:11 -07:00			`pub mod packet;`
plan.rs -> payment_plan.rs 2018-05-29 12:55:21 -07:00			`pub mod payment_plan;`
step one of lastidnotfound: record_stage->record_service, trim recorder to hashes (#1281) step one of lastidnotfound * record_stage->record_service, trim recorder to hashes * doc updates, hash multiple without alloc() cc #1171 2018-09-20 15:02:24 -07:00			`pub mod poh;`
			`pub mod poh_service;`
Rename Historian to RecordStage Historian was a legacy name. The new name reflects the new pipelined architecture. 2018-05-14 13:19:11 -07:00			`pub mod record_stage;`
logger -> recorder Free up namespace for a traditional runtime logger. 2018-03-19 09:16:16 -07:00			`pub mod recorder;`
Implements recvmmsg() for UDP packets (#1161) * Implemented recvmmsg() for UDP packets - This change implements binding between libc API for recvmmsg() - The function can receive multiple packets using one system call Fixes #1141 * Added unit tests for recvmmsg() * Added recv_mmsg() wrapper for non Linux OS * Address review comments for recvmmsg() * Remove unnecessary imports * Moved target specific dependencies to the function 2018-09-13 14:41:28 -07:00			`pub mod recvmmsg;`
Use a Drop trait to keep track of lifetimes for recycled objects. * Move recycler instances to the point of allocation * sinks no longer need to call `recycle` * Remove the recycler arguments from all the apis that no longer need them 2018-09-18 08:02:57 -07:00			`pub mod recycler;`
fmt 2018-05-22 15:18:07 -07:00			`pub mod replicate_stage;`
Placeholder storage contract and replicator client (#1286) * Add hooks for executing the storage contract * Add store_ledger stage Similar to replicate_stage but no voting/banking stuff, just convert blobs to entries and write the ledger out * Add storage_addr to tests and add new NodeInfo constructor to reduce duplication... 2018-09-21 15:32:15 -07:00			`pub mod replicator;`
Move thin client messages into their own module 2018-05-12 16:41:27 -07:00			`pub mod request;`
Move RequestProcessor into its own module 2018-05-12 16:50:55 -07:00			`pub mod request_processor;`
Free up name 'thin_client_service' 2018-05-12 09:31:28 -07:00			`pub mod request_stage;`
Fast UdpSocket reader * message needs to fit into 256 bytes * allocator to keep track of blocks of messages * udp socket receiver server that fills up the block as fast as possible * udp socket sender server that sends out the block as fast as possible 2018-03-07 13:47:13 -08:00			`pub mod result;`
Rename WindowStage to RetransmitStage The window is used for both broadcasting from leader to validator and retransmitting between validators. 2018-08-09 12:03:34 -07:00			`pub mod retransmit_stage;`
Implement json-rpc functionality 2018-08-10 16:05:23 -07:00			`pub mod rpc;`
Free up the name 'tpu' 2018-05-12 09:53:25 -07:00			`pub mod rpu;`
Add Service trait Added a consistent interface to all the microservices. 2018-07-03 21:14:08 -07:00			`pub mod service;`
Nightly rustfmt Format code with the nightly version of rustfmt, which sorts imports. 2018-03-26 21:03:26 -07:00			`pub mod signature;`
Rename sigverify modules 2018-05-25 15:52:17 -07:00			`pub mod sigverify;`
			`pub mod sigverify_stage;`
Placeholder storage contract and replicator client (#1286) * Add hooks for executing the storage contract * Add store_ledger stage Similar to replicate_stage but no voting/banking stuff, just convert blobs to entries and write the ledger out * Add storage_addr to tests and add new NodeInfo constructor to reduce duplication... 2018-09-21 15:32:15 -07:00			`pub mod storage_program;`
			`pub mod store_ledger_stage;`
Nightly rustfmt Format code with the nightly version of rustfmt, which sorts imports. 2018-03-26 21:03:26 -07:00			`pub mod streamer;`
s/contract/program 2018-09-19 17:25:57 -07:00			`pub mod system_program;`
AccountantSkel -> Tpu The terms Stub and Skel come from OMG IDL and only made sense while the Stub was acting as an RPC client for the the Accountant object. Nowadays, the Stub interface looks nothing like the Accountant and meanwhile we've recognized the multithreaded implementation is more reminiscent of a pipelined CPU. Thus, we finally bite the bullet and rename our modules. AccountantSkel -> Tpu AccountantStub -> ThinClient Up next will be moving much of the TPU code into separate modules, each representing a stage of the pipeline. The interface of each will follow the precedent set by the Historian object. 2018-05-08 16:32:50 -07:00			`pub mod thin_client;`
Rework sig processing threads and add perf for process/verify 2018-05-07 16:49:15 -07:00			`pub mod timing;`
Add TPU 2018-05-14 16:36:19 -07:00			`pub mod tpu;`
cargo +nightly fmt 2018-05-08 17:59:01 -07:00			`pub mod transaction;`
Move validation processor to its own module 2018-05-11 23:31:32 -07:00			`pub mod tvu;`
Validators now vote once a second regardless 2018-07-19 21:27:35 -07:00			`pub mod vote_stage;`
Submit leader's vote after observing 2/3 validator votes (#780) * fixup! * fixups! * send the vote and count it * actually vote * test * Spelling fixes * Process the voting transaction in the leader's bank * Send tokens to the leader * Give leader tokens in more cases * Test for write_stage::leader_vote * Request airdrop inside fullnode and not the script * Change readme to indicate that drone should be up before leader And start drone before leader in snap scripts * Rename _kp => _keypair for keypairs and other review fixups * Remove empty else * tweak test_leader_vote numbers to be closer to testing 2/3 boundary * combine creating blob and transaction for leader/validator 2018-07-31 22:07:53 -07:00			`pub mod wallet;`
Move winow into its own module 2018-08-09 12:31:34 -07:00			`pub mod window;`
Split window into two modules 2018-09-07 15:00:26 -07:00			`pub mod window_service;`
Move write_service and drain_service into new write_stage module 2018-05-14 15:27:40 -07:00			`pub mod write_stage;`
Allow event log to hold events of any serializable (hashable) type 2018-02-26 15:42:31 -08:00			`extern crate bincode;`
Add PublicKey wrapper Add custom formatting for PublicKey display and debug 2018-07-31 10:26:26 -07:00			`extern crate bs58;`
Implement window service Batch out of order blobs until we have a contigious window. 2018-04-02 19:32:58 -07:00			`extern crate byteorder;`
Add method to run local drone for tests 2018-09-14 01:58:59 -07:00			`extern crate bytes;`
Add DateTime and Cancel conditions Fixes #32, #33 2018-03-07 20:55:49 -08:00			`extern crate chrono;`
Move all handling except network/gossip from /bin to wallet module 2018-09-14 01:58:39 -07:00			`extern crate clap;`
			`extern crate dirs;`
Add serialization/deseriation support to event log See bincode and serde_json for usage: https://github.com/TyOverby/bincode Fixes #1 2018-02-20 15:26:11 -08:00			`extern crate generic_array;`
tests for my IP picker 2018-09-19 18:33:40 -07:00			`extern crate ipnetwork;`
Do Proof of History verification before appending entries to the bank Note: replicate_stage is still using `process_entries()` because changing it to `process_blocks()` causes the `test_replicate` test to fail. 2018-06-29 11:58:29 -07:00			`extern crate itertools;`
Update jsonrpc dependency syntax 2018-08-13 12:59:04 -07:00			`extern crate jsonrpc_core;`
Implement json-rpc functionality 2018-08-10 16:05:23 -07:00			`#[macro_use]`
			`extern crate jsonrpc_macros;`
			`extern crate jsonrpc_http_server;`
Implements recvmmsg() for UDP packets (#1161) * Implemented recvmmsg() for UDP packets - This change implements binding between libc API for recvmmsg() - The function can receive multiple packets using one system call Fixes #1141 * Added unit tests for recvmmsg() * Added recv_mmsg() wrapper for non Linux OS * Address review comments for recvmmsg() * Remove unnecessary imports * Moved target specific dependencies to the function 2018-09-13 14:41:28 -07:00			`extern crate libc;`
Fast UdpSocket reader * message needs to fit into 256 bytes * allocator to keep track of blocks of messages * udp socket receiver server that fills up the block as fast as possible * udp socket sender server that sends out the block as fast as possible 2018-03-07 13:47:13 -08:00			`#[macro_use]`
No longer rename log crate 2018-03-19 09:18:48 -07:00			`extern crate log;`
Open multiple sockets for transaction UDP port (#1128) * Reuse UDP port and open multiple sockets for transaction address * Fixed failing crdt tests * Add tests for reusing UDP ports * Address review comments * Updated bench-streamer to use multiple receive sockets * Fix minimum number of recv sockets for bench-streamer * Address review comments Fixes #1132 * Moved bind_to function to nat.rs 2018-09-06 14:13:40 -07:00			`extern crate nix;`
Alphabetize 2018-09-14 15:31:54 -07:00			`extern crate pnet_datalink;`
Parallelize verification 2018-02-14 13:19:30 -08:00			`extern crate rayon;`
improve localnet-sanity's robustness (#1160) * fix poll_gossip_for_leader() loop to actually wait for 30 seconds * reduce reuseaddr use to only when necessary, try to avoid already bound sockets * move nat.rs to netutil.rs * add gossip tracing to thin_client and bench-tps 2018-09-08 12:50:43 -07:00			`extern crate reqwest;`
Extend the event log with a Claim event to claim possession Unlike a Discovery event, a Claim event associates a public key with a hash. It's intended to to be used to claim ownership of some hashable data. For example, a graphic designer could claim copyright by hashing some image they created, signing it with their private key, and publishing the hash-signature pair via the historian. If someone else tries to claim it as their own, the designer can point to the historian's log as cryptographically secure evidence that the designer's copy existed before anyone else's. Note there's nothing here that verifies the first claim is the actual content owner, only that the first claim almost certainly happened before a second. 2018-02-24 05:53:36 -08:00			`extern crate ring;`
Add serialization/deseriation support to event log See bincode and serde_json for usage: https://github.com/TyOverby/bincode Fixes #1 2018-02-20 15:26:11 -08:00			`extern crate serde;`
			`#[macro_use]`
			`extern crate serde_derive;`
Initialize the testnode from a log $ cargo run --bin silk-genesis-file-demo > demo-genesis.json $ cat demo-genesis.json \| cargo run --bin silk-genesis-block > demo-genesis.log $ cat demo-genesis.log \| cargo run --bin silk-testnode 2018-03-05 14:34:15 -08:00			`extern crate serde_json;`
Use sha256 hashes instead of Rust's builtin hasher. Causes a 20x performance degradation. Enabling asm did not speed things up. 2018-02-19 15:17:13 -08:00			`extern crate sha2;`
Open multiple sockets for transaction UDP port (#1128) * Reuse UDP port and open multiple sockets for transaction address * Fixed failing crdt tests * Add tests for reusing UDP ports * Address review comments * Updated bench-streamer to use multiple receive sockets * Fix minimum number of recv sockets for bench-streamer * Address review comments Fixes #1132 * Moved bind_to function to nat.rs 2018-09-06 14:13:40 -07:00			`extern crate socket2;`
Add hostname to metrics on panic 2018-08-10 15:59:41 -07:00			`extern crate sys_info;`
Add method to run local drone for tests 2018-09-14 01:58:59 -07:00			`extern crate tokio;`
			`extern crate tokio_codec;`
Extend the event log with a Claim event to claim possession Unlike a Discovery event, a Claim event associates a public key with a hash. It's intended to to be used to claim ownership of some hashable data. For example, a graphic designer could claim copyright by hashing some image they created, signing it with their private key, and publishing the hash-signature pair via the historian. If someone else tries to claim it as their own, the designer can point to the historian's log as cryptographically secure evidence that the designer's copy existed before anyone else's. Note there's nothing here that verifies the first claim is the actual content owner, only that the first claim almost certainly happened before a second. 2018-02-24 05:53:36 -08:00			`extern crate untrusted;`
Fast UdpSocket reader * message needs to fit into 256 bytes * allocator to keep track of blocks of messages * udp socket receiver server that fills up the block as fast as possible * udp socket sender server that sends out the block as fast as possible 2018-03-07 13:47:13 -08:00
			`#[cfg(test)]`
			`#[macro_use]`
			`extern crate matches;`
Rework sig processing threads and add perf for process/verify 2018-05-07 16:49:15 -07:00
Metrics v0.1 2018-07-05 21:40:09 -07:00			`extern crate influx_db_client;`
Rework sig processing threads and add perf for process/verify 2018-05-07 16:49:15 -07:00			`extern crate rand;`