solana/src/lib.rs

//! The `solana` library implements the Solana high-performance blockchain architecture.
//! It includes a full Rust implementation of the architecture (see
//! [Fullnode](server/struct.Fullnode.html)) as well as hooks to GPU implementations of its most
//! paralellizable components (i.e. [SigVerify](sigverify/index.html)).  It also includes
//! command-line tools to spin up fullnodes and a Rust library
//! (see [ThinClient](thin_client/struct.ThinClient.html)) to interact with them.
//!

#![cfg_attr(feature = "unstable", feature(test))]
#[macro_use]
pub mod counter;
pub mod accounts;
pub mod bank;
pub mod bank_forks;
pub mod banking_stage;
pub mod blob_fetch_stage;
pub mod bloom;
pub mod broadcast_service;
#[cfg(feature = "chacha")]
pub mod chacha;
#[cfg(all(feature = "chacha", feature = "cuda"))]
pub mod chacha_cuda;
pub mod client;
pub mod cluster_info_vote_listener;
pub mod crds;
pub mod crds_gossip;
pub mod crds_gossip_error;
pub mod crds_gossip_pull;
pub mod crds_gossip_push;
pub mod crds_value;
#[macro_use]
pub mod contact_info;
pub mod blocktree;
pub mod blocktree_processor;
pub mod cluster_info;
pub mod db_window;
pub mod entry;
pub mod entry_stream;
pub mod entry_stream_stage;
#[cfg(feature = "erasure")]
pub mod erasure;
pub mod fetch_stage;
pub mod fullnode;
pub mod gen_keys;
pub mod genesis_block;
pub mod gossip_service;
pub mod last_id_queue;
pub mod leader_confirmation_service;
pub mod leader_scheduler;
pub mod local_vote_signer_service;
pub mod packet;
pub mod poh;
pub mod poh_recorder;
pub mod poh_service;
pub mod recvmmsg;
pub mod repair_service;
pub mod replay_stage;
pub mod replicator;
pub mod result;
pub mod retransmit_stage;
pub mod rpc;
pub mod rpc_mock;
pub mod rpc_pubsub;
pub mod rpc_pubsub_service;
pub mod rpc_request;
pub mod rpc_service;
pub mod rpc_status;
pub mod rpc_subscriptions;
pub mod service;
pub mod sigverify;
pub mod sigverify_stage;
pub mod status_cache;
pub mod storage_stage;
pub mod streamer;
pub mod test_tx;
pub mod thin_client;
pub mod tpu;
pub mod tpu_forwarder;
pub mod tvu;
pub mod voting_keypair;
#[cfg(test)]
pub mod window;
pub mod window_service;

#[cfg(test)]
#[cfg(any(feature = "chacha", feature = "cuda"))]
#[macro_use]
extern crate hex_literal;

#[macro_use]
extern crate log;

#[macro_use]
extern crate serde_derive;
#[macro_use]
extern crate serde_json;

#[cfg(test)]
#[macro_use]
extern crate matches;
Better docs for the bank 2018-06-06 08:49:22 -07:00			//! The `solana` library implements the Solana high-performance blockchain architecture.
Cleanup top-level lib doc 2018-06-07 13:51:29 -07:00			`//! It includes a full Rust implementation of the architecture (see`
Fix broken doc 2018-08-22 09:42:28 -07:00			`//! [Fullnode](server/struct.Fullnode.html)) as well as hooks to GPU implementations of its most`
Cleanup top-level lib doc 2018-06-07 13:51:29 -07:00			`//! paralellizable components (i.e. [SigVerify](sigverify/index.html)). It also includes`
			`//! command-line tools to spin up fullnodes and a Rust library`
			`//! (see [ThinClient](thin_client/struct.ThinClient.html)) to interact with them.`
Better docs for the bank 2018-06-06 08:49:22 -07:00			`//!`

Parallelize verification 2018-02-14 13:19:30 -08:00			`#![cfg_attr(feature = "unstable", feature(test))]`
perf counters 2018-05-30 21:24:21 -07:00			`#[macro_use]`
			`pub mod counter;`
Accounts is to big, should be its own module (#2198) Account module is to big, should be in its own module. 2018-12-17 12:41:23 -08:00			`pub mod accounts;`
accountant -> bank 2018-05-14 14:33:11 -07:00			`pub mod bank;`
Add a way to make a DAG of checkpointed Banks 2019-02-16 19:58:07 -08:00			`pub mod bank_forks;`
Extract event processing from request_stage 2018-05-14 16:31:13 -07:00			`pub mod banking_stage;`
TVU rework (#352) Refactored TVU, into stages * blob fetch stage for blobs * window stage for maintaining the blob window * pulled out NCP out of the TVU so they can be separate units TVU is now just the fetch -> window -> request and bank processing 2018-06-13 21:52:23 -07:00			`pub mod blob_fetch_stage;`
Add scalable gossip library (#1546) * Cluster Replicated Data Store Separate the data storage and merge strategy from the network IO boundary. Implement an eager push overlay for transporting recent messages. Simulation shows fast convergence with 20k nodes. 2018-11-15 13:23:26 -08:00			`pub mod bloom;`
Rename Ncp to GossipService And BroadcastStage to BroadcastService since it's not included in the TPU pipeline. 2018-12-06 12:52:47 -08:00			`pub mod broadcast_service;`
Add chacha module to encrypt ledger files 2018-09-27 10:48:40 -07:00			`#[cfg(feature = "chacha")]`
			`pub mod chacha;`
Exclude chacha_cuda when chacha is disabled 2018-10-24 16:49:21 -07:00			`#[cfg(all(feature = "chacha", feature = "cuda"))]`
Initial validator code for rust side hooks for chacha cuda parallel encrypt 2018-10-08 13:12:33 -07:00			`pub mod chacha_cuda;`
Submit leader's vote after observing 2/3 validator votes (#780) * fixup! * fixups! * send the vote and count it * actually vote * test * Spelling fixes * Process the voting transaction in the leader's bank * Send tokens to the leader * Give leader tokens in more cases * Test for write_stage::leader_vote * Request airdrop inside fullnode and not the script * Change readme to indicate that drone should be up before leader And start drone before leader in snap scripts * Rename _kp => _keypair for keypairs and other review fixups * Remove empty else * tweak test_leader_vote numbers to be closer to testing 2/3 boundary * combine creating blob and transaction for leader/validator 2018-07-31 22:07:53 -07:00			`pub mod client;`
Push and query the ClusterInfo for votes. (#2622) 2019-01-31 15:51:29 -08:00			`pub mod cluster_info_vote_listener;`
Add scalable gossip library (#1546) * Cluster Replicated Data Store Separate the data storage and merge strategy from the network IO boundary. Implement an eager push overlay for transporting recent messages. Simulation shows fast convergence with 20k nodes. 2018-11-15 13:23:26 -08:00			`pub mod crds;`
			`pub mod crds_gossip;`
			`pub mod crds_gossip_error;`
			`pub mod crds_gossip_pull;`
			`pub mod crds_gossip_push;`
			`pub mod crds_value;`
Open multiple sockets for transaction UDP port (#1128) * Reuse UDP port and open multiple sockets for transaction address * Fixed failing crdt tests * Add tests for reusing UDP ports * Address review comments * Updated bench-streamer to use multiple receive sockets * Fix minimum number of recv sockets for bench-streamer * Address review comments Fixes #1132 * Moved bind_to function to nat.rs 2018-09-06 14:13:40 -07:00			`#[macro_use]`
Add scalable gossip library (#1546) * Cluster Replicated Data Store Separate the data storage and merge strategy from the network IO boundary. Implement an eager push overlay for transporting recent messages. Simulation shows fast convergence with 20k nodes. 2018-11-15 13:23:26 -08:00			`pub mod contact_info;`
Rename db_ledger to blocktree (#2698) 2019-02-07 20:52:39 -08:00			`pub mod blocktree;`
Extract process_ledger from Bank Fullnode was the only real consumer of process_ledger and it was only there to process a Blocktree. Blocktree is a tree, and a ledger is a sequence, so something's clearly not right here. Drop all other dependencies on process_ledger (only one test) so that it can be fixed up in isolation. 2019-02-16 02:26:21 -08:00			`pub mod blocktree_processor;`
Add scalable gossip library (#1546) * Cluster Replicated Data Store Separate the data storage and merge strategy from the network IO boundary. Implement an eager push overlay for transporting recent messages. Simulation shows fast convergence with 20k nodes. 2018-11-15 13:23:26 -08:00			`pub mod cluster_info;`
Rocks db window utils (#1851) * Implement new ledger module based on RocksDb * Add db_window module for windowing functions from RocksDb 2018-11-19 23:20:18 -08:00			`pub mod db_window;`
Move entry into its own module Hmm, Logger doesn't depend on log. 2018-03-06 16:31:17 -08:00			`pub mod entry;`
Stream entries (#2582) * Add entry streaming option * Fix tests * Remove obsolete comment * Move entry stream functionality to struct w/ trait in order to test without i/o 2019-01-29 00:21:27 -08:00			`pub mod entry_stream;`
Move EntryStream into its own Tvu stage 2019-02-11 10:36:59 -08:00			`pub mod entry_stream_stage;`
improve the error messages 2018-04-21 06:12:57 -07:00			`#[cfg(feature = "erasure")]`
			`pub mod erasure;`
Finally made fetch happen 2018-05-29 10:18:12 -07:00			`pub mod fetch_stage;`
fullnode lib 2018-07-02 11:20:35 -07:00			`pub mod fullnode;`
Rename the module that now contains only GenKeys 2019-01-25 21:58:35 -08:00			`pub mod gen_keys;`
Genesis block is now a json file 2019-01-24 12:04:04 -08:00			`pub mod genesis_block;`
Rename Ncp to GossipService And BroadcastStage to BroadcastService since it's not included in the TPU pipeline. 2018-12-06 12:52:47 -08:00			`pub mod gossip_service;`
StatusDeque split into separate objects with their own root checkpoint strategy (#2613) Split up StatusDeque into different modules * LastIdQueue tracks last_ids * StatusCache keeps track of signature statuses * StatusCache stores success as a bit in a bloom filter * Overhead for 1m Ok transactions is 4mb in memory * Less concurrency between the objects, last_id and status_cache are read and written to at different points in the pipeline * Each object has its own strategy for merging into the root checkpoint 2019-01-31 06:53:52 -08:00			`pub mod last_id_queue;`
Remove 'Compute' from name ComputeLeaderConfirmationService struct names should be a noun 2019-02-17 19:22:29 -08:00			`pub mod leader_confirmation_service;`
Added LeaderScheduler module and tests 2018-09-24 21:31:20 -07:00			`pub mod leader_scheduler;`
Refactor vote signer code (#2368) * Refactor vote signer code * fixed test compilation errors * address clippy errors * fix missing macro_use * move macro use * review comments 2019-01-10 09:21:38 -08:00			`pub mod local_vote_signer_service;`
Change for cuda verify integration 2018-03-26 21:07:11 -07:00			`pub mod packet;`
step one of lastidnotfound: record_stage->record_service, trim recorder to hashes (#1281) step one of lastidnotfound * record_stage->record_service, trim recorder to hashes * doc updates, hash multiple without alloc() cc #1171 2018-09-20 15:02:24 -07:00			`pub mod poh;`
Synchronize PoH, bank last_id queue and ledger entry channel. PoH, bank's last_id queue and the Entry channel need to have a synchronized order of ids. 2018-09-26 05:52:13 -07:00			`pub mod poh_recorder;`
Create Poh Service (#1604) * Create new Poh Service, replace tick generation in BankingStage 2018-10-25 14:56:21 -07:00			`pub mod poh_service;`
Implements recvmmsg() for UDP packets (#1161) * Implemented recvmmsg() for UDP packets - This change implements binding between libc API for recvmmsg() - The function can receive multiple packets using one system call Fixes #1141 * Added unit tests for recvmmsg() * Added recv_mmsg() wrapper for non Linux OS * Address review comments for recvmmsg() * Remove unnecessary imports * Moved target specific dependencies to the function 2018-09-13 14:41:28 -07:00			`pub mod recvmmsg;`
Support multiple forks in the ledger (#2277) * Modify db_ledger to support per_slot metadata, add signal for updates, and add chaining to slots in db_ledger * Modify replay stage to ask db_ledger for updates based on slots * Add repair send/receive metrics * Add repair service, remove old repair code * Fix tmp_copy_ledger and setup for tests to account for multiple slots and tick limits within slots 2019-02-07 15:10:54 -08:00			`pub mod repair_service;`
Free up the term 'replicate' for exclusive use in replicator Also, align Sockets field names with ContactInfo. 2018-12-07 14:09:29 -08:00			`pub mod replay_stage;`
Placeholder storage contract and replicator client (#1286) * Add hooks for executing the storage contract * Add store_ledger stage Similar to replicate_stage but no voting/banking stuff, just convert blobs to entries and write the ledger out * Add storage_addr to tests and add new NodeInfo constructor to reduce duplication... 2018-09-21 15:32:15 -07:00			`pub mod replicator;`
Fast UdpSocket reader * message needs to fit into 256 bytes * allocator to keep track of blocks of messages * udp socket receiver server that fills up the block as fast as possible * udp socket sender server that sends out the block as fast as possible 2018-03-07 13:47:13 -08:00			`pub mod result;`
Rename WindowStage to RetransmitStage The window is used for both broadcasting from leader to validator and retransmitting between validators. 2018-08-09 12:03:34 -07:00			`pub mod retransmit_stage;`
Implement json-rpc functionality 2018-08-10 16:05:23 -07:00			`pub mod rpc;`
Improve Wallet coverage (#2385) * Add trait for RpcRequestHandler trait for RpcClient and add MockRpcClient for unit tests * Add request_airdrop integration test * Add timestamp_tx, witness_tx, and cancel_tx to wallet integration tests; add wallet integration tests to test-stable * Add test cases * Ignore plentiful sleeps in unit tests 2019-01-13 23:10:03 -08:00			`pub mod rpc_mock;`
pubsub.rs -> rpc_pubsub.rs 2018-10-12 08:17:28 -07:00			`pub mod rpc_pubsub;`
Move RpcPubSubService into its own module 2019-02-17 09:09:46 -08:00			`pub mod rpc_pubsub_service;`
Move rpc request methods from wallet into separate module 2018-10-08 11:10:47 -07:00			`pub mod rpc_request;`
Move RpcService into its own module 2019-02-17 09:29:08 -08:00			`pub mod rpc_service;`
Move RpcSignatureStatus into its own module And fixup some imports from previous commits. 2019-02-17 10:17:58 -08:00			`pub mod rpc_status;`
Move RpcSubscriptions into its own module 2019-02-17 08:38:36 -08:00			`pub mod rpc_subscriptions;`
Add Service trait Added a consistent interface to all the microservices. 2018-07-03 21:14:08 -07:00			`pub mod service;`
Rename sigverify modules 2018-05-25 15:52:17 -07:00			`pub mod sigverify;`
			`pub mod sigverify_stage;`
StatusDeque split into separate objects with their own root checkpoint strategy (#2613) Split up StatusDeque into different modules * LastIdQueue tracks last_ids * StatusCache keeps track of signature statuses * StatusCache stores success as a bit in a bloom filter * Overhead for 1m Ok transactions is 4mb in memory * Less concurrency between the objects, last_id and status_cache are read and written to at different points in the pipeline * Each object has its own strategy for merging into the root checkpoint 2019-01-31 06:53:52 -08:00			`pub mod status_cache;`
Add storage stage which does storage mining verification for validators 2018-10-08 13:12:33 -07:00			`pub mod storage_stage;`
Nightly rustfmt Format code with the nightly version of rustfmt, which sorts imports. 2018-03-26 21:03:26 -07:00			`pub mod streamer;`
Move system_transaction out of src/ 2018-12-04 15:37:11 -08:00			`pub mod test_tx;`
AccountantSkel -> Tpu The terms Stub and Skel come from OMG IDL and only made sense while the Stub was acting as an RPC client for the the Accountant object. Nowadays, the Stub interface looks nothing like the Accountant and meanwhile we've recognized the multithreaded implementation is more reminiscent of a pipelined CPU. Thus, we finally bite the bullet and rename our modules. AccountantSkel -> Tpu AccountantStub -> ThinClient Up next will be moving much of the TPU code into separate modules, each representing a stage of the pipeline. The interface of each will follow the precedent set by the Historian object. 2018-05-08 16:32:50 -07:00			`pub mod thin_client;`
Add TPU 2018-05-14 16:36:19 -07:00			`pub mod tpu;`
add tpu_forwarding, simplify ClusterInfo::new() from Result<Self> to Self 2018-11-16 16:42:29 -08:00			`pub mod tpu_forwarder;`
Move validation processor to its own module 2018-05-11 23:31:32 -07:00			`pub mod tvu;`
Rename VoteSignerProxy to VotingKeypair Works just like a normal Keypair, but will only sign voting transactions. 2019-01-31 20:12:51 -08:00			`pub mod voting_keypair;`
revert revert of kill window (#2427) * remove window code from most places * window used only for testing * remove unecessary clippy directives 2019-01-15 10:51:53 -08:00			`#[cfg(test)]`
Move winow into its own module 2018-08-09 12:31:34 -07:00			`pub mod window;`
Split window into two modules 2018-09-07 15:00:26 -07:00			`pub mod window_service;`
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms 2018-12-08 21:44:20 -08:00
Storage mining fixups... * Use IV to make unique identies * Use hex! macro for hex literal and not string converted to u8 slice * fix sha sampling to control init/end of sha state 2018-10-19 10:39:13 -07:00			`#[cfg(test)]`
			`#[cfg(any(feature = "chacha", feature = "cuda"))]`
			`#[macro_use]`
			`extern crate hex_literal;`
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms 2018-12-08 21:44:20 -08:00
Fast UdpSocket reader * message needs to fit into 256 bytes * allocator to keep track of blocks of messages * udp socket receiver server that fills up the block as fast as possible * udp socket sender server that sends out the block as fast as possible 2018-03-07 13:47:13 -08:00			`#[macro_use]`
No longer rename log crate 2018-03-19 09:18:48 -07:00			`extern crate log;`
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms 2018-12-08 21:44:20 -08:00
Add serialization/deseriation support to event log See bincode and serde_json for usage: https://github.com/TyOverby/bincode Fixes #1 2018-02-20 15:26:11 -08:00			`#[macro_use]`
			`extern crate serde_derive;`
Request/reqwest improvements - Use json macro to simplify request builds - Add proxy option for reqwest to use TLS - Add rpc port options for configured nodes 2018-09-21 14:27:03 -07:00			`#[macro_use]`
Initialize the testnode from a log $ cargo run --bin silk-genesis-file-demo > demo-genesis.json $ cat demo-genesis.json \| cargo run --bin silk-genesis-block > demo-genesis.log $ cat demo-genesis.log \| cargo run --bin silk-testnode 2018-03-05 14:34:15 -08:00			`extern crate serde_json;`
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms 2018-12-08 21:44:20 -08:00
Fast UdpSocket reader * message needs to fit into 256 bytes * allocator to keep track of blocks of messages * udp socket receiver server that fills up the block as fast as possible * udp socket sender server that sends out the block as fast as possible 2018-03-07 13:47:13 -08:00			`#[cfg(test)]`
			`#[macro_use]`
			`extern crate matches;`