Commit Graph

1790 Commits

Author SHA1 Message Date
Sagar Dhawan 12ae7b9a6b
Add test for tvu POH verification (#1844) 2018-11-16 15:48:10 -08:00
Michael Vines 6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
Michael Vines a0dd8617be Remove airdrop from fullnode 2018-11-16 13:25:55 -08:00
Rob Walker 1576072edb
remove spurious eprintln!() 2018-11-16 10:21:58 -08:00
carllin 03d206a7ca
Check for valid tvu, not tpu in broadcast (#1836) 2018-11-15 23:30:22 -08:00
Sathish c973de1d76
Decouple log and metrics rate (#1839)
Use separate env for log and metrics rate.

Set default log level to WARN if unset.
2018-11-15 22:27:16 -08:00
Stephen Akridge 71336965a6 Limit targets to 4 in bench-tps
Transaction got bigger so can only fit 4 targets in a
Transaction now.
2018-11-15 20:25:07 -08:00
Michael Vines e791d0f74d Drone now returns signed airdrop transactions 2018-11-15 17:13:13 -08:00
Stephen Akridge 3543a9a49f Add check for missing signature with fee'ed transaction
And update fetch-perf-libs version
2018-11-15 16:23:13 -08:00
Stephen Akridge 7dd198a99e Change signed_key to index into account_keys
If index is within the signed keys range.
2018-11-15 16:23:13 -08:00
Stephen Akridge e048116ab2 Remove signed_keys
Use first signatures.len() of account_keys for signing
2018-11-15 16:23:13 -08:00
Stephen Akridge cda9ad8565 Multiple signatures for transactions
With multiple instructions in a TX may need
multiple signatures.

Fixes #1531
2018-11-15 16:23:13 -08:00
carllin 928f375683
Rocks db (#1792)
* Add rocksdb crate

* Implement new ledger module based on RocksDb
2018-11-15 15:53:31 -08:00
Rob Walker d3e521f70e
accept other socket errors, ignore unless out of tries (#1835) 2018-11-15 15:49:37 -08:00
Michael Vines a7ee428214 Fix build 2018-11-15 14:06:57 -08:00
anatoly yakovenko a41254e18c
Add scalable gossip library (#1546)
* Cluster Replicated Data Store

Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.

Simulation shows fast convergence with 20k nodes.
2018-11-15 13:23:26 -08:00
Sagar Dhawan c81a3f6ced
Fix RPC address clashes on local multi-node testnet (#1821)
* Fix RPC address clashes on local multi-node testnet
2018-11-15 10:42:02 -08:00
Tyera Eulberg a5412fc0cd Fix find port functions 2018-11-15 10:45:39 -07:00
Michael Vines 5a8938209b Expose tick_height to native programs 2018-11-14 10:33:27 -08:00
Michael Vines e33f3a2562 Publish expected native program entrypoint in sdk/ 2018-11-14 10:33:27 -08:00
Sathish 9fe210c454
Add host information to db entries (#1778)
Add new field to each db entry identifying the host
that it originated from.
2018-11-13 21:54:15 -08:00
Jack May e7b6c8b7e0 Accounts get kicked if no tokens 2018-11-13 17:23:13 -08:00
Jack May 4e553ea095 test_replicate fails locally, ignore 2018-11-13 17:13:25 -08:00
Jack May 0c46f15f94 test_rpc_new fails locally, ignore for now 2018-11-13 17:12:25 -08:00
Sathish 7b92497d21
Update counters irrespective of logging level (#1799) 2018-11-13 16:55:14 -08:00
Sagar Dhawan 729d28d910 Add poh verification before processing entries
- Replicate stage now verifies entries delivered
  by the window
- Minor refactor of entries_from_blobs
2018-11-13 14:17:00 -08:00
carllin 6335be803c
Broadcast last tick before leader rotation (#1766)
* Broadcast last tick before leader rotation to everybody on network

* Add test

* Refactor broadcast
2018-11-13 02:21:37 -08:00
Sagar Dhawan a77b1ff767
Revert "Migrate from ring to ed25519-dalek" (#1798)
* Revert "Migrate from ring to ed25519-dalek"

This reverts commit 7c610b216b.

* Fix test failures with revert
2018-11-12 22:34:43 -08:00
Tyera Eulberg c7a67b5a02 Add deploy command to test 2018-11-12 18:21:16 -07:00
Tyera Eulberg 0e749dad4c Use cluster_info to get rpc address 2018-11-12 18:21:16 -07:00
Rob Walker fa72160c95
add last_id to Entry, PohEntry (#1783)
add prev_id to Entry, PohEntry
2018-11-12 17:03:23 -08:00
Greg Fitzgerald 126f065cc9 Extract complex loop from execute_instruction 2018-11-12 14:47:23 -08:00
carllin 4d98da44e3
Fix possibility of a vote error breaking ledger (#1768)
* Fix possibility of a vote error breaking ledger

* Add test
2018-11-12 11:40:32 -08:00
Michael Vines 522876c808 Rename Account.program_id to Account.owner 2018-11-12 10:59:01 -08:00
Michael Vines 7d05cc8c5d Add missing account fields 2018-11-12 10:59:01 -08:00
Michael Vines 49f4be6a2b codemod --extensions rs loader_program_id loader 2018-11-12 10:59:01 -08:00
Greg Fitzgerald 5fce8d2ce1 Don't ignore VoteProgram errors 2018-11-11 22:18:06 -07:00
Greg Fitzgerald 2696b22348 Cleanup TVU diagram 2018-11-11 20:55:21 -08:00
Greg Fitzgerald 5df4754579 Don't call instructions transactions 2018-11-11 20:07:15 -08:00
Tyera Eulberg a00284c727 Remove userdata diff and make helper fn 2018-11-11 18:57:28 -07:00
Tyera Eulberg 3832602ec4 Move notifications after store_accounts 2018-11-11 18:57:28 -07:00
Greg Fitzgerald fea86b2955 No longer serialize as JSON-encoded pkcs8
That's supposed to be an ASCII format, but we're not making use
of it. We can switch back to that some day, but if we do, it shouldn't
be JSON-encoded.
2018-11-11 09:22:52 -07:00
Greg Fitzgerald 7c610b216b Migrate from ring to ed25519-dalek
Why?

* Pure Rust, no BoringSSL (or OpenSSL) dependency
* Those avx2 benchmarks
* ring includes far more than what we need
* ring author won't add release tags: https://github.com/briansmith/ring#versioning--stability
2018-11-11 09:22:52 -07:00
Greg Fitzgerald cdf1a96e23 Revert "V1 Window/Ledger based on RocksDb (#1712)"
This reverts commit bfcdec95cb.
2018-11-09 20:25:53 -07:00
carllin bfcdec95cb
V1 Window/Ledger based on RocksDb (#1712)
* Add rocksdb

* Implement new ledger module based on RocksDb
2018-11-09 18:30:26 -08:00
Greg Fitzgerald fc55835932 Revert "Boot rpc_port"
This reverts commit 1984b6db06dae4f6b655b9076e657e389d786105.
2018-11-09 17:52:10 -07:00
Greg Fitzgerald 3772910bf2 Boot rpc_port 2018-11-09 17:52:10 -07:00
Greg Fitzgerald 24379c14dc Fix clippy warnings 2018-11-09 17:52:10 -07:00
Greg Fitzgerald 23846bcf1c Don't require a cluster to query for one's own pubkey 2018-11-09 17:52:10 -07:00
Greg Fitzgerald 9dd0a6e6a7 Boot drone_addr and rpc_addr from config
WalletConfig is intended for the validated command-line input.
2018-11-09 17:52:10 -07:00
Greg Fitzgerald 5ca473ac2d Don't get the network from parse_args 2018-11-09 17:52:10 -07:00
Greg Fitzgerald f5827d4a83 Fix typo 2018-11-08 17:15:48 -07:00
Greg Fitzgerald b0f8a983c4
Add the solana-wallet documentation (#1744)
* Add the solana-wallet documentation

There doesn't seem to be a way to publish bin docs to crates.io.
Until there is, we can include CLI documentation is the appendix
of the markdown book.

* A command to generate all the usage docs

Usage:

$ scripts/wallet-help.sh >> src/wallet.md
2018-11-08 15:42:20 -07:00
Rob Walker d831c5dcc9
remove dead poh code (#1747) 2018-11-08 12:55:23 -08:00
Greg Fitzgerald ce474eaf54 Better titles for tpu and tvu 2018-11-08 11:33:52 -07:00
Greg Fitzgerald 0da1c06b15 Add disk to the hardware used by both Tpu and Tvu 2018-11-08 11:33:52 -07:00
Greg Fitzgerald 01edc94a4b Move description of the Rust flavor of stages to service.rs 2018-11-08 11:33:52 -07:00
Greg Fitzgerald f96563c3f2 Add documentation for pipelining 2018-11-08 11:33:52 -07:00
Rob Walker 30697f63f1
add support for slots in erasure (#1736) 2018-11-08 10:20:03 -08:00
Greg Fitzgerald 2a6046de8e
Cleanup TVU code to look like its block diagram (#1737)
* Reorg TVU code to look like TVU diagram

And move channel creation into LedgerWriteStage so that it can
be used in the same was as all the other stages.

* Delete commented out code
2018-11-07 19:25:36 -07:00
Greg Fitzgerald 43bd28cdfa Add loader_ prefix to LoaderTransaction methods 2018-11-07 15:06:38 -07:00
Rob Walker 6c10458b5b
leader slots in Blobs (#1732)
* add leader slot to Blobs
* remove get_X() methods in favor of X() methods for Blob
* add slot to get_scheduled_leader()
2018-11-07 13:18:14 -08:00
Stephen Akridge 7780d9bab8 Add ledger write and storage stage to TVU documentation 2018-11-07 12:07:12 -08:00
Greg Fitzgerald 038a46b5ef Integrate the markdown book into the codebase
This implies that the book should describe exactly what is implemented,
and will not lead the way and eventually bitrot as the RFCs do.
2018-11-07 10:58:47 -07:00
Greg Fitzgerald 8be7c13d2d
Stub out architecture book (#1674)
* Stub out architecture documentation

* Add book HTML generation and book tests to CI

* Add heading

* Better table of contents

* Reference existing documentation

Move ASCII art from code comments into rendered SVG

* Attempt to fix CI

* Add lamport docs

And truncate lines to 80 characters

* Fix links

And reference shorter, newer description of PoH.

* Replace ASCII art with SVG

* Streamline for Pillbox

* Update path before optional install

* Use $CARGO_HOME instead of $HOME

* Delete code

Attempt to describe all data structures without code.

* Boot RPU from docs, add JsonRpcService

Also, use Rust naming conventions in the block diagrams to
minimize the jump from docs to code.

* Latest code uses tick_height

* Rename bob/ folder to art/

A home for any ASCII art

* Import JSON RPC API

* More mdbook docs

* Add Ncp

* Cleanup links

* Move pipelining description into fullnode description

* Move high-level transaction docs into top-level doc

* Delete unused files
2018-11-06 18:00:58 -07:00
Tyera Eulberg d7ea66b6a1 RPC and Pubsub, bind to 0.0.0.0 2018-11-06 15:45:36 -07:00
Sathish 371c69d425
Add ledger write stage counters (#1713) 2018-11-06 14:44:54 -08:00
Rob Walker 9b43b00d5c
remove tick_count, leader_scheduler, from broadcast code (#1725) 2018-11-06 13:17:41 -08:00
Rob Walker 76694bfcf4
remove entry_writer.rs (#1720) 2018-11-06 12:42:31 -08:00
Greg Fitzgerald bfad138bb3 Pass any serializable to Transaction constructor 2018-11-06 11:23:59 -07:00
Greg Fitzgerald 54968b59bb Update last_id between client retries
Fixes #1694
2018-11-06 09:06:15 -07:00
Tyera Eulberg f683817b48 Remove RPU; replace with RPC 2018-11-05 20:30:47 -07:00
Rob Walker 22abc27be4
add tests for bank.purge() (#1711) 2018-11-05 16:43:27 -08:00
Greg Fitzgerald c9138f964b Change token type from i64 to u64
Fixes #1526
2018-11-05 15:25:26 -07:00
Rob Walker 1fbf1d2cf2
Add checkpoint, rollback to to bank (#1662)
add linked-list capability to accounts

change accounts from a linked list to a VecDeque

add checkpoint and rollback for lastids

add subscriber notifications for rollbacks

checkpoint transaction count, too
2018-11-05 09:47:41 -08:00
Rob Walker 1d7722043f genesis has 3 entries now 2018-11-02 22:02:13 -07:00
Rob Walker 95f9488a70
use default buffer size for index, use BLOB_DATA_SIZE for data buffer (#1693) 2018-11-02 21:52:57 -07:00
Greg Fitzgerald e7cbbd8d45 cargo fmt 2018-11-02 19:54:49 -07:00
Greg Fitzgerald c8c255ad73 Rename Budget to BudgetExpr 2018-11-02 19:54:49 -07:00
Michael Vines a264f8fa9b Fix |cargo test| 2018-11-02 19:04:59 -07:00
Michael Vines 40e945b0c8 Move token_program from src/ to programs/native/ 2018-11-02 18:13:02 -07:00
carllin 0636399b7a
Compute finality computation in new ComputeLeaderFinalityService (#1652)
* Move finality computation into a service run from the banking stage, ComputeLeaderFinalityService

* Change last ids nth to tick height, remove separate tick height from bank
2018-11-02 15:49:14 -07:00
carllin 298bd6479a
Add first leader to genesis (#1681)
* Add first leader to genesis entries, consume in genesis.sh

* Set bootstrap leader in the bank on startup, remove instantiation of bootstrap leader from bin/fullnode

* Remove need to initialize bootstrap leader in leader_scheduler, now can be read from genesis entries

* Add separate interface new_with_leader() in mint for creating genesis leader entries
2018-11-02 14:32:05 -07:00
Tyera Eulberg 879431ebcd Add timeout to TcpStream connect, and rename test 2018-11-01 14:13:19 -06:00
Tyera Eulberg af1283e92c Improve airdrop confirmation logic 2018-11-01 11:28:33 -06:00
Sathish dffa2eb04f
Do not parallelize deserialize operation (#1663)
Deserialize operations are faster when done serially with the
MT banking stage and helps with performance improvement with
reduced thread context switches.
2018-10-31 22:12:15 -07:00
carllin 00889c5139
Fix bad function arguments (#1682) 2018-10-31 19:55:58 -07:00
Stephen Akridge af8dc3fd83 Fix snap build
cuda and chacha features required for chacha_cuda
2018-10-31 17:59:31 -07:00
carllin ba884b4e36
Add thin client test for vote functionality, fix sizing errors in vote contract (#1643)
* Added tests to thin client to test VoteContract calls, fix VoteContract sizing errors

* Calculate upper bound on VoteProgram size at runtime, add test for serializing/deserializing a max sized VoteProgram state
2018-10-31 17:47:50 -07:00
Tyera Eulberg 6ddd494826 Improve rpc logging 2018-10-31 15:21:55 -06:00
Stephen Akridge aa2fd3f3bb Storage RFC grammar 2018-10-31 13:44:21 -07:00
Stephen Akridge cf00354f42 Add storage stage which does storage mining verification for validators 2018-10-31 13:44:21 -07:00
carllin 47f1fa3f2e
Remove purging of leader id from cluster info (#1642) 2018-10-31 12:30:48 -07:00
Michael Vines 74b4ecb7f3 Upgrade to influx_db_client@0.3.6 2018-10-30 19:44:09 -07:00
Michael Vines 8d16f69bb9 Improve account subscribe/unsubscribe logging 2018-10-30 12:03:35 -07:00
Michael Vines 3a73a09391 Avoid panicking when a native library doesn't exist 2018-10-30 12:03:35 -07:00
Michael Vines 009c71f7e2 Demote info logs 2018-10-30 12:03:35 -07:00
Michael Vines 073d39df44 Add solana_ prefix to loaders so their logs appear in the default RUST_LOG config 2018-10-30 12:03:35 -07:00
Michael Vines ae7222f0df Work around influxdb panic 2018-10-30 12:03:35 -07:00
Rob Walker 13bfdde228
remove ledger tail code, WINDOW_SIZE begone (#1617)
* remove WINDOW_SIZE, use window.window_size()
* move ledger tail, redundant with ledger-based repair
2018-10-30 10:05:18 -07:00