Commit Graph

1839 Commits

Author SHA1 Message Date
Michael Vines a13e25f083 Ignore flaky test_tpu_forwarder 2018-11-26 09:27:21 -08:00
Greg Fitzgerald 3d3a30e200 Fix mdbook test 2018-11-26 07:51:10 -08:00
Greg Fitzgerald 76b83ac0f4 Move testnet demos into the book
Have git readme focus on fullnode development and the book focus on
users.
2018-11-26 07:51:10 -08:00
Greg Fitzgerald 903a9bfd05 s/contract/program/ 2018-11-26 08:20:42 -07:00
Greg Fitzgerald 655ee1a64b Fix typos 2018-11-26 08:20:42 -07:00
Greg Fitzgerald e0e6c3fdb2 Extract execute_instruction() to seed new runtime module
Fixes #1528
2018-11-26 08:20:42 -07:00
Greg Fitzgerald 31f00974f2 Hoist the lookup of executable accounts 2018-11-26 08:20:42 -07:00
Greg Fitzgerald c3218bb9c2 Hoist tick_height 2018-11-26 08:20:42 -07:00
Greg Fitzgerald d2972024de Uppercase acronyms
Looks like there will be very little Rust code in the markdown book
so switching back to English capitalization conventions.
2018-11-25 22:58:07 -07:00
Greg Fitzgerald dc0a2ca656
Move disclaimer down a bit
Odd to see a disclaimer before knowing anything about what you're reading
2018-11-25 20:27:35 -07:00
Greg Fitzgerald c3fb9d5549
Cleanup book (#1904)
* Cleanup book

* Distinguish upstream from downstream validators
* Add BroadcastStage to Fullnode/Tpu diagrams
* First attempt to re-describe the runtime

* Reorg book

Push back details of the fullnode implementation
2018-11-25 16:58:38 -07:00
Sathish f5b5c54d7d
Update condition for nosigverify (#1903) 2018-11-25 13:11:07 -08:00
Sathish 9f0b06bb86 Filter out leader node while retransmitting blobs (#1894) 2018-11-24 20:33:49 -08:00
carllin 57a384d6a0
Rocks db window service (#1888)
* Add db_window module for windowing functions from RocksDb

* Replace window with db_window functions in window_service

* Fix tests

* Make note of change in db_window

* Create RocksDb ledger in bin/fullnode

* Make db_ledger functions generic

* Add db_ledger to bin/replicator
2018-11-24 19:32:33 -08:00
Greg Fitzgerald 69802e141f Add the story of how this codebase came to be 2018-11-24 14:39:53 -07:00
Greg Fitzgerald 6fc02b7424 Detect legacy programs upfront 2018-11-24 11:56:51 -07:00
Greg Fitzgerald 30cdd85028 Implement the same interface in all builtin programs 2018-11-24 11:56:51 -07:00
Greg Fitzgerald 871dd47019 Extract the part of execute_instruction that should only return a ProgramError
TODO: hoist load_executable_accounts() and then change
process_instruction() to return ProgramError.
2018-11-24 11:56:51 -07:00
Greg Fitzgerald 37f8dd57e2 Extract ProgramError from BankError 2018-11-24 11:56:51 -07:00
Greg Fitzgerald f827bfd83f Remove instruction index parameter 2018-11-24 11:56:51 -07:00
Greg Fitzgerald b3af930153 Rename process_transaction to process_instruction 2018-11-24 11:56:51 -07:00
Greg Fitzgerald cd488b7d07 Hoist program static methods to top-level functions 2018-11-24 11:56:51 -07:00
Rob Walker e2373ff51a add nosigverify command line option to ease debug 2018-11-23 16:55:04 -08:00
Greg Fitzgerald b3d2c900cd Rename BudgetState to BudgetProgram 2018-11-23 13:25:17 -07:00
Michael Vines d5adec20a3 get_ip_addr: Fall back to loopback if no better option exists 2018-11-23 13:24:41 -05:00
carllin 942256a647
Add db_ledger benchmarks (#1875)
* Add db_ledger benchmarks

* ignore benches in CI, due to timeouts
2018-11-23 06:12:43 -08:00
carllin a3321a5d80
Fix endianess in db_ledger to account for the default byte-comparator used by Rocksdb (#1885) 2018-11-22 01:35:19 -08:00
carllin 521de13571
Add maximum repair length to db_window (#1886)
* Add maximum repair length to db_window
2018-11-21 23:44:49 -08:00
Greg Fitzgerald 3abf6a8a30 Reorg the markdown book to cater to app devs
First, talk about how a client interacts with Solana to do useful
things. Then describe how the fullnode you're talking to works and
why it's so very fast.  Last, why that fullnode you don't trust
does what you asked it to anyway.
2018-11-21 15:49:57 -08:00
jackcmay 59163e2dd9
Optimize some CI stuff (#1880)
* CI Optimizations
2018-11-21 12:16:16 -08:00
carllin d9dabdfc74
Rocks db window utils (#1851)
* Implement new ledger module based on RocksDb

* Add db_window module for windowing functions from RocksDb
2018-11-19 23:20:18 -08:00
Rob Walker 6b910d1bd4 add tpu_forwarding, simplify ClusterInfo::new() from Result<Self> to Self 2018-11-19 20:45:49 -08:00
Rob Walker bbd9ea8c00
Delete settings.rs.foo 2018-11-19 13:39:08 -08:00
Anatoly Yakovenko fc67a968e8 Use known keys in the unit test to avoid random false positives. 2018-11-19 13:41:24 -07:00
Rob Walker 3d113611cc
remove Result<> return from ClusterInfo::new() (#1869)
strip Result<> for ClusterInfo::new()
2018-11-19 11:25:14 -08:00
Sagar Dhawan 07667771ef
Fix Gossip Pushes going to invalid addresses (#1858) 2018-11-17 19:57:28 -08:00
Michael Vines 3822c29415 Route program_id to program entrypoint 2018-11-17 19:42:03 -08:00
Greg Fitzgerald ff386d6585 Add disclaimer to markdown book
copy-paste from readme
2018-11-17 19:56:08 -07:00
Greg Fitzgerald 88ddb31477 teminology cleanup: leader slots and voting rounds 2018-11-17 18:56:13 -07:00
Greg Fitzgerald 077d1a41f1 Add too book 2018-11-17 18:56:13 -07:00
Greg Fitzgerald 857ab8662e backticks and missing variable descriptions 2018-11-17 18:56:13 -07:00
Greg Fitzgerald a17f9bd0f4 Work towards adding leader rotation to the book 2018-11-17 18:56:13 -07:00
Greg Fitzgerald f4b9e93b11 Migrate storage RFC to book 2018-11-17 18:55:08 -07:00
Greg Fitzgerald 2c11bf2e66 Various book cleanup
* Merge Leader and Validator diagrams
* New sdk-tools diagram
* Move terminology to just after introduction
* Purge use of LAMPORT as an acronym
* Add notes about persistent storage
2018-11-17 17:50:29 -08:00
Michael Vines 719e14b30a Add an explicit state of a reserved signature
An RPC client that fetches the signature status before the bank finishes
executing the corresponding Transaction should receive SignatureNotFound
instead of Confirmed
2018-11-17 16:40:23 -08:00
Michael Vines d1f01b5209 Fix clippy lint 2018-11-17 15:54:21 -08:00
Michael Vines d96a6b42a5 Move drone into its own crate 2018-11-16 20:42:21 -08:00
carllin cf95708c18 Set drone address to always be the initial network entry point (#1847)
* Set drone address to always be the initial network entry point, so that even when leaders rotate the client can still find the drone

* Extract drone address as a separate argument to bench-tps

* Add drone port to client.sh instead of setting it in bench-tps

* Add drone entrypoint to scripts

* Fix build error
2018-11-16 19:56:26 -08:00
Michael Vines e1c7b99450
Accounts get kicked if no tokens 2018-11-16 18:53:37 -08:00
Sagar Dhawan 12ae7b9a6b
Add test for tvu POH verification (#1844) 2018-11-16 15:48:10 -08:00
Michael Vines 6ac5700f2e Move metrics into its own crate 2018-11-16 15:10:07 -08:00
Michael Vines a0dd8617be Remove airdrop from fullnode 2018-11-16 13:25:55 -08:00
Rob Walker 1576072edb
remove spurious eprintln!() 2018-11-16 10:21:58 -08:00
carllin 03d206a7ca
Check for valid tvu, not tpu in broadcast (#1836) 2018-11-15 23:30:22 -08:00
Sathish c973de1d76
Decouple log and metrics rate (#1839)
Use separate env for log and metrics rate.

Set default log level to WARN if unset.
2018-11-15 22:27:16 -08:00
Stephen Akridge 71336965a6 Limit targets to 4 in bench-tps
Transaction got bigger so can only fit 4 targets in a
Transaction now.
2018-11-15 20:25:07 -08:00
Michael Vines e791d0f74d Drone now returns signed airdrop transactions 2018-11-15 17:13:13 -08:00
Stephen Akridge 3543a9a49f Add check for missing signature with fee'ed transaction
And update fetch-perf-libs version
2018-11-15 16:23:13 -08:00
Stephen Akridge 7dd198a99e Change signed_key to index into account_keys
If index is within the signed keys range.
2018-11-15 16:23:13 -08:00
Stephen Akridge e048116ab2 Remove signed_keys
Use first signatures.len() of account_keys for signing
2018-11-15 16:23:13 -08:00
Stephen Akridge cda9ad8565 Multiple signatures for transactions
With multiple instructions in a TX may need
multiple signatures.

Fixes #1531
2018-11-15 16:23:13 -08:00
carllin 928f375683
Rocks db (#1792)
* Add rocksdb crate

* Implement new ledger module based on RocksDb
2018-11-15 15:53:31 -08:00
Rob Walker d3e521f70e
accept other socket errors, ignore unless out of tries (#1835) 2018-11-15 15:49:37 -08:00
Michael Vines a7ee428214 Fix build 2018-11-15 14:06:57 -08:00
anatoly yakovenko a41254e18c
Add scalable gossip library (#1546)
* Cluster Replicated Data Store

Separate the data storage and merge strategy from the network IO boundary.
Implement an eager push overlay for transporting recent messages.

Simulation shows fast convergence with 20k nodes.
2018-11-15 13:23:26 -08:00
Sagar Dhawan c81a3f6ced
Fix RPC address clashes on local multi-node testnet (#1821)
* Fix RPC address clashes on local multi-node testnet
2018-11-15 10:42:02 -08:00
Tyera Eulberg a5412fc0cd Fix find port functions 2018-11-15 10:45:39 -07:00
Michael Vines 5a8938209b Expose tick_height to native programs 2018-11-14 10:33:27 -08:00
Michael Vines e33f3a2562 Publish expected native program entrypoint in sdk/ 2018-11-14 10:33:27 -08:00
Sathish 9fe210c454
Add host information to db entries (#1778)
Add new field to each db entry identifying the host
that it originated from.
2018-11-13 21:54:15 -08:00
Jack May e7b6c8b7e0 Accounts get kicked if no tokens 2018-11-13 17:23:13 -08:00
Jack May 4e553ea095 test_replicate fails locally, ignore 2018-11-13 17:13:25 -08:00
Jack May 0c46f15f94 test_rpc_new fails locally, ignore for now 2018-11-13 17:12:25 -08:00
Sathish 7b92497d21
Update counters irrespective of logging level (#1799) 2018-11-13 16:55:14 -08:00
Sagar Dhawan 729d28d910 Add poh verification before processing entries
- Replicate stage now verifies entries delivered
  by the window
- Minor refactor of entries_from_blobs
2018-11-13 14:17:00 -08:00
carllin 6335be803c
Broadcast last tick before leader rotation (#1766)
* Broadcast last tick before leader rotation to everybody on network

* Add test

* Refactor broadcast
2018-11-13 02:21:37 -08:00
Sagar Dhawan a77b1ff767
Revert "Migrate from ring to ed25519-dalek" (#1798)
* Revert "Migrate from ring to ed25519-dalek"

This reverts commit 7c610b216b.

* Fix test failures with revert
2018-11-12 22:34:43 -08:00
Tyera Eulberg c7a67b5a02 Add deploy command to test 2018-11-12 18:21:16 -07:00
Tyera Eulberg 0e749dad4c Use cluster_info to get rpc address 2018-11-12 18:21:16 -07:00
Rob Walker fa72160c95
add last_id to Entry, PohEntry (#1783)
add prev_id to Entry, PohEntry
2018-11-12 17:03:23 -08:00
Greg Fitzgerald 126f065cc9 Extract complex loop from execute_instruction 2018-11-12 14:47:23 -08:00
carllin 4d98da44e3
Fix possibility of a vote error breaking ledger (#1768)
* Fix possibility of a vote error breaking ledger

* Add test
2018-11-12 11:40:32 -08:00
Michael Vines 522876c808 Rename Account.program_id to Account.owner 2018-11-12 10:59:01 -08:00
Michael Vines 7d05cc8c5d Add missing account fields 2018-11-12 10:59:01 -08:00
Michael Vines 49f4be6a2b codemod --extensions rs loader_program_id loader 2018-11-12 10:59:01 -08:00
Greg Fitzgerald 5fce8d2ce1 Don't ignore VoteProgram errors 2018-11-11 22:18:06 -07:00
Greg Fitzgerald 2696b22348 Cleanup TVU diagram 2018-11-11 20:55:21 -08:00
Greg Fitzgerald 5df4754579 Don't call instructions transactions 2018-11-11 20:07:15 -08:00
Tyera Eulberg a00284c727 Remove userdata diff and make helper fn 2018-11-11 18:57:28 -07:00
Tyera Eulberg 3832602ec4 Move notifications after store_accounts 2018-11-11 18:57:28 -07:00
Greg Fitzgerald fea86b2955 No longer serialize as JSON-encoded pkcs8
That's supposed to be an ASCII format, but we're not making use
of it. We can switch back to that some day, but if we do, it shouldn't
be JSON-encoded.
2018-11-11 09:22:52 -07:00
Greg Fitzgerald 7c610b216b Migrate from ring to ed25519-dalek
Why?

* Pure Rust, no BoringSSL (or OpenSSL) dependency
* Those avx2 benchmarks
* ring includes far more than what we need
* ring author won't add release tags: https://github.com/briansmith/ring#versioning--stability
2018-11-11 09:22:52 -07:00
Greg Fitzgerald cdf1a96e23 Revert "V1 Window/Ledger based on RocksDb (#1712)"
This reverts commit bfcdec95cb.
2018-11-09 20:25:53 -07:00
carllin bfcdec95cb
V1 Window/Ledger based on RocksDb (#1712)
* Add rocksdb

* Implement new ledger module based on RocksDb
2018-11-09 18:30:26 -08:00
Greg Fitzgerald fc55835932 Revert "Boot rpc_port"
This reverts commit 1984b6db06dae4f6b655b9076e657e389d786105.
2018-11-09 17:52:10 -07:00
Greg Fitzgerald 3772910bf2 Boot rpc_port 2018-11-09 17:52:10 -07:00
Greg Fitzgerald 24379c14dc Fix clippy warnings 2018-11-09 17:52:10 -07:00
Greg Fitzgerald 23846bcf1c Don't require a cluster to query for one's own pubkey 2018-11-09 17:52:10 -07:00
Greg Fitzgerald 9dd0a6e6a7 Boot drone_addr and rpc_addr from config
WalletConfig is intended for the validated command-line input.
2018-11-09 17:52:10 -07:00
Greg Fitzgerald 5ca473ac2d Don't get the network from parse_args 2018-11-09 17:52:10 -07:00