Sagar Dhawan
0bea870b22
Dynamic N layer 'avalanche' broadcast and retransmit ( #2058 )
...
* Dynamic N layer avalanche broadcast and retransmit
2019-01-02 14:16:15 +05:30
Michael Vines
58f2598d5d
Revert "Validators make a transaction to advertise their storage last_id"
...
This reverts commit a1759aed19
.
2018-12-23 14:02:09 -08:00
carllin
58a4905916
Make reconstruct_entries_from_blobs() support Blobs and borrowed SharedBlobs, make distinction between to_blobs and to_shared_blobs ( #2270 )
2018-12-22 19:30:30 -08:00
Michael Vines
2c9607d5da
Rename getConfirmation -> getConfirmationTime
2018-12-22 12:47:02 -08:00
Stephen Akridge
a1759aed19
Validators make a transaction to advertise their storage last_id
...
* Also implement more storage contract logic
* Add transactions for proof validation,
* Move storage state members into system storage account userdata
2018-12-21 15:45:30 -08:00
Sathish
1a3387706d
Spawn threads based on cpu count ( #2232 )
2018-12-21 13:55:45 -08:00
Pankaj Garg
951d6398a0
Rename finality to confirmation ( #2250 )
...
* Rename finality to confirmation
* fix cargo fmt errors
2018-12-20 15:47:48 -08:00
Pankaj Garg
7c98545b33
Use newer votes to calculate confirmation time ( #2247 )
2018-12-20 15:27:47 -08:00
Pankaj Garg
bb1060bdad
Reduce ticks per block to increase voting frequency ( #2242 )
2018-12-20 14:43:03 -08:00
Michael Vines
7ad45a91ec
Fix compile error
2018-12-20 13:47:36 -08:00
Michael Vines
034c5d0422
db_ledger now fully encapsulates rocksdb
2018-12-20 12:32:25 -08:00
carllin
7148c14178
Debug broadcast ( #2233 )
...
* Account for duplicate blobs in process_blobs
* Increase max bytes for level base to match write buffer
2018-12-20 12:12:04 -08:00
Michael Vines
93fb61dc8f
Re-export rocksdb::DBRawIterator until it can be encapsulated
2018-12-20 10:38:03 -08:00
Michael Vines
b36ceb5be4
Remove rocksdb dependency from result.rs
2018-12-20 10:38:03 -08:00
Michael Vines
37d7ad819b
Purge DB::destroy() usage
2018-12-20 10:38:03 -08:00
carllin
666af1e62d
Debug broadcast ( #2208 )
...
* Add per cf rocksdb options, increase compaction and flush threads
* Change broadcast stage to bulk write blobs
* add db_ledger function specifically for broadcast
* fix broken tests
* fix benches
2018-12-19 16:11:47 -08:00
Rob Walker
b4f4347d6e
add some more tests ( #2217 )
2018-12-18 17:27:03 -08:00
Rob Walker
5c866dd000
test drive new coverage stuff ( #2216 )
2018-12-18 16:44:27 -08:00
Pankaj Garg
974249f2a5
Parallelize entry processing in replay stage in validators ( #2212 )
...
* Parallelize entry processing in replay stage in validators
- single threaded entry processing is not utlizing CPU cores to the fullest
* fix tests and address review comments
2018-12-18 16:06:05 -08:00
Rob Walker
a65022aed7
DbLedger doesn't need to be mut, doesn't need an RwLock ( #2215 )
...
* DbLedger doesn't need to be mut, doesn't need an RwLock
* fix erasure cases
2018-12-18 15:18:57 -08:00
Sagar Dhawan
ffdc1814c6
Add counters for gossip verification failures ( #2094 )
2018-12-17 20:12:50 -08:00
carllin
9720ac0019
Fix try_erasure() ( #2185 )
...
* Fix try_erasure bug
* Re-enable asserts in test_replicator_startup
* Add test for out of order process_blobs
2018-12-17 15:34:19 -08:00
anatoly yakovenko
9a6e27ac36
Accounts is to big, should be its own module ( #2198 )
...
Account module is to big, should be in its own module.
2018-12-17 12:41:23 -08:00
Michael Vines
07202205c4
Revert "ignore unstable tests"
...
This reverts commit bd7ef5d445071329a3b49b1f8be71b602226bbec.
2018-12-17 10:47:32 -08:00
Michael Vines
b7144560c9
Include port number when gossip bind_to fails
2018-12-17 09:31:31 -08:00
anatoly yakovenko
4be6d01dfb
Move last ids ( #2187 )
...
* Break out last_ids into its own module
* Boot SignatureNotFound from BankError
* No longer return BankError from LastIds methods
* No longer piggypack on BankError for a LastIds signature status
* Drop all dependencies on the bank
* SignatureStatus -> Status and LastIds -> StatusDeque
* Unstable tests, issue 2193
2018-12-17 07:55:56 -08:00
Michael Vines
fd562cb9e2
Rust 2018 cleanup
2018-12-14 21:57:15 -08:00
Michael Vines
5bf9a20d42
fullnode-config no longer depends on src/
2018-12-14 20:13:34 -08:00
Michael Vines
eedc8c7812
Move src/netutil.rs into its own crate
2018-12-14 20:13:34 -08:00
carllin
9ef5e51c0f
Cleanup slot remnants in db_ledger ( #2153 )
...
* Cleanup slot remnants in db_ledger
2018-12-14 17:05:41 -08:00
Sathish
8ee0e9632c
Switch to using hashbrown version of HashMap and ( #2158 )
...
HashSet for improved performance and memory usage
2018-12-14 15:10:10 -08:00
Michael Vines
6ac466c0a4
Move src/logger.rs into logger/ crate to unify logging across the workspace
2018-12-14 13:10:43 -08:00
Michael Vines
d45fcc4381
Move src/wallet.rs into wallet/ crate
2018-12-14 12:15:18 -08:00
carllin
6fcccedb70
align tick entries' tick_height with actual number of ticks in bank ( #2147 )
2018-12-14 02:25:50 -08:00
Michael Vines
0fe6d61036
Move binaries from src/bin into their own crate
2018-12-13 22:25:27 -08:00
Pankaj Garg
091b21fae7
Vote every number of ticks ( #2141 )
...
* Vote every number of ticks
* address review comments
* fix for failing leader rotation tests
* remove check for vote failure from replay tests
(as votes will be cached and transmitted when leader is available)
2018-12-13 18:43:10 -08:00
Sathish
7325b19aef
Do not allocate for each metrics submission ( #2146 )
2018-12-13 16:40:00 -08:00
Stephen Akridge
7cdbbfa88e
Storage stage updates
...
* Remove logging init from storage program: saw a crash in a test
indicating the logger being init'ed twice.
* Add entry_height mining proof to indicate which segment the result is
for
* Add an interface to get storage miner pubkeys for a given entry_height
* Add an interface to get the current storage mining entry_height
* Set the tvu socket to 0.0.0.0:0 in replicator to stop getting entries
after the desired ledger segment is downloaded.
* Use signature of PoH height to determine which block to download for
replicator.
2018-12-13 11:30:12 -08:00
Rob Walker
9880a86f80
remove prev_id, unused ( #2150 )
2018-12-13 09:24:38 -08:00
Michael Vines
647e5d76b0
Move solana-fullnode into fullnode/
2018-12-13 01:45:29 -08:00
Michael Vines
7e4af9382e
Move solana-upload-perf into upload-perf/
2018-12-13 01:06:40 -08:00
Michael Vines
282d4a3563
Move solana-keygen into keygen/
2018-12-13 01:06:40 -08:00
Michael Vines
cafeef33c3
Relocate all keypair generation into one location: sdk/src/signature.rs
2018-12-13 01:06:40 -08:00
Rob Walker
4f48f1a850
add db_ledger genesis, rework to_blob(), to_blobs() ( #2135 )
2018-12-12 20:42:12 -08:00
Carl
a05a378db4
cleanup
2018-12-12 19:12:51 -08:00
Carl
245362db96
Make a dummy version of serving repairs from db_ledger
2018-12-12 19:12:51 -08:00
Mark
b1b190b80d
Fix too many args in Tvu::new ( #2114 )
...
* Reduce args in Tvu::new under to 8
Now pass in sockets through a the crate::tvu::Sockets struct
Move ClusterInfo.keypair to pub(crate) in order to remove redundant
signing keypair parameter
* remove commented code
2018-12-12 18:57:48 -08:00
Rob Walker
3408ce89a7
add check_tick_height ( #2144 )
2018-12-12 18:52:11 -08:00
Michael Vines
75f407e191
Provide entire elf to bpf_loader
2018-12-12 17:14:50 -08:00
Mekagoza
9b81696a09
remove obsoleted TODO
2018-12-12 16:26:59 -08:00
Mekagoza
80e19e0ad7
Encapsulate accounts of solana: 🏦 :Accounts
...
Make the field private and expose an account_values() method that
returns the values iterator from the internal hashmap
2018-12-12 16:26:59 -08:00
carllin
ae903f190e
Broadcast for slots ( #2081 )
...
* Insert blobs into db_ledger in broadcast stage to support leader to validator transitions
* Add transmitting real slots to broadcast stage
* Handle real slots instead of default slots in window
* Switch to dummy repair on slots and modify erasure to support leader rotation
* Shorten length of holding locks
* Remove logger from replicator test
2018-12-12 15:58:29 -08:00
Pankaj Garg
3413ecc2bd
Change query used to find list of nodes in the network ( #2124 )
...
* Change query used to find list of nodes in the network
* include "All" option for host selection
2018-12-12 12:38:00 -08:00
Sagar Dhawan
4788a4f775
Correctly describe repair and retransmit peers ( #2110 )
2018-12-11 15:51:47 -08:00
Pankaj Garg
9243bc58db
Metrics for window repair ( #2106 )
...
* Metrics for window repair
- Also increase max repair length
* fix vote counters, and add repair window graph
* update per node graphs
* revert max repair length change
2018-12-11 15:43:41 -08:00
Rob Walker
4d67aca919
add genesis and read_ledger to db_ledger ( #2097 )
2018-12-11 09:14:23 -08:00
Rob Walker
aeee25e703
add tick_height to Entry to be able to repair by period, chain forks of Entries, etc. ( #2096 )
2018-12-10 20:03:04 -08:00
Greg Fitzgerald
01c524ddd2
Revert changes to counter names
2018-12-10 15:26:43 -07:00
Greg Fitzgerald
5e703dc70a
Free up the term 'replicate' for exclusive use in replicator
...
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
Pankaj Garg
bc96bd3410
Fix peer count in edge dashboard ( #2090 )
...
Fixes #2075
2018-12-10 14:24:32 -08:00
Michael Vines
4ea1c030bc
Give bootstrap leader one more token
2018-12-10 13:24:14 -08:00
Stephen Akridge
172e511e56
Use retry_transfer to test multiple times for replicator tokens
...
May fix failures in CI where replicator is trying to do an airdrop.
2018-12-10 12:19:00 -08:00
Tyera Eulberg
4481efd51e
Merge pull request #2084 from CriesofCarrots/fix-wallet-accountinuse
...
Fix wallet accountinuse
2018-12-10 12:20:55 -07:00
Greg Fitzgerald
0ef099421c
cargo fmt
2018-12-08 23:19:55 -07:00
Greg Fitzgerald
f1ae5b1795
Fix warnings
2018-12-08 23:19:55 -07:00
Greg Fitzgerald
a8d6c75a24
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms
2018-12-08 23:19:55 -07:00
Greg Fitzgerald
c49e2f8bbd
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition
2018-12-08 23:19:55 -07:00
Greg Fitzgerald
ec5a8141eb
cargo fix --edition
2018-12-08 23:19:55 -07:00
Sathish
2de45a4da5
Update airdrop tokens to 3 for fullnode ( #2051 )
...
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
Michael Vines
73b9ee9e84
Add solana_ prefix to native_loader program
...
This allows its logging to show up in the default RUST_LOG=solana=info
log setting
2018-12-08 11:04:45 -08:00
Greg Fitzgerald
0a83b17cdd
Upgrade to Rust 1.31.0 ( #2052 )
...
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Tyera Eulberg
571522e738
Update jsonrpc version
2018-12-07 17:47:54 -07:00
Stephen Akridge
3441d3399b
Replicator rework
...
* Move more of the replicator logic into the replicator class
* Add support for the RPC interface to query the storage last_id value
that the replicator would sign and use to pick a block.
* Fix replicator connecting to gossip and change test to exercise that
scenario.
2018-12-07 15:20:36 -08:00
Michael Vines
f4b26247c0
Genesis only needs a keypair, not the entire fullnode::Config
2018-12-06 16:31:24 -08:00
Michael Vines
c4b8f0cd2f
bench-tps will now generate an ephemeral identity if not provided with one
...
Also simplify scripts as a result
2018-12-06 16:30:48 -08:00
Greg Fitzgerald
97b1156a7a
Rename Ncp to GossipService
...
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
Michael Vines
cc2f448d92
Add fullnode --no-leader-rotation flag
2018-12-06 11:30:19 -08:00
Michael Vines
632425c7d7
Move native_loader under programs/native/
2018-12-05 14:32:42 -08:00
dependabot[bot]
ad3e36a7ab
Bump rand from 0.5.5 to 0.6.1 ( #1891 )
...
* Bump rand from 0.5.5 to 0.6.1
Bumps [rand](https://github.com/rust-random/rand ) from 0.5.5 to 0.6.1.
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/commits )
Signed-off-by: dependabot[bot] <support@dependabot.com>
* Fix conflicts and deprecated usages
* Fix benches
2018-12-05 14:12:10 -08:00
Michael Vines
1bcafca690
Find test_tx again
2018-12-05 13:29:29 -08:00
Rob Walker
e21b6d9db3
ensure we'd actually have N hashes per tick ( #2011 )
2018-12-05 12:49:41 -08:00
carllin
9c30bddb88
Rocks db erasure decoding ( #1900 )
...
* Change erasure to consume new RocksDb window
* Change tests for erasure
* Remove erasure from window
* Integrate erasure decoding back into window
* Remove corrupted blobs from ledger
* Replace Erasure result with result module's Result
2018-12-05 12:47:19 -08:00
Michael Vines
59e6bd115e
system_program must be a static lib as it allocates Account memory
2018-12-05 10:49:06 -08:00
Michael Vines
15aef079e3
Include builtin programs for ledger verification
2018-12-05 10:49:06 -08:00
Michael Vines
42689d4842
cargo fmt
2018-12-05 10:49:06 -08:00
Michael Vines
214ed3667c
Move system_transaction out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
122627dda2
Move loader_transaction out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
7af95eadcc
Move vote_transaction out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
9ee858a00c
Move budget_program out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
27d456bf93
Move storage_program out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
ea6e042a6f
Move vote_program out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
a594f56c02
Add token_program.rs to sdk/
2018-12-05 10:49:06 -08:00
Michael Vines
e6fa74fe69
Remove custom Error enum, just use ProgramError
2018-12-05 10:49:06 -08:00
Michael Vines
9a4f8199d6
Move system_program out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
ae0be1e857
Remove bpf_loader.rs
2018-12-05 10:49:06 -08:00
Michael Vines
d010cac8a5
Remove token_program.rs
2018-12-05 10:49:06 -08:00
Michael Vines
63a758508a
Add sdk native_loader.rs
2018-12-05 10:49:06 -08:00
Stephen Akridge
c1c2f1f0a9
Cleanup ad-hoc rpc address formation
...
Lots of places where we are forming rpc addresses.
2018-12-03 18:13:55 -08:00
Michael Vines
777a0a858e
Move ProgramError into sdk/
2018-12-03 13:50:00 -08:00
Michael Vines
68e99c18c0
Remove duplicate SYSTEM_PROGRAM_ID
2018-12-03 13:50:00 -08:00
Michael Vines
c99f93e40a
Remove signature.rs indirection
2018-12-03 13:50:00 -08:00
carllin
4ae58cc854
Change range of leader scheduler to match current broadcasts ( #1920 )
2018-12-03 00:10:43 -08:00
Sagar Dhawan
b21facab7b
Add metrics for prune messages ( #1981 )
2018-12-01 14:05:40 -08:00
Sagar Dhawan
34c3a0cc1f
Add signature verification to gossip ( #1937 )
2018-12-01 12:00:30 -08:00
Pankaj Garg
549bfe7412
Vote signing JSON RPC service ( #1965 )
...
* Vote signing JSON RPC service
- barebone service that listens for RPC requests
* Daemon for vote signer service
* Add request APIs for JSON RPC
* Cleanup of cargo dependencies
* Fix compiler error
2018-11-30 15:07:08 -08:00
Rob Walker
3ca826a480
re-enable test_tpu_forwarder ( #1964 )
2018-11-30 13:52:37 -08:00
Michael Vines
bad0b55ab6
Expose which keys signed the Transaction in the SDK
2018-11-30 08:16:23 -08:00
Michael Vines
0878bd53d9
Delete stub src/transaction.rs
2018-11-29 23:07:57 -08:00
Stephen Akridge
de910e1169
Make test_pubkey_distribution faster
...
multi-thread pubkey histogram generation.
2018-11-29 17:37:37 -08:00
Stephen Akridge
9684737de7
Add wait before checking confirm again
...
Otherwise we can quickly check that we
have no signature 4 times in a row.
2018-11-29 15:32:58 -08:00
Stephen Akridge
ecc87ab1aa
Add a an optional timeout to thin_client
...
Such that a negative test like test_transaction_count doesn't
have to wait num_retries * default_timeout.
2018-11-29 13:53:40 -08:00
Michael Vines
fa359c6fc4
Merge vote new and register transactions
2018-11-29 12:31:34 -08:00
Stephen Akridge
8cc751d1cc
Improve RPC service startup error messages with actual error
...
Error always fixed to message about ports but that's not the only
error that can occur.
2018-11-29 11:39:42 -08:00
Stephen Akridge
978fd6858f
Move replicator_startup_test to integration test set
...
Sometimes fails when run multithreaded with other tests.
2018-11-29 11:39:42 -08:00
Michael Vines
41689256c6
Ensure key[0] is signed
2018-11-29 10:26:46 -08:00
Pankaj Garg
194e3100a9
Additional checks in test_bank_checkpoint_zero_balance ( #1943 )
2018-11-28 12:40:34 -08:00
Rob Walker
8ad4464d4b
add tests for other "from" indexes signing (or not)
2018-11-28 07:56:04 -08:00
Rob Walker
e7b0a736f5
verify signature is on the from account
2018-11-28 07:56:04 -08:00
Rob Walker
167eb01735
optimize bench-tps and rpc_request to work on crappy WSL boxes
2018-11-27 22:45:08 -08:00
Michael Vines
8fb5d72b13
Make insufficient tokens message more helpful
2018-11-27 17:37:25 -08:00
Pankaj Garg
8947c5a4aa
Set account to default if the balance reaches 0 in a checkpoint bank ( #1932 )
...
Fixes : #1931
2018-11-27 14:17:29 -08:00
Greg Fitzgerald
a7562c9be1
Extract execute_transaction() from the bank
2018-11-27 12:35:52 -07:00
Greg Fitzgerald
08dc169f94
Hoist load_loaders()
...
This makes execute_transactions() stateless.
2018-11-27 12:35:52 -07:00
Greg Fitzgerald
f549d8ac74
Hoist loading of loaders
...
This might cause a TPS boost in batched BPF transactions, since
now it'll only clone its account once per transaction instead of
once per instruction.
2018-11-27 12:35:52 -07:00
Greg Fitzgerald
1ac7536286
Pass executable_accounts into with_subset()
2018-11-27 12:35:52 -07:00
Greg Fitzgerald
88e270723f
Move markdown book out of src/
2018-11-26 09:44:19 -08:00
Michael Vines
a13e25f083
Ignore flaky test_tpu_forwarder
2018-11-26 09:27:21 -08:00
Greg Fitzgerald
3d3a30e200
Fix mdbook test
2018-11-26 07:51:10 -08:00
Greg Fitzgerald
76b83ac0f4
Move testnet demos into the book
...
Have git readme focus on fullnode development and the book focus on
users.
2018-11-26 07:51:10 -08:00
Greg Fitzgerald
903a9bfd05
s/contract/program/
2018-11-26 08:20:42 -07:00
Greg Fitzgerald
655ee1a64b
Fix typos
2018-11-26 08:20:42 -07:00
Greg Fitzgerald
e0e6c3fdb2
Extract execute_instruction() to seed new runtime module
...
Fixes #1528
2018-11-26 08:20:42 -07:00
Greg Fitzgerald
31f00974f2
Hoist the lookup of executable accounts
2018-11-26 08:20:42 -07:00
Greg Fitzgerald
c3218bb9c2
Hoist tick_height
2018-11-26 08:20:42 -07:00
Greg Fitzgerald
d2972024de
Uppercase acronyms
...
Looks like there will be very little Rust code in the markdown book
so switching back to English capitalization conventions.
2018-11-25 22:58:07 -07:00
Greg Fitzgerald
dc0a2ca656
Move disclaimer down a bit
...
Odd to see a disclaimer before knowing anything about what you're reading
2018-11-25 20:27:35 -07:00
Greg Fitzgerald
c3fb9d5549
Cleanup book ( #1904 )
...
* Cleanup book
* Distinguish upstream from downstream validators
* Add BroadcastStage to Fullnode/Tpu diagrams
* First attempt to re-describe the runtime
* Reorg book
Push back details of the fullnode implementation
2018-11-25 16:58:38 -07:00
Sathish
f5b5c54d7d
Update condition for nosigverify ( #1903 )
2018-11-25 13:11:07 -08:00
Sathish
9f0b06bb86
Filter out leader node while retransmitting blobs ( #1894 )
2018-11-24 20:33:49 -08:00
carllin
57a384d6a0
Rocks db window service ( #1888 )
...
* Add db_window module for windowing functions from RocksDb
* Replace window with db_window functions in window_service
* Fix tests
* Make note of change in db_window
* Create RocksDb ledger in bin/fullnode
* Make db_ledger functions generic
* Add db_ledger to bin/replicator
2018-11-24 19:32:33 -08:00
Greg Fitzgerald
69802e141f
Add the story of how this codebase came to be
2018-11-24 14:39:53 -07:00
Greg Fitzgerald
6fc02b7424
Detect legacy programs upfront
2018-11-24 11:56:51 -07:00
Greg Fitzgerald
30cdd85028
Implement the same interface in all builtin programs
2018-11-24 11:56:51 -07:00
Greg Fitzgerald
871dd47019
Extract the part of execute_instruction that should only return a ProgramError
...
TODO: hoist load_executable_accounts() and then change
process_instruction() to return ProgramError.
2018-11-24 11:56:51 -07:00
Greg Fitzgerald
37f8dd57e2
Extract ProgramError from BankError
2018-11-24 11:56:51 -07:00
Greg Fitzgerald
f827bfd83f
Remove instruction index parameter
2018-11-24 11:56:51 -07:00
Greg Fitzgerald
b3af930153
Rename process_transaction to process_instruction
2018-11-24 11:56:51 -07:00
Greg Fitzgerald
cd488b7d07
Hoist program static methods to top-level functions
2018-11-24 11:56:51 -07:00
Rob Walker
e2373ff51a
add nosigverify command line option to ease debug
2018-11-23 16:55:04 -08:00
Greg Fitzgerald
b3d2c900cd
Rename BudgetState to BudgetProgram
2018-11-23 13:25:17 -07:00