0c90e1eff6
Make entry_sender optional on window_service
...
window_service in replicator has no need to consume the the produced entries.
2019-01-09 15:15:47 -08:00
491bca5e4b
Remove ledger.rs
...
Split into entry.rs for entry-constructing functions and EntrySlice
trait and db_ledger.rs for ledger helper test functions.
2019-01-09 15:15:47 -08:00
ebd676faaa
Rename Block to EntrySlice
2019-01-09 15:15:47 -08:00
045c5e8556
Remove most of the old ledger code
...
Removes LedgerWriter, read_ledger, LedgerWindow
2019-01-09 15:15:47 -08:00
45b4cf2887
Remove store_ledger_stage which is no longer needed
2019-01-09 15:15:47 -08:00
73eca72f14
Switch test to send a repair request to try and download from replicator
...
Removes need for read_ledger in the test and also tests replicator
download path.
2019-01-09 13:24:12 -08:00
94494b64d7
whack commented out, obsolete, superceded test
2019-01-09 11:30:07 -08:00
b648f37b97
encapsulate erasure_cf ( #2349 )
2019-01-09 10:21:55 -08:00
f53810fcd2
Remove unused exit variable
...
The exit variable was only used by a test.
2019-01-08 20:22:31 -08:00
3c6afe7707
Rename get_blob_bytes to read_blobs_bytes
2019-01-08 16:00:39 -08:00
09296e0d71
Fix two storage tests
...
* test_encrypt_files_many_keys_multiple_keys passing
- buffer chunk size unified between single key and multiple key path,
which shouldn't be necessary but can fix later.
* test_encrypt_file_many_keys_bad_key_length passing
2019-01-08 16:00:39 -08:00
4b3d64ec9f
Convert chacha_encrypt_file to work with db_ledger blobs directly
2019-01-08 16:00:39 -08:00
a904e15ecc
enscapsulate data_cf ( #2336 )
...
* enscapsulate data_cf
2019-01-08 15:53:44 -08:00
a82a5ae184
Delete unused code
...
The ignored test is still broken, but at least no longer creates a
window for no reason.
Also removed all remaining references to "ncp".
2019-01-08 14:09:50 -08:00
bafd90807d
encapsulate meta_cf ( #2335 )
2019-01-08 11:41:55 -08:00
0f8ea6872e
Add missing error counters and load_account test cases ( #2327 )
2019-01-08 09:20:25 -08:00
d2431128c7
Remove WriteStage from TPU/TVU diagrams
...
Fixes #2312
2019-01-08 08:42:06 -08:00
a8b9899dee
Add retry, restore ignored tests
2019-01-07 19:30:08 -08:00
d2cb4e003c
Re-enable the --lib tests
2019-01-07 15:28:20 -08:00
6000df9779
Optimize has_duplicates() for short slices
2019-01-07 13:20:04 -07:00
24963e547c
with_subset() -> get_subset_unchecked_mut()
...
A simpler, safer, and better documented use of unsafe code
2019-01-07 13:20:04 -07:00
46d44ca99c
Add make_rpc_request retry mechanism
2019-01-07 11:02:35 -08:00
0c52df7569
Consolidate locks and error handling when loading accounts( #2309 )
2019-01-06 22:06:55 -08:00
91bd38504e
Use vote signer service in fullnode ( #2009 )
...
* Use vote signer service in fullnode
* Use native types for signature and pubkey, and address other review comments
* Start local vote signer if a remote service address is not provided
* Rebased to master
* Fixes after rebase
2019-01-05 12:57:52 -08:00
b7dc9dbc76
RPC API now assumes a drone running on the bootstrap leader
2019-01-04 18:45:55 -08:00
8b357dcb32
cargo fmt
2019-01-04 16:39:04 -08:00
1f6346d880
De-dup ledgers - db_ledger is now the only ledger written to disk
2019-01-04 16:37:00 -08:00
b7bd38744c
Spelling and formatting
2019-01-04 16:04:31 -08:00
f8a67e282a
Ignore test_tpu_forwarder ( #2307 )
2019-01-04 16:02:50 -08:00
0505d7bd32
Don't double-clone every account
2019-01-03 17:42:37 -07:00
00d310f86d
Remove some metrics datapoint, as it was causing excessive logging ( #2287 )
...
- 100 nodes test was bringing down the influx DB server
2019-01-03 09:25:11 -08:00
0bea870b22
Dynamic N layer 'avalanche' broadcast and retransmit ( #2058 )
...
* Dynamic N layer avalanche broadcast and retransmit
2019-01-02 14:16:15 +05:30
58f2598d5d
Revert "Validators make a transaction to advertise their storage last_id"
...
This reverts commit a1759aed19
2018-12-23 14:02:09 -08:00
58a4905916
Make reconstruct_entries_from_blobs() support Blobs and borrowed SharedBlobs, make distinction between to_blobs and to_shared_blobs ( #2270 )
2018-12-22 19:30:30 -08:00
2c9607d5da
Rename getConfirmation -> getConfirmationTime
2018-12-22 12:47:02 -08:00
a1759aed19
Validators make a transaction to advertise their storage last_id
...
* Also implement more storage contract logic
* Add transactions for proof validation,
* Move storage state members into system storage account userdata
2018-12-21 15:45:30 -08:00
1a3387706d
Spawn threads based on cpu count ( #2232 )
2018-12-21 13:55:45 -08:00
951d6398a0
Rename finality to confirmation ( #2250 )
...
* Rename finality to confirmation
* fix cargo fmt errors
2018-12-20 15:47:48 -08:00
7c98545b33
Use newer votes to calculate confirmation time ( #2247 )
2018-12-20 15:27:47 -08:00
bb1060bdad
Reduce ticks per block to increase voting frequency ( #2242 )
2018-12-20 14:43:03 -08:00
7ad45a91ec
Fix compile error
2018-12-20 13:47:36 -08:00
034c5d0422
db_ledger now fully encapsulates rocksdb
2018-12-20 12:32:25 -08:00
7148c14178
Debug broadcast ( #2233 )
...
* Account for duplicate blobs in process_blobs
* Increase max bytes for level base to match write buffer
2018-12-20 12:12:04 -08:00
93fb61dc8f
Re-export rocksdb::DBRawIterator until it can be encapsulated
2018-12-20 10:38:03 -08:00
b36ceb5be4
Remove rocksdb dependency from result.rs
2018-12-20 10:38:03 -08:00
37d7ad819b
Purge DB::destroy() usage
2018-12-20 10:38:03 -08:00
666af1e62d
Debug broadcast ( #2208 )
...
* Add per cf rocksdb options, increase compaction and flush threads
* Change broadcast stage to bulk write blobs
* add db_ledger function specifically for broadcast
* fix broken tests
* fix benches
2018-12-19 16:11:47 -08:00
b4f4347d6e
add some more tests ( #2217 )
2018-12-18 17:27:03 -08:00
5c866dd000
test drive new coverage stuff ( #2216 )
2018-12-18 16:44:27 -08:00
974249f2a5
Parallelize entry processing in replay stage in validators ( #2212 )
...
* Parallelize entry processing in replay stage in validators
- single threaded entry processing is not utlizing CPU cores to the fullest
* fix tests and address review comments
2018-12-18 16:06:05 -08:00
a65022aed7
DbLedger doesn't need to be mut, doesn't need an RwLock ( #2215 )
...
* DbLedger doesn't need to be mut, doesn't need an RwLock
* fix erasure cases
2018-12-18 15:18:57 -08:00
ffdc1814c6
Add counters for gossip verification failures ( #2094 )
2018-12-17 20:12:50 -08:00
9720ac0019
Fix try_erasure() ( #2185 )
...
* Fix try_erasure bug
* Re-enable asserts in test_replicator_startup
* Add test for out of order process_blobs
2018-12-17 15:34:19 -08:00
9a6e27ac36
Accounts is to big, should be its own module ( #2198 )
...
Account module is to big, should be in its own module.
2018-12-17 12:41:23 -08:00
07202205c4
Revert "ignore unstable tests"
...
This reverts commit bd7ef5d445071329a3b49b1f8be71b602226bbec.
2018-12-17 10:47:32 -08:00
b7144560c9
Include port number when gossip bind_to fails
2018-12-17 09:31:31 -08:00
4be6d01dfb
Move last ids ( #2187 )
...
* Break out last_ids into its own module
* Boot SignatureNotFound from BankError
* No longer return BankError from LastIds methods
* No longer piggypack on BankError for a LastIds signature status
* Drop all dependencies on the bank
* SignatureStatus -> Status and LastIds -> StatusDeque
* Unstable tests, issue 2193
2018-12-17 07:55:56 -08:00
fd562cb9e2
Rust 2018 cleanup
2018-12-14 21:57:15 -08:00
5bf9a20d42
fullnode-config no longer depends on src/
2018-12-14 20:13:34 -08:00
eedc8c7812
Move src/netutil.rs into its own crate
2018-12-14 20:13:34 -08:00
9ef5e51c0f
Cleanup slot remnants in db_ledger ( #2153 )
...
* Cleanup slot remnants in db_ledger
2018-12-14 17:05:41 -08:00
8ee0e9632c
Switch to using hashbrown version of HashMap and ( #2158 )
...
HashSet for improved performance and memory usage
2018-12-14 15:10:10 -08:00
6ac466c0a4
Move src/logger.rs into logger/ crate to unify logging across the workspace
2018-12-14 13:10:43 -08:00
d45fcc4381
Move src/wallet.rs into wallet/ crate
2018-12-14 12:15:18 -08:00
6fcccedb70
align tick entries' tick_height with actual number of ticks in bank ( #2147 )
2018-12-14 02:25:50 -08:00
0fe6d61036
Move binaries from src/bin into their own crate
2018-12-13 22:25:27 -08:00
091b21fae7
Vote every number of ticks ( #2141 )
...
* Vote every number of ticks
* address review comments
* fix for failing leader rotation tests
* remove check for vote failure from replay tests
(as votes will be cached and transmitted when leader is available)
2018-12-13 18:43:10 -08:00
7325b19aef
Do not allocate for each metrics submission ( #2146 )
2018-12-13 16:40:00 -08:00
7cdbbfa88e
Storage stage updates
...
* Remove logging init from storage program: saw a crash in a test
indicating the logger being init'ed twice.
* Add entry_height mining proof to indicate which segment the result is
for
* Add an interface to get storage miner pubkeys for a given entry_height
* Add an interface to get the current storage mining entry_height
* Set the tvu socket to 0.0.0.0:0 in replicator to stop getting entries
after the desired ledger segment is downloaded.
* Use signature of PoH height to determine which block to download for
replicator.
2018-12-13 11:30:12 -08:00
9880a86f80
remove prev_id, unused ( #2150 )
2018-12-13 09:24:38 -08:00
647e5d76b0
Move solana-fullnode into fullnode/
2018-12-13 01:45:29 -08:00
7e4af9382e
Move solana-upload-perf into upload-perf/
2018-12-13 01:06:40 -08:00
282d4a3563
Move solana-keygen into keygen/
2018-12-13 01:06:40 -08:00
cafeef33c3
Relocate all keypair generation into one location: sdk/src/signature.rs
2018-12-13 01:06:40 -08:00
4f48f1a850
add db_ledger genesis, rework to_blob(), to_blobs() ( #2135 )
2018-12-12 20:42:12 -08:00
a05a378db4
cleanup
2018-12-12 19:12:51 -08:00
245362db96
Make a dummy version of serving repairs from db_ledger
2018-12-12 19:12:51 -08:00
b1b190b80d
Fix too many args in Tvu::new ( #2114 )
...
* Reduce args in Tvu::new under to 8
Now pass in sockets through a the crate::tvu::Sockets struct
Move ClusterInfo.keypair to pub(crate) in order to remove redundant
signing keypair parameter
* remove commented code
2018-12-12 18:57:48 -08:00
3408ce89a7
add check_tick_height ( #2144 )
2018-12-12 18:52:11 -08:00
75f407e191
Provide entire elf to bpf_loader
2018-12-12 17:14:50 -08:00
9b81696a09
remove obsoleted TODO
2018-12-12 16:26:59 -08:00
80e19e0ad7
Encapsulate accounts of solana: 🏦 :Accounts
...
Make the field private and expose an account_values() method that
returns the values iterator from the internal hashmap
2018-12-12 16:26:59 -08:00
ae903f190e
Broadcast for slots ( #2081 )
...
* Insert blobs into db_ledger in broadcast stage to support leader to validator transitions
* Add transmitting real slots to broadcast stage
* Handle real slots instead of default slots in window
* Switch to dummy repair on slots and modify erasure to support leader rotation
* Shorten length of holding locks
* Remove logger from replicator test
2018-12-12 15:58:29 -08:00
3413ecc2bd
Change query used to find list of nodes in the network ( #2124 )
...
* Change query used to find list of nodes in the network
* include "All" option for host selection
2018-12-12 12:38:00 -08:00
4788a4f775
Correctly describe repair and retransmit peers ( #2110 )
2018-12-11 15:51:47 -08:00
9243bc58db
Metrics for window repair ( #2106 )
...
* Metrics for window repair
- Also increase max repair length
* fix vote counters, and add repair window graph
* update per node graphs
* revert max repair length change
2018-12-11 15:43:41 -08:00
4d67aca919
add genesis and read_ledger to db_ledger ( #2097 )
2018-12-11 09:14:23 -08:00
aeee25e703
add tick_height to Entry to be able to repair by period, chain forks of Entries, etc. ( #2096 )
2018-12-10 20:03:04 -08:00
01c524ddd2
Revert changes to counter names
2018-12-10 15:26:43 -07:00
5e703dc70a
Free up the term 'replicate' for exclusive use in replicator
...
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
bc96bd3410
Fix peer count in edge dashboard ( #2090 )
...
Fixes #2075
2018-12-10 14:24:32 -08:00
4ea1c030bc
Give bootstrap leader one more token
2018-12-10 13:24:14 -08:00
172e511e56
Use retry_transfer to test multiple times for replicator tokens
...
May fix failures in CI where replicator is trying to do an airdrop.
2018-12-10 12:19:00 -08:00
4481efd51e
Merge pull request #2084 from CriesofCarrots/fix-wallet-accountinuse
...
Fix wallet accountinuse
2018-12-10 12:20:55 -07:00
0ef099421c
cargo fmt
2018-12-08 23:19:55 -07:00
f1ae5b1795
Fix warnings
2018-12-08 23:19:55 -07:00
a8d6c75a24
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms
2018-12-08 23:19:55 -07:00
c49e2f8bbd
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition
2018-12-08 23:19:55 -07:00
ec5a8141eb
cargo fix --edition
2018-12-08 23:19:55 -07:00
2de45a4da5
Update airdrop tokens to 3 for fullnode ( #2051 )
...
Filter out leader while computing the super majority stake
2018-12-08 16:54:42 -08:00
73b9ee9e84
Add solana_ prefix to native_loader program
...
This allows its logging to show up in the default RUST_LOG=solana=info
log setting
2018-12-08 11:04:45 -08:00
0a83b17cdd
Upgrade to Rust 1.31.0 ( #2052 )
...
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
571522e738
Update jsonrpc version
2018-12-07 17:47:54 -07:00
3441d3399b
Replicator rework
...
* Move more of the replicator logic into the replicator class
* Add support for the RPC interface to query the storage last_id value
that the replicator would sign and use to pick a block.
* Fix replicator connecting to gossip and change test to exercise that
scenario.
2018-12-07 15:20:36 -08:00
f4b26247c0
Genesis only needs a keypair, not the entire fullnode::Config
2018-12-06 16:31:24 -08:00
c4b8f0cd2f
bench-tps will now generate an ephemeral identity if not provided with one
...
Also simplify scripts as a result
2018-12-06 16:30:48 -08:00
97b1156a7a
Rename Ncp to GossipService
...
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
cc2f448d92
Add fullnode --no-leader-rotation flag
2018-12-06 11:30:19 -08:00
632425c7d7
Move native_loader under programs/native/
2018-12-05 14:32:42 -08:00
ad3e36a7ab
Bump rand from 0.5.5 to 0.6.1 ( #1891 )
...
* Bump rand from 0.5.5 to 0.6.1
Bumps [rand](https://github.com/rust-random/rand ) from 0.5.5 to 0.6.1.
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/commits )
Signed-off-by: dependabot[bot] <support@dependabot.com>
* Fix conflicts and deprecated usages
* Fix benches
2018-12-05 14:12:10 -08:00
1bcafca690
Find test_tx again
2018-12-05 13:29:29 -08:00
e21b6d9db3
ensure we'd actually have N hashes per tick ( #2011 )
2018-12-05 12:49:41 -08:00
9c30bddb88
Rocks db erasure decoding ( #1900 )
...
* Change erasure to consume new RocksDb window
* Change tests for erasure
* Remove erasure from window
* Integrate erasure decoding back into window
* Remove corrupted blobs from ledger
* Replace Erasure result with result module's Result
2018-12-05 12:47:19 -08:00
59e6bd115e
system_program must be a static lib as it allocates Account memory
2018-12-05 10:49:06 -08:00
15aef079e3
Include builtin programs for ledger verification
2018-12-05 10:49:06 -08:00
42689d4842
cargo fmt
2018-12-05 10:49:06 -08:00
214ed3667c
Move system_transaction out of src/
2018-12-05 10:49:06 -08:00
122627dda2
Move loader_transaction out of src/
2018-12-05 10:49:06 -08:00
7af95eadcc
Move vote_transaction out of src/
2018-12-05 10:49:06 -08:00
9ee858a00c
Move budget_program out of src/
2018-12-05 10:49:06 -08:00
27d456bf93
Move storage_program out of src/
2018-12-05 10:49:06 -08:00
ea6e042a6f
Move vote_program out of src/
2018-12-05 10:49:06 -08:00
a594f56c02
Add token_program.rs to sdk/
2018-12-05 10:49:06 -08:00
e6fa74fe69
Remove custom Error enum, just use ProgramError
2018-12-05 10:49:06 -08:00
9a4f8199d6
Move system_program out of src/
2018-12-05 10:49:06 -08:00
ae0be1e857
Remove bpf_loader.rs
2018-12-05 10:49:06 -08:00
d010cac8a5
Remove token_program.rs
2018-12-05 10:49:06 -08:00
63a758508a
Add sdk native_loader.rs
2018-12-05 10:49:06 -08:00
c1c2f1f0a9
Cleanup ad-hoc rpc address formation
...
Lots of places where we are forming rpc addresses.
2018-12-03 18:13:55 -08:00
777a0a858e
Move ProgramError into sdk/
2018-12-03 13:50:00 -08:00
68e99c18c0
Remove duplicate SYSTEM_PROGRAM_ID
2018-12-03 13:50:00 -08:00
c99f93e40a
Remove signature.rs indirection
2018-12-03 13:50:00 -08:00
4ae58cc854
Change range of leader scheduler to match current broadcasts ( #1920 )
2018-12-03 00:10:43 -08:00
b21facab7b
Add metrics for prune messages ( #1981 )
2018-12-01 14:05:40 -08:00
34c3a0cc1f
Add signature verification to gossip ( #1937 )
2018-12-01 12:00:30 -08:00
549bfe7412
Vote signing JSON RPC service ( #1965 )
...
* Vote signing JSON RPC service
- barebone service that listens for RPC requests
* Daemon for vote signer service
* Add request APIs for JSON RPC
* Cleanup of cargo dependencies
* Fix compiler error
2018-11-30 15:07:08 -08:00
3ca826a480
re-enable test_tpu_forwarder ( #1964 )
2018-11-30 13:52:37 -08:00
bad0b55ab6
Expose which keys signed the Transaction in the SDK
2018-11-30 08:16:23 -08:00
0878bd53d9
Delete stub src/transaction.rs
2018-11-29 23:07:57 -08:00
de910e1169
Make test_pubkey_distribution faster
...
multi-thread pubkey histogram generation.
2018-11-29 17:37:37 -08:00
9684737de7
Add wait before checking confirm again
...
Otherwise we can quickly check that we
have no signature 4 times in a row.
2018-11-29 15:32:58 -08:00
ecc87ab1aa
Add a an optional timeout to thin_client
...
Such that a negative test like test_transaction_count doesn't
have to wait num_retries * default_timeout.
2018-11-29 13:53:40 -08:00
fa359c6fc4
Merge vote new and register transactions
2018-11-29 12:31:34 -08:00
8cc751d1cc
Improve RPC service startup error messages with actual error
...
Error always fixed to message about ports but that's not the only
error that can occur.
2018-11-29 11:39:42 -08:00
978fd6858f
Move replicator_startup_test to integration test set
...
Sometimes fails when run multithreaded with other tests.
2018-11-29 11:39:42 -08:00
41689256c6
Ensure key[0] is signed
2018-11-29 10:26:46 -08:00
194e3100a9
Additional checks in test_bank_checkpoint_zero_balance ( #1943 )
2018-11-28 12:40:34 -08:00
8ad4464d4b
add tests for other "from" indexes signing (or not)
2018-11-28 07:56:04 -08:00
e7b0a736f5
verify signature is on the from account
2018-11-28 07:56:04 -08:00
167eb01735
optimize bench-tps and rpc_request to work on crappy WSL boxes
2018-11-27 22:45:08 -08:00
Stephen Akridge
Rob Walker
Greg Fitzgerald
jackcmay
Michael Vines
Pankaj Garg
Sagar Dhawan
carllin
Sathish
anatoly yakovenko
Carl
Mark
Mekagoza
Tyera Eulberg
Tyera Eulberg
dependabot[bot]