Michael Vines
f90d96367d
Add Fullnode::run() to optionally manage node role transitions automatically
2019-02-06 14:23:10 -08:00
carllin
3feda8a315
ReplayStage asking ledger for updates ( #2597 )
...
* Modify replay stage to ask db_ledger for updates instead of reading from upstream channel
* Add signal for db_ledger to update listeners about updates
* fix flaky test
2019-02-04 15:33:43 -08:00
Sagar Dhawan
9d34ded5f3
Update and fix test_broadcast_last_tick ( #2644 )
2019-02-01 17:13:15 -08:00
Sagar Dhawan
2985988f0d
Re-enable test_broadcast_last_tick ( #2639 )
2019-02-01 14:23:20 -08:00
Michael Vines
d62c9ac309
Create program/ crate avoid / crate dependency on bpfloader
...
The bpfloader crate was triggering cargo to perform excessive rebuilds
of in-workspace dependencies. Unclear why exactly, but seems related to
the special dual crate-type employed by bpfloader.
2019-02-01 12:42:46 -08:00
Michael Vines
40acaee446
Remove unnecessary abstractions and helper functions
2019-02-01 12:42:46 -08:00
Greg Fitzgerald
dad0bfe447
Replace transaction traits with structs
...
Also:
* SystemTransaction::new -> new_account
* SystemTransaction::new_create -> new_program_account
2019-02-01 11:38:25 -08:00
Greg Fitzgerald
82a2080e45
Rename VoteSignerProxy to VotingKeypair
...
Works just like a normal Keypair, but will only sign voting
transactions.
2019-02-01 07:11:17 -07:00
Michael Vines
d87c2eb903
Fullnode::new() - provide FullnodeConfig as a ref
2019-01-31 21:12:36 -08:00
carllin
37003da854
Fix potential of checking tvu bank for truth when its behind ( #2614 )
...
* Fix race between tpu and tvu, where tvu bank is not caught up to tpu bank
* Add test
* Cleanup Fullnode tests
2019-01-31 19:21:02 -08:00
Michael Vines
9767468b7f
Remove unneeded Option
2019-01-31 13:53:59 -08:00
carllin
84567d36cf
Leader scheduler groundwork for Blocktree ( #2599 )
...
* Groundwork for entry tree, align constants with definitions in the book
* Fix edge case in test, node can keep generating ticks between handle_role_transition and exit() call
2019-01-31 13:44:24 -08:00
Pankaj Garg
32162ef0f1
Connect TPU's broadcast service with TVU's blob fetch stage ( #2587 )
...
* Connect TPU's broadcast service with TVU's blob fetch stage
- This is needed since ledger is being written only in TVU now
* fix clippy warnings
* fix failing test
* fix broken tests
* fixed failing tests
2019-01-31 13:43:22 -08:00
anatoly yakovenko
2754ceec60
StatusDeque split into separate objects with their own root checkpoint strategy ( #2613 )
...
Split up StatusDeque into different modules
* LastIdQueue tracks last_ids
* StatusCache keeps track of signature statuses
* StatusCache stores success as a bit in a bloom filter
* Overhead for 1m Ok transactions is 4mb in memory
* Less concurrency between the objects, last_id and status_cache are read and written to at different points in the pipeline
* Each object has its own strategy for merging into the root checkpoint
2019-01-31 06:53:52 -08:00
Greg Fitzgerald
609e915169
Fix clippy warning
...
Always pass Arcs by reference. Then you'll only need to clone()
to cross thread boundaries.
2019-01-30 21:59:05 -07:00
Greg Fitzgerald
e25992a011
Always give Fullnode a vote signer
...
This will allow us to use the the signer's pubkey as the node id.
Disable voting with a configuration option.
2019-01-30 21:59:05 -07:00
Rob Walker
1b50fbbc90
remove Result<> from Blob accessors, add parent ( #2608 )
...
* remove Result<> from Blob accessors, add parent
* update chacha's golden
* fixup benches
2019-01-30 20:18:28 -08:00
Michael Vines
c01290438f
Move virtual genesis tick into the ledger proper as entry 0
2019-01-30 14:02:07 -08:00
Michael Vines
9e9c82869a
create_tmp_sample_ledger() need not return the genesis block
2019-01-30 14:02:07 -08:00
Michael Vines
494b143453
Delete create_tmp_genesis
2019-01-30 14:02:07 -08:00
Michael Vines
8cc1cde0fe
create_tmp_sample_ledger() now returns entry_height and last_id
2019-01-30 14:02:07 -08:00
Anatoly Yakovenko
a16e41002e
reduce gossip nodes in concurrent tests for CI
2019-01-30 10:26:28 -07:00
Michael Vines
25f25d0f82
new_fullnode: don't return the genesis_block, nobody uses it
2019-01-29 17:51:07 -08:00
Michael Vines
ae7f169027
Add FullnodeConfig struct to Fullnode::new* functions
...
This avoids having to touch *every* Fullnode::new* call site when
a new fullnode option is added
2019-01-29 09:42:48 -08:00
Tyera Eulberg
6da7a784f2
Stream entries ( #2582 )
...
* Add entry streaming option
* Fix tests
* Remove obsolete comment
* Move entry stream functionality to struct w/ trait in order to test without i/o
2019-01-29 00:21:27 -08:00
Michael Vines
12cddf725e
Harmonize Fullnode::new* function arguments
2019-01-28 22:37:56 -08:00
Michael Vines
d8861c2a5f
Wait until the leader shows up on gossip
2019-01-28 22:37:56 -08:00
Michael Vines
29ef9370a6
Remove LeaderSchedulerConfig options
2019-01-28 13:51:01 -08:00
Stephen Akridge
2262f279d5
Reduce boilerplate code with helper function to create
...
fullnode/bank/genesis
2019-01-28 13:48:58 -08:00
Greg Fitzgerald
1bae87d4b3
Add unit-test-friendly VoteSignerProxy constructor
2019-01-26 14:56:49 -07:00
Sagar Dhawan
d65e7b9fcc
Speedup rotation ( #2468 )
...
Speedup leader to validator transitions
2019-01-26 13:58:08 +05:30
Michael Vines
4bb6549895
Genesis block is now a json file
2019-01-25 09:05:15 -08:00
Stephen Akridge
9845aec007
Rename data_replicator tests module
...
replicator name is associated with storage replicators, so
data_replicator sounds like that but it is actually a bunch of gossip
tests.
2019-01-24 15:49:55 -08:00
Tyera Eulberg
b7be5b9a7a
Add no-signer argument
2019-01-24 10:47:37 -08:00
Greg Fitzgerald
73b10c196e
Disable integration test that fails in CI
2019-01-22 19:24:44 -08:00
Mark
f37eb533f1
Replicator timeout ( #2480 )
...
* Add timeout to Replicator::new; used when polling for leader
* Add timeout functionality to replicator ledger download
Shares the same timeout as polling for leader
Defaults to 30 seconds
* Add docs for Replicator::new
2019-01-21 15:37:41 -06:00
Tyera Eulberg
cb23070dfe
Remove sleeps on fullnode spin-up in integration tests
2019-01-21 13:27:31 -07:00
Greg Fitzgerald
5d9d83d312
Less clones
2019-01-21 12:56:27 -07:00
Greg Fitzgerald
823252dd41
Cleanup terminology
2019-01-21 12:56:27 -07:00
Tyera Eulberg
35764225ed
Remove socket from rpc test and move integration test
2019-01-21 12:29:04 -07:00
Stephen Akridge
1fd7bd7ede
Storage fixes
...
* replicators generate their sample values
* fixes to replicator block height logic
2019-01-18 13:05:35 -08:00
Michael Vines
6933f2bad1
Remove stale TODO
2019-01-17 23:22:07 -08:00
Michael Vines
b03d1d8894
Enable integration test logging for better debug on CI failure
2019-01-17 23:14:18 -08:00
Michael Vines
8e4a86e329
Recovery multinode tests
2019-01-17 23:14:18 -08:00
Tyera Eulberg
8af61f561b
Improve Wallet coverage ( #2385 )
...
* Add trait for RpcRequestHandler trait for RpcClient and add MockRpcClient for unit tests
* Add request_airdrop integration test
* Add timestamp_tx, witness_tx, and cancel_tx to wallet integration tests; add wallet integration tests to test-stable
* Add test cases
* Ignore plentiful sleeps in unit tests
2019-01-14 00:10:03 -07:00
Jak May
e9a0b3a8f3
Add BPF-to-BPF and PC relative call tests ( #2395 )
2019-01-11 19:33:08 -08:00
Jak May
23c43ed21b
Multi-file BPF C builds ( #2393 )
2019-01-11 15:33:21 -08:00
Pankaj Garg
d7594b19fc
Implemented a trait for vote signer service ( #2386 )
...
* Implemented a trait for vote signer service
* removes need for RPC in unit tests for vote signing
* fix build errors
* address some review comments
2019-01-11 12:58:31 -08:00
Pankaj Garg
2dbe8fc1a9
Refactor vote signer code ( #2368 )
...
* Refactor vote signer code
* fixed test compilation errors
* address clippy errors
* fix missing macro_use
* move macro use
* review comments
2019-01-10 09:21:38 -08:00
Stephen Akridge
491bca5e4b
Remove ledger.rs
...
Split into entry.rs for entry-constructing functions and EntrySlice
trait and db_ledger.rs for ledger helper test functions.
2019-01-09 15:15:47 -08:00
Stephen Akridge
73eca72f14
Switch test to send a repair request to try and download from replicator
...
Removes need for read_ledger in the test and also tests replicator
download path.
2019-01-09 13:24:12 -08:00
Michael Vines
2c52e82352
Use retry_make_rpc_request to avoid occasional CI test failures
2019-01-07 21:25:25 -08:00
Pankaj Garg
91bd38504e
Use vote signer service in fullnode ( #2009 )
...
* Use vote signer service in fullnode
* Use native types for signature and pubkey, and address other review comments
* Start local vote signer if a remote service address is not provided
* Rebased to master
* Fixes after rebase
2019-01-05 12:57:52 -08:00
Michael Vines
1f6346d880
De-dup ledgers - db_ledger is now the only ledger written to disk
2019-01-04 16:37:00 -08:00
jackcmay
a461c5682d
First stab at Rust BPF ( #2269 )
...
First stab at Rust BPF
2019-01-02 15:12:42 -08:00
Michael Vines
58f2598d5d
Revert "Validators make a transaction to advertise their storage last_id"
...
This reverts commit a1759aed19
.
2018-12-23 14:02:09 -08:00
carllin
58a4905916
Make reconstruct_entries_from_blobs() support Blobs and borrowed SharedBlobs, make distinction between to_blobs and to_shared_blobs ( #2270 )
2018-12-22 19:30:30 -08:00
Michael Vines
2c9607d5da
Rename getConfirmation -> getConfirmationTime
2018-12-22 12:47:02 -08:00
Stephen Akridge
a1759aed19
Validators make a transaction to advertise their storage last_id
...
* Also implement more storage contract logic
* Add transactions for proof validation,
* Move storage state members into system storage account userdata
2018-12-21 15:45:30 -08:00
Pankaj Garg
951d6398a0
Rename finality to confirmation ( #2250 )
...
* Rename finality to confirmation
* fix cargo fmt errors
2018-12-20 15:47:48 -08:00
Michael Vines
37d7ad819b
Purge DB::destroy() usage
2018-12-20 10:38:03 -08:00
carllin
9720ac0019
Fix try_erasure() ( #2185 )
...
* Fix try_erasure bug
* Re-enable asserts in test_replicator_startup
* Add test for out of order process_blobs
2018-12-17 15:34:19 -08:00
anatoly yakovenko
4be6d01dfb
Move last ids ( #2187 )
...
* Break out last_ids into its own module
* Boot SignatureNotFound from BankError
* No longer return BankError from LastIds methods
* No longer piggypack on BankError for a LastIds signature status
* Drop all dependencies on the bank
* SignatureStatus -> Status and LastIds -> StatusDeque
* Unstable tests, issue 2193
2018-12-17 07:55:56 -08:00
Michael Vines
6ac466c0a4
Move src/logger.rs into logger/ crate to unify logging across the workspace
2018-12-14 13:10:43 -08:00
anatoly yakovenko
a22e1199cf
Add fork selection RFC ( #2061 )
...
RFC and simulation for fork generation.
2018-12-14 11:15:23 -08:00
Stephen Akridge
85398c728a
Disable assert in replicator startup test
2018-12-13 16:50:30 -08:00
Stephen Akridge
7cdbbfa88e
Storage stage updates
...
* Remove logging init from storage program: saw a crash in a test
indicating the logger being init'ed twice.
* Add entry_height mining proof to indicate which segment the result is
for
* Add an interface to get storage miner pubkeys for a given entry_height
* Add an interface to get the current storage mining entry_height
* Set the tvu socket to 0.0.0.0:0 in replicator to stop getting entries
after the desired ledger segment is downloaded.
* Use signature of PoH height to determine which block to download for
replicator.
2018-12-13 11:30:12 -08:00
Carl
a05a378db4
cleanup
2018-12-12 19:12:51 -08:00
Carl
245362db96
Make a dummy version of serving repairs from db_ledger
2018-12-12 19:12:51 -08:00
carllin
ae903f190e
Broadcast for slots ( #2081 )
...
* Insert blobs into db_ledger in broadcast stage to support leader to validator transitions
* Add transmitting real slots to broadcast stage
* Handle real slots instead of default slots in window
* Switch to dummy repair on slots and modify erasure to support leader rotation
* Shorten length of holding locks
* Remove logger from replicator test
2018-12-12 15:58:29 -08:00
jackcmay
cefbb7c27d
Fix shared object relcations with multiple static arrays ( #2121 )
2018-12-12 08:41:45 -08:00
jackcmay
935524f20c
Fix eh frame relocation ( #2109 )
...
* Exclude .eh_frame
2018-12-11 12:14:41 -08:00
jackcmay
e3dfd7b1ab
Allow BPF structure passing and returning ( #2100 )
...
* Add BPF struct passing and returning tests
2018-12-11 09:03:37 -08:00
Greg Fitzgerald
5e703dc70a
Free up the term 'replicate' for exclusive use in replicator
...
Also, align Sockets field names with ContactInfo.
2018-12-10 15:26:43 -07:00
Stephen Akridge
172e511e56
Use retry_transfer to test multiple times for replicator tokens
...
May fix failures in CI where replicator is trying to do an airdrop.
2018-12-10 12:19:00 -08:00
Greg Fitzgerald
0ef099421c
cargo fmt
2018-12-08 23:19:55 -07:00
Greg Fitzgerald
a8d6c75a24
cargo +nightly fix --features=bpf_c,cuda,erasure,chacha --edition-idioms
2018-12-08 23:19:55 -07:00
Michael Vines
73b9ee9e84
Add solana_ prefix to native_loader program
...
This allows its logging to show up in the default RUST_LOG=solana=info
log setting
2018-12-08 11:04:45 -08:00
Greg Fitzgerald
0a83b17cdd
Upgrade to Rust 1.31.0 ( #2052 )
...
* Upgrade to Rust 1.31.0
* Upgrade nightly
* Fix all clippy warnings
* Revert relaxed version check and update
2018-12-07 20:01:28 -07:00
Stephen Akridge
3441d3399b
Replicator rework
...
* Move more of the replicator logic into the replicator class
* Add support for the RPC interface to query the storage last_id value
that the replicator would sign and use to pick a block.
* Fix replicator connecting to gossip and change test to exercise that
scenario.
2018-12-07 15:20:36 -08:00
Greg Fitzgerald
97b1156a7a
Rename Ncp to GossipService
...
And BroadcastStage to BroadcastService since it's not included in the
TPU pipeline.
2018-12-06 15:48:19 -07:00
Michael Vines
632425c7d7
Move native_loader under programs/native/
2018-12-05 14:32:42 -08:00
Michael Vines
214ed3667c
Move system_transaction out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
122627dda2
Move loader_transaction out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
ea6e042a6f
Move vote_program out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
9a4f8199d6
Move system_program out of src/
2018-12-05 10:49:06 -08:00
Michael Vines
ae0be1e857
Remove bpf_loader.rs
2018-12-05 10:49:06 -08:00
jackcmay
0cb5ae41c6
Enable BPF shared objects ( #2012 )
...
* Switch to BPF ELF shared objects (.so)
2018-12-04 22:03:32 -08:00
Michael Vines
c99f93e40a
Remove signature.rs indirection
2018-12-03 13:50:00 -08:00
Sagar Dhawan
34c3a0cc1f
Add signature verification to gossip ( #1937 )
2018-12-01 12:00:30 -08:00
Michael Vines
0878bd53d9
Delete stub src/transaction.rs
2018-11-29 23:07:57 -08:00
Stephen Akridge
978fd6858f
Move replicator_startup_test to integration test set
...
Sometimes fails when run multithreaded with other tests.
2018-11-29 11:39:42 -08:00
carllin
3f9ad1253d
Re-enable fixed tests ( #1907 )
2018-11-25 20:51:55 -08:00
Sathish
9f0b06bb86
Filter out leader node while retransmitting blobs ( #1894 )
2018-11-24 20:33:49 -08:00
carllin
57a384d6a0
Rocks db window service ( #1888 )
...
* Add db_window module for windowing functions from RocksDb
* Replace window with db_window functions in window_service
* Fix tests
* Make note of change in db_window
* Create RocksDb ledger in bin/fullnode
* Make db_ledger functions generic
* Add db_ledger to bin/replicator
2018-11-24 19:32:33 -08:00
carllin
942256a647
Add db_ledger benchmarks ( #1875 )
...
* Add db_ledger benchmarks
* ignore benches in CI, due to timeouts
2018-11-23 06:12:43 -08:00
Rob Walker
3d113611cc
remove Result<> return from ClusterInfo::new() ( #1869 )
...
strip Result<> for ClusterInfo::new()
2018-11-19 11:25:14 -08:00
Sagar Dhawan
07667771ef
Fix Gossip Pushes going to invalid addresses ( #1858 )
2018-11-17 19:57:28 -08:00
Michael Vines
d96a6b42a5
Move drone into its own crate
2018-11-16 20:42:21 -08:00
Michael Vines
6ac5700f2e
Move metrics into its own crate
2018-11-16 15:10:07 -08:00