Commit Graph

78 Commits

Author SHA1 Message Date
Tyera Eulberg f683817b48 Remove RPU; replace with RPC 2018-11-05 20:30:47 -07:00
carllin 298bd6479a
Add first leader to genesis (#1681)
* Add first leader to genesis entries, consume in genesis.sh

* Set bootstrap leader in the bank on startup, remove instantiation of bootstrap leader from bin/fullnode

* Remove need to initialize bootstrap leader in leader_scheduler, now can be read from genesis entries

* Add separate interface new_with_leader() in mint for creating genesis leader entries
2018-11-02 14:32:05 -07:00
Michael Vines e47fcb196b s/solana_program_interface/solana[_-]sdk/g 2018-10-25 12:31:45 -07:00
carllin 0bd1412562
Switch leader scheduler to use PoH ticks instead of Entry height (#1519)
* Add PoH height to process_ledger()

* Moved broadcast_stage Leader Scheduling logic to use Poh height instead of entry_height

* Moved LeaderScheduler logic to PoH in ReplicateStage

* Fix Leader scheduling tests to use PoH instead of entry height

* Change is_leader detection in repair() to use PoH instead of entry height

* Add tests to LeaderScheduler for new functionality

* fix Entry::new and genesis block PoH counts

* Moved LeaderScheduler to PoH ticks

* Cleanup to resolve PR comments
2018-10-18 22:57:48 -07:00
carllin 9931ac9780
Leader scheduler plumbing (#1440)
* Added LeaderScheduler module and tests

* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage

* Add LeaderScheduler plumbing for Tvu, window, and tests

* Fix bank and switch tests to use new LeaderScheduler

* move leader rotation check from window service to replicate stage

* Add replicate_stage leader rotation exit test

* removed leader scheduler from the window service and associated modules/tests

* Corrected is_leader calculation in repair() function in window.rs

* Integrate LeaderScheduler with write_stage for leader to validator transitions

* Integrated LeaderScheduler with BroadcastStage

* Removed gossip leader rotation from crdt

* Add multi validator, leader test

* Comments and cleanup

* Remove unneeded checks from broadcast stage

* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role

* Set new leader in validator -> validator transitions

* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail

* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops

* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
2018-10-10 16:49:41 -07:00
Pankaj Garg e10574c64d Remove recycler and it's usage
- The memory usage due to recycler was high, and incrementing with
  time.
2018-09-27 10:42:37 -06:00
jackcmay 9c47e022dc
break dependency of programs on solana core (#1371)
* break dependency of programs on Solana core
2018-09-27 07:49:26 -07:00
Greg Fitzgerald b7ae5b712a Move Pubkey into its own module 2018-09-26 20:40:40 -06:00
carllin e7383a7e66
Validator to leader (#1303)
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode

* Added logic to shutdown Tvu once ReplicateStage finishes

* Added test for successfully shutting down validator and starting up leader

* Add test for leader validator interaction

* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning

* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
2018-09-25 15:41:29 -07:00
Rob Walker 9e3d2956d8 remove last recycle? 2018-09-24 08:09:41 -06:00
Greg Fitzgerald 6073cd57fa Boot Recycler::recycle() 2018-09-20 17:08:51 -06:00
Anatoly Yakovenko 431692d9d0 Use a Drop trait to keep track of lifetimes for recycled objects.
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
2018-09-19 16:59:42 -06:00
anatoly yakovenko 6ec0e42220
budget as separate contract and system call contract (#1189)
* budget and system contracts and verification

* contract check_id methods
* system call contract
* verify contract execution rules
* move system into its own file
* allocate before transfer for budget
* store error in budget context
* budget contract and tests without bank
* moved budget of of bank
2018-09-17 13:36:31 -07:00
sakridge 4dc30ea104
Add recycler stats (#1187) 2018-09-13 14:49:48 -07:00
Pankaj Garg 90df6237c6
Implements recvmmsg() for UDP packets (#1161)
* Implemented recvmmsg() for UDP packets

- This change implements binding between libc API for recvmmsg()
- The function can receive multiple packets using one system call

Fixes #1141

* Added unit tests for recvmmsg()

* Added recv_mmsg() wrapper for non Linux OS

* Address review comments for recvmmsg()

* Remove unnecessary imports

* Moved target specific dependencies to the function
2018-09-13 14:41:28 -07:00
sakridge 1a4cd763f8
Fix missing recycle in recv_from (#1205)
In the error case that i>0 (we have blobs to send)
we break out of the loop and do not push the allocated r
to the v array. We should recycle this blob, otherwise it
will be dropped.
2018-09-13 08:29:18 -07:00
Rob Walker 9ab5692acf
fix "leak" in Blob::recv_from (#1198)
* fix "leak" in Blob::recv_from

fixes #1199
2018-09-12 14:45:43 -07:00
sakridge 297f859631
Change '>=' back to '>' to fix recycling of blobs/packets (#1192)
Recycler will have a strong ref to the item so it will be at
least 1, >= will always prevent recycling.
2018-09-11 16:52:45 -07:00
anatoly yakovenko c34d911eaf
Migrate Budget DSL to use the Account state (#979)
* Migrate Budget DSL to use the Account state instead of global bank data structures.

* Serialize Instruction into Transaction::userdata.
* Store the pending set in the Account::userdata
* Enforce the token balance rules on contract execution. This becomes the entry point for generic contracts.
* This pr will have a performance impact on the bank. The next set of changes will fix this by locking each account during multi threaded execution of all the contracts.
* With this change a contract transaction needs to store its state under an address. That address could be the destination of the tokens, or any random address. For the latter, an extra step would be needed to claim the tokens which isn't implemented by budget_dsl at the moment.
* test tracking issue 1157
2018-09-07 20:18:36 -07:00
Rob Walker 226d3b9471
Trace recycle() calls (#968)
* trace recycle() calls fixes #810
2018-09-05 05:07:02 +09:00
Greg Fitzgerald 8cc030ef84 Use Vec instead of VecDeque for SharedBlobs 2018-09-04 07:50:23 -10:00
Rob Walker 63e44dcc35 continue rendezvous refactor for gossip and repair
* remove trailing whitespace in ci/audit.sh

  * code review fixups
     * rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
     * remove out-of-date TODO in localnet-sanity.sh

  * remove features=test and code that was using it (localhost prohibitions in
      crdt) added TODO in crdt.rs, maybe we should boot localhost in production
      networks?

  * boot tvu_window from NodeInfo: instead, send repair requests from the repair
      socket (to gossip on peer) and answer repair requests via the sockaddr
      from the repair request

  * remove various unused pub functions

  * banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
2018-08-31 23:21:07 +09:00
Greg Fitzgerald 90ae662e4d Fix packet header offset
And update transaction offsets to use the same approach as packet.rs.
Maybe this should be serialized_size(), but thanks to this
GenericArray update, those values are the same.
2018-08-26 14:27:19 -06:00
sakridge cabe80b129
Increment counter by number of packets received (#1049)
So that we can see the total packets/s
2018-08-23 12:32:50 -07:00
Rob Walker bc5f29150b fix erasure, remove Entry "pad"
* fixes #997
 * Entry pad is no longer required since erasure coding aligns data length
2018-08-18 02:22:52 -07:00
Rob Walker 46016b8c7e crashes generate_coding() 2018-08-18 02:22:52 -07:00
Rob Walker 5dbecd6b6b add logging, more conservative reset 2018-08-18 02:22:52 -07:00
Rob Walker 5f6cbe0cf8 fixups 2018-08-13 21:07:26 -07:00
Rob Walker f167b0c2c5 fixups 2018-08-13 21:07:26 -07:00
Rob Walker f784500fbb fixups
fixes #907
2018-08-13 21:07:26 -07:00
Rob Walker 83df47323a initialize recycled data 2018-08-13 21:07:26 -07:00
Greg Fitzgerald ad331e6d56 Rename PublicKey type to Pubkey
Recognize pubkey as a noun meaning the public key of a keypair.
2018-08-09 13:41:37 -06:00
Stephen Akridge 38be61bd22 Check for log level before doing perf counter work
Perf counters, especially when running the dynamic test can cause
functions like crdt::apply_updates to be really slow (>500ms).
2018-08-08 00:16:53 -07:00
Stephen Akridge a200cedb4b Lower UDP data size to 64k - 128 bytes
rust API gives errors for packets larger than ~65500 and
wikipedia says 65507 is the max size, lowering this avoids the errors.
2018-08-07 18:39:36 -07:00
Rob Walker f2b1a04bca cargo fmt fixups 2018-08-03 11:59:25 -07:00
Rob Walker dd819cec3d fix off by one in packet.rs 2018-07-26 09:24:44 -07:00
Stephen Akridge 212874e155 Use BlobError for get_size return 2018-07-25 15:54:04 -07:00
Rob Walker 9c7913ac9e trying to raise an error 2018-07-25 08:12:20 -07:00
Rob Walker 8b01e6ac0b implement Blob::get_size(), the counterpart of Blob::set_size() 2018-07-25 08:12:20 -07:00
Rob Walker 3d80926508 fixups 2018-07-24 13:04:34 -07:00
Michael Vines 5deb34e5bd Little more trace! logging 2018-07-18 12:54:50 -07:00
anatoly yakovenko d8c9655128
Dynamic test assert (#643)
* log responder error to warn

* log responder error to warn

* fixup!

* fixed assert

* fixed bad ports issue

* comments

* test for dummy address in Crdt::new instaad of NodeInfo::new

* return error if ContactInfo supplied to Crdt::new cannot be used to connect to network

* comments
2018-07-16 19:31:52 -07:00
anatoly yakovenko 09f2d273c5
less intrusive counters (#655)
* less intrusive counters

* fixed arg

* tests

* comments
2018-07-16 18:33:50 -07:00
Greg Fitzgerald 73ae3c3301 Apply most of clippy's feedback 2018-07-12 09:40:40 -06:00
Stephen Akridge bed5438831 Improved streamer debug messages
distinguish between threads
2018-07-11 18:26:16 +02:00
Anatoly Yakovenko 03a8a5ed55 only submit to influx when we log
test accumilated value logging

lots of counters

higher influx rate

fix counter name

replicate-transactions
2018-07-11 07:53:39 -06:00
Greg Fitzgerald 231040b93e Add tests 2018-06-28 12:28:43 -07:00
Greg Fitzgerald 7c74afc35a Relax recycler
Instead of asserting ref count is 1 before recycling, allow users
to recycle items early. If it turns out that was too early, and
allocate() wants to return it, then boot it and take a memory
allocation performance hit instead.
2018-06-28 12:28:43 -07:00
Greg Fitzgerald 4aedd3f1b6 Cleanup type aliases and imports 2018-06-27 15:06:18 -06:00
Greg Fitzgerald 1691060a22 Assert recycler is given last reference to data
This patch likely fixes the sporadic failures in the following tests:

```
test server::tests::validator_exit ... FAILED
test streamer::test::streamer_send_test ... FAILED
test thin_client::tests::test_bad_sig ... FAILED
test drone::tests::test_send_airdrop ... FAILED
test thin_client::tests::test_thin_client ... FAILED
```
2018-06-26 16:51:07 -06:00