Generate tick entry ids and only register ticks as the last_id expected by the bank. Since the bank is MT, the in-flight pipeline of transactions cannot be close to the end of the queue or there is a high possibility that a starved thread will encode an expired last_id into the ledger. The banking_stage therefore uses a shorter age limit for encoded last_ids then the validators.
Bench client doesn't send transactions that are older then 30 seconds.
* Added LeaderScheduler module and tests
* plumbing for LeaderScheduler in Fullnode + tests. Add vote processing for active set to ReplicateStage and WriteStage
* Add LeaderScheduler plumbing for Tvu, window, and tests
* Fix bank and switch tests to use new LeaderScheduler
* move leader rotation check from window service to replicate stage
* Add replicate_stage leader rotation exit test
* removed leader scheduler from the window service and associated modules/tests
* Corrected is_leader calculation in repair() function in window.rs
* Integrate LeaderScheduler with write_stage for leader to validator transitions
* Integrated LeaderScheduler with BroadcastStage
* Removed gossip leader rotation from crdt
* Add multi validator, leader test
* Comments and cleanup
* Remove unneeded checks from broadcast stage
* Fix case where a validator/leader need to immediately transition on startup after reading ledger and seeing they are not in the correct role
* Set new leader in validator -> validator transitions
* Clean up for PR comments, refactor LeaderScheduler from process_entry/process_ledger_tail
* Cleaned out LeaderScheduler options, implemented LeaderScheduler strategy that only picks the bootstrap leader to support existing tests, drone/airdrops
* Ignore test_full_leader_validator_network test due to bug where the next leader in line fails to get the last entry before rotation (b/c it hasn't started up yet). Added a test test_dropped_handoff_recovery go track this bug
* Add check in window_service to exit in checks for leader rotation, and propagate that service exit up to fullnode
* Added logic to shutdown Tvu once ReplicateStage finishes
* Added test for successfully shutting down validator and starting up leader
* Add test for leader validator interaction
* fix streamer to check for exit signal before checking socket again to prevent busy leaders from never returning
* PR comments - Rewrite make_consecutive_blobs() function, revert genesis function change
lastidnotfound step 2:
* move "record stage", aka poh_service into banking stage
* remove Entry.has_more, is incompatible with leader rotation
* rewrite entry_next_hash in terms of Poh
* simplify and unify transaction hashing (no embedded nulls)
* register_last_entry from banking stage, fixes#1171 (w00t!)
* new PoH doesn't generate empty ledger entries, so some fixes necessary in
multinode tests that rely on that (e.g. giving validators airdrops)
* make window repair less patient, if we've been waiting for an answer,
don't be shy about most recent blobs
* delete recorder and record stage
* make more verbost thin_client error reporting
* more tracing in window (sigh)
* Move recycler instances to the point of allocation
* sinks no longer need to call `recycle`
* Remove the recycler arguments from all the apis that no longer need them
* Use multiple sockets for receiving blobs on validators
- The blobs that are broadcasted by leader or retransmitted by peer
validators are received on replicate_port
- Using reuse_addr/reuse_port, multiple sockets can be opened for
the same port
- This allows the kernel to queue data to user space app on multiple
socket queues, preventing over-running one queue
- This helps with reducing packets dropped due to queue over-runs
Fixes#1224
* Fixed failing tests
* Migrate Budget DSL to use the Account state instead of global bank data structures.
* Serialize Instruction into Transaction::userdata.
* Store the pending set in the Account::userdata
* Enforce the token balance rules on contract execution. This becomes the entry point for generic contracts.
* This pr will have a performance impact on the bank. The next set of changes will fix this by locking each account during multi threaded execution of all the contracts.
* With this change a contract transaction needs to store its state under an address. That address could be the destination of the tokens, or any random address. For the latter, an extra step would be needed to claim the tokens which isn't implemented by budget_dsl at the moment.
* test tracking issue 1157
* rename NodeInfo field of Node from "data" to "info"
(touches a lot of files)
* update client to use gossip to find leader, a la drone
* rework multinode scripts
* move more stuff into rust
* added usage to all
* no more rsync unless you're a validator (TODO: whack that, too)
* fullnode doesn't bail if drone isn't up yet, just keeps trying
* drone doesn't bail if network isn't up yet, just keeps trying
* remove trailing whitespace in ci/audit.sh
* code review fixups
* rename GOSSIP_PORT_RANGE => SOLANA_PORT_RANGE
* remove out-of-date TODO in localnet-sanity.sh
* remove features=test and code that was using it (localhost prohibitions in
crdt) added TODO in crdt.rs, maybe we should boot localhost in production
networks?
* boot tvu_window from NodeInfo: instead, send repair requests from the repair
socket (to gossip on peer) and answer repair requests via the sockaddr
from the repair request
* remove various unused pub functions
* banish SocketAddr parse().unwrap() to a macro that can also accept simpler stuff
* move gossip/NCP off assuming anything about its address
* use a single socket to send and receive gossip
* remove --addr/-a from CLIs
* rearrange networking utility code
* use Arc<UdpSocket> to share the Sync-safe UdpSocket among threads
* rename TestNode to Node
TODO:
* re-enable 127.0.0.1 as a valid address in crdt
* change repair request/response to a similar, single socket
* pick cloned sockets or Arc<UdpSocket> for all these (rpu uses tryclone())
* update contact_info with network truthiness instead of what the node
says?
comments
fixups!
fixups!
fixups for a real Result<> from get_balance()
on 2nd thought, be more rigorous
Merge branch 'rob-solana-accounts_with_state' into accounts_with_state
update
review comments
comments
get rid of option
- Some nodes don't have leader information while leader is broadcasting
blobs to those nodes. Such blobs are not retransmitted. This change
rertansmits the blobs once the leader's identity is know.
clear flags on fresh blobs, lest they sometimes impersonate coding blobs...
fix bug: advance *received whether the blob_index is in the window or not,
failure to do so results in a stalled repair request pipeline
Don't recover() for copy(), as copy() is already tolerant of things
recover() guards against. Note: recover() is problematic if the ledger is
"live", i.e. is currently being written to.
* add recover_ledger() to deal with expected common ledger corruptions
* add verify_ledger() for future use cases (ledger-tool)
* increase ledger testing
* allow replicate stage to run without a ledger
* ledger-tool to output valid json
* fixup!
* fixups!
* send the vote and count it
* actually vote
* test
* Spelling fixes
* Process the voting transaction in the leader's bank
* Send tokens to the leader
* Give leader tokens in more cases
* Test for write_stage::leader_vote
* Request airdrop inside fullnode and not the script
* Change readme to indicate that drone should be up before leader
And start drone before leader in snap scripts
* Rename _kp => _keypair for keypairs and other review fixups
* Remove empty else
* tweak test_leader_vote numbers to be closer to testing 2/3 boundary
* combine creating blob and transaction for leader/validator
- No sigverify if feature sigverify_cpu_disable is used
- Purge validators in the test if lag count increases beyond
SOLANA_DYNAMIC_NODES_PURGE_LAG environment variable
- Other useful log messages in the test
Clippy told us to change function parameters to references, but
wasn't able to then tell us that the clone() before borrowing
was superfluous. This patch removes those by hand.
No expectation of a performance improvement here, since we were
just cloning reference counts. Just removes a bunch of noise.
* log responder error to warn
* log responder error to warn
* fixup!
* fixed assert
* fixed bad ports issue
* comments
* test for dummy address in Crdt::new instaad of NodeInfo::new
* return error if ContactInfo supplied to Crdt::new cannot be used to connect to network
* comments
wip
voting
wip
move voting into the replicate stage
update
fixup!
fixup!
fixup!
fixup!
fixup!
fixup!
fixup!
fixup!
fixup!
fixup!
update
fixup!
fixup!
fixup!
tpu processing votes in entries before write stage
fixup!
fixup!
txs
make sure validators have an account
fixup!
fixup!
fixup!
exit fullnode correctly
exit on exit not err
try 50
add delay for voting
300
300
startup logs
par start
100
no rayon
retry longer
log leader drop
fix distance
50 nodes
100
handle deserialize error
update
fix broadcast
new table every time
tweaks
table
update
try shuffle table
skip kill
skip add
purge test
fixed tests
rebase 2
fixed tests
fixed rebase
cleanup
ok for blobs to be longer then window
fix init window
60 nodes
UPnP is now used to request a port on the NAT be forwarded to the local machine.
This obviously only works for NATs that support UPnP, and thus is not a panacea
for all NAT-related connectivity issues.
Notable hacks in this patch include a transmit/receive UDP socket pair to work
around current protocol limitations whereby the full node assumes its peer can
receive on the same UDP port it transmitted from.
We added them thinking it'd be a good stepping stone towards an
asynchronous thin client, but it's used inconsistently and where
it used, the function is still synchronous, which is just confusing.
Stephen Akridge
anatoly yakovenko
carllin
Michael Vines
Greg Fitzgerald
jackcmay
Pankaj Garg
jackcmay
Anatoly Yakovenko
Rob Walker
Carl
Tyera Eulberg
Stephen Akridge
Kazuyoshi Kato
Tyera Eulberg
Martius Lim
Greg Fitzgerald
anatoly yakovenko