Commit Graph

2712 Commits

Author SHA1 Message Date
Lijun Wang f14928a970
Stream additional block metadata via plugin (#22023)
* Stream additional block metadata through plugin
blockhash, block_height, block_time, rewards are streamed
2021-12-29 15:12:01 -08:00
Justin Starry b1d9a2e60e
Don't forward packets received from TPU forwards port (#22078)
* Don't forward packets received from TPU forwards port

* Add banking stage test
2021-12-29 19:34:31 +01:00
dependabot[bot] d20a3774db
Bump lru from 0.7.1 to 0.7.2 (#22161)
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.7.1 to 0.7.2.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.1...0.7.2)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-29 10:47:37 -07:00
carllin eaa8c67bde
Count compute units even when transaction errors (#22059) 2021-12-28 17:05:11 -05:00
carllin f061059e45
Prevent log spam (#22148) 2021-12-28 17:04:48 -05:00
Tao Zhu 3d6ab96587 push live packets straight to buffer, leader only process packets from buffer 2021-12-28 15:21:24 -06:00
Yueh-Hsuan Chiang b89cd8cd1a
Avoid cloning Vec<Entry> when calling entries_to_test_shreds() (#22093) 2021-12-24 12:32:43 -08:00
Justin Starry 93c776ce19
Refactor packet deduplication and harden bench test (#22080) 2021-12-22 23:05:10 -06:00
Tao Zhu dd80a525ef
Leader QoS service metrics (#21708)
* - qos_service metrics tagged with leader thread ids to separate gossip/tpu votes and transactions;
- qos_service metrics is reported with bank slot;
- replaced timer-based reporting with signal via channel; removed async report test as qos_service now lives within a thread

* - add tpu live packets (eg, not buffered packets) states to qos metrics reporting
2021-12-22 21:39:59 +00:00
behzad nouri f67ecd5c18
removes unused Cargo dependencies (#22022)
Also moving some to [dev-dependencies] so that they are not propagated
to other packages which depend on the package.
2021-12-22 17:01:33 +00:00
behzad nouri 4d62f03297
uses enum instead of trait for VoteTransaction (#22019)
Box<dyn Trait> involves runtime dispatch, has significant overhead and
is slow. It also requires hacky boilerplate code for implementing Clone
or other basic traits:
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L70-L102

Only limited known types can be VoteTransaction and they are all defined
in the same crate. So using a trait here only adds overhead.
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L125-L165
https://github.com/solana-labs/solana/blob/e92a81b74/programs/vote/src/vote_state/mod.rs#L221-L264
2021-12-22 14:25:46 +00:00
Tao Zhu 9c5d82557a skip reporting all-zero stats 2021-12-21 16:20:36 -06:00
dependabot[bot] 69d0b08dd8
chore: bump lru from 0.7.0 to 0.7.1 (#22018)
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.7.0...0.7.1)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-20 21:56:56 -07:00
behzad nouri 65d59f4ef0
tracks erasure coding shreds' indices explicitly (#21822)
The indices for erasure coding shreds are tied to data shreds:
https://github.com/solana-labs/solana/blob/90f41fd9b/ledger/src/shred.rs#L921

However with the upcoming changes to erasure schema, there will be more
erasure coding shreds than data shreds and we can no longer infer coding
shreds indices from data shreds.

The commit adds constructs to track coding shreds indices explicitly.
2021-12-19 22:37:55 +00:00
behzad nouri 7476dfeec0
removes Select in favor of recv_timeout/try_iter (#21981)
crossbeam_channel::Select::ready_timeout might return with success spuriously.
2021-12-18 17:39:07 +00:00
Jeff Biseda 3fe942ab30
new net-stats require a new table (#21996) 2021-12-18 00:13:16 -08:00
dependabot[bot] 8f547a6c98
chore: bump serde from 1.0.131 to 1.0.132 (#21989)
* chore: bump serde from 1.0.131 to 1.0.132

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.131 to 1.0.132.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.131...v1.0.132)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-18 07:45:20 +00:00
carllin 7f6fb6937a
Ensure AncestorHashesSerice selects an open port (#21919) 2021-12-18 00:44:01 -05:00
Jeff Biseda 97a1fa10a6
streamer send destination metrics for repair, gossip (#21564) 2021-12-17 15:21:05 -08:00
segfaultdoctor 76098dd42a
RPC Block Subscription (#21787)
* add stuff

* compiling

* add notify block

* wip

* feat: add blockSubscribe pubsub method

* address PR comments

Co-authored-by: Lucas B <buffalu@jito.network>
Co-authored-by: Zano <segfaultdoctor@protonmail.com>
2021-12-17 16:03:09 -07:00
behzad nouri 89d66c3210
removes next_shred_index from return value of entries to shreds api (#21961)
next-shred-index is already readily available from returned data shreds.
The commit simplifies the api for upcoming changes to erasure coding
schema which will require explicit tracking of indices for coding shreds
as well as data shreds.
2021-12-17 15:01:55 +00:00
Jeff Biseda 7ec39f5a1e
time based retransmit in replay_stage (#21498) 2021-12-17 05:44:40 -08:00
carllin 385efae4b3
Remove need to send bank in retransmit request from ReplayStage (#21943)
* Remove need to send bank in retransmitter
2021-12-16 21:11:01 -05:00
Justin Starry 6ff0be6a82
Clean up demote program write lock feature (#21949)
* Clean up demote program write lock feature

* fix test
2021-12-16 17:27:22 -05:00
steviez e83ca4bb28
Clean up test_ledger_cleanup_compaction prints (#21875)
- Use info!()/warn!() over println!()/eprintln!()
- Make status prints consistent
- Add default RUST_LOG filter to see test printouts
- Adjust reported data to show shreds and rates we care about
2021-12-16 11:24:29 -06:00
carllin cb395abff7
Fix subtraction overflow (#21871) 2021-12-14 14:24:22 -05:00
behzad nouri 8d980f07ba
uses Option<Slot> for SlotMeta.parent_slot (#21808)
SlotMeta.parent_slot for the head of a detached chain of slots is
unknown and that is indicated by u64::MAX which lacks type-safety:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L203-L205

The commit changes the type to Option<Slot>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-14 18:57:11 +00:00
behzad nouri 4ceb2689f5
adds ShredId uniquely identifying each shred (#21820) 2021-12-14 17:34:02 +00:00
steviez 4a9d7318d1
Rework test parameters to be shreds instead of entries (#21780)
The number of shreds that result from a given number of entries is
variable and in our test case, somewhat unintuitive to think about when
trying to determine how much data we're pushing into the blockstore. So,
this change converts the unit of test parameters from entries to shreds.

This change also cleans up some variable naming for clarity and prints.
2021-12-13 23:34:43 -06:00
dependabot[bot] 018b54dbd7
chore: bump serde_json from 1.0.72 to 1.0.73 (#21856)
* chore: bump serde_json from 1.0.72 to 1.0.73

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.72...v1.0.73)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 04:08:22 +00:00
dependabot[bot] b81124deec
chore: bump etcd-client from 0.8.1 to 0.8.2 (#21825)
Bumps [etcd-client](https://github.com/etcdv3/etcd-client) from 0.8.1 to 0.8.2.
- [Release notes](https://github.com/etcdv3/etcd-client/releases)
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.8.1...0.8.2)

---
updated-dependencies:
- dependency-name: etcd-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-13 16:20:22 -07:00
dependabot[bot] 8aa3d690b5
chore: bump dashmap from 4.0.2 to 5.0.0 (#21824)
* chore: bump dashmap from 4.0.2 to 5.0.0

Bumps [dashmap](https://github.com/xacrimon/dashmap) from 4.0.2 to 5.0.0.
- [Release notes](https://github.com/xacrimon/dashmap/releases)
- [Commits](https://github.com/xacrimon/dashmap/commits/v5.0.0)

---
updated-dependencies:
- dependency-name: dashmap
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-13 13:23:58 -07:00
Jeff Washington (jwash) 90f41fd9b7
use cost model to limit new account creation (#21369)
* use cost model to limit new account creation

* handle every system instruction

* remove &

* simplify match

* simplify match

* add datapoint for account data size

* add postgres error handling

* handle accounts:unlock_accounts
2021-12-12 14:57:18 -06:00
behzad nouri e08139f949
uses Option<u64> for SlotMeta.last_index (#21775)
SlotMeta.last_index may be unknown and current code is using u64::MAX to
indicate that:
https://github.com/solana-labs/solana/blob/6c108c8fc/ledger/src/blockstore_meta.rs#L169-L174

This lacks type-safety and can introduce bugs if not always checked for
Several instances of slot_meta.last_index + 1 are also subject to
overflow.

This commit updates the type to Option<u64>. Backward compatibility is
maintained by customizing serde serialize/deserialize implementations.
2021-12-11 14:47:20 +00:00
Justin Starry 254ef3e7b6
Rename Packets to PacketBatch (#21794) 2021-12-11 09:44:15 -05:00
Yueh-Hsuan Chiang 65194c7ae8
Add NUM_WRITERS to ledger_cleanup to enable multiple writers. (#21729)
Summary:
* Add NUM_WRITERS to ledger_cleanup to enable multiple writers.
  (Note that our insert_shreds() is still single threaded because
   it has a lock that limits only one writer at a time.)

* Make pre-generated slots more performent by directly inserting
  into the shared queue.  Otherwise, the main-thread which
  prepares the slots will be slower than the writers.

* Correct the shred insertion time -- before this diff it did not
  wait for joining all writer threads.
2021-12-10 09:42:51 -08:00
dependabot[bot] d870f566ef
chore: bump serde from 1.0.130 to 1.0.131 (#21758)
* chore: bump serde from 1.0.130 to 1.0.131

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.130 to 1.0.131.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.130...v1.0.131)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-10 08:47:46 +00:00
behzad nouri 8063273d09
adds more sanity checks to shreds (#21675) 2021-12-09 16:43:57 +00:00
Ashwin Sekar f0acf7681e
Add vote instructions that directly update on chain vote state (#21531)
* Add vote state instructions

UpdateVoteState and UpdateVoteStateSwitch

* cargo tree

* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
dependabot[bot] f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 (#21643)
* chore: bump itertools from 0.10.1 to 0.10.3

Bumps [itertools](https://github.com/rust-itertools/itertools) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases)
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3)

---
updated-dependencies:
- dependency-name: itertools
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-06 23:23:26 -07:00
behzad nouri cd17f63d81
adds back position field to coding-shred-header (#21600)
https://github.com/solana-labs/solana/pull/17004
removed position field from coding-shred-header because as it stands the
field is redundant and unused.
However, with the upcoming changes to erasure coding schema this field
will no longer be redundant and needs to be populated.
2021-12-05 14:42:09 +00:00
Trent Nelson d6f22433d0 Bump version to v1.10.0 2021-12-04 20:17:54 +00:00
Jeff Biseda 9c6b95e1e1
fix distance calculation in get_closest_completion (#21601) 2021-12-03 22:36:46 -08:00
dependabot[bot] 446089edb4
chore: bump retain_mut from 0.1.4 to 0.1.5 (#21599)
Bumps [retain_mut](https://github.com/upsuper/retain_mut) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/upsuper/retain_mut/releases)
- [Commits](https://github.com/upsuper/retain_mut/compare/v0.1.4...v0.1.5)

---
updated-dependencies:
- dependency-name: retain_mut
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-12-03 22:11:42 -07:00
Justin Starry 1430b58a6d
Remove deprecated slow epoch boundary methods (#21568) 2021-12-03 17:59:10 +00:00
Michael Vines b8837c04ec Reformat imports to a consistent style for imports
rustfmt.toml configuration:
  imports_granularity = "One"
  group_imports = "One"
2021-12-03 09:19:13 -08:00
Alexander Meißner b78f5b6032
Refactor: Cleanup InstructionProcessor (#21404)
* Moves create_message(), native_invoke() and process_cross_program_instruction()
from the InstructionProcessor to the InvokeContext so that they can have a useful "self" parameter.

* Moves InstructionProcessor into InvokeContext and Bank.

* Moves ExecuteDetailsTimings into its own file.

* Moves Executor into invoke_context.rs

* Moves PreAccount into its own file.

* impl AbiExample for BuiltinPrograms
2021-12-01 08:54:42 +01:00
Michael Vines dd12d90eac Upgrade to Rust 2021 2021-11-30 20:43:46 -08:00
dependabot[bot] de8edad30b
chore: bump etcd-client from 0.7.2 to 0.8.1 (#21513)
Bumps [etcd-client](https://github.com/etcdv3/etcd-client) from 0.7.2 to 0.8.1.
- [Release notes](https://github.com/etcdv3/etcd-client/releases)
- [Commits](https://github.com/etcdv3/etcd-client/compare/0.7.2...0.8.1)

---
updated-dependencies:
- dependency-name: etcd-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-11-30 18:16:32 +00:00
dependabot[bot] a125e7b8c2
chore: bump serde_json from 1.0.71 to 1.0.72 (#21426)
* chore: bump serde_json from 1.0.71 to 1.0.72

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00