Commit Graph

1984 Commits

Author SHA1 Message Date
behzad nouri d6fbf3fb17
adds new contact-info with forward compatible sockets (#29596)
The commit implement new ContactInfo where
* Ports and IP addresses are specified separately so that unique IP
  addresses can only be specified once.
* Different sockets (tvu, tpu, etc) are specified by opaque u8 tags so
  that adding and removing sockets is backward and forward compatible.
* solana_version::Version is also embedded in so that it won't need to
  be gossiped separately.
* NodeInstance is also rolled in by adding a field identifying when the
  instance was first created so that it won't need to be gossiped
  separately.

Update plan:
* Once the cluster is able to ingest the new type (i.e. this patch), a
  2nd patch will start gossiping the new ContactInfo along with the
  LegacyContactInfo.
* Once all nodes in the cluster gossip the new ContactInfo, a 3rd patch
  will start solely using the new ContactInfo while still gossiping the
  old LegacyContactInfo.
* Once all nodes in the cluster solely use the new ContactInfo, a 4th
  patch will stop gossiping the old LegacyContactInfo.
2023-01-26 17:02:18 +00:00
Ryo Onodera 40bbf99c74
Add fully-reproducible online tracer for banking (#29196)
* Add fully-reproducible online tracer for banking

* Don't use eprintln!()...

* Update programs/sbf/Cargo.lock...

* Remove meaningless assert_eq

* Group test-only code under aptly named mod

* Remove needless overflow handling in receive_until

* Delay stat aggregation as it's possible now

* Use Cow to avoid needless heap allocs

* Properly consume metrics action as soon as hold

* Trace UnprocessedTransactionStorage::len() instead

* Loosen joining api over type safety for replaystage

* Introce hash event to override these when simulating

* Use serde_with/serde_as instead of hacky workaround

* Update another Cargo.lock...

* Add detailed comment for Packet::buffer serialize

* Rename sender_overhead_minimized_receiver_loop()

* Use type interference for TraceError

* Another minor rename

* Retire now useless ForEach to simplify code

* Use type alias as much as possible

* Properly translate and propagate tracing errors

* Clarify --enable-banking-trace with better naming

* Consider unclean (signal-based) node restarts..

* Tweak logging and cli

* Remove Bank events as it's not needed anymore

* Make tpu own banking tracer thread

* Reduce diff a bit..

* Use latest serde_with

* Finally use the published rolling-file crate

* Make test code change more consistent

* Revive dead and non-terminating test code path...

* Dispose batches early now that possible

* Split off thread handle very early at ::new()

* Tweak message for TooSmallDirByteLimitl

* Remove too much of indirection

* Remove needless pub from ::channel()

* Clarify test comments

* Avoid needless event creation if tracer is disabled

* Write tests around file rotation and spill-over

* Remove unneeded PathBuf::clone()s...

* Introduce inner struct instead of tuple...

* Remove unused enum BankStatus...

* Avoid .unwrap() for the case of disabled tracer...
2023-01-25 21:54:38 +09:00
valiksinev 5f7fea100a
Big integer modular exponentiation (EIP-198) (#28503)
* big_mod_exp impl

* fix programs/sbf/Cargo.lock

* ComputeBudget impl

* update compute_budget

* compute_budget update

* fix build

* fix tests

* fix cargo clippy

* fix clippy

* fix bpf_loader dependency sorting

* fix sorting

* fix merge from master

* fix cargo fmt

* fix C-tests

* fix cargo fmt

* comments apply

* fix programs/sbf/Cargo.lock

* update compude_budget cost

* remove whitespaces

* fix cargo fmt

Co-authored-by: sinev-valentine <sinev-valentine@yandex.ru>
2023-01-20 18:42:37 +09:00
Ryo Onodera 254381e3ca
Bump chrono to v0.4.23 (#29709)
* Bump chrono to v0.4.23

* fmt...
2023-01-18 13:54:02 +09:00
dependabot[bot] d32256d2af
Bump pem from 1.1.0 to 1.1.1 (#29673)
* Bump pem from 1.1.0 to 1.1.1

Bumps [pem](https://github.com/jcreekmore/pem-rs) from 1.1.0 to 1.1.1.
- [Release notes](https://github.com/jcreekmore/pem-rs/releases)
- [Changelog](https://github.com/jcreekmore/pem-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jcreekmore/pem-rs/compare/v1.1.0...v1.1.1)

---
updated-dependencies:
- dependency-name: pem
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-12 19:39:02 -07:00
dependabot[bot] 10f8b5faa3
Bump memmap2 from 0.5.3 to 0.5.8 (#29674)
* Bump memmap2 from 0.5.3 to 0.5.8

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.3 to 0.5.8.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.3...v0.5.8)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-12 19:38:38 -07:00
dependabot[bot] a08124e7bb
Bump tonic from 0.8.0 to 0.8.2 (#29644)
* Bump tonic from 0.8.0 to 0.8.2

Bumps [tonic](https://github.com/hyperium/tonic) from 0.8.0 to 0.8.2.
- [Release notes](https://github.com/hyperium/tonic/releases)
- [Changelog](https://github.com/hyperium/tonic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/tonic/compare/v0.8.0...v0.8.2)

---
updated-dependencies:
- dependency-name: tonic
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-12 00:13:51 -07:00
Lijun Wang 7c8b846344
Update quinn versions (#29603)
* chore: bump quinn-udp from 0.1.3 to 0.3.2

Bumps [quinn-udp](https://github.com/quinn-rs/quinn) from 0.1.3 to 0.3.2.
- [Release notes](https://github.com/quinn-rs/quinn/releases)
- [Commits](https://github.com/quinn-rs/quinn/commits)

---
updated-dependencies:
- dependency-name: quinn-udp
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Try to use quinn 0.9.3 and quinn-proto 0.9.2

* Update streamer and client for quic to support qunn 0.9.3

* Update Cargo.lock

* Fixed unit test failure for quic tests

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-11 10:08:22 -08:00
dependabot[bot] fadf027851
Bump semver from 1.0.14 to 1.0.16 (#29645)
* Bump semver from 1.0.14 to 1.0.16

Bumps [semver](https://github.com/dtolnay/semver) from 1.0.14 to 1.0.16.
- [Release notes](https://github.com/dtolnay/semver/releases)
- [Commits](https://github.com/dtolnay/semver/compare/1.0.14...1.0.16)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-11 10:07:58 -07:00
dependabot[bot] 758311b073
chore: bump either from 1.7.0 to 1.8.0 (#29425)
* chore: bump either from 1.7.0 to 1.8.0

Bumps [either](https://github.com/bluss/either) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/bluss/either/releases)
- [Commits](https://github.com/bluss/either/compare/1.7.0...1.8.0)

---
updated-dependencies:
- dependency-name: either
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-11 09:48:12 -07:00
HaoranYi d693167dfb
Feature: filter stake by min delegation for rewards (#29618)
filter stake by min delegation for rewards
2023-01-11 08:55:18 -06:00
Yihau Chen b1b683d2af
chore: bump hyper to 0.4.23 (#29619) 2023-01-11 14:19:32 +08:00
dependabot[bot] 864a7d8d6f
Bump bzip2 from 0.4.3 to 0.4.4 (#29630)
* Bump bzip2 from 0.4.3 to 0.4.4

Bumps [bzip2](https://github.com/alexcrichton/bzip2-rs) from 0.4.3 to 0.4.4.
- [Release notes](https://github.com/alexcrichton/bzip2-rs/releases)
- [Commits](https://github.com/alexcrichton/bzip2-rs/commits/0.4.4)

---
updated-dependencies:
- dependency-name: bzip2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-10 23:08:22 -07:00
dependabot[bot] 04b8e8f5be
chore: bump cc from 1.0.73 to 1.0.78 (#29387)
* chore: bump cc from 1.0.73 to 1.0.78

Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.0.73 to 1.0.78.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.73...1.0.78)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2023-01-10 22:50:03 -07:00
Dmitri Makarov cc4e134079
Add an option to install a specific version of sbf-tools (#29544) 2023-01-10 06:14:24 -08:00
Yihau Chen 94cb88ffad
add solana-ledger-tool bigtable copy (#28122)
* init copy cmd

* extract creating emulator connection logic

* extract copy args as struct

* add new_for_emulator

* add tryFrom confirmed block to versioned block

* implement bigtable copy command

* use 'force' flag to force upload

* use unwrap_or

* remove redundant importing

* fix nightly lint

* explicit transactions missing error

* process ending_slot

* prevent start slot > end slot

* print skip slots in debug level

* fix destination bigtable should not be readonly

* combine is-emulator and endpoint to emulated source. conflict with crenditial path

* wording

* log some error messages with error level

* nightly lint

* add dry-run

* extract create bigtable instances logic

* use a lighter way to check block

* use the latest futures version which is used in the repo

* use futures = "0.3"

Co-authored-by: Tyera <teulberg@gmail.com>

* wording

Co-authored-by: Tyera <teulberg@gmail.com>

* wording

Co-authored-by: Tyera <teulberg@gmail.com>

Co-authored-by: Tyera <teulberg@gmail.com>
2023-01-09 11:23:35 +08:00
Tyera 7689044521
Fix solana-stake-accounts version (#29542)
Fix clap inputs
2023-01-05 19:49:28 -07:00
Jon Cinque aa593a31f8
logger: Update to env_logger 0.9.3 (#29510) 2023-01-04 21:35:37 +01:00
Yihau Chen d078c041d1
chore: add missing members back to workspace.members (#29450)
* frozen-abi/macro

* program-runtime

* sdk/macro

* sdk/program

* storage-bigtable/build-proto

* fix sorting
2023-01-04 13:22:00 +08:00
Jon Cinque 968b158494
vote: Prevent commission update in the second half of epochs (#29362)
* vote: Prevent commission update in the second half of epochs

* Address feedback

* Fix tests

* Make the feature enabled by single-contributor

* Use a cooler pubkey
2022-12-23 12:24:39 +01:00
dependabot[bot] 65391a70cb
chore: bump futures-util from 0.3.24 to 0.3.25 (#29337)
* chore: bump futures-util from 0.3.24 to 0.3.25

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.24 to 0.3.25.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.24...0.3.25)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-12-22 22:42:18 -07:00
Lijun Wang 0a7d8520b8
Make connection cache to support specified client endpoint (#29240)
ConnectionCache is being used for managing connections for sending messages using quic. In the current implementation the connection's endpoint is created in a lazy initialized fashion and set to one per connection pool. In repair we need all connections to use the same Endpoint so that the server can send back the response to the same Endpoint.
2022-12-22 12:35:32 -08:00
Tyera c8dd5bbd48
Move getSecondaryIndexKeySize RPC to admin_rpc_service (#29003)
* Move getSecondaryIndexKeySize to admin_rpc_service

* Add AdminRpc test framework

* Move test to admin_rpc_service

* Update test

* Remove full-api infra
2022-12-19 19:54:05 -07:00
dependabot[bot] 6ebbaf30c9
chore: bump rcgen from 0.9.2 to 0.9.3 (#28021)
* chore: bump rcgen from 0.9.2 to 0.9.3

Bumps [rcgen](https://github.com/est31/rcgen) from 0.9.2 to 0.9.3.
- [Release notes](https://github.com/est31/rcgen/releases)
- [Changelog](https://github.com/est31/rcgen/blob/master/CHANGELOG.md)
- [Commits](https://github.com/est31/rcgen/compare/v0.9.2...v0.9.3)

---
updated-dependencies:
- dependency-name: rcgen
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-12-20 02:12:53 +00:00
Yihau Chen 90f74302cf
chore: improve solana-client-test (#29255)
* separate test port

* make server check more deterministic

* remove serial_test

* use atomic auto incremental port

* make NEXT_RPC_PUBSUB_PORT as an global static variable

* make check_server_is_ready become check_server_is_ready_or_panic

* use processed commitment in test_rpc_client

* lint
2022-12-15 16:27:31 +08:00
dependabot[bot] dca5d7f9b4
chore: bump test-case from 2.1.0 to 2.2.2 (#28184)
Bumps [test-case](https://github.com/frondeus/test-case) from 2.1.0 to 2.2.2.
- [Release notes](https://github.com/frondeus/test-case/releases)
- [Changelog](https://github.com/frondeus/test-case/blob/master/CHANGELOG.md)
- [Commits](https://github.com/frondeus/test-case/compare/v2.1.0...v2.2.2)

---
updated-dependencies:
- dependency-name: test-case
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 16:07:50 +00:00
Alexander Meißner 013f784e92
Bump solana_rbpf to v0.2.38 (#29137)
* Refactor register_syscalls() => create_loader().

* Bumps solana_rbpf to v0.2.38

* Replaces Config::encrypt_environment_registers by Config::runtime_environment_key.

* Adds pc parameter to MemoryMapping::map()

* Removes debugger from rbpf CLI.

* Adds debugger to rbpf CLI again.

* Renames vm.context_object => vm.env.context_object_pointer.

* Renames SyscallRegistry => BuiltInProgram.
Renames SyscallFunction => BuiltInFunction.
Renames register_syscall_by_name() => register_function_by_name().

* Removes ContextObject from Analysis.

* Renames Config::error_on_syscall_bpf_function_hash_collisions => Config::external_internal_function_hash_collision.
2022-12-12 08:47:09 +01:00
HaoranYi 33b15240ac
Revert #28945 (#29127)
revert #28945
2022-12-06 17:08:56 -06:00
haoran f716cad4af don't use procfs as it is not supported on mac and windows.
make open_fd stats only on linux platform
2022-12-06 13:31:50 -06:00
haoran fc97d818b6 share code 2022-12-06 13:31:50 -06:00
Haoran Yi e1ba5a2a63 add monitoring for open file descriptors stat 2022-12-06 13:31:50 -06:00
Haoran Yi 703511c3e8 log number of open files when panic 2022-12-06 13:31:50 -06:00
Tyera 3d6eb16753
Bump spl-ata to v1.1.2 (#28979) 2022-11-30 03:46:42 +00:00
hanako mumei 015784ad07 update to spl-token-2022 0.5.0
* support CpiGuard and PermanentDelegate extensions in transaction-status and account-decoder
* update transaction-status and account-decoder to new ConfidentialTransfer interfaces
2022-11-28 11:30:41 -08:00
ananas-block afc3fee937
added alt_bn syscalls (#27961)
* added alt_bn128_syscalls

* increased regression build redundancy to > 10
2022-11-21 17:43:06 +01:00
Yihau Chen eaea3ec5bc
bump solana_rbpf to v0.2.37 (#28857) 2022-11-19 11:30:36 +08:00
Tyera c32377b5af
Split out quic- and udp-client definitions (#28762)
* Move ConnectionCache back to solana-client, and duplicate ThinClient, TpuClient there

* Dedupe thin_client modules

* Dedupe tpu_client modules

* Move TpuClient to TpuConnectionCache

* Move ThinClient to TpuConnectionCache

* Move TpuConnection and quic/udp trait implementations back to solana-client

* Remove enum_dispatch from solana-tpu-client

* Move udp-client to its own crate

* Move quic-client to its own crate
2022-11-18 12:21:45 -07:00
Justin Starry 7371608722
Support versioned transactions in program test framework (#28739)
* Support versioned transactions in program test framework

* use working bank

* Update to process_transaction_with_metadata

* Migrate client apis from Transaction to Into<VersionedTransaction>

* feedback
2022-11-18 23:43:52 +08:00
Tyera Eulberg 8b63f73100
Remove Default impls for TpuConnectionCache and ConnectionPool clients (#28788)
* Remove TpuConnectionCache impl Default

* Add fallible ctors for Quic/UdpClients

* Add Quic/TpuClient ctor trait

* Remove Quic/UdpClient impl Default
2022-11-17 20:13:43 -07:00
Brooks Prumo d798e751a0
Disables EAH with short epochs (#28803) 2022-11-15 13:26:19 -05:00
Alexander Meißner ff1ff587d1
Bump solana_rbpf to 0.2.36 (#28794)
* Bumps solana_rbpf to v0.2.36

* Removes ThisInstructionMeter.

* Removes one "unsafe" expression.

* Removes redundant call to solana_rbpf:🧝:register_bpf_function().

* Adjusts SyscallFunction and SyscallRegistry.

* Inlines ProgramEnvironment into EbpfVm.

* Refactors trait SyscallConsume into fn consume_compute_meter().

* Inlines ComputeMeter into InvokeContext.

* Removes solana-metrics dependency from bpf_loader.

* Replaces RBPF tracer functionality by the debugger.

* Take compute_units_consumed from execute_program().

* Merges execute_program_interpreted() and execute_program_jit().
2022-11-15 15:21:11 +01:00
Dmitri Makarov 0564d2a785 Add to cargo-test-sbf an option to test specific packages 2022-11-04 16:35:46 +00:00
Jon Cinque 9d7e999c1d
bench-tps: Transition to using spl-instruction-padding (#28496)
* bench-tps: Transition to using spl-instruction-padding

* Use macros for conversions
2022-10-26 11:16:57 -04:00
Dmitri Makarov f48ed1f331 Bump solana_rbpf to 0.2.35
- recognize new SBF machine code in ELF header
2022-10-25 15:28:02 +00:00
Ashwin Sekar f207af765e
Split out voting and banking threads in banking stage (#27931)
* Split out voting and banking threads in banking stage

Additionally this allows us to aggressively prune the buffer for voting threads
as with the new vote state only the latest vote from each validator is
necessary.

* Update local cluster test to use new Vote ix

* Encapsulate transaction storage filtering better

* Address pr comments

* Commit cargo lock change

* clippy

* Remove unsafe impls

* pr comments

* compute_sanitized_transaction -> build_sanitized_transaction

* &Arc -> Arc

* Move test

* Refactor metrics enums

* clippy
2022-10-20 21:10:48 +00:00
Yihau Chen a1b03ec1b4
add test_save for install config (#28397) 2022-10-20 17:34:08 +08:00
Lijun Wang a6884d682c Restore the dev-dependency 2022-10-19 16:46:36 -07:00
Lijun Wang a378284621 removed crossbeam 2022-10-19 16:46:36 -07:00
Alexander Meißner 5aca7df232
Bank::create_executor() (#28474)
* Moves the integration test test_bpf_loader_upgradeable_deploy_with_max_len() from the bpf_loader into the runtime.

* Adds Bank::create_executor().

* Adds a test for Bank::create_executor().
2022-10-19 22:28:57 +02:00
Alexey Skibin 108a02cfd4
Add off-chain messages support (#27456) 2022-10-17 11:19:12 -04:00