Commit Graph

16030 Commits

Author SHA1 Message Date
Jeff Washington (jwash) db40d06a39
AcctIdx: introduce startup to bucket map (#20004) 2021-09-18 12:55:57 -05:00
Jeff Washington (jwash) bed0049a51
AcctIdx: add age per item in in_mem acct idx (#19981) 2021-09-18 09:56:26 -05:00
Jeff Washington (jwash) 8df8f4396d
AcctIdx: start calling flush on in mem cache buckets (#19966)
* AcctIdx: start calling flush on in mem cache buckets

* fix orderings
2021-09-18 09:56:08 -05:00
Jeff Washington (jwash) 4089f8b06b
AcctIdx: with multiple bg threads, only log stats once per time interval (#19984) 2021-09-18 09:55:20 -05:00
Jeff Washington (jwash) 4e038e94fd
add 'drives' to AccountsIndexConfig (#19989) 2021-09-18 09:54:57 -05:00
Jeff Washington (jwash) f500c99a6d
pass AcctIdxConfig to BucketMapHolder (#19997) 2021-09-18 09:54:00 -05:00
Jeff Washington (jwash) bd86f41e18
acct idx tests use 2 bins instead of 8k (#19996) 2021-09-18 09:53:44 -05:00
Jon Cinque e9b066d497
client: Add retry logic on Pubsub 429s (#19990) 2021-09-18 12:40:43 +02:00
Alexander Meißner 36f46e1c31
CPI Account Reuse (#19762)
* Removes two account copy steps from InstructionProcessor::native_invoke().

* Moves gathering of keyed_accounts, caller_write_privileges and program_indices into InstructionProcessor::create_message().

* Explicitly routes the serialized account lengths to enable sharing of existing account structures.

* Recycles existing account structs in CPI syscall.
2021-09-18 08:09:47 +02:00
Justin Starry efd024510a
Reduce payer balance needed to deploy programs (#19645)
* Reduce payer balance needed to deploy programs

* Fix test and unbalanced ix error

* fix test

* fix up tests
2021-09-17 19:49:53 -05:00
carllin e6b4dd3866
Add bank to banking stage regardless of if there is a working bank (#19855) 2021-09-17 16:55:53 -07:00
Jeff Washington (jwash) 4dc2f08198
AcctIdx: hold_range_in_memory (#19955) 2021-09-17 22:19:29 +00:00
Jeff Washington (jwash) 24b136a993
AcctIdx: configurable flush threads (#19983) 2021-09-17 22:02:43 +00:00
visortelle 073c5359b0
Fix broken links in terminology.md (#19978) 2021-09-17 15:55:31 -06:00
Tyera Eulberg c4fe352965 Tiny toml fmt fix 2021-09-17 14:16:14 -06:00
Jeff Washington (jwash) b2152be3b2
introduce bucket map (#19848)
* introduce bucket map

* rename BucketMap bits to num_buckets_pow2

* use u64::BITS

* Store the number of buckets in BucketMapConfig as a regular number

* remove redundant type aliases

* use Slot from sdk

* use Arc::clone() instead

* fixup erase drives

* rename num_buckets to max_buckets

* add doc to BucketMapConfig::new()

* add more documentation

* rename to DEFAULT_CAPACITY_POW2

* doc

* add more traits while we can

* rename capacity to capacity_pow2

* fix a naming for max_buckets_pow2

* remove unused/incorrect DataBucket::bytes

* rework benches a bit

* fixup bench docs

* rename create_bucket_capacity_pow2 to bucket_capacity_when_created_pow2

* rename BucketMapKeyValue to BucketItem

* rename to items_in_range

* remove values()

* remove addref and unref

* remove more addref and unref

* resurect addref and unref since tests use 'em for now

* rename to BucketStorage

* move stats in bucket_stats

* remove specializations (i don't think they are needed)

* move MaxSearch and RefCount into lib.rs

* move BucketItem to bucket_item.rs

* add doc

* keys no longer returns an option

* Revert "remove specializations (i don't think they are needed)"

This reverts commit b22f78e072cf0f7107851b08e58c2e3fead3f64d.

Co-authored-by: Brooks Prumo <brooks@solana.com>
2021-09-17 15:11:27 -05:00
Jeff Washington (jwash) cddb9da4f0
AcctIdx: central age functions/state (#19980) 2021-09-17 15:11:07 -05:00
Pavel Strakhov 65227f44dc
Optimize RPC pubsub for multiple clients with the same subscription (#18943)
* reimplement rpc pubsub with a broadcast queue

* update tests for new pubsub implementation

* fix: fix review suggestions

* chore(rpc): add additional pubsub metrics

* integrate max subscriptions check into SubscriptionTracker to reduce locking

* separate subscription control from tracker

* limit memory usage of items in pubsub broadcast queue, improve error handling

* add more pubsub metrics

* add final count metrics to pubsub

* add metric for total number of subscriptions

* fix small review suggestions

* remove by_params from SubscriptionTracker and add node_progress_watchers map instead

* add subscription tracker tests

* add metrics for number of pubsub notifications as a counter

* ignore clippy lint in TokenCounter

* fix underflow in token counter

* reduce queue capacity in pubsub tests

* fix(rpc): fix test timeouts

* fix race in account subscription test

* Add RpcSubscriptions::new_for_tests

Co-authored-by: Pavel Strakhov <p.strakhov@iconic.vc>
Co-authored-by: Nikita Podoliako <n.podoliako@zubr.io>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-09-17 13:40:14 -06:00
Jack May fc2bf2d3b6
Cleanup and standardize precompiles (#19918) 2021-09-17 11:36:57 -07:00
Jeff Washington (jwash) 99f5684dc4
AcctIdx: handle future config defaults (#19967) 2021-09-17 13:12:06 -05:00
Josh 0404e75e96
feat: make confirm transaction timeout configurable (#19954) 2021-09-17 09:54:25 -07:00
Jeff Washington (jwash) 9998e16df3
AcctIdx: prepare to spin up multiple bg threads (#19969) 2021-09-17 11:53:25 -05:00
Jack May 7b0bf64404
cleanup old features (#19956) 2021-09-17 09:46:49 -07:00
Jeff Washington (jwash) 37d6f5013f
AcctIdx: initialize acct idx by config (#19968) 2021-09-17 16:39:41 +00:00
visortelle 1ec22572f2
Fix typos in terminology.md (#19977) 2021-09-17 10:30:10 -06:00
dependabot[bot] b8106e2450
chore: bump memmap2 from 0.3.1 to 0.4.0 (#19974)
* chore: bump memmap2 from 0.3.1 to 0.4.0

Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases)
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.3.1...v0.4.0)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-09-17 10:00:12 -06:00
Jeff Washington (jwash) 376b20c3d2
AcctIdx: add some bucket flush helpers (#19965) 2021-09-17 10:41:30 -05:00
Jon Cinque 9eb98adf97
stake: Add BorshDeserialize trait to structs (#19958) 2021-09-17 10:14:23 +02:00
dependabot[bot] 40cc243017
chore:(deps): bump @solana/web3.js from 1.26.0 to 1.27.0 in /explorer (#19975)
Bumps [@solana/web3.js](https://github.com/solana-labs/solana-web3.js) from 1.26.0 to 1.27.0.
- [Release notes](https://github.com/solana-labs/solana-web3.js/releases)
- [Changelog](https://github.com/solana-labs/solana-web3.js/blob/master/.releaserc.json)
- [Commits](https://github.com/solana-labs/solana-web3.js/compare/v1.26.0...v1.27.0)

---
updated-dependencies:
- dependency-name: "@solana/web3.js"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-17 08:13:40 +00:00
dependabot[bot] a4a8560311
chore:(deps): bump sass from 1.41.0 to 1.41.1 in /explorer (#19972)
Bumps [sass](https://github.com/sass/dart-sass) from 1.41.0 to 1.41.1.
- [Release notes](https://github.com/sass/dart-sass/releases)
- [Changelog](https://github.com/sass/dart-sass/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sass/dart-sass/compare/1.41.0...1.41.1)

---
updated-dependencies:
- dependency-name: sass
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-17 08:07:10 +00:00
dependabot[bot] d184eab019
chore:(deps): bump @types/node from 16.9.1 to 16.9.2 in /explorer (#19973)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.9.1 to 16.9.2.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-17 08:07:02 +00:00
Jeff Washington (jwash) 4d8b3aa578
AcctsIdx: introduce dirty state per pubkey (#19960) 2021-09-16 21:36:23 -05:00
dependabot[bot] 53fbbfe56c
chore: bump prettier from 2.4.0 to 2.4.1 in /web3.js (#19961)
Bumps [prettier](https://github.com/prettier/prettier) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/2.4.0...2.4.1)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-16 18:39:18 -05:00
Justin Starry c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` (#19931)
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback
2021-09-16 23:37:45 +00:00
dependabot[bot] 92510a2831
chore: bump @types/node from 16.9.1 to 16.9.2 in /web3.js (#19957)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 16.9.1 to 16.9.2.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-16 16:22:41 -07:00
sakridge c8f76b8bd0
Fix tpu service fanout (#19944) 2021-09-17 01:01:56 +02:00
Jeff Washington (jwash) 58f25a8752
AcctIdx: move background() to AccountsIndexStorage (#19948) 2021-09-16 17:52:06 -05:00
dependabot[bot] 9f66965eac
chore: bump http-server from 13.0.1 to 13.0.2 in /web3.js (#19953)
Bumps [http-server](https://github.com/http-party/http-server) from 13.0.1 to 13.0.2.
- [Release notes](https://github.com/http-party/http-server/releases)
- [Commits](https://github.com/http-party/http-server/compare/v13.0.1...v13.0.2)

---
updated-dependencies:
- dependency-name: http-server
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-16 15:24:19 -07:00
Jeff Washington (jwash) 66e0fafc21
AccountsIndexStorage holds InMemAccountsIndex[] (#19947) 2021-09-16 21:12:22 +00:00
Josh 49d3d79459
web3.js: add accounts support to simulateTransaction (#19590)
* feat: add accounts support to simulateTransaction

* feat: introduce test for simulateTransaction on Message objects

* feat: populate transaction from message defaults to no signatures

* fix: remove unused constant

* fix: small formatting error

* fix: eslint and prettier were fighting over ternary indentation

* fix: make simulated transaction result accounts nullable
2021-09-16 14:10:28 -07:00
dependabot[bot] 1a91621c29
chore: bump @typescript-eslint/parser from 4.31.0 to 4.31.1 in /web3.js (#19921)
Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 4.31.0 to 4.31.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v4.31.1/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-16 14:06:11 -07:00
dependabot[bot] 68da1909de
chore: bump serde_json from 1.0.67 to 1.0.68 (#19942)
* chore: bump serde_json from 1.0.67 to 1.0.68

Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.67 to 1.0.68.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.67...v1.0.68)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-16 20:23:25 +00:00
Christian Kamm f1bbf1d8b0
rpc: performance fix for getProgramAccounts (#19941)
* rpc: performance fix for getProgramAccounts

The accounts were gradually pushed into a vector, which produced
significant slowdowns for very large responses.

* rpc: rewrite loops using iterators

Co-authored-by: Christian Kamm <ckamm@delightful-solutions.de>
2021-09-16 19:40:01 +00:00
dependabot[bot] 43d4aa7900
chore: bump socket2 from 0.4.1 to 0.4.2 (#19937)
* chore: bump socket2 from 0.4.1 to 0.4.2

Bumps [socket2](https://github.com/rust-lang/socket2) from 0.4.1 to 0.4.2.
- [Release notes](https://github.com/rust-lang/socket2/releases)
- [Changelog](https://github.com/rust-lang/socket2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/socket2/compare/v0.4.1...v0.4.2)

---
updated-dependencies:
- dependency-name: socket2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-16 18:56:36 +00:00
Brooks Prumo 4c69229a47
Add incremental snapshot CLI args to ledger-tool (#19889)
This commit only adds args to specify the maximum full/incremental
snapshot archives to retain when purging old snapshot archives.  It
purposely does *not* add a way to generate new incremental snapshots.

Fixes #19857
2021-09-16 10:56:18 -05:00
dependabot[bot] ea15c0b40b
chore: bump libc from 0.2.101 to 0.2.102 (#19924)
* chore: bump libc from 0.2.101 to 0.2.102

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.101 to 0.2.102.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.101...0.2.102)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-16 09:45:01 -06:00
dependabot[bot] 3b06214590
chore:(deps): bump prettier from 2.4.0 to 2.4.1 in /explorer (#19938)
Bumps [prettier](https://github.com/prettier/prettier) from 2.4.0 to 2.4.1.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/2.4.0...2.4.1)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-16 09:06:12 -05:00
sakridge dc69cc1ae4
Only allow votes when root distance gets too high (#19917) 2021-09-16 15:12:26 +02:00
Sean Young 2762f6f96f Fix solana docker image
The docker image fails with:

/usr/bin/solana-run.sh: line 66: ./fetch-spl.sh: No such file or directory

In the solana docker image, scripts/run.sh is copied to
/usr/bin/solana-run.sh and fetch-spl.sh to /usr/bin/fetch-spl.sh. This
means that the line:

cd "$(dirname "$0")/.."

means we're doing a "cd /usr", which means we can't find fetch-spl.sh or
spl-genesis-args.sh (i.e., the error above).
2021-09-16 11:32:47 +01:00
dependabot[bot] 4675fa6c46
chore:(deps): bump @blockworks-foundation/mango-client in /explorer (#19939)
Bumps [@blockworks-foundation/mango-client](https://github.com/blockworks-foundation/mango-client-v3) from 3.0.17 to 3.0.18.
- [Release notes](https://github.com/blockworks-foundation/mango-client-v3/releases)
- [Commits](https://github.com/blockworks-foundation/mango-client-v3/commits)

---
updated-dependencies:
- dependency-name: "@blockworks-foundation/mango-client"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-09-16 08:07:31 +00:00