db40cb41b2
Handle new security advisories ( #19126 )
...
* Update hyper 0.14
* Add audit ignores
* Bump assert_cmd and remove audit ignore
* Remove unnecessary unmaintained dependency
* Remove pin-project-lite warning
2021-08-09 19:36:03 +00:00
e4b66a5913
Fix deserialization of RPC errors in HttpSender ( #19110 )
...
Fixes #15576
2021-08-09 10:45:00 -07:00
f5ce2c2d63
Add incinerator to explorer search results ( #19125 )
2021-08-09 16:10:42 +00:00
c7b986220e
Accounts::new_with_config_for_tests ( #19104 )
2021-08-09 09:01:12 -05:00
1951fe4e80
accounts default stragglers ( #19101 )
2021-08-09 09:00:30 -05:00
db14bcc43b
AccountsIndex RefCount() returns u64 instead of atomic ( #19088 )
2021-08-09 08:59:56 -05:00
9616ae0f2c
signal to upsert whether reclaims are important ( #18900 )
...
* signal to upsert whether reclaims are important
* reclaims_must_be_empty -> previous_slot_entry_was_cached
* UPSERT_RECLAIMS_MUST_BE_EMPTY_FALSE -> UPSERT_PREVIOUS_SLOT_ENTRY_WAS_CACHED_FALSE
2021-08-09 08:58:59 -05:00
94c40ad3d2
chore:(deps): bump @solana/spl-token-registry in /explorer ( #19118 )
...
Bumps [@solana/spl-token-registry](https://github.com/solana-labs/token-list ) from 0.2.210 to 0.2.212.
- [Release notes](https://github.com/solana-labs/token-list/releases )
- [Changelog](https://github.com/solana-labs/token-list/blob/main/CHANGELOG.md )
- [Commits](https://github.com/solana-labs/token-list/compare/v0.2.210...v0.2.212 )
---
updated-dependencies:
- dependency-name: "@solana/spl-token-registry"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-09 08:07:34 +00:00
fd937548a0
Move SnapshotArchiveInfo and friends into its own module ( #19114 )
2021-08-08 07:57:06 -05:00
aa688e4549
fix typo ( #19111 )
2021-08-06 19:52:54 -07:00
00890957ee
Add snapshot_utils::bank_from_latest_snapshot_archives() ( #18983 )
...
While reviewing PR #18565 , as issue was brought up to refactor some code
around verifying the bank after rebuilding from snapshots. A new
top-level function has been added to get the latest snapshot archives
and load the bank then verify. Additionally, new tests have been
written and existing tests have been updated to use this new function.
Fixes #18973
While resolving the issue, it became clear there was some additional
low-hanging fruit this change enabled. Specifically, the functions
`bank_to_xxx_snapshot_archive()` now return their respective
`SnapshotArchiveInfo`. And on the flip side,
`bank_from_snapshot_archives()` now takes `SnapshotArchiveInfo`s instead
of separate paths and archive formats. This bundling simplifies bank
rebuilding.
2021-08-06 20:16:06 -05:00
7923c26939
AccountsDb::new_with_config_for_tests ( #19103 )
2021-08-06 17:18:16 -05:00
f771063275
get rid of Accounts new and default ( #19100 )
2021-08-06 15:52:27 -05:00
216a1b3d74
remove Bank::default() ( #19099 )
2021-08-06 13:10:13 -05:00
c12289fd1b
remove AccountsDb::new() from public api ( #19098 )
2021-08-06 13:07:50 -05:00
41f4973f0d
accounts default refactoring stragglers ( #19097 )
2021-08-06 12:36:42 -05:00
397801a2d8
Extract tower storage details from Tower struct
2021-08-06 10:04:37 -07:00
ca37873e16
rework bank::new_with_paths ( #19087 )
...
* rework bank::new_with_paths
* missing 1 bench
2021-08-06 09:30:40 -05:00
8878f526ce
rework AccountsIndex traits ( #19089 )
2021-08-06 08:39:34 -05:00
fcb4ccd413
chore:(deps): bump @types/react from 17.0.15 to 17.0.16 in /explorer ( #19094 )
...
Bumps [@types/react](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react ) from 17.0.15 to 17.0.16.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases )
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react )
---
updated-dependencies:
- dependency-name: "@types/react"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-06 08:08:42 +00:00
592013eaf4
Clean within shrink_all_slots ( #19042 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-08-05 23:26:38 -07:00
0028442e14
rework defaults and construction of accountsdb, accounts, bank ( #19083 )
...
* rework defaults and construction of accountsdb, accounts, bank
* merge issues
2021-08-05 17:27:13 -05:00
e4be00fece
falls back on working-bank if root-bank::epoch-staked-nodes is none
...
bank.get_leader_schedule_epoch(shred_slot)
is one epoch after epoch_schedule.get_epoch(shred_slot).
At epoch boundaries, shred is already one epoch after the root-slot. So
we need epoch-stakes 2 epochs ahead of the root. But the root bank only
has epoch-stakes for one epoch ahead, and as a result looking up epoch
staked-nodes from the root-bank fails.
To be backward compatible with the current master code, this commit
implements a fallback on working-bank if epoch staked-nodes obtained
from the root-bank is none.
2021-08-05 21:47:33 +00:00
eaf927cf49
allows only one thread to update cluster-nodes cache entry for an epoch
...
If two threads simultaneously call into ClusterNodesCache::get for the
same epoch, and the cache entry is outdated, then both threads recompute
cluster-nodes for the epoch and redundantly overwrite each other.
This commit wraps ClusterNodesCache entries in Arc<Mutex<...>>, so that
when needed only one thread does the computations to update the entry.
2021-08-05 21:47:33 +00:00
fb69f45f14
adds fallback & metric for when epoch staked-nodes are none
2021-08-05 21:47:33 +00:00
50d0e830c9
unifies cluster-nodes computation & caching across turbine stages
...
Broadcast-stage is using epoch_staked_nodes based on the same slot that
shreds belong to:
https://github.com/solana-labs/solana/blob/049fb0417/core/src/broadcast_stage/standard_broadcast_run.rs#L208-L228
https://github.com/solana-labs/solana/blob/0cf52e206/core/src/broadcast_stage.rs#L342-L349
But retransmit-stage is using bank-epoch of the working-bank:
https://github.com/solana-labs/solana/blob/19bd30262/core/src/retransmit_stage.rs#L272-L289
So the two are not consistent at epoch boundaries where some nodes may
have a working bank (or similarly a root bank) lagging other nodes. As a
result the node which obtains a packet may construct turbine broadcast
tree inconsistently with its parent node in the tree and so some packets
may fail to reach all nodes in the tree.
2021-08-05 21:47:33 +00:00
aa32738dd5
uses cluster-nodes cache in broadcast-stage
...
* Current caching mechanism does not update cluster-nodes when the epoch
(and so epoch staked nodes) changes:
https://github.com/solana-labs/solana/blob/19bd30262/core/src/broadcast_stage/standard_broadcast_run.rs#L332-L344
* Additionally, the cache update has a concurrency bug in which the
thread which does compare_and_swap may be blocked when it tries to
obtain the write-lock on cache, while other threads will keep running
ahead with the outdated cache (since the atomic timestamp is already
updated).
In the new ClusterNodesCache, entries are keyed by epoch, and so if
epoch changes cluster-nodes will be recalculated. The time-to-live
eviction policy is also encapsulated and rigidly enforced.
2021-08-05 21:47:33 +00:00
30bec3921e
uses cluster-nodes cache in retransmit stage
...
The new cluster-nodes cache will:
* ensure cluster-nodes are recalculated if the epoch (and so the epoch
staked nodes) changes.
* encapsulate time-to-live eviction policy.
2021-08-05 21:47:33 +00:00
ecc1c7957f
implements cluster-nodes cache
...
Cluster nodes are cached keyed by the respective epoch from which stakes
are obtained, and so if epoch changes cluster-nodes will be recomputed.
A time-to-live eviction policy is enforced to refresh entries in case
gossip contact-infos are updated.
2021-08-05 21:47:33 +00:00
44b11154ca
sends slots (instead of stakes) through broadcast flow
...
Current broadcast code is computing stakes for each slot before sending
them down the channel:
https://github.com/solana-labs/solana/blob/049fb0417/core/src/broadcast_stage/standard_broadcast_run.rs#L208-L228
https://github.com/solana-labs/solana/blob/0cf52e206/core/src/broadcast_stage.rs#L342-L349
Since the stakes are a function of epoch the slot belongs to (and so
does not necessarily change from one slot to another), forwarding the
slot itself would allow better caching downstream.
In addition we need to invalidate the cache if the epoch changes (which
the current code does not do), and that requires to know which slot (and
so epoch) current broadcasted shreds belong to:
https://github.com/solana-labs/solana/blob/19bd30262/core/src/broadcast_stage/standard_broadcast_run.rs#L332-L344
2021-08-05 21:47:33 +00:00
b67ffab370
Add more API documentation for Rust RpcClient ( #19021 )
...
* Add doc links to Transaction API docs
* Add more RpcClient API docs
* Reflow some rpc_client docs
* Update client/src/rpc_client.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update client/src/rpc_client.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update client/src/rpc_client.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update sdk/src/transaction.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update RpcClient docs per review
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-08-05 15:03:33 -06:00
e368f10973
add _for_tests to new_no_wallclock_throttle ( #19086 )
2021-08-05 14:50:25 -05:00
a9014ceceb
Bank::default_for_tests() ( #19084 )
2021-08-05 11:53:29 -05:00
24207a09ac
remove AccountsIndex::default ( #19082 )
...
* accounts_db calls AccountsDb::new(bins)
* remove AccountsIndex::default
2021-08-05 11:38:53 -05:00
5cf28689e6
accounts_db calls AccountsDb::new(bins) ( #19068 )
2021-08-05 11:15:26 -05:00
40914de811
updates cluster-slots with root-bank instead of root-slot + bank-forks
...
ClusterSlots::update is taking both root-slot and bank-forks only to
later lookup root-bank from bank-forks, which is redundant. Also
potentially by the time bank-forks is locked to obtain root-bank,
root-slot may have already changed and so be inconsistent with the
root-slot passed in as the argument.
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L32-L39
https://github.com/solana-labs/solana/blob/6d95d679c/core/src/cluster_slots.rs#L122
2021-08-05 14:43:06 +00:00
2fc112edcf
removes unused code from cluster-slots
2021-08-05 14:43:06 +00:00
67788ad206
move AccountsIndex upsert into static WriteAccountMapEntry ( #18899 )
...
* rework accounts index to push upsert deeper
* clean up return value of upsert_existing_key
* upsert_existing_key -> update_key_if_exists
* upsert_new_key -> upsert
* upsert_item -> lock_and_update_slot_list
* update_static -> update_slot_list
2021-08-05 08:45:08 -05:00
bf16b0517c
add _for_tests to setup_bank_and_vote_pubkeys ( #19060 )
2021-08-05 08:43:35 -05:00
087db70df6
add traits required by IsCached ( #19066 )
2021-08-05 08:43:00 -05:00
14361906ca
for all tests, bank::new -> bank::new_for_tests ( #19064 )
2021-08-05 08:42:38 -05:00
367d5f62ce
chore:(deps): bump @solana/spl-token-registry in /explorer ( #19077 )
...
Bumps [@solana/spl-token-registry](https://github.com/solana-labs/token-list ) from 0.2.208 to 0.2.210.
- [Release notes](https://github.com/solana-labs/token-list/releases )
- [Changelog](https://github.com/solana-labs/token-list/blob/main/CHANGELOG.md )
- [Commits](https://github.com/solana-labs/token-list/compare/v0.2.208...v0.2.210 )
---
updated-dependencies:
- dependency-name: "@solana/spl-token-registry"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-05 08:34:30 +00:00
6e285c42ea
chore:(deps): bump @types/node from 16.4.10 to 16.4.12 in /explorer ( #19076 )
...
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node ) from 16.4.10 to 16.4.12.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases )
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node )
---
updated-dependencies:
- dependency-name: "@types/node"
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-05 08:30:56 +00:00
80e770dfa0
chore: bump @babel/preset-env from 7.14.9 to 7.15.0 in /web3.js ( #19073 )
...
Bumps [@babel/preset-env](https://github.com/babel/babel/tree/HEAD/packages/babel-preset-env ) from 7.14.9 to 7.15.0.
- [Release notes](https://github.com/babel/babel/releases )
- [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md )
- [Commits](https://github.com/babel/babel/commits/v7.15.0/packages/babel-preset-env )
---
updated-dependencies:
- dependency-name: "@babel/preset-env"
dependency-type: direct:development
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-08-05 08:16:48 +00:00
dfe99efa7c
introduce AccountsIndex::default_for_tests() ( #19067 )
2021-08-04 21:58:53 -05:00
5a4979f25f
Handle 0-lamport account in index generation ( #19041 )
...
* Handle 0-lamport account in index generation
* rename duplicate to dirty keys
Co-authored-by: Carl Lin <carl@solana.com>
2021-08-04 23:33:47 +00:00
bde9b4de94
Bank::new -> Bank::new_for_benches ( #19063 )
2021-08-04 17:30:43 -05:00
3280ae3e9f
add validator option --accounts-db-skip-shrink ( #19028 )
...
* add validator option --accounts-db-skip-shrink
* typo
2021-08-04 17:28:33 -05:00
68cc71409e
Do not shell out for tar ( #19043 )
...
When making a snapshot archive, we used to shell out and call `tar -S`
for sparse file support. The tar crate supports sparse files, so no
need to do this anymore.
Fixes #10860
2021-08-04 17:07:55 -05:00
a1112254a5
Fix wrong old snapshot archives getting purged ( #19061 )
...
I introduced a bug where old snapshot archives were incorrectly purged.
Instead of purged to oldest, I was purged the newest...
The fix is to add a `reverse()` in the purge logic, and I've added a
test to catch this bug in the future.
Fixes #19057
2021-08-04 16:42:42 -05:00
Tyera Eulberg
Brian Anderson
Justin Starry
Jeff Washington (jwash)
dependabot[bot]
Brooks Prumo
0xlucius
Michael Vines
sakridge
behzad nouri