7b0bf64404
cleanup old features ( #19956 )
2021-09-17 09:46:49 -07:00
37d6f5013f
AcctIdx: initialize acct idx by config ( #19968 )
2021-09-17 16:39:41 +00:00
b8106e2450
chore: bump memmap2 from 0.3.1 to 0.4.0 ( #19974 )
...
* chore: bump memmap2 from 0.3.1 to 0.4.0
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.3.1...v0.4.0 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-09-17 10:00:12 -06:00
376b20c3d2
AcctIdx: add some bucket flush helpers ( #19965 )
2021-09-17 10:41:30 -05:00
4d8b3aa578
AcctsIdx: introduce dirty state per pubkey ( #19960 )
2021-09-16 21:36:23 -05:00
58f25a8752
AcctIdx: move background() to AccountsIndexStorage ( #19948 )
2021-09-16 17:52:06 -05:00
66e0fafc21
AccountsIndexStorage holds InMemAccountsIndex[] ( #19947 )
2021-09-16 21:12:22 +00:00
dc69cc1ae4
Only allow votes when root distance gets too high ( #19917 )
2021-09-16 15:12:26 +02:00
bd434d92e1
nit: misspelling ( #19929 )
2021-09-16 01:27:26 +00:00
fd60ef78eb
chore: bump flate2 from 1.0.21 to 1.0.22 ( #19863 )
...
* chore: bump flate2 from 1.0.21 to 1.0.22
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.21 to 1.0.22.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.21...1.0.22 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-15 14:30:30 -06:00
eddd583cd5
AcctIdx: pass bins to BucketMapHolder ( #19906 )
2021-09-15 18:07:53 +00:00
99f2c746d1
AcctIdx: Put InMemAcctIndex behind arc ( #19904 )
2021-09-15 17:36:08 +00:00
b467e7fb3b
AcctIdx: remove auto Debug trait ( #19905 )
2021-09-15 09:54:16 -05:00
8e3c420414
Add logging after taking a bank snapshot ( #19891 )
2021-09-15 07:39:21 -05:00
d3c80be7d5
AcctIdx: use stats() accessor ( #19890 )
2021-09-15 05:28:44 +00:00
691bea8776
extract map to map() ( #19883 )
2021-09-14 23:08:17 -05:00
229b378ed1
in-mem acct idx hash map uses rwlock ( #19878 )
2021-09-14 18:01:34 -05:00
fc647eed19
introduce trait IndexValue for accounts index ( #19859 )
2021-09-14 17:51:07 -05:00
21bc7e94f5
Add logging of incremental snapshots to purge_old_snapshot_archives() ( #19874 )
2021-09-14 17:49:15 -05:00
60276da1fd
rework acct idx entry to prepare for hashmap ( #19875 )
2021-09-14 15:27:23 -05:00
2a795614d3
accounts index keys returns by value ( #19858 )
2021-09-14 09:38:59 -05:00
00d7981f64
Fix native invoke writable privileges ( #19750 )
...
* Fix native invoke writable privileges
* build downstream spl bpf programs for tests
2021-09-13 22:57:37 -07:00
910f241c3f
add <T> through accounts index ( #19852 )
2021-09-13 22:59:03 -05:00
b57e86abf2
cache account hash info ( #19426 )
...
* cache account hash info
* ledger_path -> accounts_hash_cache_path
2021-09-13 20:39:26 -05:00
dca49a614f
size and per-bin stats ( #19842 )
2021-09-13 19:26:49 -05:00
87a7f00926
Track reset bank in PohRecorder ( #19810 )
2021-09-13 16:55:35 -07:00
3071c4caad
Set is_startup=true for exhaustively_free_unused_resource() ( #19846 )
2021-09-13 18:08:06 -05:00
e1a038e791
add stats inserts, deletes, updates_in_mem ( #19807 )
2021-09-13 15:59:01 -05:00
ef749a2506
Flush accounts cache before calling clean_accounts() ( #19839 )
2021-09-13 14:56:13 -05:00
c05226620f
metrics on accounts index entry ( #19806 )
2021-09-13 08:33:45 -05:00
3f6eb96d6e
Call `Bank::update_accounts_hash()` before taking bank snapshot ( #19765 )
...
This is a bugfix.
When `load_frozen_forks()` called `snapshot_bank()`, the accounts hash
was not computed. So then when a snapshot archive was eventually
created, its hash would be all 1s, which is clearly wrong. The fix is
to update the bank's accounts hash before taking the bank snapshot.
2021-09-13 08:27:50 -05:00
d388c5c257
move update_key_if_exists to InMemAccountsIndex ( #19803 )
2021-09-13 08:27:21 -05:00
3cea535fdb
InMemAccountsIndex::remove is no longer called ( #19805 )
2021-09-12 22:01:41 -05:00
b992c02708
in_mem_accounts_index filters by range ( #19779 )
2021-09-12 21:57:15 -05:00
0263ffb2ed
move upsert and a handful of helpers to InMemAccountsIndex ( #19799 )
2021-09-12 21:54:09 -05:00
3617d43e76
accounts index stats ( #19797 )
2021-09-12 22:14:59 +00:00
d9674f7ff0
consolidate another use of remove_if_slot_list_empty ( #19804 )
2021-09-12 19:05:44 +00:00
62c8bcf565
Add default() to SnapshotConfig ( #19776 )
2021-09-12 13:44:27 -05:00
c9a3b8941f
move insert_new_entry_if_missing_with_lock into InMemAccountsIndex ( #19800 )
2021-09-12 13:39:29 -05:00
a295febf04
move remove_if_slot_list_empty to InMemAccountsIndex ( #19801 )
2021-09-12 10:26:04 -05:00
361101bd31
rename generic V -> T. Matches intent better. V is now a larger type that contains T ( #19798 )
2021-09-12 10:25:09 -05:00
c9261e034e
remove dev deps
2021-09-11 14:33:53 -07:00
c3a610374a
unused deps
2021-09-11 14:33:53 -07:00
91c3b18e1e
InMemAccountsIndex iter returns vec<owned> ( #19778 )
2021-09-11 20:33:04 +00:00
7de2236284
accounts index get returns value ( #19777 )
2021-09-11 13:28:56 -05:00
ed7a6c6732
push use of AccountMapEntry one level deeper ( #19775 )
2021-09-11 02:09:51 +00:00
cba834808a
refactor generate_index zero lamports ( #19769 )
2021-09-10 23:46:08 +00:00
595bba95b4
accounts index traits ( #19768 )
2021-09-10 18:32:10 -05:00
9899cd359a
accounts index iterator uses copy of pubkey and account map entry ( #19767 )
2021-09-10 17:52:49 -05:00
11b10439b4
factor out InMemAccountsIndex to prepare for disk index ( #19773 )
2021-09-10 17:52:25 -05:00
7aa5f6b833
Add CLI args for incremental snapshots ( #19694 )
...
Add `--incremental-snapshots` flag to enable incremental snapshots.
This will allow setting `--full-snapshot-interval-slots` and
`--incremental-snapshot-interval-slots`.
Also added `--maximum-incremental-snapshots-to-retain`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-10 15:59:26 -05:00
2de2c6ddd7
log accounts loaded for shrink stats ( #19747 )
2021-09-10 12:51:58 -05:00
098585234d
Add return data implementation
...
This consists of:
- syscalls
- passing return data from invoked to invoker
- printing to stable log
- rust and C SDK changes
2021-09-10 14:25:54 +01:00
88c1b8f047
Index loaders / executable accounts ( #19469 )
...
* Appends loaders / executable_accounts to accounts in transaction loading.
* Adds indices to loaders / executable_accounts.
* Moves MessageProcessor::create_keyed_accounts() into InvokeContext::push().
* Removes "executable_accounts",
now referenced by transaction wide index into "accounts".
* Removes create_pre_accounts() from InstructionProcessor,
as it is already in MessageProcessor.
* Collect program account indices directly in load_executable_accounts().
2021-09-10 08:36:21 +02:00
df5befdaca
metrics on rent partitions ( #19728 )
2021-09-09 10:50:35 -05:00
8489ee7df9
Revert "collect rent in parallel for partitions ( #19478 )" ( #19723 )
...
This reverts commit 1560d5e3c3
2021-09-08 21:27:08 -05:00
97418bede3
introduce BinnedHashData type to improve some code readability ( #19722 )
2021-09-08 19:25:52 -05:00
1cf7ae51f8
add comment ( #19705 )
2021-09-08 19:25:20 -05:00
001ed66b4d
minor constant cleanup ( #19721 )
2021-09-08 18:26:10 -05:00
38bbb77989
Return error if Transaction contains writable executable or ProgramData accounts ( #19629 )
...
* Return error if Transaction locks an executable as writable
* Return error if a ProgramData account is writable but the upgradable loader isn't present
* Remove unreachable clause
* Fixup bpf tests
* Review comments
* Add new TransactionError
* Disallow writes to any upgradeable-loader account when loader not present; remove is_upgradeable_loader_present exception for all other executables
2021-09-08 15:21:52 -06:00
3a8c678f62
Remove some copying ( #19691 )
2021-09-08 18:32:38 +02:00
456bf15012
AccountsIndexConfig -> AccountsDbConfig ( #19687 )
2021-09-08 04:30:38 +00:00
d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it ( #19686 )
2021-09-07 20:09:40 -05:00
4a5f83d3a7
Add fn to check when to take snapshots ( #19682 )
2021-09-07 23:26:35 +00:00
a0552e5b46
Make startup aware of Incremental Snapshots ( #19600 )
2021-09-07 20:43:43 +00:00
982454a455
refactor do_shrink_slot_stores ( #19560 )
2021-09-07 15:10:49 -05:00
1560d5e3c3
collect rent in parallel for partitions ( #19478 )
...
* collect rent in parallel for partitions
* combine use
* use fg accounts db thread pool
2021-09-07 11:51:59 -05:00
9d9482b9d8
Plumb `maximum_incremental_snapshot_archives_to_retain` ( #19640 )
2021-09-06 18:01:56 -05:00
8b9e472a6c
feat: add ed25519 signature verify program
...
Solang requires a method for verify ed25519 signatures. Add a new
builtin program at address Ed25519SigVerify111111111111111111111111111
which takes any number of ed25519 signature, public key, and message.
If any of the signatures fails to verify, an error is returned.
The changes for the web3.js package will go into another commit, since
the tests test against a released solana node. Adding web3.js ed25519
testing will break CI.
2021-09-05 18:59:37 +01:00
d461a9ac10
verify_precompiles needs FeatureSet
...
Rather than pass in individual features, pass in the entire feature set
so that we can add the ed25519 program feature in a later commit.
2021-09-05 18:59:37 +01:00
333e5a9446
Make purge_old_snapshot_archives aware of maximum_incremental_snapshot_archives_to_retain ( #19615 )
2021-09-04 07:37:29 -05:00
decec3cd8b
Demote write locks on transaction program ids ( #19593 )
...
* Add feature
* Demote write lock on program ids
* Fixup bpf tests
* Update MappedMessage::is_writable
* Comma nit
* Review comments
2021-09-04 03:05:30 +00:00
7578db7ee3
add lowest_pubkey_from_bin ( #19617 )
2021-09-04 02:20:55 +00:00
b3fae0a01a
move constants in functions to global space ( #19622 )
2021-09-03 19:46:42 -05:00
e6055010eb
accounts index iterator takes &Range ( #19618 )
2021-09-03 18:00:49 -05:00
2c3bdedea3
stake: Advance `credits_observed` on activation epoch ( #19309 )
...
* stake: Advance `credits_observed` on activation epoch
* Add test for merging stakes just after activation
2021-09-03 23:20:56 +02:00
5e25ee5ebe
Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig ( #19612 )
2021-09-03 20:21:32 +00:00
7ab0aec61f
Rename maximum_full_snapshot_archives_to_retain ( #19610 )
...
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
2021-09-03 11:28:10 -05:00
df040c05da
get rid of unneeded mut and refactor addref ( #19594 )
2021-09-03 10:45:37 -05:00
170927b7c4
nit: Move builtins types to builtins file ( #19597 )
2021-09-02 21:29:11 -07:00
3936c10aa0
rename var ( #19588 )
2021-09-03 00:55:35 +00:00
9cc57fb076
remove file that accidentally got added ( #19586 )
2021-09-03 00:15:25 +00:00
e9374d32a3
Revert "Make startup aware of Incremental Snapshots ( #19550 )" ( #19599 )
...
This reverts commit d45ced0a5d
2021-09-02 19:14:41 -05:00
d45ced0a5d
Make startup aware of Incremental Snapshots ( #19550 )
2021-09-02 19:05:15 -05:00
57f51352f6
refactor AccountEntry addref/unref ( #19583 )
2021-09-02 23:25:27 +00:00
682daf1117
cleanup of allocate_accounts_index ( #19585 )
2021-09-02 22:58:07 +00:00
78585a992b
add IsCached trait bounds to AccountsIndex ( #19584 )
2021-09-02 22:45:35 +00:00
2874f6b0a2
create remove_if_slot_list_empty to define accounts index behavior ( #19581 )
2021-09-02 16:54:37 -05:00
0571962776
accounts index test uses # test bins ( #19580 )
2021-09-02 16:32:42 -05:00
6ab8286e8d
add logging for a handful of tests ( #19582 )
2021-09-02 16:32:19 -05:00
f5388cfe71
tests which assert or panic in other threads now will report an error ( #19532 )
...
* tests which assert or panic in other threads now will report an error
* rename, add assert detecting time out
2021-09-02 11:26:57 -05:00
7eb793a55e
add debugging to test ( #19563 )
...
* add debugging to test
* update cargo
2021-09-02 02:52:16 +00:00
ed2c071fe1
remove unused is_startup ( #19561 )
2021-09-01 18:39:00 -05:00
8378e8790f
Accountsdb replication installment 2 ( #19325 )
...
This is the 2nd installment for the AccountsDb replication.
Summary of Changes
The basic google protocol buffer protocol for replicating updated slots and accounts. tonic/tokio is used for transporting the messages.
The basic framework of the client and server for replicating slots and accounts -- the persisting of accounts in the replica-side will be done at the next PR -- right now -- the accounts are streamed to the replica-node and dumped. Replication for information about Bank is also not done in this PR -- to be addressed in the next PR to limit the change size.
Functionality used by both the client and server side are encapsulated in the replica-lib crate.
There is no impact to the existing validator by default.
Tests:
Observe the confirmed slots replicated to the replica-node.
Observe the accounts for the confirmed slot are received at the replica-node side.
2021-09-01 14:10:16 -07:00
a5f74d86c8
make test pass with 1 bin to remove noisy failure during testing ( #19554 )
2021-09-01 15:58:56 -05:00
dd9481c403
parallelize index_read in shrink ( #19506 )
2021-09-01 15:40:59 -05:00
82a6bbe068
add Debug trait to range for accounts index ( #19531 )
2021-09-01 08:13:56 -05:00
12dc8749cf
add Copy trait to AccountInfo for fast copies to mmapped file ( #19524 )
2021-09-01 08:13:08 -05:00
0088aefa24
Fix tests that make assumptions about tx fee rate ( #19537 )
2021-08-31 22:02:14 -06:00
1d5a8ebc6a
Revert "Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )" ( #19529 )
...
This reverts commit 4d361af976
2021-08-31 22:03:19 -05:00
Jack May
Jeff Washington (jwash)
dependabot[bot]
sakridge
Brooks Prumo
carllin
Giles Cope
Sean Young
Alexander Meißner
Tyera Eulberg
Jon Cinque
Brooks Prumo
Lijun Wang