1d02dba06f
remove unnecessary clone ( #17559 )
2021-05-28 10:30:01 -05:00
55c22d3b76
add check_hash to non-index hash calculation ( #17558 )
2021-05-28 10:24:40 -05:00
1953543274
Refactor bank_forks_utils::load() and some snapshot logic ( #17492 )
...
Refactor a few functions that are on the load-from-snapshot path, to facilitate
adding in incremental snapshots more easily.
Additionally, add some tests and doc comments.
2021-05-27 12:00:27 -05:00
8eab0e8602
Bump version to v1.8.0 ( #17541 )
2021-05-27 08:51:53 -07:00
49402b7d82
Only add keys with multiple index entries to the clean set ( #17473 )
2021-05-27 16:31:24 +02:00
3973bf08f1
move local out of call ( #17519 )
2021-05-27 08:53:11 -05:00
9541411c15
Plumb transaction-level rewards (aka "rent debits") into the `getTransaction` RPC method
2021-05-27 03:05:05 +00:00
cbce440af4
simulateTransaction can now return accounts modified by the simulation
2021-05-26 14:20:23 -07:00
3f9e3c7375
Remove copied shrink code ( #17385 )
2021-05-26 19:27:18 +02:00
6b9d8d41a3
add --limit_load_slot_count_from_snapshot to ledger-tool ( #17417 )
2021-05-26 10:36:12 -05:00
2a6c5ed0ac
add tests from `RentDebits`
2021-05-26 08:19:10 +00:00
97eab7edf9
runtime: add rent debit charges to block metadata
2021-05-26 08:19:10 +00:00
e9bc1c6b07
Add last valid block height to rpc Fees ( #17506 )
...
* Add last_valid_block_height to fees rpc
* Add getBlockHeight rpc
* Update docs
2021-05-26 07:26:19 +00:00
a5c2067273
fixup! Ancestors to HashSet: From(Vec) ( #17447 ) ( #17481 )
2021-05-25 12:24:34 -05:00
98f1b11edd
Ancestors to HashSet: From(Vec) ( #17447 )
...
* ancestors.large_range_slots becomes HashSet
* Ancestors to HashSet: From(Vec)
2021-05-25 11:01:11 -05:00
d39a327138
ancestors.large_range_slots becomes HashSet ( #17446 )
2021-05-25 11:00:50 -05:00
40bb6577fc
initialize purge stats ( #17345 )
2021-05-25 09:24:38 -05:00
d4fc9e7ed4
ad metric for collect ( #17418 )
2021-05-24 18:14:52 -05:00
dd22ae047b
ancestors is set instead of map ( #17363 )
2021-05-24 17:01:02 -05:00
d8bc56fa51
Refactor purge_slots_from_cache_and_store() and handle_reclaims() ( #17319 )
2021-05-24 13:51:17 -07:00
ad1f24d487
runtime: Executor usage counts retain only single-epoch memory ( #17162 )
2021-05-24 12:01:56 -07:00
a8dca3976b
Refactor genesis download/load/check functions ( #17276 )
...
* Refactor genesis ingest functions
* Consolidate genesis.bin/genesis.tar.bz2 references
2021-05-24 16:45:36 +02:00
855ae79598
Fix InvokeContext::push() account_deps ( #17350 )
...
* Reverts aliasing of account_deps with the previous invocation stack frame in InvokeContext::push().
* Adds explicit assert of programdata account content in test_program_bpf_upgrade_via_cpi.
2021-05-21 22:34:07 +02:00
3f3324231d
Hold read lock during startup shrink ( #17309 )
...
* hold read lock during account scan of shrink
* rename and improve rusty
2021-05-21 13:59:32 -05:00
44831c18d2
reuse work in is_non_loader_key ( #16521 )
2021-05-21 10:07:04 -05:00
8143ee5502
add AccountsIndex AccountMapsReadLock type ( #17365 )
2021-05-21 10:02:14 -05:00
2f55547d37
remove flatten in storage scan ( #17354 )
...
* remove flatten in storage scan
* fix tests
2021-05-21 09:26:46 -05:00
2c99b23ad7
Add get_sysvar() helper to sdk
2021-05-21 01:30:12 +00:00
45552d271a
take mutex once in process_dead_slots ( #17357 )
2021-05-20 19:50:58 -05:00
0486df02ba
don't limit to thread pool when cleaning on startup ( #17317 )
2021-05-20 14:36:35 -05:00
33ab9c4e8d
batch insert account_index items in generate_index ( #17290 )
2021-05-20 10:29:13 -05:00
9b74988fc6
calculate_accounts_hash does not throttle threads on startup ( #17338 )
2021-05-20 10:25:54 -05:00
a9aa533684
replace Ancestors HashMap for performance ( #16477 )
...
* replace Ancestors HashMap for performance
* add ancestors module
2021-05-20 10:11:56 -05:00
a544010b03
don't log shrink metrics on first call ( #17328 )
...
* don't log shrink metrics on first call
* simplify logic
2021-05-20 00:15:35 +00:00
32ec8341f9
generate_index inserts ideal initial data ( #17247 )
...
* improve insert into map initially
* rework towards single code path
* rename
* update test
2021-05-19 16:21:24 -05:00
b5302e7619
add calc_stored_meta_us metric ( #17318 )
2021-05-19 12:29:41 -05:00
c20b27bc8f
shink all in parallel on startup ( #17308 )
2021-05-19 12:15:24 -05:00
ed9cbd50f0
move Ancestors to its own module ( #17316 )
2021-05-19 11:50:34 -05:00
0781fe1b4f
Upgrade Rust to 1.52.0 ( #17096 )
...
* Upgrade Rust to 1.52.0
update nightly_version to newly pushed docker image
fix clippy lint errors
1.52 comes with grcov 0.8.0, include this version to script
* upgrade to Rust 1.52.1
* disabling Serum from downstream projects until it is upgraded to Rust 1.52.1
2021-05-19 09:31:47 -05:00
4788976517
rework slot list update ( #17232 )
2021-05-19 08:48:46 -05:00
7fe24c455c
simplify generate_index ( #17292 )
2021-05-19 08:11:41 -05:00
827355a6b1
Create solana-rpc crate and move subscriptions ( #17320 )
...
* Move non_circulating_supply to runtime
* Add solana-rpc crate and move max_slots
* Move subscriptions to solana-rpc
* Single use statements
2021-05-19 00:54:28 -06:00
a3c0833a1c
tests: cleanup test ( #17305 )
...
* tests: cleanup test
* 2 more replacements
2021-05-18 18:08:37 -05:00
75335b4f58
datapoint for verify_snapshot_bank ( #17306 )
2021-05-18 12:14:43 -05:00
2175fc0625
calculate account infos before lock ( #17245 )
2021-05-18 08:06:19 -05:00
3a56a56d69
StoredMetaWriteVersion ( #17293 )
2021-05-17 23:58:36 +00:00
bcbe155575
Minor test cleanup and comments ( #17283 )
2021-05-18 07:22:43 +09:00
d461aba6a4
don't try to build secondary indexes if we don't need them ( #17246 )
2021-05-17 17:21:15 -05:00
9d6837c904
fix comment ( #17286 )
2021-05-17 14:06:25 -05:00
d1db5448b9
hold lock to speed up insert ( #17194 )
...
* hold lock to speed up insert
* add tests
2021-05-17 11:58:33 -05:00
3745e0babc
minor refactoring of AccountsIndex in preparation of bigger things ( #17237 )
2021-05-14 15:27:10 -05:00
fd88db7339
ancestors::new -> default ( #17195 )
2021-05-14 15:20:59 -05:00
c6f7867aa1
log mismatch ( #17209 )
2021-05-14 09:50:11 -05:00
76dfacca61
Add documentation for BankForks::prune_non_rooted() ( #17213 )
2021-05-14 07:47:48 -05:00
27004f1b76
Return error for excluded secondary-index keys ( #17193 )
...
* Add runtime helpers to check secondary indexes for key
* Add custom rpc error
* Check secondary-index key inclusion in rpc
* Clone complete AccountSecondaryIndexes into rpc to avoid bank query
2021-05-13 21:04:21 +00:00
3dbc7744ab
metrics for generating index time ( #17192 )
...
* metrics for generating index time
* update metrics to include scan time
2021-05-13 14:32:19 -05:00
239ab8799c
Remove bloat from secondary indexes ( #17048 )
2021-05-12 15:29:30 -07:00
9c42a89a43
Issue #17008 -- make snapshot archives to hold on to configurable. ( #17158 )
...
* purge_old_snapshot_archives is changed to take an extra argument 'maximum_snapshots_to_retain' to control the max number of latest snapshot archives to retain. Note the oldest snapshot is always retained as before and is not subjected to this new options.
* The validator and ledger-tool executables are modified with a CLI argument --maximum-snapshots-to-retain. And the options are propagated down the call chains. Their corresponding shell scripts were changed accordingly.
* SnapshotConfig is modified to have an extra field for the maximum_snapshots_to_retain
* Unit tests are developed to cover purge_old_snapshot_archives
2021-05-12 10:32:27 -07:00
7d96f78821
include/exclude keys on account secondary index ( #17110 )
...
* AccountSecondaryIndexes.include/exclude
* use normal scan if key is not indexed
* add a test to ask for a scan for an excluded secondary index
* fix up cli args
2021-05-11 22:06:22 +00:00
f39dda00e0
type AccountSecondaryIndexes = HashSet ( #17108 )
2021-05-10 14:22:48 +00:00
a6a1355b80
fix secondary index test ( #17111 )
2021-05-10 08:54:30 -05:00
c616c34825
Add comment for another unspoken eager collection subtlety ( #17137 )
2021-05-10 22:44:46 +09:00
225ec00ec4
add comment ( #17059 )
2021-05-06 15:17:41 -05:00
dc0429f5e6
add metric for assumption ( #17061 )
2021-05-06 15:04:13 -05:00
edc2ab3e48
Test account index and store alignment ( #17038 )
...
* Use ReclaimResult::Default() instead of building subtypes
* Add test to ensure account_db store and index are aligned
2021-05-06 14:13:44 -05:00
d19526e6c2
Dump rent_collector/inflation with ledger-tool cap ( #17069 )
2021-05-06 19:29:46 +09:00
ffbe8906ed
get root lock once ( #16829 )
2021-05-05 15:17:45 -05:00
e6f49a3e79
Clean unrooted unfrozen banks ( #16580 ) ( #17003 )
...
Cleanup pubkeys when an unrooted, unfrozen bank is dropped. This is a
continuation of PR #16911 .
2021-05-05 15:02:02 -05:00
ab7c96aa81
insert accounts in parallel when building initial index ( #17040 )
...
* insert accounts in parallel when building initial index
* rename nits from pr review
* rename nits from pr review
* rename nits from pr review
* rename nits from pr review
2021-05-05 17:08:45 +00:00
3e0fed48e7
Don't recognize temp snapshots as possible snapshots to open
2021-05-05 08:45:03 -07:00
5786be13a4
flatten_hash_intermediate sets capacity first ( #17013 )
...
* flatten_hash_intermediate sets capacity first
* use iterator instead of for
2021-05-05 09:07:05 -05:00
d94e4ef7c9
More docs for bank.rs ( #17006 )
2021-05-03 17:37:18 +00:00
541aa5ad85
tests: lamports -> lamports() ( #16982 )
2021-05-03 10:45:54 -05:00
744ac1535f
tests: lamports -> ReadableAccount ( #16983 )
2021-05-03 10:45:15 -05:00
1fe131a5bb
Readable account trait for stored account meta ( #16690 )
...
* ReadableAccount for StoredAccountMeta
* add test
2021-04-30 16:25:02 -05:00
1a9954f85b
bank deposit checked_add error ( #16917 )
...
* bank deposit checked_add error
* add id
* rename variables
* rename error and metric
2021-04-30 16:22:17 -05:00
01308cd890
distribute_rent_to_validators checked_add_lamports unwrap ( #16847 )
...
* distribute_rent_to_validators checked_add_lamports unwrap
* make rent disappear on add failure
* add pubkey to message
* update message text
* don't store account that we failed to transfer to
* format
2021-04-30 16:19:20 -05:00
763c04adf3
lamports = -> .set_lamports() ( #16980 )
2021-04-30 21:17:05 +00:00
17e6bd579f
Clean unrooted dropped banks ( #16580 ) ( #16911 )
...
In a scenario where a bank is unrooted and dropped, any keys that exist
_only_ in that bank are now cleaned up.
This work was originally based on PR #15106 .
2021-04-30 15:34:38 -05:00
6f88aeac56
tests: .lamports= -> .set_lamports() ( #16977 )
...
* .lamports= -> .set_lamports()
* another
* fix
* another
2021-04-30 18:44:15 +00:00
0af84bb91e
lamports= -> .set_lamports() ( #16978 )
2021-04-30 18:20:54 +00:00
c4943f3cb0
checked_sub_lamports ( #16932 )
2021-04-30 09:56:34 -05:00
63c56b57a9
checked_sub_lamports with unwrap ( #16922 )
...
* checked_sub_lamports with unwrap
* avoid unwrap
2021-04-29 15:04:28 -05:00
c9c94a7ef2
checked_sub_lamports ( #16923 )
2021-04-29 12:38:21 -05:00
ca7b36ad8f
lamports -> lamports() ( #16920 )
2021-04-29 10:44:46 -05:00
23d67e4ac7
stretchy roots tracker ( #16830 )
...
* stretchy roots tracker
* rename hash to hash_set in tests
* update comment
* try 2 widths in test
* bool iter
* add assert
* helper function for bitfield insert/remove
* introduce RollingBitFieldTester
* another bool iter replacement
* map cleanup
* map to cloned
2021-04-29 09:11:28 -05:00
a7070a5ca7
set_lamports() ( #16921 )
2021-04-28 21:30:06 +00:00
21c75d9d29
lamports = -> set_lamports ( #16919 )
2021-04-28 15:39:54 -05:00
f533d3be77
Write account path impl ReadableAccount ( #16779 )
2021-04-28 15:29:22 -05:00
9070191b8b
stats on clean for removing roots ( #16849 )
...
* stats on clean for removing roots
* rename
* accumulate and swap metrics
2021-04-28 13:24:01 -05:00
77f1ffd84b
Collect uncleaned pubkeys from all slots ( #16786 )
...
While working on another issue (#16580 ), the list of uncleaned pubkeys
returned from `remove_uncleaned_slots_and_collect_pubkeys_up_to_slot()`
did not include unrooted slots. This meant that during cleaning,
unrooted slots would not have their pubkeys cleaned up properly.
Now, return all uncleaned pubkeys, regardless if the slot is rooted or
not. Additionally, update the tests to have unrooted slots to ensure
this behavior.
This is part two of PR #16879 , and originally based on PR #15106 .
2021-04-28 10:57:42 -05:00
a7a671b3aa
AccountSharedData.set_executable() ( #16881 )
2021-04-28 14:07:43 +00:00
2021255f91
executable() ( #16880 )
2021-04-28 08:53:54 -05:00
da3342759b
private AccountSharedData.rent_epoch ( #16877 )
2021-04-28 08:52:20 -05:00
6381ee38eb
reclaims unref accounts from index ( #16838 )
2021-04-28 08:50:38 -05:00
1864bc2080
write Option<AccountSharedData> ( #16874 )
...
* write Option<&AccountSharedData>
* add comment
2021-04-28 08:47:26 -05:00
1eaff394da
Refactor `collect_uncleaned_pubkeys_to_slot()` ( #16879 )
...
* Refactor `collect_uncleaned_pubkeys_to_slot()`
While working on another issue (#16580 ), I came across
`collect_unclean_pubkeys_to_slot()` and had difficulty understanding it.
Since the function does a few logically separate things, I split the
function up. I also added documentation, removed an unused return value,
and renamed the functions to be more specific.
This commit is to split up an existing PR (#16786 ), where I had both this
aesthetic change _and_ a behavioral change.
2021-04-28 08:16:12 -05:00
1bd623cd15
private AccountSharedData.rent_epoch ( #16844 )
2021-04-27 13:51:13 -05:00
8f56c116d7
tests: lamports += to checked_add ( #16842 )
2021-04-27 10:23:07 -05:00
d533f77301
bank.withdraw uses checked_sub_lamports ( #16848 )
...
* bank.withdraw uses checked_sub_lamports
* retain previous codepath for error
* map_err from clippy
2021-04-27 09:58:41 -05:00
Jeff Washington (jwash)
Brooks Prumo
Michael Vines
sakridge
Trent Nelson
Tyera Eulberg
steviez
carllin
jon-chuang
Alexander Meißner
Tao Zhu
Ryo Onodera
Lijun Wang