Jeff Washington (jwash)
ebea3297f9
AcctIdx: generate index inserts/updates directly to disk ( #21363 )
...
* when initially creating account index, write directly to disk
* AcctIdx: generate index inserts/updates directly to disk
2021-11-19 17:17:07 -06:00
Brooks Prumo
0bb059185c
Add lamports_in_rent_paying_accounts to TotalAccountsStats ( #21368 )
2021-11-19 17:02:23 -06:00
Alexander Meißner
8a50b6302f
Fix #21346 ( #21362 )
...
Fixes the empty transaction bug in ThisInvokeContext::push() and adds a test for it to the bank.
2021-11-19 20:43:42 +01:00
Jeff Washington (jwash)
79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort ( #21327 )
...
* ScanConfig -> &ScanConfig
* add --accounts-index-scan-results-limit-mb to allow scans to abort
* feedback
2021-11-19 09:00:19 -06:00
Brooks Prumo
c24e30f689
Constrain memory ordering on AccountsDb::next_id ( #21311 )
2021-11-18 20:34:37 -06:00
Brooks Prumo
f7152c889c
Constrain memory ordering on AppendVec::id ( #21310 )
2021-11-18 20:34:20 -06:00
Brooks Prumo
5788973153
Constrain memory ordering on AccountStorageEntry::alive_bytes ( #21309 )
2021-11-18 20:34:08 -06:00
Jack May
03c36d240a
Nonce naming cleanup ( #21336 )
2021-11-18 16:07:17 -08:00
carllin
b30c94ce55
ClusterInfoVoteListener send only missing votes to BankingStage ( #20873 )
2021-11-18 15:20:41 -08:00
Jeff Washington (jwash)
a272e19f8d
AcctIdx: report on partial progress during flushing ( #21306 )
2021-11-18 11:01:19 -06:00
Lijun Wang
89c45a57f8
Refactor slot status notification to decouple from accounts notifications ( #21308 )
...
Problem
Slot status can be used of in other scenarios in addition to account information such as transactions, blocks. The current implementation is too tightly coupled.
Summary of Changes
Decouple the slot status notification from accounts notification. Created a new slot status notification module.
2021-11-17 17:11:38 -08:00
Jeff Washington (jwash)
9cb74e8421
remove store_cached stall ( #21326 )
2021-11-17 18:30:40 -06:00
Jeff Washington (jwash)
83de2f7376
fix initial massive metrics spike ( #21304 )
2021-11-17 17:16:48 -06:00
Jeff Washington (jwash)
0f69a14247
Add ability to abort scan ( #21314 )
2021-11-17 13:10:29 -06:00
Alexander Meißner
e540b1cf3c
Refactor: Move sdk::process_instruction in program-runtime-crate ( #21180 )
...
* Moves the Executor dyn Trait to instruction_processor.rs
* Moves the Logger dyn Trait as well as the ic_msg and ic_logger_msg macros to log_collector.rs,
and moves the stable_log to stable_log.rs
* Moves the ComputeMeter dyn Trait to invoke_context.rs
* Moves the InvokeContext dyn Trait and the ProcessInstructionWithContext type to invoke_context.rs
* Updates cargo files.
* Re-export InvokeContext in program-test
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-11-17 19:35:07 +01:00
Brooks Prumo
c3e5927d16
Constrain memory ordering on AccountsDb::write_version ( #21312 )
2021-11-17 07:11:16 -06:00
Jack May
cb0bb5bd1e
Nonce accounts must be writeable ( #21260 )
...
* Nonce accounts must be writeable
* feedback
* feedback
2021-11-16 23:01:00 +00:00
Jeff Washington (jwash)
ae497715cc
HashCalc: Perf: use refs for hashes ( #21280 )
2021-11-16 10:30:55 -06:00
sakridge
398af132a5
More set_root metrics ( #21286 )
2021-11-15 16:28:18 -07:00
Ivan Mironov
9b1bf98aa2
Accept only decimal digits in file names inside snapshot ( #21213 )
...
This also should make snapshot validation a bit faster.
2021-11-15 17:02:15 +00:00
Jeff Washington (jwash)
39340ed25b
throttle store_cached when cache size is too large ( #21188 )
...
* throttle store_cached when cache size is too large
* reduce max delay
* 100ms max
* 10ms max delay
2021-11-13 14:00:37 -06:00
Jeff Washington (jwash)
f8dcb2f38b
report mem stats ( #21258 )
2021-11-13 00:59:41 +00:00
Jack May
778b5d736c
Load nonce with fixed root ( #21248 )
2021-11-12 14:12:34 -08:00
Jeff Washington (jwash)
1509513339
hash calc: filler_accounts_enabled ( #21243 )
2021-11-12 09:53:21 -06:00
Tao Zhu
11153e1f87
refactor cost calculation ( #21062 )
...
* - cache calculated transaction cost to allow sharing;
- atomic cost tracking op;
- only lock accounts for transactions eligible for current block;
- moved qos service and stats reporting to its own model;
- add cost_weight default to neutral (as 1), vote has zero weight;
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update core/src/qos_service.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update core/src/qos_service.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-12 01:04:53 -06:00
Justin Starry
ef29d2d172
Refactor vote state to remove double negative ( #21244 )
2021-11-12 00:26:43 +00:00
Jack May
131c0d78c4
Fix compute budget bump ( #21238 )
2021-11-11 14:09:28 -08:00
Jeff Washington (jwash)
c0f9e351db
hashcalc eliminate zeros optimizations ( #21209 )
2021-11-11 09:44:32 +00:00
Brooks Prumo
8b2133b078
Refactor total stats calculation into shared fn ( #21212 )
2021-11-09 19:13:37 -06:00
dependabot[bot]
48265ada39
chore: bump blake3 from 1.1.0 to 1.2.0 ( #21203 )
...
* chore: bump blake3 from 1.1.0 to 1.2.0
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.1.0...1.2.0 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Make versions consistent
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-11-08 15:27:37 -07:00
Jeff Washington (jwash)
5827cf4c51
hash calc eliminate zero refactor get_item ( #21036 )
2021-11-08 20:27:48 +00:00
carllin
706b60b5c8
Add `set_root` bank drop logging ( #21144 )
2021-11-06 01:05:36 +00:00
Jeff Washington (jwash)
1b46d1d54d
hashcalc eliminate zeros cleanup ( #21175 )
2021-11-05 10:54:49 -05:00
Ivan Mironov
c78f474373
Add validator option to change niceness of snapshot packager thread
2021-11-04 17:16:46 -06:00
Alexander Meißner
7200c5106e
Replaces MockInvokeContext by ThisInvokeContext in tests ( #20881 )
...
* Replaces MockInvokeContext by ThisInvokeContext in BpfLoader, SystemInstructionProcessor, CLIs, ConfigProcessor, StakeProcessor and VoteProcessor.
* Finally, removes MockInvokeContext, MockComputeMeter and MockLogger.
* Adjusts assert_instruction_count test.
* Moves ThisInvokeContext to the program-runtime crate.
2021-11-04 21:47:32 +01:00
Justin Starry
0597594943
Fix rent debits test ( #21177 )
2021-11-04 20:03:19 +00:00
Jack May
4e527b84b0
Remove unused upgrade epoch handling ( #21164 )
2021-11-04 10:44:36 -07:00
Jeff Washington (jwash)
171243a370
add rent exempt count to get_total_accounts_stats ( #21153 )
2021-11-04 11:17:19 -05:00
Jeff Washington (jwash)
e765542cd8
eliminate pubkey copy in eliminate_zeros ( #21032 )
...
* rework eliminate zero loop
* eliminate pubkey copy in eliminate_zeros
2021-11-04 11:16:36 -05:00
Jeff Washington (jwash)
76e533be46
LoadedAccountAccessor::Cached does not store Pubkey ( #21030 )
2021-11-04 10:28:04 -05:00
Justin Starry
aaccba8377
Refactor `RentDebits` to use hashmap instead of vec
2021-11-04 15:02:24 +00:00
Jeff Washington (jwash)
b1e3a82c11
refactor rent collector and logs ( #21155 )
2021-11-04 06:28:41 -05:00
Alexander Meißner
29ad081555
Stop caching sysvars, instead load them ahead of time. ( #21108 )
2021-11-04 09:48:34 +01:00
Brooks Prumo
c326f41dfe
Add bank::get_total_accounts_stats() ( #21159 )
2021-11-03 19:23:45 -05:00
Jeff Washington (jwash)
c59c56c0f8
log rent exempt accounts ( #21137 )
2021-11-03 14:44:52 -05:00
Jeff Washington (jwash)
c0952831be
correctly clean filler accounts ( #21139 )
2021-11-03 11:27:15 -05:00
Justin Starry
140a5f633d
Simplify replay vote tracking by using packet metadata ( #21112 )
2021-11-03 09:02:48 +00:00
sakridge
1e5212e60d
Add set_root timing metrics ( #21119 )
2021-11-02 18:23:35 +01:00
Jack May
bced07a099
Update fee api to use blockhash ( #21054 )
2021-10-29 13:52:59 -07:00
Jack May
4aa12a52b6
Feature cleanup ( #21038 )
2021-10-28 09:04:03 -07:00
Jeff Washington (jwash)
7a41b2c5d1
optimize get_accounts_delta_hash ( #21027 )
2021-10-28 10:40:32 -05:00
Jeff Washington (jwash)
e9ab214237
filler accts: only add filler accts to slots in the current epoch ( #21024 )
2021-10-28 10:26:30 -05:00
Jeff Washington (jwash)
6a1ff60c61
percent stats 0-1 -> 0-100 ( #21028 )
2021-10-27 18:08:27 -05:00
Justin Starry
036d7fcc81
Clean up sanitized tx creation for tests ( #21006 )
2021-10-27 18:09:16 +01:00
Jeff Washington (jwash)
9d330fc638
FillerAccts: use variable cycle partitions ( #20963 )
2021-10-27 11:18:27 -05:00
Jack May
1e2bef76e3
Add compute budget program as a noop ( #20992 )
2021-10-26 20:08:59 -07:00
Jeff Washington (jwash)
4d6190a1ba
change bg waiting metrics to percent ( #20955 )
...
* Revert part of "AcctIdx: cleanup bg threads (#20731 )"
This reverts part of commit 47de4f31b6
.
* change bg waiting metrics to percent
2021-10-26 15:09:17 -05:00
Jack May
2515f6a04f
Update deprecation versions ( #20959 )
2021-10-26 09:06:41 -07:00
Lijun Wang
bbe3ce3db5
Accountsdb plugin write ordering ( #20948 )
...
Use the write_version in the Accounts's meta data so that account write with lower write_version would not overwrite the higher ones.
2021-10-25 14:07:56 -07:00
Alexander Meißner
cf0fd5b2ca
Moves NonceKeyedAccount from the SDK to the Runtime. ( #20954 )
2021-10-25 22:58:18 +02:00
Jeff Washington (jwash)
9e85499d4c
fix hash calc # storages, # slots ( #20832 )
2021-10-25 19:37:50 +00:00
Jeff Washington (jwash)
43ea579f63
add cli for --accounts-hash-num-passes ( #20827 )
2021-10-25 09:45:46 -05:00
Tao Zhu
c2bfce90b3
- cost_tracker is data member of a bank, it can report metrics when bank is frozen ( #20802 )
...
- removed cost_tracker_stats and histogram
- move stats reporting outside of bank freeze
2021-10-24 22:19:23 -05:00
Lijun Wang
f14365f4b4
Accountsdb plugin postgres -- bulk insertion at startup ( #20763 )
...
Use bulk insertion to Postgres at startup to reduce time taken for initial snapshot restore for postgres plugin. Avoid duplicate writes of accounts at startup. Doing account plugin notification and indexing in parallel.
Improved error handling for postgres plugin to show the real db issues for debug purpose
Added more metrics for postgres plugin.
Refactored plugin centric code out to a sub module from accounts_db and added unit tests
2021-10-24 12:43:33 -07:00
Jack May
bfbbc53dac
Divorce the runtime from FeeCalculator ( #20737 )
2021-10-22 14:32:40 -07:00
Justin Starry
735016661b
Report timing info for stakes cache updates from txs ( #20856 )
2021-10-22 12:49:02 -04:00
dependabot[bot]
1d9c1ccfd2
chore: bump blake3 from 1.0.0 to 1.1.0 ( #20877 )
...
* chore: bump blake3 from 1.0.0 to 1.1.0
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.0.0...1.1.0 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-22 09:44:42 -06:00
Alexander Meißner
97c2732d02
Refactor: Cleanup InvokeContext ( #20785 )
...
* Move blockhash and fee_calculator in ThisInvokeContext instead of using a reference.
* Moves tx_wide_compute_cap into InvokeContext::push().
* Adds ThisInvokeContext::new_mock() constructor.
* Adds missing loader account in uses of MockInvokeContext.
* Use keyed_account_at_index() when accessing keyed_accounts.
* Makes sysvar interface consistent between ThisInvokeContext and MockInvokeContext,
in order to add InvokeContext::get_sysvars().
* Adds InvokeContext::set_blockhash() and InvokeContext ::set_fee_calculator().
* Adds new_mock_with_features.
* Makes ancestors optional in ThisInvokeContext.
* Adds prepare_mock_invoke_context() and mock_process_instruction().
2021-10-21 20:57:42 +02:00
Jack May
a8098f37d0
add checked instructions sysvar api ( #20790 )
2021-10-19 21:01:58 -07:00
Jack May
58164517e4
Add program heap bump instruction ( #20607 )
2021-10-19 21:01:39 -07:00
Jon Cinque
cb2bd65858
runtime: Add foundation stake pool withdraw authority ( #20797 )
2021-10-19 21:49:21 +00:00
Jeff Washington (jwash)
47a58a38c2
clean_accounts calls AcctIdx: get_many ( #20715 )
...
* AcctIdx: get_many
* keep read lock
* AcctIdx: get_many optionally adds to cache
* rename
2021-10-19 15:54:06 -05:00
Jeff Washington (jwash)
0acbfdfcb9
hash calc: sort time is redundantly counted as part of 'total time' ( #20791 )
2021-10-19 15:52:43 -05:00
Tao Zhu
7496b5784b
- make cost_tracker a member of bank, remove shared instance from TPU; ( #20627 )
...
- decouple cost_model from cost_tracker; allowing one cost_model
instance being shared within a validator;
- update cost_model api to calculate_cost(&self...)->transaction_cost
2021-10-19 14:37:33 -05:00
Michael Vines
2b76ea51b4
Reduce visibility of `Hash` struct contents
2021-10-19 09:30:47 -07:00
Jeff Washington (jwash)
46bf7d4a4a
AcctIdx: support 2^24 bins ( #20739 )
2021-10-18 14:05:16 -05:00
Brooks Prumo
751b45df64
Make pub get_incremental_snapshot_archives() ( #20725 )
2021-10-18 13:48:50 -05:00
sakridge
c9b367c350
Minimum of 1 lamport for rent ( #20650 )
2021-10-18 19:09:53 +02:00
Jeff Washington (jwash)
6ec159a1ae
parallelize update_index ( #20601 )
2021-10-18 09:07:01 -05:00
Jeff Washington (jwash)
0d934d311e
sort shrink acct indexes ( #20708 )
2021-10-18 09:06:32 -05:00
Alexander Meißner
e98cfbb570
Removes deprecated BpfComputeBudget. ( #20728 )
2021-10-18 09:55:41 +02:00
Jeff Washington (jwash)
70b2e5fef2
hash calc uses all threads during startup ( #20735 )
2021-10-15 17:41:23 -05:00
Jeff Washington (jwash)
47de4f31b6
AcctIdx: cleanup bg threads ( #20731 )
2021-10-15 16:15:11 -05:00
Jeff Washington (jwash)
254ad45878
sort pubkeys for clean ( #20706 )
2021-10-15 14:31:06 -05:00
Jeff Washington (jwash)
6d7da6dbee
clean:clone uncleaned_roots for loop ( #20714 )
2021-10-15 13:18:00 -05:00
Jeff Washington (jwash)
5a6d732d48
AcctIdx: separate account index by startup and !startup ( #20711 )
2021-10-15 13:04:32 -05:00
Jeff Washington (jwash)
9d62963240
AcctIdx: get_internal allows not adding to in_mem ( #20716 )
2021-10-15 13:04:05 -05:00
Jeff Washington (jwash)
1e064c32e2
pull append_vec_id() out of loop ( #20713 )
2021-10-15 10:41:23 -05:00
Jeff Washington (jwash)
00b9056004
clean metric: total_us ( #20707 )
2021-10-15 10:41:05 -05:00
Jeff Washington (jwash)
b80fd7566a
add worker threads during startup ( #20700 )
2021-10-15 10:40:36 -05:00
Tao Zhu
e225ed7ab0
update config by testnet ( #20699 )
2021-10-15 15:32:46 +00:00
Jeff Washington (jwash)
7eead4e25f
clean stats on acct idx lookup ( #20632 )
2021-10-14 08:37:42 -05:00
Jeff Washington (jwash)
c43b6e0248
log metric for clean_accounts.delta_insert_us ( #20634 )
2021-10-14 08:37:14 -05:00
Jeff Washington (jwash)
9b042fed30
AcctIdx: support unref first class for perf ( #20638 )
2021-10-14 08:36:53 -05:00
Jack May
da45be366a
Remove blockhash from fee calculation ( #20641 )
2021-10-13 13:10:58 -07:00
Tao Zhu
220fd41bbc
fix unstable test ( #20645 )
2021-10-13 11:43:18 -05:00
Jeff Washington (jwash)
aec9d8bf2f
add clean stats ( #20631 )
2021-10-13 08:22:11 -05:00
Alexander Meißner
1d813ea078
Refactor: Make program_id always last in program chain ( #20598 )
...
* Replaces program_id field in InvokeContextStackFrame by index.
* Swaps order of program account and programdata account.
* Removes program_id parameter from InvokeContext::push().
2021-10-13 08:58:20 +02:00
Jeff Washington (jwash)
7631011d8c
AcctIdx: add flush remove stat ( #20625 )
2021-10-12 16:13:32 -05:00
Alexander Meißner
06cad19c91
Adds missing program_id in InvokeContext::push() calls. ( #20621 )
2021-10-12 19:26:55 +02:00