2bbe1d875a
Deserialize accounts before acquiring stakes cache lock ( #21733 )
...
* Deserialize stored accounts before locking stakes cache
* fix test
2021-12-10 16:02:35 -05:00
9b41ddd9ba
Add address lookup table program ( #21616 )
...
* Add address lookup table program
* feedback
2021-12-10 16:02:16 -05:00
ec7e17787e
Compute accounts data len during generate_index() ( #21757 )
2021-12-10 13:27:59 -06:00
80eae20610
Fix master fmt
2021-12-10 11:33:39 -07:00
350845c513
Move type alias and use it more broadly ( #21763 )
2021-12-10 10:46:21 -07:00
fd175c1ea9
Add StakesCache struct to abstract away locking ( #21738 )
2021-12-10 10:51:33 -05:00
622fd7c7ec
testing for the latest changes
2021-12-10 19:44:22 +05:30
d870f566ef
chore: bump serde from 1.0.130 to 1.0.131 ( #21758 )
...
* chore: bump serde from 1.0.130 to 1.0.131
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.130 to 1.0.131.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.130...v1.0.131 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-10 08:47:46 +00:00
c1386d66e6
Nits in message-processor ( #21755 )
...
* Fixup typo
* Simplify types slightly
2021-12-09 23:38:36 -07:00
16a6dceb6b
impl better debug for Ancestors ( #21737 )
2021-12-09 18:36:13 -06:00
1528f85112
Add return types to generate_index() ( #21735 )
2021-12-09 17:15:13 -06:00
6fc329180b
Add more reporting for invalid stake cache members and prune them ( #21654 )
...
* Add more reporting for invalid stake cache members
* feedback
2021-12-09 14:12:11 -05:00
a2df1eb502
AcctIdx: disk generate index and filler accounts use more threads ( #21566 )
2021-12-09 11:54:14 -06:00
58a7d3fc0e
AcctIdx: disable direct to disk for filler account upsert ( #21710 )
2021-12-09 10:14:57 -06:00
0090916735
skip clean on startup, too ( #21718 )
2021-12-09 09:36:02 -06:00
e020960f49
AcctIdx: generate index goes to mem not disk ( #21709 )
2021-12-08 19:48:12 -06:00
54862eba0d
AcctIdx: env var to enable testing of disk buckets ( #21494 )
2021-12-08 19:47:25 -06:00
181c0092d6
AcctIdx: resize in-mem after startup for disk index ( #21676 )
2021-12-08 16:52:22 -06:00
8d1e5ac294
refactor and test scan abort code ( #21390 )
2021-12-08 14:09:34 -06:00
f0acf7681e
Add vote instructions that directly update on chain vote state ( #21531 )
...
* Add vote state instructions
UpdateVoteState and UpdateVoteStateSwitch
* cargo tree
* extract vote state version conversion to common fn
2021-12-07 16:47:26 -08:00
1df88837c8
- Implicitly fixes invoke_context.return_data not being reset between instructions in process_message. ( #21671 )
...
- Lets InvokeContext::process_cross_program_instruction() handle the first invocation depth too.
- Marks InvokeContext::verify(), InvokeContext::verify_and_update() and InvokeContext::process_executable_chain() private.
- Renames InvokeContext::process_cross_program_instruction() to InvokeContext::process_instruction().
- Removes InvokeContext::new_mock_with_sysvars().
2021-12-07 23:00:04 +01:00
f6801a4af4
chore: bump itertools from 0.10.1 to 0.10.3 ( #21643 )
...
* chore: bump itertools from 0.10.1 to 0.10.3
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.1 to 0.10.3.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.1...v0.10.3 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-06 23:23:26 -07:00
da4015a959
Ensure that StakeDelegations and StakeHistory serde ( #21640 )
...
Add tests to StakeDelegations and StakeHistory to ensure that the outer
types serialize and deserialize correctly to/from the inner types.
2021-12-06 17:34:10 -06:00
a1adcb23b6
Remove activated feature for filtering invalid stakes from rewards ( #21641 )
2021-12-06 17:56:05 -05:00
3dab1e711d
Move transaction error code into new module ( #21635 )
2021-12-06 12:45:33 -05:00
d6f22433d0
Bump version to v1.10.0
2021-12-04 20:17:54 +00:00
46fe56171b
Make StakeHistory clone-on-write ( #21573 )
2021-12-03 12:10:29 -06:00
1430b58a6d
Remove deprecated slow epoch boundary methods ( #21568 )
2021-12-03 17:59:10 +00:00
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
0ef1b25e4b
Make StakeDelegations clone-on-write ( #21542 )
2021-12-03 08:54:38 -06:00
1a4a039913
Adds feature reject_empty_instruction_without_program. ( #21591 )
2021-12-03 15:47:18 +01:00
a9d5ef2055
Cleanup: InvokeContext accessors ( #21574 )
...
* Removes blockhash accessors from InvokeContext.
* Removes lamports_per_signature accessors from InvokeContext.
* Removes return_data accessors from InvokeContext.
* Removes feature_set accessor from InvokeContext.
* Removes instruction_recorders and instruction_index accessors from InvokeContext.
* Moves get_sysvars() into InvokeContext.
* Removes compute_meter parameter from InvokeContext::new().
* Removes InvokeContext::new_mock_with_sysvars_and_features().
* Removes InvokeContext::update_timing().
2021-12-03 12:15:22 +01:00
bdc33ba0a1
Remove unnecessary Option ( #21569 )
2021-12-02 20:19:22 +00:00
bf5b7f5d7f
report compute units without tx_wide_compute_cap feature enabled ( #21421 )
2021-12-02 12:14:57 -06:00
f0b32b75ab
AcctIdx: use ZeroLamport trait ( #21552 )
2021-12-02 12:10:11 -06:00
314605e149
AcctIdx: test cleanup ( #21550 )
2021-12-02 12:09:37 -06:00
976eb81d4f
Cleanup the bank's use of nonces ( #21246 )
2021-12-02 09:57:05 -08:00
bfdb775ffc
Unifies ThisInvokeContext and dyn trait InvokeContext. ( #21563 )
2021-12-02 18:47:16 +01:00
8dfa83c579
AcctIdx: minor code cleanup ( #21547 )
2021-12-02 11:05:54 -06:00
6bb884836c
address review feedback - use u64 instead of usize
2021-12-02 08:12:30 -08:00
adbf31b98c
avoid usize overflow for MAXIMUM_APPEND_VEC_FILE_SIZE on 32-bit platforms
2021-12-02 08:12:30 -08:00
c8a52337c8
AcctIdx: remove unused function ( #21546 )
2021-12-01 17:14:49 -06:00
1fae3d24de
bank: reuse last hash/fees helper to instatiate `InvokeContext`
2021-12-01 12:59:20 -07:00
91c6a10cda
bank: get hash and fee rate from the same source when storing accounts
2021-12-01 12:59:20 -07:00
308d7d40d0
aggressively flush cache based on cache size ( #21332 )
...
* aggressively flush cache based on cache size
* find_older_frozen_slots -> cached_frozen_slots
* remove 'WRITE_CACHE_LIMIT_BYTES_DEFAULT'
* tweaks to stats
* fix tests
2021-12-01 13:10:48 -06:00
1eefdeba85
AcctIdx: cleanup trait ( #21522 )
2021-12-01 12:19:06 -06:00
4a1ef12bd9
Refactor fns that calculate stake and voter balances ( #21529 )
2021-12-01 10:29:24 -06:00
b108d7ddaa
AcctIdx: insert goes directly to disk to avoid unnecessary allocations ( #21490 )
...
* AcctIdx: upsert avoids unnecessary allocation (during startup)
* feedback
2021-12-01 10:21:01 -06:00
cb368e6554
Pass Epoch by value in StakeHistory::get() ( #21523 )
2021-12-01 08:57:29 -06:00
b78f5b6032
Refactor: Cleanup InstructionProcessor ( #21404 )
...
* Moves create_message(), native_invoke() and process_cross_program_instruction()
from the InstructionProcessor to the InvokeContext so that they can have a useful "self" parameter.
* Moves InstructionProcessor into InvokeContext and Bank.
* Moves ExecuteDetailsTimings into its own file.
* Moves Executor into invoke_context.rs
* Moves PreAccount into its own file.
* impl AbiExample for BuiltinPrograms
2021-12-01 08:54:42 +01:00
e922c2da9d
Update to Rust 1.56.1
2021-11-30 23:28:07 -08:00
dd12d90eac
Upgrade to Rust 2021
2021-11-30 20:43:46 -08:00
8cf36e5cb0
Add GPU sigverify for verify path ( #20851 )
...
Allows the use of GPU acceleration in verifying the signatures in Entry's after deserialization in the replay stage
Co-authored-by: Stephen Akridge <sakridge@gmail.com>
Co-authored-by: Ryan Leung <ryan.leung@solana.com>
2021-11-30 21:16:13 -05:00
d8fb7ce511
AcctIdx: upsert avoids unnecessary allocation ( #21488 )
2021-11-30 11:36:46 -06:00
7ec88226ee
AcctIdx: stats are usize ( #21493 )
...
* AcctIdx: stats are usize
* rename test function
2021-11-30 11:35:21 -06:00
2be859d304
add test for aborting index scan ( #21487 )
2021-11-29 16:13:39 -06:00
ba9dfa0d22
Remove frozen account support
2021-11-29 08:38:11 -08:00
41cc7af7cd
bank: `TransactionLogCollector` - only push address mappings after pushing a log message
2021-11-25 04:50:51 +00:00
e558ad4c71
bank: don't panic when getting logs from malformed TransactionLogCollector
2021-11-25 04:50:51 +00:00
dafdc15dfe
bank: Add failing test for `TransactionLogCollector::get_logs_for_address()`
2021-11-25 04:50:51 +00:00
efedb55705
bank: factor tx log getter out to method on `TransactionLogCollector`
2021-11-25 04:50:51 +00:00
9edfc5936d
Refactor accounts.rs with Justin's comments to improve lock accounts ( #21406 )
...
with results code path.
- fix a bug that could unlock accounts that weren't locked
- add test to the refactored function
- skip enumerating transaction accounts if qos results is an error
- add #[must_use] annotation
- avoid clone error in results
- add qos error code to unlock_accounts match statement
- remove unnecessary AbiExample
2021-11-23 21:17:55 +00:00
22a2537aac
Refactor: Cleanup program-runtime dyn Traits ( #21395 )
...
* Unifies dyn Trait ComputeMeter, ThisComputeMeter and TransactionComputeMeter.
* Unifies dyn Trait Logger and ThisLogger.
* Moves Logger to log_collector.rs
* Unifies Logger and LogCollector.
* Removes inner RefCell from LogCollector.
* Adds the log::debug!() message to ic_logger_msg!() again.
2021-11-23 13:23:40 +01:00
b87ebf9e58
AcctIdx: PreAllocatedAccountMapEntry does not make unnecessary Arc ( #21364 )
2021-11-22 15:52:45 -06:00
a762066310
AcctIdx: don't log disk stats when disk isn't enabled ( #21371 )
2021-11-22 15:26:25 -06:00
c4d68063c7
Add timing for accounts add_root ( #21379 )
2021-11-22 18:29:45 +01:00
cd6f931223
Add timing metrics to bank creation and rewards calculation ( #21066 )
2021-11-21 15:11:02 +01:00
02bc4e3fc1
spl-token: New program feature flag ( #21354 )
...
* spl-token: Add feature flag for new release
* Remove all spl token version declarations
2021-11-21 14:27:03 +01:00
ebea3297f9
AcctIdx: generate index inserts/updates directly to disk ( #21363 )
...
* when initially creating account index, write directly to disk
* AcctIdx: generate index inserts/updates directly to disk
2021-11-19 17:17:07 -06:00
0bb059185c
Add lamports_in_rent_paying_accounts to TotalAccountsStats ( #21368 )
2021-11-19 17:02:23 -06:00
8a50b6302f
Fix #21346 ( #21362 )
...
Fixes the empty transaction bug in ThisInvokeContext::push() and adds a test for it to the bank.
2021-11-19 20:43:42 +01:00
79d21d6805
add --accounts-index-scan-results-limit-mb to allow scans to abort ( #21327 )
...
* ScanConfig -> &ScanConfig
* add --accounts-index-scan-results-limit-mb to allow scans to abort
* feedback
2021-11-19 09:00:19 -06:00
c24e30f689
Constrain memory ordering on AccountsDb::next_id ( #21311 )
2021-11-18 20:34:37 -06:00
f7152c889c
Constrain memory ordering on AppendVec::id ( #21310 )
2021-11-18 20:34:20 -06:00
5788973153
Constrain memory ordering on AccountStorageEntry::alive_bytes ( #21309 )
2021-11-18 20:34:08 -06:00
03c36d240a
Nonce naming cleanup ( #21336 )
2021-11-18 16:07:17 -08:00
b30c94ce55
ClusterInfoVoteListener send only missing votes to BankingStage ( #20873 )
2021-11-18 15:20:41 -08:00
a272e19f8d
AcctIdx: report on partial progress during flushing ( #21306 )
2021-11-18 11:01:19 -06:00
89c45a57f8
Refactor slot status notification to decouple from accounts notifications ( #21308 )
...
Problem
Slot status can be used of in other scenarios in addition to account information such as transactions, blocks. The current implementation is too tightly coupled.
Summary of Changes
Decouple the slot status notification from accounts notification. Created a new slot status notification module.
2021-11-17 17:11:38 -08:00
9cb74e8421
remove store_cached stall ( #21326 )
2021-11-17 18:30:40 -06:00
83de2f7376
fix initial massive metrics spike ( #21304 )
2021-11-17 17:16:48 -06:00
0f69a14247
Add ability to abort scan ( #21314 )
2021-11-17 13:10:29 -06:00
e540b1cf3c
Refactor: Move sdk::process_instruction in program-runtime-crate ( #21180 )
...
* Moves the Executor dyn Trait to instruction_processor.rs
* Moves the Logger dyn Trait as well as the ic_msg and ic_logger_msg macros to log_collector.rs,
and moves the stable_log to stable_log.rs
* Moves the ComputeMeter dyn Trait to invoke_context.rs
* Moves the InvokeContext dyn Trait and the ProcessInstructionWithContext type to invoke_context.rs
* Updates cargo files.
* Re-export InvokeContext in program-test
Co-authored-by: Jon Cinque <jon.cinque@gmail.com>
2021-11-17 19:35:07 +01:00
c3e5927d16
Constrain memory ordering on AccountsDb::write_version ( #21312 )
2021-11-17 07:11:16 -06:00
cb0bb5bd1e
Nonce accounts must be writeable ( #21260 )
...
* Nonce accounts must be writeable
* feedback
* feedback
2021-11-16 23:01:00 +00:00
ae497715cc
HashCalc: Perf: use refs for hashes ( #21280 )
2021-11-16 10:30:55 -06:00
398af132a5
More set_root metrics ( #21286 )
2021-11-15 16:28:18 -07:00
9b1bf98aa2
Accept only decimal digits in file names inside snapshot ( #21213 )
...
This also should make snapshot validation a bit faster.
2021-11-15 17:02:15 +00:00
39340ed25b
throttle store_cached when cache size is too large ( #21188 )
...
* throttle store_cached when cache size is too large
* reduce max delay
* 100ms max
* 10ms max delay
2021-11-13 14:00:37 -06:00
f8dcb2f38b
report mem stats ( #21258 )
2021-11-13 00:59:41 +00:00
778b5d736c
Load nonce with fixed root ( #21248 )
2021-11-12 14:12:34 -08:00
1509513339
hash calc: filler_accounts_enabled ( #21243 )
2021-11-12 09:53:21 -06:00
11153e1f87
refactor cost calculation ( #21062 )
...
* - cache calculated transaction cost to allow sharing;
- atomic cost tracking op;
- only lock accounts for transactions eligible for current block;
- moved qos service and stats reporting to its own model;
- add cost_weight default to neutral (as 1), vote has zero weight;
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update core/src/qos_service.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* Update core/src/qos_service.rs
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-11-12 01:04:53 -06:00
ef29d2d172
Refactor vote state to remove double negative ( #21244 )
2021-11-12 00:26:43 +00:00
131c0d78c4
Fix compute budget bump ( #21238 )
2021-11-11 14:09:28 -08:00
c0f9e351db
hashcalc eliminate zeros optimizations ( #21209 )
2021-11-11 09:44:32 +00:00
8b2133b078
Refactor total stats calculation into shared fn ( #21212 )
2021-11-09 19:13:37 -06:00
48265ada39
chore: bump blake3 from 1.1.0 to 1.2.0 ( #21203 )
...
* chore: bump blake3 from 1.1.0 to 1.2.0
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.1.0...1.2.0 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Make versions consistent
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-11-08 15:27:37 -07:00
5827cf4c51
hash calc eliminate zero refactor get_item ( #21036 )
2021-11-08 20:27:48 +00:00
706b60b5c8
Add `set_root` bank drop logging ( #21144 )
2021-11-06 01:05:36 +00:00
1b46d1d54d
hashcalc eliminate zeros cleanup ( #21175 )
2021-11-05 10:54:49 -05:00
c78f474373
Add validator option to change niceness of snapshot packager thread
2021-11-04 17:16:46 -06:00
7200c5106e
Replaces MockInvokeContext by ThisInvokeContext in tests ( #20881 )
...
* Replaces MockInvokeContext by ThisInvokeContext in BpfLoader, SystemInstructionProcessor, CLIs, ConfigProcessor, StakeProcessor and VoteProcessor.
* Finally, removes MockInvokeContext, MockComputeMeter and MockLogger.
* Adjusts assert_instruction_count test.
* Moves ThisInvokeContext to the program-runtime crate.
2021-11-04 21:47:32 +01:00
0597594943
Fix rent debits test ( #21177 )
2021-11-04 20:03:19 +00:00
4e527b84b0
Remove unused upgrade epoch handling ( #21164 )
2021-11-04 10:44:36 -07:00
171243a370
add rent exempt count to get_total_accounts_stats ( #21153 )
2021-11-04 11:17:19 -05:00
e765542cd8
eliminate pubkey copy in eliminate_zeros ( #21032 )
...
* rework eliminate zero loop
* eliminate pubkey copy in eliminate_zeros
2021-11-04 11:16:36 -05:00
76e533be46
LoadedAccountAccessor::Cached does not store Pubkey ( #21030 )
2021-11-04 10:28:04 -05:00
aaccba8377
Refactor `RentDebits` to use hashmap instead of vec
2021-11-04 15:02:24 +00:00
b1e3a82c11
refactor rent collector and logs ( #21155 )
2021-11-04 06:28:41 -05:00
29ad081555
Stop caching sysvars, instead load them ahead of time. ( #21108 )
2021-11-04 09:48:34 +01:00
c326f41dfe
Add bank::get_total_accounts_stats() ( #21159 )
2021-11-03 19:23:45 -05:00
c59c56c0f8
log rent exempt accounts ( #21137 )
2021-11-03 14:44:52 -05:00
c0952831be
correctly clean filler accounts ( #21139 )
2021-11-03 11:27:15 -05:00
140a5f633d
Simplify replay vote tracking by using packet metadata ( #21112 )
2021-11-03 09:02:48 +00:00
1e5212e60d
Add set_root timing metrics ( #21119 )
2021-11-02 18:23:35 +01:00
bced07a099
Update fee api to use blockhash ( #21054 )
2021-10-29 13:52:59 -07:00
4aa12a52b6
Feature cleanup ( #21038 )
2021-10-28 09:04:03 -07:00
7a41b2c5d1
optimize get_accounts_delta_hash ( #21027 )
2021-10-28 10:40:32 -05:00
e9ab214237
filler accts: only add filler accts to slots in the current epoch ( #21024 )
2021-10-28 10:26:30 -05:00
6a1ff60c61
percent stats 0-1 -> 0-100 ( #21028 )
2021-10-27 18:08:27 -05:00
036d7fcc81
Clean up sanitized tx creation for tests ( #21006 )
2021-10-27 18:09:16 +01:00
9d330fc638
FillerAccts: use variable cycle partitions ( #20963 )
2021-10-27 11:18:27 -05:00
1e2bef76e3
Add compute budget program as a noop ( #20992 )
2021-10-26 20:08:59 -07:00
4d6190a1ba
change bg waiting metrics to percent ( #20955 )
...
* Revert part of "AcctIdx: cleanup bg threads (#20731 )"
This reverts part of commit 47de4f31b6
2021-10-26 15:09:17 -05:00
2515f6a04f
Update deprecation versions ( #20959 )
2021-10-26 09:06:41 -07:00
bbe3ce3db5
Accountsdb plugin write ordering ( #20948 )
...
Use the write_version in the Accounts's meta data so that account write with lower write_version would not overwrite the higher ones.
2021-10-25 14:07:56 -07:00
cf0fd5b2ca
Moves NonceKeyedAccount from the SDK to the Runtime. ( #20954 )
2021-10-25 22:58:18 +02:00
9e85499d4c
fix hash calc # storages, # slots ( #20832 )
2021-10-25 19:37:50 +00:00
43ea579f63
add cli for --accounts-hash-num-passes ( #20827 )
2021-10-25 09:45:46 -05:00
c2bfce90b3
- cost_tracker is data member of a bank, it can report metrics when bank is frozen ( #20802 )
...
- removed cost_tracker_stats and histogram
- move stats reporting outside of bank freeze
2021-10-24 22:19:23 -05:00
f14365f4b4
Accountsdb plugin postgres -- bulk insertion at startup ( #20763 )
...
Use bulk insertion to Postgres at startup to reduce time taken for initial snapshot restore for postgres plugin. Avoid duplicate writes of accounts at startup. Doing account plugin notification and indexing in parallel.
Improved error handling for postgres plugin to show the real db issues for debug purpose
Added more metrics for postgres plugin.
Refactored plugin centric code out to a sub module from accounts_db and added unit tests
2021-10-24 12:43:33 -07:00
bfbbc53dac
Divorce the runtime from FeeCalculator ( #20737 )
2021-10-22 14:32:40 -07:00
735016661b
Report timing info for stakes cache updates from txs ( #20856 )
2021-10-22 12:49:02 -04:00
1d9c1ccfd2
chore: bump blake3 from 1.0.0 to 1.1.0 ( #20877 )
...
* chore: bump blake3 from 1.0.0 to 1.1.0
Bumps [blake3](https://github.com/BLAKE3-team/BLAKE3 ) from 1.0.0 to 1.1.0.
- [Release notes](https://github.com/BLAKE3-team/BLAKE3/releases )
- [Commits](https://github.com/BLAKE3-team/BLAKE3/compare/1.0.0...1.1.0 )
---
updated-dependencies:
- dependency-name: blake3
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-22 09:44:42 -06:00
97c2732d02
Refactor: Cleanup InvokeContext ( #20785 )
...
* Move blockhash and fee_calculator in ThisInvokeContext instead of using a reference.
* Moves tx_wide_compute_cap into InvokeContext::push().
* Adds ThisInvokeContext::new_mock() constructor.
* Adds missing loader account in uses of MockInvokeContext.
* Use keyed_account_at_index() when accessing keyed_accounts.
* Makes sysvar interface consistent between ThisInvokeContext and MockInvokeContext,
in order to add InvokeContext::get_sysvars().
* Adds InvokeContext::set_blockhash() and InvokeContext ::set_fee_calculator().
* Adds new_mock_with_features.
* Makes ancestors optional in ThisInvokeContext.
* Adds prepare_mock_invoke_context() and mock_process_instruction().
2021-10-21 20:57:42 +02:00
a8098f37d0
add checked instructions sysvar api ( #20790 )
2021-10-19 21:01:58 -07:00
58164517e4
Add program heap bump instruction ( #20607 )
2021-10-19 21:01:39 -07:00
cb2bd65858
runtime: Add foundation stake pool withdraw authority ( #20797 )
2021-10-19 21:49:21 +00:00
47a58a38c2
clean_accounts calls AcctIdx: get_many ( #20715 )
...
* AcctIdx: get_many
* keep read lock
* AcctIdx: get_many optionally adds to cache
* rename
2021-10-19 15:54:06 -05:00
0acbfdfcb9
hash calc: sort time is redundantly counted as part of 'total time' ( #20791 )
2021-10-19 15:52:43 -05:00
7496b5784b
- make cost_tracker a member of bank, remove shared instance from TPU; ( #20627 )
...
- decouple cost_model from cost_tracker; allowing one cost_model
instance being shared within a validator;
- update cost_model api to calculate_cost(&self...)->transaction_cost
2021-10-19 14:37:33 -05:00
2b76ea51b4
Reduce visibility of `Hash` struct contents
2021-10-19 09:30:47 -07:00
46bf7d4a4a
AcctIdx: support 2^24 bins ( #20739 )
2021-10-18 14:05:16 -05:00
751b45df64
Make pub get_incremental_snapshot_archives() ( #20725 )
2021-10-18 13:48:50 -05:00
c9b367c350
Minimum of 1 lamport for rent ( #20650 )
2021-10-18 19:09:53 +02:00
6ec159a1ae
parallelize update_index ( #20601 )
2021-10-18 09:07:01 -05:00
0d934d311e
sort shrink acct indexes ( #20708 )
2021-10-18 09:06:32 -05:00
e98cfbb570
Removes deprecated BpfComputeBudget. ( #20728 )
2021-10-18 09:55:41 +02:00
70b2e5fef2
hash calc uses all threads during startup ( #20735 )
2021-10-15 17:41:23 -05:00
47de4f31b6
AcctIdx: cleanup bg threads ( #20731 )
2021-10-15 16:15:11 -05:00
254ad45878
sort pubkeys for clean ( #20706 )
2021-10-15 14:31:06 -05:00
6d7da6dbee
clean:clone uncleaned_roots for loop ( #20714 )
2021-10-15 13:18:00 -05:00
5a6d732d48
AcctIdx: separate account index by startup and !startup ( #20711 )
2021-10-15 13:04:32 -05:00
9d62963240
AcctIdx: get_internal allows not adding to in_mem ( #20716 )
2021-10-15 13:04:05 -05:00
1e064c32e2
pull append_vec_id() out of loop ( #20713 )
2021-10-15 10:41:23 -05:00
00b9056004
clean metric: total_us ( #20707 )
2021-10-15 10:41:05 -05:00
b80fd7566a
add worker threads during startup ( #20700 )
2021-10-15 10:40:36 -05:00
e225ed7ab0
update config by testnet ( #20699 )
2021-10-15 15:32:46 +00:00
7eead4e25f
clean stats on acct idx lookup ( #20632 )
2021-10-14 08:37:42 -05:00
c43b6e0248
log metric for clean_accounts.delta_insert_us ( #20634 )
2021-10-14 08:37:14 -05:00
9b042fed30
AcctIdx: support unref first class for perf ( #20638 )
2021-10-14 08:36:53 -05:00
da45be366a
Remove blockhash from fee calculation ( #20641 )
2021-10-13 13:10:58 -07:00
220fd41bbc
fix unstable test ( #20645 )
2021-10-13 11:43:18 -05:00
aec9d8bf2f
add clean stats ( #20631 )
2021-10-13 08:22:11 -05:00
1d813ea078
Refactor: Make program_id always last in program chain ( #20598 )
...
* Replaces program_id field in InvokeContextStackFrame by index.
* Swaps order of program account and programdata account.
* Removes program_id parameter from InvokeContext::push().
2021-10-13 08:58:20 +02:00
7631011d8c
AcctIdx: add flush remove stat ( #20625 )
2021-10-12 16:13:32 -05:00
06cad19c91
Adds missing program_id in InvokeContext::push() calls. ( #20621 )
2021-10-12 19:26:55 +02:00
005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; ( #20527 )
...
- move related modules to runtime
2021-10-12 08:51:33 -05:00
b0ee5d6cfa
Add debug guard to catch invalid params ( #20603 )
2021-10-11 16:47:17 -07:00
0da677e213
AcctIdx: write disk only when removing from mem ( #20578 )
2021-10-11 14:47:06 -05:00
9d94e43839
chore: bump ouroboros from 0.12.0 to 0.13.0 ( #20591 )
...
* chore: bump ouroboros from 0.12.0 to 0.13.0
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-10-11 11:54:37 -06:00
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
f30f3bddbb
Refactor: Remove program_id from process_instruction() ( #20540 )
...
* Replaces usage of program_id parameter by invoke_context.get_caller()?.
* Removes "pubkey: &Pubkey" parameter from "process_instruction()".
2021-10-10 22:29:18 +02:00
8cf222b8bd
AcctIdx: combine if ( #20545 )
2021-10-10 13:48:18 -05:00
33d8c07364
AcctIdx: combine scan and update loops ( #20546 )
2021-10-10 13:47:52 -05:00
db9336c99e
Remove feature switch handling for checking vote init ( #20557 )
2021-10-08 23:35:26 -04:00
0545306f0c
AcctIdx: remove stats from write lock section ( #20548 )
2021-10-08 15:27:38 -05:00
5440c1d2e1
SnapshotPackagerService pushes incremental snapshot hashes to CRDS ( #20442 )
...
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!
This commit does two main things:
1. SnapshotPackagerService now knows about incremental snapshot hashes,
and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
hashes need to be passed all the way to SnapshotPackagerService so it
can push these starting hashes to CRDS. Those values have been piped
through.
Fixes #20441 and #20423
2021-10-08 15:14:56 -05:00
4f6a0b2650
AcctIdx: stats for buckets on disk, add median ( #20528 )
2021-10-08 13:58:38 -05:00
2c3d52b4cc
AcctIdx: avoid overcounting inserts ( #20544 )
2021-10-08 13:57:47 -05:00
4e65487d2f
Refactor: process_instruction() ( #20448 )
...
* Adds first_instruction_account parameter to process_instruction().
* Removes InvokeContext::remove_first_keyed_account() from all BPF loaders.
* Removes InvokeContext::remove_first_keyed_account() from all builtin programs.
* Removes InvokeContext::remove_first_keyed_account() from all mock ups.
* Deprecates InvokeContext::remove_first_keyed_account().
* Documents index base of keyed_account_at_index().
* Adds dynamic offset to call sites of "keyed_account_at_index()".
2021-10-08 11:41:07 +02:00
069586c171
AcctIdx: set startup around clean and shrink at startup to flush cache ( #20526 )
2021-10-07 22:57:40 +00:00
177a375479
Tpu vote 1.7 ( #20187 ) ( #20494 )
...
* Add separate vote processing tpu port
* Add feature to send to tpu vote port
* Add vote rejecting sigverify mode
* use packet.meta.is_simple_vote_tx in place of deserialization
* consolidate code that identifies vote tx atcommon path for cpu and gpu
* new key for feature set
* banking forward tpu vote
* add tpu vote port to dockerfile and other review changes
* Simplify thread id compare
* fix a test; updated cluster_info ABI change
Co-authored-by: Tao Zhu <tao@solana.com>
Co-authored-by: sakridge <sakridge@gmail.com>
2021-10-07 09:38:23 +00:00
df39b37cb8
improve clean stats ( #20469 )
2021-10-06 20:04:26 -05:00
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
3b70cdb8d8
Upgrade to Rust 1.55.0
2021-10-06 17:48:58 -07:00
785fcb63f5
Remove support for dynamically loaded native programs ( #20444 )
2021-10-06 14:53:23 -07:00
7027d56064
Resolve nightly-2021-10-05 clippy complaints
2021-10-06 10:37:58 -07:00
895b46e0c3
simplify AccountsDb::new_with_config ( #20473 )
2021-10-06 11:41:23 -05:00
129716f3f0
Optimize stakes cache and rewards at epoch boundaries ( #20432 )
...
* Optimize stakes cache and rewards at epoch boundaries
* Fetch from accounts db
* Add cli flag for disabling epoch boundary optimization
2021-10-06 00:53:26 -04:00
250a8503fe
Make rewards tracer async friendly ( #20452 )
2021-10-05 21:30:08 -04:00
d03bf2bbfe
AcctIdx: metrics fixes ( #20449 )
2021-10-05 22:26:29 +00:00
4267419349
AcctIdx: sleep thread when enough threads are running to complete the remaining work ( #20446 )
2021-10-05 16:48:23 -05:00
0d98a91511
AcctIdx: metric for bucket grow time ( #20445 )
2021-10-05 20:22:01 +00:00
64cf354651
Add clean metrics ( #20410 )
2021-10-04 18:21:45 -05:00
2d78f8ad2a
parallelize unref_from_storage from clean_dead_slots_from_accounts_index ( #20411 )
2021-10-04 18:20:10 -05:00
0ddb34a0b4
Add struct and convenience methods to track stake activation status ( #20392 )
...
* Add struct and convenience methods to track stake activation status
* fix nits
* rename
2021-10-04 18:59:11 -04:00
b6a6d6986d
lift store.slot() from loop ( #20409 )
2021-10-04 16:07:01 -05:00
8da2eb980a
AcctIdx: introduce BucketApi for access to a specific bucket ( #20359 )
2021-10-04 11:48:09 -05:00
e31c065544
AcctIdx: when disk index insert needs to grow, drop locks, grow, then restart flush ( #20333 )
2021-09-30 19:27:09 -04:00
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
9ef1f06b3f
chore: bump ouroboros from 0.11.1 to 0.12.0 ( #20344 )
...
* chore: bump ouroboros from 0.11.1 to 0.12.0
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.11.1 to 0.12.0.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-30 13:20:05 -06:00
8db9586599
AcctIdx: randomly evict items from in mem ( #20319 )
2021-09-30 02:51:54 +00:00
5e05f12c48
AcctIdx: bucket perf improvements ( #20328 )
2021-09-30 01:53:26 +00:00
59112adac1
AcctIdx: us stats were showing ms ( #20324 )
2021-09-29 22:18:07 +00:00
2ec74474e9
AcctIdx: optimize should_remove_from_mem ( #20327 )
2021-09-29 16:41:54 -05:00
4de5fff3ca
Simplifies return_data accessors in InvokeContext. ( #20290 )
2021-09-29 19:11:06 +02:00
57c8abf499
Refactor: Merge message processor into invoke context ( #20308 )
...
* Inlines MessageProcessor::execute_instruction() in MessageProcessor::process_message().
* Moves MessageProcessor::create_pre_accounts() into ThisInvokeContext::push().
* Move instruction_recorders slice into InvokeContext.
* Makes account_indices optional in InvokeContext::push().
* Separates initial InvokeContext::push() from ThisInvokeContext::new().
* invoke_context.pop() the base invocation frame.
* Zip message.instructions.iter() and program_indices.iter().
* Moves ThisInvokeContext::new() to the beginning of the loop inside MessageProcessor::process_message().
* Hoists ThisInvokeContext::new() out of loop inside MessageProcessor::process_message().
* Removes unnecessary clone() from ThisInvokeContext::new() in MessageProcessor ::process_message().
* Stop ignoring errors from MessageProcessor::create_pre_accounts().
* Moves MessageProcessor::verify_account_references() and MessageProcessor::verify() into InvokeContext::verify().
2021-09-29 12:05:25 -05:00
062d8e6c5a
AcctIdx: fix logic on should_remove_from_mem ( #20302 )
2021-09-29 13:07:40 +00:00
bd9b2f6f39
AcctIdx: throttling bg flushing to spread out work over interval ( #20176 )
...
* AcctIdx: throttling bg flushing to spread out work over interval
* prevent hang
2021-09-29 07:42:54 -05:00
8fee9a2e1a
Dont call precompiled programs ( #19930 )
2021-09-29 06:25:08 +00:00
491877de3d
Revert "Refactor: Merge MessageProcessor into InvokeContext ( #20165 )" ( #20301 )
...
This reverts commit df6905c3a6
2021-09-28 16:59:01 -07:00
fa5b091b4c
prior to panicing with cap mismatch, try other calculation ( #20292 )
2021-09-28 16:09:36 -05:00
320beb76f6
AcctIdx: implement wait_for_idle ( #20256 )
2021-09-28 11:07:19 -05:00
2cd9dc99b6
Restore ability for programs to upgrade themselves ( #20265 )
...
* Make helper associated fn
* Add feature definition
* Add handling to preserve program-id write lock when upgradeable loader is present; restore bpf upgrade-self test
* Use single feature
2021-09-28 09:59:08 -06:00
4e27543415
Allow programs to realloc their accounts within limits ( #19475 )
2021-09-28 01:13:03 -07:00
e50a26e6cf
AcctIdx: add age_interval_ms() to remove constant use ( #20264 )
2021-09-28 01:22:07 +00:00
af309c126c
AcctIdx: refactor get to use more refs ( #20268 )
2021-09-28 00:18:36 +00:00
34406d4815
chore: bump ouroboros from 0.10.1 to 0.11.1 ( #20244 )
...
* chore: bump ouroboros from 0.10.1 to 0.11.1
Bumps [ouroboros](https://github.com/joshua-maros/ouroboros ) from 0.10.1 to 0.11.1.
- [Release notes](https://github.com/joshua-maros/ouroboros/releases )
- [Commits](https://github.com/joshua-maros/ouroboros/commits )
---
updated-dependencies:
- dependency-name: ouroboros
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-27 23:26:08 +00:00
f6812523cd
handle race condition with flush and delete ( #20255 )
2021-09-27 16:55:15 -05:00
bbd2f9672d
clone -> Arc::clone to make cost clear ( #20262 )
2021-09-27 16:49:53 -05:00
4f83818fdb
get rid of WriteAccountMapEntry instance ( #19093 )
2021-09-27 16:24:19 -05:00
e4795ae7ed
AcctIdx: keep cached or multiple slot lists in cache ( #20257 )
2021-09-27 16:09:26 -05:00
9d91a6f1b7
shrink slots at startup uses more threads ( #20251 )
2021-09-27 14:14:55 -05:00
8266864adc
AcctIdx: maybe advance age when a bg thread wakes up ( #20250 )
...
* fix math error with stats calc
* AcctIdx: maybe report stats when a bg thread wakes up
2021-09-27 18:39:41 +00:00
95fbc29500
Add stats when getting snapshot storages ( #20234 )
2021-09-27 13:07:00 -05:00
46263d405e
fix math error with stats calc ( #20253 )
2021-09-27 11:55:19 -05:00
bc50fd4ad6
allocate threads for accounts idx based on cpus ( #20118 )
2021-09-27 11:25:06 -05:00
025261c05e
AcctIdx: when no disk index, always wait for stats ( #20248 )
2021-09-27 11:24:36 -05:00
833f7761d2
AcctIdx: upsert uses get from mem first ( #20186 )
2021-09-27 11:23:53 -05:00
d8f98b3dcb
AcctIdx: avoid crazy #s on ms_per_age stat ( #20184 )
2021-09-27 08:25:23 -05:00
df6905c3a6
Refactor: Merge MessageProcessor into InvokeContext ( #20165 )
...
* Inlines MessageProcessor::execute_instruction() in MessageProcessor::process_message().
* Moves MessageProcessor::create_pre_accounts() into ThisInvokeContext::push().
* Hoists ThisInvokeContext::new() out of loop inside MessageProcessor::process_message().
* Moves MessageProcessor::verify_account_references() and MessageProcessor::verify() into InvokeContext::verify().
2021-09-27 08:28:45 +02:00
c6d189dac2
bg threads wait until next expected event ( #20126 )
2021-09-24 16:00:41 -05:00
122206dbb1
AcctIdx: bg loop sleeps when all buckets flushed at current age ( #20168 )
2021-09-24 13:19:06 -05:00
1d13594c1c
AcctIdx: store # threads in BucketHolder. Used later. ( #20151 )
2021-09-23 19:56:44 -05:00
b3bb079d9f
AcctIdx: condense upsert to always use read lock ( #20148 )
2021-09-23 19:19:27 -05:00
1347b507da
Handle duplicate AppendVec IDs ( #20096 )
...
When reconstructing the AccountsDb, if the storages came from full and
incremental snapshots generated on different nodes, it's possible that
the AppendVec IDs could overlap/have duplicates, which would cause the
reconstruction to fail.
This commit handles this issue by unconditionally remapping the
AppendVec ID for every AppendVec.
Fixes #17088
2021-09-23 17:34:32 -05:00
2ae1e80013
AcctIdx: items needs to load from disk ( #20138 )
2021-09-23 17:33:54 -05:00
334d38d200
AcctIdx: generation per thread starts at a random bucket ( #20122 )
2021-09-23 19:59:37 +00:00
254d9c8903
range takes a ref ( #20140 )
2021-09-23 13:57:56 -05:00
093dd68214
AcctIdx: stats for how much time bg threads wait ( #20127 )
2021-09-23 18:37:14 +00:00
fe28f17718
AcctIdx: startup causes us to flush remove everything asap ( #20121 )
2021-09-23 12:17:52 -05:00
5dae615208
AcctIdx: set age of cache accesses to future ( #20125 )
2021-09-23 11:43:48 -05:00
fb55e9a9b6
AcctIdx: when stop_flush is disabled, prepare to age ( #20113 )
2021-09-23 11:42:32 -05:00
5bbb0da7b8
AcctIdx: stats for in-mem ( #20114 )
2021-09-23 08:14:53 -05:00
cb9d93525b
AcctIdx: metrics for loading from disk ( #20124 )
2021-09-23 08:14:08 -05:00
4d3e32803f
AcctIdx: PreAllocatedAccountMapEntry ( #20117 )
2021-09-23 08:11:32 -05:00
7b365c564f
runtime: remove inactive delegation from stakes cache
2021-09-22 20:50:58 -06:00
476124de50
AcctIdx: report disk bucket stats ( #20120 )
2021-09-22 18:17:20 -05:00
f7eebadbed
AcctIdx: remove old items from cache ( #20076 )
2021-09-22 21:37:15 +00:00
0eb0d7f73b
AcctIdx: consolidate lock_and_update_slot_list ( #20090 )
2021-09-22 18:40:19 +00:00
4fb77183ef
AcctIdx: move background() to bucket holder ( #20085 )
2021-09-22 17:40:30 +00:00
681e8728a1
get rid of logs left in test ( #20116 )
2021-09-22 12:39:48 -05:00
bbd7bc2985
AcctIdx: Fix bad test ( #20115 )
2021-09-22 12:39:31 -05:00
1eeddf2e23
orderings on AcctIdx dirty ( #20087 )
2021-09-22 11:24:16 -05:00
681d6d121e
AcctIdx: 2 upserts use disk ( #20072 )
2021-09-22 11:14:08 -05:00
deab944b30
AcctIdx: flush inserts to disk ( #20074 )
2021-09-22 09:55:40 -05:00
804ed825f7
AcctIdx: rework timing of bg loop ( #20077 )
2021-09-22 09:39:49 -05:00
a6aef76f52
AcctIdx: put_range_in_cache loads from disk ( #20073 )
2021-09-22 09:37:34 -05:00
c944bfb1d9
AcctIdx tests use disk buckets ( #20070 )
2021-09-22 08:55:58 -05:00
9e9417b474
AcctIdx: initialze stats with # bins ( #20088 )
2021-09-22 08:55:07 -05:00
b918c2f4cd
AcctIdx: remove_if_slot_list_empty uses disk backing ( #20069 )
2021-09-22 08:48:44 -05:00
6c518102dd
AcctIdx: fix some missing dirty calls ( #20089 )
2021-09-22 08:45:08 -05:00
1b58166e9d
chore: bump memmap2 from 0.4.0 to 0.5.0 ( #20057 )
...
* chore: bump memmap2 from 0.4.0 to 0.5.0
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-21 17:20:49 -06:00
e6795e6fdd
AcctIdx: get can load from disk ( #20067 )
2021-09-21 12:40:07 -05:00
7f3d445af5
AcctIdx: Orderings and cleanup ( #20046 )
2021-09-21 10:52:39 -05:00
bbe151e594
AcctIdx: only flush if disk index being used ( #20045 )
2021-09-21 08:42:13 -05:00
38ea3aa31a
AcctIdx: keys impl when items are on disk ( #20039 )
2021-09-21 08:41:39 -05:00
fb71e80962
AcctIdx: accessors for age ( #20023 )
2021-09-21 08:41:17 -05:00
c3679ab9bd
AcctIdx: first pass at aging buckets ( #20008 )
2021-09-21 08:40:55 -05:00
b507715d44
CPI without Account Refs ( #20034 )
...
* Removes search for accounts and unsafe lifetime transmute in InvokeContext::push().
* Replaces accounts by account_indices in verify_and_update() and process_cross_program_instruction().
Co-authored-by: Justin Starry <justin.m.starry@gmail.com>
2021-09-21 14:41:02 +02:00
94330de843
AcctIdx: delete from disk index ( #20035 )
2021-09-20 16:19:21 -05:00
6399a87485
AcctIdx: report age in stats ( #20025 )
2021-09-20 14:30:12 -05:00
e9ee90a121
AcctIdx: age adds time component ( #20024 )
2021-09-20 14:29:34 -05:00
d842705622
AcctIdx: when bg processing in multiple threads, avoid bin collisions ( #20019 )
2021-09-20 11:50:22 -05:00
e6934e7247
AcctIdx: consolidate to correct CondVar ( #20017 )
2021-09-20 09:58:20 -05:00
55d9ff9899
WaitableCondVar supports notify_one ( #20013 )
2021-09-20 09:57:57 -05:00
bc7983b13e
AcctIdx: metric for active bg threads ( #20026 )
2021-09-20 09:44:40 -05:00
2a42f8a06e
AcctIdx: disk index now holds a bucket map ( #19995 )
2021-09-20 08:40:10 -05:00
af5b0d42a8
AcctIdx: initial index items have future age to flush ( #20010 )
2021-09-19 20:22:09 -05:00
c1d181add5
add --accounts-index-memory-limit-mb ( #19269 )
2021-09-19 18:00:15 -05:00
f579f73700
AcctIdx: introduce age per in-mem bucket ( #19982 )
2021-09-18 22:09:37 -05:00
742155c214
call set_startup and add metrics on generate_index ( #20006 )
2021-09-18 22:08:58 -05:00
db40d06a39
AcctIdx: introduce startup to bucket map ( #20004 )
2021-09-18 12:55:57 -05:00
bed0049a51
AcctIdx: add age per item in in_mem acct idx ( #19981 )
2021-09-18 09:56:26 -05:00
8df8f4396d
AcctIdx: start calling flush on in mem cache buckets ( #19966 )
...
* AcctIdx: start calling flush on in mem cache buckets
* fix orderings
2021-09-18 09:56:08 -05:00
4089f8b06b
AcctIdx: with multiple bg threads, only log stats once per time interval ( #19984 )
2021-09-18 09:55:20 -05:00
4e038e94fd
add 'drives' to AccountsIndexConfig ( #19989 )
2021-09-18 09:54:57 -05:00
f500c99a6d
pass AcctIdxConfig to BucketMapHolder ( #19997 )
2021-09-18 09:54:00 -05:00
bd86f41e18
acct idx tests use 2 bins instead of 8k ( #19996 )
2021-09-18 09:53:44 -05:00
4dc2f08198
AcctIdx: hold_range_in_memory ( #19955 )
2021-09-17 22:19:29 +00:00
24b136a993
AcctIdx: configurable flush threads ( #19983 )
2021-09-17 22:02:43 +00:00
b2152be3b2
introduce bucket map ( #19848 )
...
* introduce bucket map
* rename BucketMap bits to num_buckets_pow2
* use u64::BITS
* Store the number of buckets in BucketMapConfig as a regular number
* remove redundant type aliases
* use Slot from sdk
* use Arc::clone() instead
* fixup erase drives
* rename num_buckets to max_buckets
* add doc to BucketMapConfig::new()
* add more documentation
* rename to DEFAULT_CAPACITY_POW2
* doc
* add more traits while we can
* rename capacity to capacity_pow2
* fix a naming for max_buckets_pow2
* remove unused/incorrect DataBucket::bytes
* rework benches a bit
* fixup bench docs
* rename create_bucket_capacity_pow2 to bucket_capacity_when_created_pow2
* rename BucketMapKeyValue to BucketItem
* rename to items_in_range
* remove values()
* remove addref and unref
* remove more addref and unref
* resurect addref and unref since tests use 'em for now
* rename to BucketStorage
* move stats in bucket_stats
* remove specializations (i don't think they are needed)
* move MaxSearch and RefCount into lib.rs
* move BucketItem to bucket_item.rs
* add doc
* keys no longer returns an option
* Revert "remove specializations (i don't think they are needed)"
This reverts commit b22f78e072cf0f7107851b08e58c2e3fead3f64d.
Co-authored-by: Brooks Prumo <brooks@solana.com>
2021-09-17 15:11:27 -05:00
cddb9da4f0
AcctIdx: central age functions/state ( #19980 )
2021-09-17 15:11:07 -05:00
99f5684dc4
AcctIdx: handle future config defaults ( #19967 )
2021-09-17 13:12:06 -05:00
9998e16df3
AcctIdx: prepare to spin up multiple bg threads ( #19969 )
2021-09-17 11:53:25 -05:00
7b0bf64404
cleanup old features ( #19956 )
2021-09-17 09:46:49 -07:00
37d6f5013f
AcctIdx: initialize acct idx by config ( #19968 )
2021-09-17 16:39:41 +00:00
b8106e2450
chore: bump memmap2 from 0.3.1 to 0.4.0 ( #19974 )
...
* chore: bump memmap2 from 0.3.1 to 0.4.0
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.3.1...v0.4.0 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-09-17 10:00:12 -06:00
376b20c3d2
AcctIdx: add some bucket flush helpers ( #19965 )
2021-09-17 10:41:30 -05:00
4d8b3aa578
AcctsIdx: introduce dirty state per pubkey ( #19960 )
2021-09-16 21:36:23 -05:00
58f25a8752
AcctIdx: move background() to AccountsIndexStorage ( #19948 )
2021-09-16 17:52:06 -05:00
66e0fafc21
AccountsIndexStorage holds InMemAccountsIndex[] ( #19947 )
2021-09-16 21:12:22 +00:00
dc69cc1ae4
Only allow votes when root distance gets too high ( #19917 )
2021-09-16 15:12:26 +02:00
bd434d92e1
nit: misspelling ( #19929 )
2021-09-16 01:27:26 +00:00
fd60ef78eb
chore: bump flate2 from 1.0.21 to 1.0.22 ( #19863 )
...
* chore: bump flate2 from 1.0.21 to 1.0.22
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.21 to 1.0.22.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.21...1.0.22 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-09-15 14:30:30 -06:00
eddd583cd5
AcctIdx: pass bins to BucketMapHolder ( #19906 )
2021-09-15 18:07:53 +00:00
99f2c746d1
AcctIdx: Put InMemAcctIndex behind arc ( #19904 )
2021-09-15 17:36:08 +00:00
b467e7fb3b
AcctIdx: remove auto Debug trait ( #19905 )
2021-09-15 09:54:16 -05:00
8e3c420414
Add logging after taking a bank snapshot ( #19891 )
2021-09-15 07:39:21 -05:00
d3c80be7d5
AcctIdx: use stats() accessor ( #19890 )
2021-09-15 05:28:44 +00:00
691bea8776
extract map to map() ( #19883 )
2021-09-14 23:08:17 -05:00
229b378ed1
in-mem acct idx hash map uses rwlock ( #19878 )
2021-09-14 18:01:34 -05:00
fc647eed19
introduce trait IndexValue for accounts index ( #19859 )
2021-09-14 17:51:07 -05:00
21bc7e94f5
Add logging of incremental snapshots to purge_old_snapshot_archives() ( #19874 )
2021-09-14 17:49:15 -05:00
60276da1fd
rework acct idx entry to prepare for hashmap ( #19875 )
2021-09-14 15:27:23 -05:00
2a795614d3
accounts index keys returns by value ( #19858 )
2021-09-14 09:38:59 -05:00
00d7981f64
Fix native invoke writable privileges ( #19750 )
...
* Fix native invoke writable privileges
* build downstream spl bpf programs for tests
2021-09-13 22:57:37 -07:00
910f241c3f
add <T> through accounts index ( #19852 )
2021-09-13 22:59:03 -05:00
b57e86abf2
cache account hash info ( #19426 )
...
* cache account hash info
* ledger_path -> accounts_hash_cache_path
2021-09-13 20:39:26 -05:00
dca49a614f
size and per-bin stats ( #19842 )
2021-09-13 19:26:49 -05:00
87a7f00926
Track reset bank in PohRecorder ( #19810 )
2021-09-13 16:55:35 -07:00
3071c4caad
Set is_startup=true for exhaustively_free_unused_resource() ( #19846 )
2021-09-13 18:08:06 -05:00
e1a038e791
add stats inserts, deletes, updates_in_mem ( #19807 )
2021-09-13 15:59:01 -05:00
ef749a2506
Flush accounts cache before calling clean_accounts() ( #19839 )
2021-09-13 14:56:13 -05:00
c05226620f
metrics on accounts index entry ( #19806 )
2021-09-13 08:33:45 -05:00
3f6eb96d6e
Call `Bank::update_accounts_hash()` before taking bank snapshot ( #19765 )
...
This is a bugfix.
When `load_frozen_forks()` called `snapshot_bank()`, the accounts hash
was not computed. So then when a snapshot archive was eventually
created, its hash would be all 1s, which is clearly wrong. The fix is
to update the bank's accounts hash before taking the bank snapshot.
2021-09-13 08:27:50 -05:00
d388c5c257
move update_key_if_exists to InMemAccountsIndex ( #19803 )
2021-09-13 08:27:21 -05:00
3cea535fdb
InMemAccountsIndex::remove is no longer called ( #19805 )
2021-09-12 22:01:41 -05:00
b992c02708
in_mem_accounts_index filters by range ( #19779 )
2021-09-12 21:57:15 -05:00
0263ffb2ed
move upsert and a handful of helpers to InMemAccountsIndex ( #19799 )
2021-09-12 21:54:09 -05:00
3617d43e76
accounts index stats ( #19797 )
2021-09-12 22:14:59 +00:00
d9674f7ff0
consolidate another use of remove_if_slot_list_empty ( #19804 )
2021-09-12 19:05:44 +00:00
62c8bcf565
Add default() to SnapshotConfig ( #19776 )
2021-09-12 13:44:27 -05:00
c9a3b8941f
move insert_new_entry_if_missing_with_lock into InMemAccountsIndex ( #19800 )
2021-09-12 13:39:29 -05:00
a295febf04
move remove_if_slot_list_empty to InMemAccountsIndex ( #19801 )
2021-09-12 10:26:04 -05:00
361101bd31
rename generic V -> T. Matches intent better. V is now a larger type that contains T ( #19798 )
2021-09-12 10:25:09 -05:00
c9261e034e
remove dev deps
2021-09-11 14:33:53 -07:00
c3a610374a
unused deps
2021-09-11 14:33:53 -07:00
91c3b18e1e
InMemAccountsIndex iter returns vec<owned> ( #19778 )
2021-09-11 20:33:04 +00:00
7de2236284
accounts index get returns value ( #19777 )
2021-09-11 13:28:56 -05:00
ed7a6c6732
push use of AccountMapEntry one level deeper ( #19775 )
2021-09-11 02:09:51 +00:00
cba834808a
refactor generate_index zero lamports ( #19769 )
2021-09-10 23:46:08 +00:00
595bba95b4
accounts index traits ( #19768 )
2021-09-10 18:32:10 -05:00
9899cd359a
accounts index iterator uses copy of pubkey and account map entry ( #19767 )
2021-09-10 17:52:49 -05:00
11b10439b4
factor out InMemAccountsIndex to prepare for disk index ( #19773 )
2021-09-10 17:52:25 -05:00
7aa5f6b833
Add CLI args for incremental snapshots ( #19694 )
...
Add `--incremental-snapshots` flag to enable incremental snapshots.
This will allow setting `--full-snapshot-interval-slots` and
`--incremental-snapshot-interval-slots`.
Also added `--maximum-incremental-snapshots-to-retain`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-09-10 15:59:26 -05:00
2de2c6ddd7
log accounts loaded for shrink stats ( #19747 )
2021-09-10 12:51:58 -05:00
098585234d
Add return data implementation
...
This consists of:
- syscalls
- passing return data from invoked to invoker
- printing to stable log
- rust and C SDK changes
2021-09-10 14:25:54 +01:00
88c1b8f047
Index loaders / executable accounts ( #19469 )
...
* Appends loaders / executable_accounts to accounts in transaction loading.
* Adds indices to loaders / executable_accounts.
* Moves MessageProcessor::create_keyed_accounts() into InvokeContext::push().
* Removes "executable_accounts",
now referenced by transaction wide index into "accounts".
* Removes create_pre_accounts() from InstructionProcessor,
as it is already in MessageProcessor.
* Collect program account indices directly in load_executable_accounts().
2021-09-10 08:36:21 +02:00
df5befdaca
metrics on rent partitions ( #19728 )
2021-09-09 10:50:35 -05:00
8489ee7df9
Revert "collect rent in parallel for partitions ( #19478 )" ( #19723 )
...
This reverts commit 1560d5e3c3
2021-09-08 21:27:08 -05:00
97418bede3
introduce BinnedHashData type to improve some code readability ( #19722 )
2021-09-08 19:25:52 -05:00
1cf7ae51f8
add comment ( #19705 )
2021-09-08 19:25:20 -05:00
001ed66b4d
minor constant cleanup ( #19721 )
2021-09-08 18:26:10 -05:00
38bbb77989
Return error if Transaction contains writable executable or ProgramData accounts ( #19629 )
...
* Return error if Transaction locks an executable as writable
* Return error if a ProgramData account is writable but the upgradable loader isn't present
* Remove unreachable clause
* Fixup bpf tests
* Review comments
* Add new TransactionError
* Disallow writes to any upgradeable-loader account when loader not present; remove is_upgradeable_loader_present exception for all other executables
2021-09-08 15:21:52 -06:00
3a8c678f62
Remove some copying ( #19691 )
2021-09-08 18:32:38 +02:00
456bf15012
AccountsIndexConfig -> AccountsDbConfig ( #19687 )
2021-09-08 04:30:38 +00:00
d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it ( #19686 )
2021-09-07 20:09:40 -05:00
4a5f83d3a7
Add fn to check when to take snapshots ( #19682 )
2021-09-07 23:26:35 +00:00
a0552e5b46
Make startup aware of Incremental Snapshots ( #19600 )
2021-09-07 20:43:43 +00:00
982454a455
refactor do_shrink_slot_stores ( #19560 )
2021-09-07 15:10:49 -05:00
1560d5e3c3
collect rent in parallel for partitions ( #19478 )
...
* collect rent in parallel for partitions
* combine use
* use fg accounts db thread pool
2021-09-07 11:51:59 -05:00
9d9482b9d8
Plumb `maximum_incremental_snapshot_archives_to_retain` ( #19640 )
2021-09-06 18:01:56 -05:00
8b9e472a6c
feat: add ed25519 signature verify program
...
Solang requires a method for verify ed25519 signatures. Add a new
builtin program at address Ed25519SigVerify111111111111111111111111111
which takes any number of ed25519 signature, public key, and message.
If any of the signatures fails to verify, an error is returned.
The changes for the web3.js package will go into another commit, since
the tests test against a released solana node. Adding web3.js ed25519
testing will break CI.
2021-09-05 18:59:37 +01:00
d461a9ac10
verify_precompiles needs FeatureSet
...
Rather than pass in individual features, pass in the entire feature set
so that we can add the ed25519 program feature in a later commit.
2021-09-05 18:59:37 +01:00
333e5a9446
Make purge_old_snapshot_archives aware of maximum_incremental_snapshot_archives_to_retain ( #19615 )
2021-09-04 07:37:29 -05:00
decec3cd8b
Demote write locks on transaction program ids ( #19593 )
...
* Add feature
* Demote write lock on program ids
* Fixup bpf tests
* Update MappedMessage::is_writable
* Comma nit
* Review comments
2021-09-04 03:05:30 +00:00
7578db7ee3
add lowest_pubkey_from_bin ( #19617 )
2021-09-04 02:20:55 +00:00
b3fae0a01a
move constants in functions to global space ( #19622 )
2021-09-03 19:46:42 -05:00
e6055010eb
accounts index iterator takes &Range ( #19618 )
2021-09-03 18:00:49 -05:00
2c3bdedea3
stake: Advance `credits_observed` on activation epoch ( #19309 )
...
* stake: Advance `credits_observed` on activation epoch
* Add test for merging stakes just after activation
2021-09-03 23:20:56 +02:00
5e25ee5ebe
Add maximum_incremental_snapshot_archives_to_retain to SnapshotConfig ( #19612 )
2021-09-03 20:21:32 +00:00
7ab0aec61f
Rename maximum_full_snapshot_archives_to_retain ( #19610 )
...
To prepare for adding maximum_incremental_snapshot_archives_to_retain,
rename the current field in SnapshotConfig.
2021-09-03 11:28:10 -05:00
df040c05da
get rid of unneeded mut and refactor addref ( #19594 )
2021-09-03 10:45:37 -05:00
170927b7c4
nit: Move builtins types to builtins file ( #19597 )
2021-09-02 21:29:11 -07:00
3936c10aa0
rename var ( #19588 )
2021-09-03 00:55:35 +00:00
9cc57fb076
remove file that accidentally got added ( #19586 )
2021-09-03 00:15:25 +00:00
e9374d32a3
Revert "Make startup aware of Incremental Snapshots ( #19550 )" ( #19599 )
...
This reverts commit d45ced0a5d
2021-09-02 19:14:41 -05:00
d45ced0a5d
Make startup aware of Incremental Snapshots ( #19550 )
2021-09-02 19:05:15 -05:00
57f51352f6
refactor AccountEntry addref/unref ( #19583 )
2021-09-02 23:25:27 +00:00
682daf1117
cleanup of allocate_accounts_index ( #19585 )
2021-09-02 22:58:07 +00:00
78585a992b
add IsCached trait bounds to AccountsIndex ( #19584 )
2021-09-02 22:45:35 +00:00
2874f6b0a2
create remove_if_slot_list_empty to define accounts index behavior ( #19581 )
2021-09-02 16:54:37 -05:00
0571962776
accounts index test uses # test bins ( #19580 )
2021-09-02 16:32:42 -05:00
6ab8286e8d
add logging for a handful of tests ( #19582 )
2021-09-02 16:32:19 -05:00
f5388cfe71
tests which assert or panic in other threads now will report an error ( #19532 )
...
* tests which assert or panic in other threads now will report an error
* rename, add assert detecting time out
2021-09-02 11:26:57 -05:00
7eb793a55e
add debugging to test ( #19563 )
...
* add debugging to test
* update cargo
2021-09-02 02:52:16 +00:00
ed2c071fe1
remove unused is_startup ( #19561 )
2021-09-01 18:39:00 -05:00
8378e8790f
Accountsdb replication installment 2 ( #19325 )
...
This is the 2nd installment for the AccountsDb replication.
Summary of Changes
The basic google protocol buffer protocol for replicating updated slots and accounts. tonic/tokio is used for transporting the messages.
The basic framework of the client and server for replicating slots and accounts -- the persisting of accounts in the replica-side will be done at the next PR -- right now -- the accounts are streamed to the replica-node and dumped. Replication for information about Bank is also not done in this PR -- to be addressed in the next PR to limit the change size.
Functionality used by both the client and server side are encapsulated in the replica-lib crate.
There is no impact to the existing validator by default.
Tests:
Observe the confirmed slots replicated to the replica-node.
Observe the accounts for the confirmed slot are received at the replica-node side.
2021-09-01 14:10:16 -07:00
a5f74d86c8
make test pass with 1 bin to remove noisy failure during testing ( #19554 )
2021-09-01 15:58:56 -05:00
dd9481c403
parallelize index_read in shrink ( #19506 )
2021-09-01 15:40:59 -05:00
82a6bbe068
add Debug trait to range for accounts index ( #19531 )
2021-09-01 08:13:56 -05:00
12dc8749cf
add Copy trait to AccountInfo for fast copies to mmapped file ( #19524 )
2021-09-01 08:13:08 -05:00
0088aefa24
Fix tests that make assumptions about tx fee rate ( #19537 )
2021-08-31 22:02:14 -06:00
1d5a8ebc6a
Revert "Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )" ( #19529 )
...
This reverts commit 4d361af976
2021-08-31 22:03:19 -05:00
2df96cd81e
remove type specifics to make funciton more flexible ( #19536 )
2021-08-31 20:03:42 -05:00
fe9ee9134a
Make background services aware of incremental snapshots ( #19401 )
...
AccountsBackgroundService now knows about incremental snapshots. It is
now also in charge of deciding if an AccountsPackage is destined to be a
SnapshotPackage or not (or just used by AccountsHashVerifier).
!!! New behavior changes !!!
Taking snapshots (both bank and archive) **MUST** succeed.
This is required because of how the last full snapshot slot is
calculated, which is used by AccountsBackgroundService when calling
`clean_accounts()`.
File system calls are now unwrapped and will result in a crash. As Trent told me:
>Well I think if a snapshot fails due to some IO error, it's very likely that the operator is going to have to intervene before it works. We should exit error in this case, otherwise the validator might happily spin for several more hours, never successfully writing a complete snapshot, before something else brings it down. This would leave the validator's last local snapshot many more slots behind than it would be had we exited outright and potentially force the operator to abandon ledger continuity in favor of a quick catchup
Other errors will set the `exit` flag to `true`, and the node will gracefully shutdown.
Fixes #19167
Fixes #19168
2021-08-31 18:33:27 -05:00
718ab7c12e
Revert "Add snapshot_runtime_info module ( #18199 )"
...
This reverts commit c1b9d40a64
2021-08-31 17:50:18 -05:00
5cf8d8795b
Feature cleanup ( #19528 )
2021-08-31 14:51:26 -07:00
e1939688c2
parallel shrink ( #19507 )
2021-08-31 11:57:34 -05:00
0e94b8fd4e
remove redundant cancelled_shrink ( #19522 )
2021-08-31 11:51:48 -05:00
9d3afba045
Debug shrink stats ( #19505 )
...
* add some shrink metrics
* renames
2021-08-31 09:04:56 -05:00
09458cc802
Shrink mmap leakage ( #19373 )
...
* Putting stores satisfying future shrink into the candidate list.
* Fixed unit tests for select_shrink_candidates
* Added metrics on stores readded back for next round
* Ehance the metrics on accountsdb to report the total bytes and alive bytes and alive ratio
* Enhance select_shrink_candidates metrics
2021-08-31 00:55:16 -07:00
a655b01700
some basic accounts index refactoring ( #19510 )
2021-08-30 18:40:10 -05:00
6cfd44b811
chore: bump flate2 from 1.0.20 to 1.0.21 ( #19500 )
...
* chore: bump flate2 from 1.0.20 to 1.0.21
Bumps [flate2](https://github.com/rust-lang/flate2-rs ) from 1.0.20 to 1.0.21.
- [Release notes](https://github.com/rust-lang/flate2-rs/releases )
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.0.20...1.0.21 )
---
updated-dependencies:
- dependency-name: flate2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-30 16:48:02 -06:00
6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 ( #19497 )
...
* chore: bump serde from 1.0.129 to 1.0.130
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-30 18:19:34 +00:00
8ad52fa095
implements copy-on-write for vote-accounts ( #19362 )
...
Bank::vote_accounts redundantly clones vote-accounts HashMap even though
an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/95c998a19/runtime/src/bank.rs#L5174-L5186
This commit implements copy-on-write semantics for vote-accounts by
wrapping the underlying HashMap in Arc<...>.
2021-08-30 15:54:01 +00:00
2d7f036afd
Add solana-program-runtime crate ( #19438 )
2021-08-27 00:30:36 +00:00
02b050e0f5
replace AccountsIndex btree with hashmap of 8k bins ( #19212 )
2021-08-26 18:12:43 -05:00
98bc694606
hash calculation adds really old slots to dirty_stores ( #19434 )
2021-08-26 14:32:43 -05:00
535de3b302
add test for test_clean_nonrooted ( #19409 )
2021-08-26 13:56:09 -05:00
e492638b7d
chore: bump libc from 0.2.100 to 0.2.101 ( #19442 )
...
* chore: bump libc from 0.2.100 to 0.2.101
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.100 to 0.2.101.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.100...0.2.101 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-26 12:38:34 -06:00
46890ac197
chore: bump serde from 1.0.128 to 1.0.129 ( #19395 )
...
* chore: bump serde from 1.0.128 to 1.0.129
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.128 to 1.0.129.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.128...v1.0.129 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-24 12:32:30 -06:00
c0c95e88d8
chore: bump libc from 0.2.99 to 0.2.100 ( #19371 )
...
* chore: bump libc from 0.2.99 to 0.2.100
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.99 to 0.2.100.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.99...0.2.100 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-08-23 09:34:32 -06:00
aea9960c75
chore: bump serde from 1.0.127 to 1.0.128 ( #19370 )
...
* chore: bump serde from 1.0.127 to 1.0.128
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.127 to 1.0.128.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.127...v1.0.128 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
2021-08-23 09:33:34 -06:00
1a4bede918
Remove filter_snapshot_storages_for_incremental_snapshot() ( #19349 )
...
Now that we can get just the subset of snapshot storages we want, the extra
filtering afterwards is redundant.
2021-08-23 09:45:29 -05:00
6d939811e9
Name snapshots consistently ( #19346 )
...
#### Problem
Snapshot names are overloaded, and there are multiple terms that mean the same thing. This is confusing. Here's a list of ones in the codebase that I've found:
```
- snapshot_dir
- snapshots_dir
- snapshot_path
- snapshot_output_dir
- snapshot_package_output_path
- snapshot_archives_dir
```
#### Summary of Changes
For all the ones that are about the directory where snapshot archives are stored, ensure they are `snapshot_archives_dir`. For the ones about the (bank) snapshots directory, set to `bank_snapshots_dir`.
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-08-21 15:41:03 -05:00
234461f779
Add base_slot to get_snapshot_storages() ( #19348 )
2021-08-20 16:23:43 -05:00
4d361af976
Add LastFullSnapshotSlot to SnapshotConfig ( #19341 )
2021-08-20 17:06:53 +00:00
3e5ba8dcaa
bug: `sysvar::Instructions` is not owned by `Sysvar1111111111111111111111111111111111111` ( #19242 )
...
* Fix instructions sysvar owner
* Update feature switch address
Co-authored-by: Justin Starry <justin@solana.com>
2021-08-20 08:32:28 -07:00
73aa004c59
stake: Remove v2 program references ( #19308 )
...
* stake: Remove v2 program references
* Remove stake v2 feature, along with stake rewrite
2021-08-20 01:08:44 -04:00
3ec33e7d02
Fail secp256k1 if the instruction data looks incorrect ( #19300 )
2021-08-19 13:13:54 -07:00
89a31ff473
change untar to use unpack instead of unpack_in ( #19216 )
...
* change untar to use unpack instead of unpack_in
* hacky, but maybe passes tests
* chore: bump tar from 0.4.35 to 0.4.37
Bumps [tar](https://github.com/alexcrichton/tar-rs ) from 0.4.35 to 0.4.37.
- [Release notes](https://github.com/alexcrichton/tar-rs/releases )
- [Commits](https://github.com/alexcrichton/tar-rs/compare/0.4.35...0.4.37 )
---
updated-dependencies:
- dependency-name: tar
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* cleanup
* cleanup, add validate_inside_dst
* collapse use
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
* delete comment line
* add comments
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Tyera Eulberg <teulberg@gmail.com>
2021-08-18 15:49:02 -05:00
6327e006df
CalculateHashIntermediate::new_without_slot -> new ( #19274 )
2021-08-18 09:07:34 -05:00
c167211611
Expose genesis block time via rpc ( #19267 )
...
* Expose genesis_creation_time from Bank
* Backfill genesis_creation_time for block/block-time requests of slot 0
2021-08-17 16:29:34 -06:00
c50b01cb60
Store versioned transactions in the ledger, disabled by default ( #19139 )
...
* Add support for versioned transactions, but disable by default
* merge conflicts
* trent's feedback
* bump Cargo.lock
* Fix transaction error encoding
* Rename legacy_transaction method
* cargo clippy
* Clean up casts, int arithmetic, and unused methods
* Check for duplicates in sanitized message conversion
* fix clippy
* fix new test
* Fix bpf conditional compilation for message module
2021-08-17 15:17:56 -07:00
098e2b2de3
chore: bump memmap2 from 0.3.0 to 0.3.1 ( #19255 )
...
* chore: bump memmap2 from 0.3.0 to 0.3.1
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs ) from 0.3.0 to 0.3.1.
- [Release notes](https://github.com/RazrFalcon/memmap2-rs/releases )
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md )
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v.0.3.0...v0.3.1 )
---
updated-dependencies:
- dependency-name: memmap2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-17 15:49:50 -06:00
7c70f2158b
accounts_index_bins to AccountsIndexConfig ( #19257 )
...
* accounts_index_bins to AccountsIndexConfig
* rename param bins -> config
* rename BINS_FOR* to ACCOUNTS_INDEX_CONFIG_FOR*
2021-08-17 14:50:01 -05:00
f9986c66b8
Make SnapshotPackagerService aware of Incremental Snapshots ( #19254 )
...
Add a field to SnapshotPackage that is an enum for SnapshotType, so archive_snapshot_package() will do the right thing.
Fixes #19166
2021-08-17 13:01:59 -05:00
7481d9b66b
exhaustively_free_unused_resource uses more threads ( #19264 )
2021-08-17 12:38:41 -05:00
39a3d5b8a9
implements AsRef (instead of Borrow) for VoteAccounts ( #19252 )
...
According to the docs:
> if you want to borrow only a single field of a struct you can
> implement AsRef, but not Borrow.
https://doc.rust-lang.org/std/convert/trait.AsRef.html
2021-08-16 18:21:33 +00:00
176036aa58
Rename AccountsPacakge to SnapshotPackage and AccountsPackagePre to AccountsPackage ( #19231 )
...
Renaming these types to better communicate their usages, which will
further diverge as incremental snapshot support is added.
With the new names, AccountsPacakge now refers to the type between
AccountsBackgroundProcess and AccountsHashVerifier, and SnapshotPackage
refers to the type between AccountsHashVerifier and
SnapshotPackagerService.
2021-08-13 16:08:09 -05:00
1a3f29e22d
rework bank test to specifically use BTreeMap ( #19225 )
2021-08-13 13:30:48 -05:00
01b00bc593
Fix crossed PRs; remove deprecated fee_calculator usage ( #19228 )
2021-08-13 11:14:00 -06:00
0b50bb2b20
Deprecate FeeCalculator returning APIs ( #19120 )
2021-08-13 09:08:20 -07:00
26e963f436
remove par iter for 32k wide code path ( #18764 )
2021-08-13 10:07:47 -05:00
772341dccb
fixup! Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots ( #18870 ) ( #19213 )
2021-08-12 17:29:58 -05:00
5fb6b341c4
Handle cleaning of zero-lamport accounts w.r.t. Incremental Snapshots ( #18870 )
...
* Handle cleaning zero-lamport accounts
Handle cleaning zero-lamport accounts in slots higher than the last full
snapshot slot. This is part of the Incremental Snapshot work.
Fixes #18825
2021-08-12 15:56:08 -05:00
414d904959
Reject blocks for costs above the max block cost ( #18994 )
...
* added realtime cost checking logic to reject block that would exceed max limit:
- defines max limits at block_cost_limits.rs
- right after each bath's execution, accumulate its cost and check again
limit, return error if limit is exceeded
* update abi that changed due to adding additional TransactionError
* To avoid counting stats mltiple times, only accumulate execute-timing when a bank is completed
* gate it by a feature
* move cost const def into block_cost_limits.rs
* redefine the cost for signature and account access, removed signer part as it is not well defined for now
* check if per_program_timings of execute_timings before sending
2021-08-12 10:48:47 -05:00
446816de52
Remove redundant is_stake check ( #19185 )
2021-08-11 18:38:03 +00:00
5970083b4d
Use last_valid_block_height in services and client apps ( #19163 )
...
* Add deprecated tag to Bank::get_blockhash_last_valid_slot
* Update SendTransactionService to use last_valid_block_height
* Update solana-tokens to use last_valid_block_height
* Remove dangling file
* Update solana program to use last_valid_block_height
* Update Banks crates to use last_valid_block_height
2021-08-11 01:04:00 -06:00
00e5e12906
renames solana_runtime::vote_account::VoteAccount
...
Rename:
VoteAccount -> VoteAccountInner # the private type
ArcVoteAccount -> VoteAccount # the public type
2021-08-10 22:54:17 +00:00
1403eaeefc
makes solana_runtime::vote_account::VoteAccount private
...
VoteAccount is an implementation detail, and should not be public.
Only ArcVoteAccount is the public type.
2021-08-10 22:54:17 +00:00
faf99f4760
Move Bank::get_incremental_snapshot_storages() into snapshot_utils ( #19155 )
...
Filtering out storages for incremental snapshots will be needed by the
background services for incremental snapshot support, but there is not a
Bank at that point. Since the filtering doesn't apply only to Bank, and
more to snapshots, move the functionality into snapshot_utils.
2021-08-10 14:04:11 -05:00
536b763751
chore: bump libc from 0.2.98 to 0.2.99 ( #19142 )
...
* chore: bump libc from 0.2.98 to 0.2.99
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.98 to 0.2.99.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.98...0.2.99 )
---
updated-dependencies:
- dependency-name: libc
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-08-10 10:54:54 -06:00
651343688d
Use option for account index bucket parameter ( #19150 )
2021-08-10 11:32:25 -05:00
05d92402f1
Simplify AccountsPackagePre::new() and friends ( #19134 )
2021-08-10 11:10:15 -05:00
f302774cf7
implements copy-on-write for staked-nodes ( #19090 )
...
Bank::staked_nodes and Bank::epoch_staked_nodes redundantly clone
staked-nodes HashMap even though an immutable reference will suffice:
https://github.com/solana-labs/solana/blob/a9014cece/runtime/src/vote_account.rs#L77
This commit implements copy-on-write semantics for staked-nodes by
wrapping the underlying HashMap in Arc<...>.
2021-08-10 12:59:12 +00:00
Justin Starry
Brooks Prumo
Tyera Eulberg
Tyera Eulberg
axleiro
dependabot[bot]
Jeff Washington (jwash)
Ashwin Sekar
Alexander Meißner
Trent Nelson
Michael Vines
Tao Zhu
Jack May
Anton Lazarev
ryleung-solana
sakridge
Jon Cinque
carllin
Lijun Wang
Jeff Washington (jwash)
Ivan Mironov
Giles Cope
Sean Young
Brooks Prumo
behzad nouri
jon-chuang
steviez