7ba3e710d3
latest_slot uses > current_max ( #16374 )
2021-04-12 10:52:24 -05:00
6930a77a0f
prepare replace Ancestors HashMap for performance ( #16476 )
2021-04-12 10:51:57 -05:00
dc7030ffaa
Allow fork choice to support multiple versions of a slot ( #16266 )
2021-04-12 01:00:59 -07:00
ac5462e7a2
Merge pull request from GHSA-fmvj-vqp5-qqh9
...
* Sanitize permissions
* Forbid creating directories under ledger/rocksdb/
* hardened_unpack: Disallow dirs under rocksdb/ in genesis
* hardened_unpack: expand valid genesis entry test coverage
* hardened_unpack: rework old-style bsd directory entry rejection
Co-authored-by: Ivan Mironov <mironov.ivan@gmail.com>
2021-04-10 00:57:32 -06:00
b08cff9e77
Simplify some pattern-matches ( #16402 )
...
When those match an exact combinator on Option / Result.
Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust ).
2021-04-08 12:40:37 -06:00
e69f37a464
Remove old function: account_balance_for_capitalization ( #16383 )
...
This function currently returns one of its' parameters and is thus useless.
2021-04-07 23:57:40 -05:00
476fd40948
Add program-test tests to bpf programs ( #16407 )
2021-04-07 01:47:15 -07:00
92f4018b07
Cleanup unsupported sysvars ( #16390 )
...
* Cleanup unsupported sysvars
* fix ser description
2021-04-06 00:08:03 -07:00
95dc7b5449
reduce lock contention on latest_root ( #16306 )
2021-04-05 11:35:14 -05:00
7a997759fa
reduce locking on get_rooted_entries ( #16304 )
2021-04-05 11:26:53 -05:00
0596cf5405
Remove unprocessed transactions from log notifications ( #16349 )
2021-04-04 23:44:59 +08:00
3996b699dc
read only account cache for executable accounts - improve replay ( #16150 )
...
* read only account cache
* tests
* clippy
* cleanup
* new file, add tests
* remove copy/paste code from test
* remove dead code
* all loads use cache
* remove stale comments
* add metrics logging for read only cache size
* report read only cache hits and misses
* consistency
* formatting
* rename, add comment
* u64
* better interaction with existing cache
* lru list saved between cleans
2021-04-01 07:16:34 -05:00
3f63ed9a72
removes OrderedIterator and transaction batch iteration order ( #16153 )
...
In TransactionBatch,
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/transaction_batch.rs#L4-L11
lock_results[i] is aligned with transactions[iteration_order[i]]:
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2414-L2424
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/accounts.rs#L788-L817
However load_and_execute_transactions is iterating over
lock_results[iteration_order[i]]
https://github.com/solana-labs/solana/blob/e50f59844/runtime/src/bank.rs#L2878-L2889
and then returning i as for the index of the retryable transaction.
If iteratorion_order is [1, 2, 0], and i is 0, then:
lock_results[iteration_order[i]] = lock_results[1]
which corresponds to
transactions[iteration_order[1]] = transactions[2]
so neither i = 0, nor iteration_order[i] = 1 gives the correct index for the
corresponding transaction (which is 2).
This commit removes OrderedIterator and transaction batch iteration order
entirely. There is only one place in blockstore processor which the
iteration order is not ordinal:
https://github.com/solana-labs/solana/blob/e50f59844/ledger/src/blockstore_processor.rs#L269-L271
It seems like, instead of using an iteration order, that can shuffle entry
transactions in-place.
2021-03-31 23:59:19 +00:00
416312b30b
update timings only on outer call ( #16275 )
...
* update timings only on outer call
* test & format
2021-03-31 18:02:59 -05:00
f374b35944
Hash stored accounts in bg ( #16157 )
...
* lazy calculate account hash
* push to bg thread
* remove deadlock
* logs
* format
* some cleanup on aisle 9
* format, fix up some metrics
* fix test, remove legacy function only there for tests
* cleanup
* remove unused store_hasher
* Switch to crossbeam
* clippy
* format
* use iter()
* rework from feedback
* hash_slot -> slot
* hash(cluster_type)
Co-authored-by: Carl Lin <carl@solana.com>
2021-03-31 15:39:34 -05:00
a5dcee254d
get rid of unnecessary data copy ( #16214 )
...
* get rid of unnecessary data copy
* preserve rent_epoch
2021-03-31 09:11:39 -05:00
54c68ea83f
Drop write lock on sysvars ( #15497 )
...
* Drop write lock on sysvars
* adds env var for demoting sysvar write lock demotion
* moves demote logic to is_writable
* feature gates sysvar write lock demotion
* adds builtins to write lock demotion
* adds system program id to builtins
* adds Feature111...
* adds an abi-freeze test
* mvines set of builtin program keys
Co-authored-by: Michael Vines <mvines@gmail.com>
* update tests
* adds bpf loader keys
* Add test sysvar
* Plumb demote_sysvar to is_writable
* more plumbing of demote_sysvar_write_locks to is_writable
* patches test_program_bpf_instruction_introspection
* hard codes demote_sysvar_write_locks to false for serialization/encoding methods
* Revert "hard codes demote_sysvar_write_locks to false for serialization/encoding methods"
This reverts commit ae3e2d2e777437bddd753933097a210dcbc1b1fc.
* change the hardcoded ones to demote_sysvar_write_locks=true
* Use data_as_mut_slice
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-30 10:05:09 -07:00
5e5b63712b
Status cache improvements ( #16174 )
2021-03-28 19:10:14 -07:00
66c42f62d8
make AccountSharedData.data private to abstract storage ( #16091 )
...
* format
* 2 more tests
* use
2021-03-25 11:04:20 -05:00
6d5c6c17c5
Simplify account.rent_epoch handling for sysvar rent ( #16049 )
...
* Add some code for special local testing
* Add comment to store_account_and_update_capitalization
* Simplify account.rent_epoch handling for sysvar rent
* Introduce *_for_test functions
* Add deprecation messages to existing api
2021-03-25 15:23:20 +09:00
26af2d4c26
Up store limit for test_account_grow ( #16099 )
...
There can be more than 5 stores
2021-03-24 15:24:17 -07:00
2d24d13046
AccountsSharedData: data copy on write ( #15800 )
...
* Arc<AccountData>
* try custom serializer
* adapt test from Behzad's change
* clippy
* simplify serialization
* remove abi example derive
* refactor 'take'
* remove serialization
* remove serialize calls
* remove account_data
* remove intos
* remove left over file
2021-03-23 15:19:31 -05:00
fde43a906d
revert test to use Account for serialization ( #16063 )
2021-03-22 22:48:09 +00:00
4beb39f7a1
multi-pass bin scanning ( #15377 )
...
* multi-pass bin scanning
* pr feedback
* format
* fix typo
* adjust metrics for code changes
* merge errors
2021-03-18 10:32:07 -05:00
f548a04fae
Allow unbounded wallclock processing time in tests ( #15961 )
2021-03-17 15:48:50 -07:00
a5d144b00f
Revert to removing only tmp-
2021-03-17 10:31:01 -07:00
0209d334bd
Add option for separate snapshot location
...
(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
2021-03-17 10:31:01 -07:00
a2eb655322
=1.7.0
2021-03-16 07:51:07 +00:00
efee8b62d7
a few missed set_data calls ( #15846 )
...
* a few missed set_data calls
* another set data call
2021-03-15 21:57:23 -05:00
e35d738906
rework BankClient.get_account_data ( #15853 )
2021-03-15 19:27:53 -05:00
c1ba265dd9
Wallclock BankingStage Throttle ( #15731 )
2021-03-15 17:11:15 -07:00
0c9ca5522c
Bump version to v1.7.0
2021-03-13 09:01:21 +00:00
918d04e3f0
Add more slot update notifications ( #15734 )
...
* Add more slot update notifications
* fix merge
* Address feedback and add integration test
* switch to datapoint
* remove unused shred method
* fix clippy
* new thread for rpc completed slots
* remove extra constant
* fixes
* rely on channel closing
* fix check
2021-03-12 21:44:06 +08:00
4bbeb9c033
Remove old feature: simple_capitalization ( #15763 )
...
* Remove old feature: simple_capitalization
* Fix another failing test in core
* Finish up test cleanup
* Further clean up a bit
2021-03-12 11:12:40 +09:00
952c3bcbb7
AccountSharedData construction ( #15790 )
2021-03-11 18:09:04 -06:00
3419a5446e
AccountSharedData.set_data ( #15781 )
...
* account.set_data and resize_data
* remove data_resize
2021-03-11 16:40:45 -06:00
1061d021c9
Unpack snapshot AppendVecs directly into account paths
2021-03-11 08:25:24 -08:00
24d18b3cf2
docs: add docs links for crates published to crates.io
2021-03-11 08:38:18 +00:00
478518308d
Share RO and Executable accounts within invocations ( #15799 )
2021-03-10 23:04:00 -08:00
65dd177601
AppendVec cleanup: remove dead code, rename non-idiomatic "new" function
2021-03-10 14:50:59 -08:00
1135ffd595
mut data refs as slice ( #15782 )
2021-03-10 15:28:03 -06:00
9c1198c0c7
Improve load_largest_accounts more ( #15785 )
...
* Add load_largest_accounts bench
* Check lamports before address filter
* Use BinaryHeap, add Accounts test
* Use pubkey reference in the min-heap
Also, flatten code with early returns
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2021-03-10 11:22:02 -07:00
369e13b111
cleanup old runtime features ( #15787 )
2021-03-10 09:48:41 -08:00
3355efbb5a
Fix test to use AccountSharedData
2021-03-09 20:47:54 -07:00
5991cef5f5
Improve load_largest_accounts and add Bank test ( #15775 )
2021-03-10 00:23:41 +00:00
52e54e1100
account.data -> data() ( #15778 )
2021-03-09 22:31:33 +00:00
8a3135d17b
Account->AccountSharedData ( #15691 )
2021-03-09 15:06:07 -06:00
e3e8179f2d
Sdk: start to appease clippy's integer arithmetic check ( #15736 )
...
* Move to module-level clippy allowance
* Fix stake_weighted_timestamp math
* Fix genesis_config
* Fix shred_version
* Fix hard_forks
* Fix process_instruction
* Add ArithmeticOverflow ix error
* Fix nonce_keyed_account
* Update BankSlotDelta frozen abi due to new ix error
2021-03-08 18:37:57 -07:00
beac2d6d3f
Remove stale comment
2021-03-08 10:18:32 +09:00
8b0c6db871
Remove old feature: cumulative_rent_related_fixes ( #15754 )
2021-03-08 09:58:50 +09:00
c5371fdc85
More debug on shrink and flush ( #15518 )
2021-03-05 14:10:34 -08:00
13b9bb6790
refactor function to store into account after data is finished ( #15710 )
2021-03-05 11:02:35 -06:00
edd159e7d7
Fix typos ( #15721 )
2021-03-05 07:16:53 +00:00
872f7117c3
Use OrderedIterator to produce TransactionLogInfo ( #15712 )
...
* Add failing test
* Fix iteration_order issue with stored logs
2021-03-04 15:26:19 -07:00
1e2f5a5f55
Permit the snapshots/status_cache file to be sparse
2021-03-04 12:36:34 -08:00
b53cb8eb2d
Deprecate Instruction::new ( #15695 )
2021-03-04 05:46:48 +00:00
be35c1c1b7
add execute detail timings ( #15638 )
2021-03-03 17:07:45 -06:00
9bbe015a05
Document and clean up AppendVec ( #15640 )
...
* Document AppendVec
* Remove the almost-duplicate state in AppendVec
AppendVec was maintaining two offsets, `current_len` and
`append_offset`. Despite the different looking names, the two
values have the same meaning, but were updated at slightly different
times. When appending a batch of accounts, `current_len` updates would
be immediately available to other threads after each append, whereas
`append_offset` would only be updated after its mutex was unlocked.
`append_offset` is redundant. By removing it, we eliminate potential
bugs and no longer need to suppress clippy warnings.
* Remove get_mut() from AppendVec design
Only the offset into the AppendVec memory is thread-safe. The memory
itself is only threadsafe because it is append-only and is otherwise
unprotected. Adding get_mut() would only be safe if the memory was
protected by a ReadWrite lock.
2021-03-03 12:51:48 -08:00
a9c8dbfd0c
Add error reporting to system program ( #15644 )
2021-03-02 21:44:25 -08:00
a2c47db920
extract expected cluster type ( #15615 )
...
* extract expected cluster type
* fix clippy warning
* more clippy
2021-03-02 13:55:34 -06:00
43663b1750
Fix clippy warnings ( #15632 )
2021-03-02 08:58:14 -08:00
da47ab7e50
fix indentation format problem ( #15618 )
...
* fix indentation format problem
* fix indentation format problem
2021-03-02 10:46:00 -06:00
aad79d9d5c
Remove unused Accounts state ( #15611 )
...
* Remove dead state from Accounts
* Refactor to remove reference to self.slot
Account::default() has zero lamports, so would return None anyway
2021-03-02 08:28:25 -08:00
2463cc1e6a
Fix typos ( #15610 )
2021-03-02 06:36:49 -08:00
97eaf3c334
Fix finalize_dead_slot_removal() of cached slots decrementing refcount ( #15534 )
2021-02-26 17:49:37 -08:00
8399851d11
check program owners ( #15495 )
...
* check program owners
* BankSlotDelta should change because InstructionError variant added
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-02-26 22:21:34 +00:00
21b43009f6
Introduce ttl eviction for RecycleStore ( #15513 )
2021-02-25 17:27:27 +09:00
eddb7f98f5
Check vote account initialization ( #15503 )
...
* Check account data_len on Vote account init
* Check account data populated on update_cached_accounts
2021-02-24 10:00:48 -07:00
2f46da346d
gracefully handle vote account without authorized voter ( #15501 )
2021-02-24 00:00:24 -08:00
9a7cd8885d
Remove read only locks when they hit ref count zero, cleanup accounts locking ( #15449 )
2021-02-22 23:56:43 -08:00
5df36aec7d
Pacify clippy
2021-02-19 20:08:41 -08:00
5d31763a3b
Update ABI digest
2021-02-19 10:54:39 -08:00
02432a548f
Cleanup old features ( #15391 )
2021-02-18 09:56:11 -08:00
e3006e3b46
correctly reserve space for pre_accounts vector ( #15378 )
2021-02-18 09:37:24 -06:00
0f6f6080f3
sdk: Add Borsh support for types and utilities ( #15290 )
...
* sdk: Add Borsh to Pubkey
* Add serialization error for easier borsh integration
* Add Borsh usage to banks-client and sdk
* Rename SerializationError -> IOError
* Add new errors to proto
* Update Cargo lock
* Update Cargo.lock based on CI
* Clippy
* Update ABI on bank
* Address review feedback
* Update sanity program instruction count test
2021-02-18 11:14:56 +01:00
7f7370c306
Re-allow clippy::integer_arithmetic at crate-level
2021-02-17 13:55:08 -07:00
9ba69a7381
Adapt to fs_extra 1.2.0
2021-02-17 12:44:58 -08:00
7fee0bc69c
move hashing functionality to accounts_hash ( #15353 )
2021-02-17 00:29:50 +00:00
8367740ff9
compute merkle root on chunks of fanout^3 ( #15344 )
...
* compute merkle root on chunks of fanout^3
* improve test_accountsdb_compute_merkle_root_large
2021-02-16 17:03:35 -06:00
ba02452d75
add validator flag no-accounts-db-index-hashing ( #15350 )
...
* add validator flag no_accounts_db_index_hashing
* add validator flag no_accounts_db_index_hashing
2021-02-16 21:13:48 +00:00
d83aeebcca
change 2d cumulative calculation ( #15346 )
2021-02-16 14:24:45 -06:00
708b8380b7
assert 2 items in stores don't have same slot and version with different values ( #15250 )
...
* assert 2 items in stores don't have same slot and version with different values
* add message to assert
2021-02-16 18:12:59 +00:00
53959b4bbc
divide account scan into bins for more efficient downstream processing ( #15303 )
2021-02-15 21:27:39 -06:00
78a176fc00
rename CumulativeOffset in preparation of 2d support ( #15338 )
2021-02-15 20:57:05 -06:00
30f18319f2
Track RecycleStore basic stats with needed refactor ( #15291 )
...
* Track RecycleStore basic stats with needed refactor
* Fix another wrong metrics def
2021-02-15 16:09:17 +09:00
866ce236b1
add another hash check ( #15278 )
...
* add another hash check
* update comment
2021-02-12 18:35:11 -06:00
b8448f4189
merkle root code no longer adds lamports ( #15298 )
2021-02-12 17:30:14 -06:00
2e7aebf0bb
Rework spl_token_v2_self_transfer_fix to avoid any SPL Token downtime
2021-02-12 22:46:39 +00:00
8944efddf7
combine flatten and hash ( #15269 )
2021-02-12 11:35:54 -06:00
eced3776b0
add metrics to hash calculation and consolidate ( #15165 )
...
* add metrics to hash calculation
* add metrics to hash calculation
* create HashStats struct to consolidate hash timing
* formatting
2021-02-11 20:06:04 -06:00
eb54e544c2
add div_ceil ( #15267 )
2021-02-11 20:05:41 -06:00
59e55f0371
remove hash accumulation with lamports ( #15275 )
2021-02-11 20:05:22 -06:00
47c60f8e98
Clean up mainnet-beta inflation candidate features
2021-02-10 17:46:22 -08:00
e59a24d9f9
pass expected capitalization to hash calculation to improve assert msg ( #15191 )
...
* cleanup if
* pass expected capitalization to hash calculation to improve assert message
* fix bank function
* one more level
* calculate_accounts_hash_helper
* add slot to error message
* success
2021-02-10 14:38:00 -06:00
da6753b8c0
Warp timestamp and extend max-allowable-drift for accommodate slow blocks ( #15204 )
...
* Remove timestamp_correction feature gating
* Remove timestamp_bounding feature gating
* Remove unused deprecated ledger code
* Remove unused deprecated unbounded-timestamp code
* Enable independent adjustment of fast/slow timestamp bounding
* Update timestamp bounds to 25% fast, 80% slow; warp timestamp
* Update bank hash test
* Add PR number to feature
Co-authored-by: Michael Vines <mvines@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-02-09 15:49:00 -07:00
8424fe2c12
use index version of calculating hash ( #15189 )
...
* use index version of calculating hash
* invert const
* formatting
2021-02-09 10:37:50 -06:00
600cea274d
change store account scan to not use dashmap ( #15104 )
...
* change store account scan to not use dashmap
* add test_accountsdb_de_dup_accounts_from_stores
* add tests
* add test_accountsdb_flatten_hash_intermediate
* add tests
* add sort test
* add test
* clippy
* first_slice -> is_first_slice
* comment
* use partial_cmp
2021-02-05 21:16:30 -06:00
fbf9dc47e9
Threadpool2 ( #15151 )
...
* rework thread pool for hash calculation
* rename
2021-02-05 18:48:16 -06:00
fabecdc86c
use thread pool for non-index hash calculations ( #15149 )
2021-02-05 19:48:55 +00:00
6fd5ec0e4c
caches descendants in bank forks ( #15107 )
2021-02-05 18:00:45 +00:00
f85be6259b
sentinel value for zero lamport accounts in hash scanning ( #15097 )
...
* sentinel value for zero lamport accounts in hash scanning
* fix test
2021-02-05 11:23:48 -06:00
c69027e5aa
chore: bump serde from 1.0.118 to 1.0.122 ( #15126 )
...
* chore: bump serde from 1.0.118 to 1.0.122
* Update programs/bpf/Cargo.lock
2021-02-05 19:23:24 +09:00
7d9f5ad525
remove unused arg from function ( #15096 )
2021-02-04 20:46:30 -06:00
f0d58f5549
move timer end outside if ( #15087 )
2021-02-04 20:43:23 -06:00
e4d0d4bfae
Fix pubkey refcount for shrink + clean ( #14987 )
2021-02-04 12:44:19 -08:00
f49a70e626
borrow storages ( #15088 )
2021-02-04 11:29:03 -06:00
e5225b7e68
Add ref count from storage ( #15078 )
2021-02-04 09:11:05 -08:00
e7f53182e2
fix merge conflict ( #15085 )
2021-02-04 09:20:50 -06:00
600ff0d915
calculate hash from store instead of index ( #15034 )
...
* calculate hash from store instead of index
* restore update hash in abs
2021-02-04 09:00:33 -06:00
2970b59853
Don't load all accounts into memory for capitalization check ( #14957 )
...
* Don't load all accounts into memory for capitalization check
2021-02-03 15:00:42 -08:00
8e93a784f3
program-test: Add ability to warp to the future ( #14998 )
...
* program-test: Add ability to warp to the future
* Make `start_local_server` take by value
* Remove clear_invoke_context
2021-02-03 23:31:36 +01:00
2b9a0311d3
Don't reset accounts if the remove_account comes from a clean ( #15022 )
...
Store may be in-use with a snapshot creation, so don't disturb
it's state.
2021-02-03 13:06:28 -08:00
37aac5a12d
Cleanup v1 shrink path ( #15009 )
...
move legacy functions to another impl block
2021-02-02 13:29:46 -08:00
22800054e6
streamline calculate_accounts_hash ( #14980 )
2021-02-02 10:58:19 -06:00
31168fe343
Disable AppendVec warn! for now ( #14996 )
...
* Disable AppendVed warn! for now
* Fix version...
* Update append_vec.rs
2021-02-02 14:52:30 +00:00
1df93fa2be
chore: bump serde from 1.0.112 to 1.0.118 ( #14828 )
...
* chore: bump serde from 1.0.112 to 1.0.122
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.112 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.122 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Update frozen_abi digest following serde update
* Revert "chore: bump serde from 1.0.112 to 1.0.122"
This reverts commit a3ef4442a4c985144ae2bd7ceaf8899a7ab8d7c0.
* Revert "[auto-commit] Update all Cargo lock files"
This reverts commit c41c3b005fb1ccade55155302c52cd5736c4b55f.
* chore: bump serde from 1.0.112 to 1.0.118
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.112 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.118 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Remove serum-dex pinning
* blind commit!
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-02-02 23:28:16 +09:00
73d9186502
More rich runtime logging ( #14938 )
2021-02-01 11:40:49 -08:00
63c44bd690
Use helper for count() in accountsDB ( #14953 )
2021-01-31 09:32:02 -08:00
07cef5a557
Buffer authority must match upgrade authority for deploys and upgrades ( #14923 )
2021-01-29 12:43:42 -08:00
65315fa4c2
Use already-generated key set to populate dirty keys for clean ( #14905 )
...
Don't need to scan the stores again when we already found the key
set of updates per slot. Just insert it earlier.
2021-01-29 11:44:43 -08:00
85b5dbead6
Add feature for pending SPL Token self-transfer fix
2021-01-29 04:19:33 +00:00
0b1015f7d3
Richer runtime failure logging ( #14875 )
2021-01-28 10:04:54 -08:00
a53b8558cd
Only mmap file from snapshot once ( #14815 )
2021-01-28 08:15:33 -08:00
63429507b2
Add syscall feature activation test ( #14890 )
2021-01-27 17:21:25 -08:00
72f10f5f29
Aggregate purge and shrink metrics ( #14763 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-27 01:39:47 -08:00
daddcd361a
Snapshots missing slots from accounts cache clean optimization ( #14852 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-26 20:08:55 -08:00
ffa5c7dcc8
Deprecate commitment variants ( #14797 )
...
* Deprecate commitment variants
* Add new CommitmentConfig builders
* Add helpers to avoid allowing deprecated variants
* Remove deprecated transaction-status code
* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist
* Remove deprecated banks code
* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now
* Heavier hand with rpc/rpc-subscription commitment
* Remove deprecated variants from local-cluster
* Remove deprecated variants from various tools
* Remove deprecated variants from validator
* Update docs
* Remove deprecated client code
* Add new variants to cli; remove deprecated variants as possible
* Don't send new commitment variants to old clusters
* Retain deprecated method in test_validator_saves_tower
* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test
* Refactor node version check to handle commitment variants and transaction encoding
* Hide deprecated variants from cli help
* Add cli App comments
2021-01-26 19:23:07 +00:00
2b50433099
Remove legacy_stake program
2021-01-25 19:34:40 -08:00
1b85114a9c
remove legacy merkle root ( #14772 )
...
* remove legacy merkle root
f78197a
* clippy
* compile error
* borrow error
* derministic results
* clippy
* borrow
2021-01-25 15:26:08 -06:00
5700e835ef
Refactors tuple of TransactionLoadResult into a struct. ( #14773 )
2021-01-25 10:35:08 +01:00
0d32a0e0f4
Partial clean ( #14800 )
...
* Revert "Revert "Partial accounts clean (#14652 )" (#14777 )"
This reverts commit ad2e10e17b
2021-01-24 09:50:19 -08:00
cbffab7850
Upgrade to Rust v1.49.0
2021-01-23 19:16:36 -08:00
96b27ca0cb
Remove zero_lamport_pubkeys reference ( #14808 )
2021-01-23 18:31:55 +00:00
424bb797a6
Speed up generate_index ( #14792 )
2021-01-23 08:05:05 -08:00
c77461e428
Remove unnecesary flushes in previous roots ( #14596 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-23 04:02:44 -08:00
bf1943e489
Add solana-test-validator --warp-slot argument
2021-01-22 21:17:02 -08:00
dd5a2ef05f
Add convenience function to create a snapshot archive out of any Bank
2021-01-22 21:17:02 -08:00
ad2e10e17b
Revert "Partial accounts clean ( #14652 )" ( #14777 )
...
This reverts commit ca7914aa3d
2021-01-22 18:05:49 -08:00
77572a7c53
Track account writable deescalation ( #14626 )
2021-01-22 15:28:01 -08:00
18bd0c9a5b
speed up merkle calculation ( #14710 )
2021-01-22 09:21:50 -06:00
ca7914aa3d
Partial accounts clean ( #14652 )
...
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
2021-01-21 11:01:53 -08:00
072e5e54d8
Add generalized voting process to enable full inflation
2021-01-20 23:37:21 -08:00
2745b79b74
Parallel cache scan ( #14544 )
...
* Parallel cache scan
* PR comments
* PR comments
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-20 08:50:17 +00:00
e3bd9e5300
Prevent the invoke and upgrade of programs in the same tx batch ( #14653 )
...
* Prevent the invoke and upgrade of programs in the same tx batch
* Pass program address as writable in the upgrade instruction
2021-01-19 16:24:44 -08:00
5f14f45282
More generic accounts purge functions ( #14595 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-17 20:31:03 -08:00
66b54b852d
Update-executable flag in pre-accounts ( #14622 )
2021-01-15 17:48:29 -08:00
4ecc0f4fa6
chore: bump rayon from 1.4.1 to 1.5.0 ( #14177 )
...
* chore: bump rayon from 1.4.1 to 1.5.0
Bumps [rayon](https://github.com/rayon-rs/rayon ) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases )
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md )
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.4.1...rayon-core-v1.5.0 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-01-15 18:07:35 +09:00
907f518f6d
Add load/execute/store timings ( #14561 )
2021-01-14 14:14:16 -08:00
6e8a1ba7de
Load executable accounts from invoke context ( #14574 )
2021-01-14 00:19:22 -08:00
6dfad0652f
Cache account stores, flush from AccountsBackgroundService ( #13140 )
2021-01-11 17:00:23 -08:00
4a66e3eddc
Use standard tmp-snapshot- file prefix for the "new_state" archive for better cleanup/consistency
2021-01-11 16:51:26 -08:00
b0e6e29527
Update timestamp max allowable drift to 50% of PoH ( #14531 )
...
* Repurpose warp-timestamp feature for general bump
* Change max_allowable_drift to 50%
* Fill in PR#
* Fix rpc test setup
2021-01-11 23:27:30 +00:00
8ad5931bfc
Check native account owner ( #14535 )
2021-01-11 22:36:52 +00:00
872a249e97
Restore snapshot hard linking
2021-01-11 12:11:25 -08:00
a95675a7ce
Avoid tmp snapshot backlog in SnapshotPackagerService under high load ( #14516 )
2021-01-11 10:21:15 -08:00
7be6770808
Rename CompressionType to ArchiveFormat
2021-01-09 09:07:49 -08:00
141e6706e6
Rename AccountsPackage::root to AccountsPackage::slot
2021-01-09 09:07:49 -08:00
9f70f7dc3e
Reduce TempDir exposure
2021-01-09 09:07:49 -08:00
f2a7f561a0
Replace incorrect symlink_dir usage with symlink_file
2021-01-09 09:07:49 -08:00
fe0ba4a429
Create account paths once
2021-01-09 09:07:49 -08:00
58487c6360
Add buffer authority to upgradeable loader ( #14482 )
2021-01-08 09:37:57 -08:00
c9df6134fa
Save 7G mem on mainnet fixing AccIndex overalloc. ( #14435 )
2021-01-06 09:48:59 +09:00
a8b5a32b50
Gate cpi program account passing ( #14443 )
2021-01-05 21:53:41 +00:00
53d65009a0
Lower recycle store count ( #14429 )
...
Too many stores can cause swap usage which
is detrimental to account store times.
2021-01-05 12:21:17 -08:00
0619805806
Upgradeable programs needs program account's address as program id ( #14417 )
2021-01-04 21:45:05 +00:00
db6ee289c9
Don't bother restoring snapshots, they're never used
2021-01-02 17:26:33 -08:00
b6dcdb90e8
Remove dead code
2021-01-02 17:26:33 -08:00
5affd8aa72
Add secondary indexes ( #14212 )
2020-12-31 18:06:03 -08:00
1c61d005b5
Re-enable shrink test ( #14331 )
2020-12-30 08:25:45 -08:00
7893e2e307
Remove Testnet-specific old code ( #14305 )
2020-12-29 05:11:46 +09:00
addffd7694
Log error from AppendVec removal & a panic clean ( #14302 )
2020-12-29 04:59:09 +09:00
c693ffaa08
Fix subtraction overflow in metrics ( #14290 )
2020-12-27 16:26:22 -08:00
49019c6613
obtains staked-nodes from the root-bank ( #14257 )
...
... as opposed to the working bank
2020-12-27 13:28:05 +00:00
ee0a80a092
Prevent bpf loader impersonators ( #14278 )
2020-12-24 03:04:48 +00:00
b1d702a618
Don't use caller passed executable account ( #14276 )
2020-12-23 14:34:14 -08:00
5945305b1d
Save cloning program account data ( #14251 )
2020-12-23 10:18:14 -08:00
5b903318b2
vote: Add helper for creating current-versioned states
2020-12-22 19:37:26 -07:00
3881ae10fb
Deinitialize nonce data upon zero balance
2020-12-22 19:37:26 -07:00
a14cfd660a
removes &Arc<Self> receivers ( #14234 )
2020-12-22 23:51:53 +00:00
c276513b84
Ignore long/hanging test ( #14261 )
2020-12-22 16:13:03 -07:00
7042f11791
Feature-gate stake-program-v3 ( #14232 )
...
* Remove deprecated legacy stake program
* Add legacy stake program
* Strip out duplicative legacy code
* Feature-deploy stake-program-v3
* Add ownership check in stake processor
2020-12-22 11:20:38 -07:00
ab205b682a
Upgradeable programs called same as non-upgradeable ( #14239 )
...
* Upgradeable programs called same as non-upgradeable
* nudge
2020-12-22 09:26:55 -08:00
baa9602411
Add shrink paths ( #14238 )
2020-12-21 21:33:37 -08:00
7b08cb1f0d
improves performance in replay-stage ( #14217 )
...
bank::vote_accounts returns a hash-map which is slow to iterate, but all uses
only require an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/runtime/src/bank.rs#L4300-L4306
Similarly, calculate_stake_weighted_timestamp takes a hash-map whereas it only
requires an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/sdk/src/stake_weighted_timestamp.rs#L21-L28
2020-12-21 19:18:19 +00:00
93ae177503
Do not delete ALL other snapshots before downloading a new snapshot
2020-12-21 08:58:52 +00:00
e15f95a36f
Fix timestamp handling on ledger warp ( #14210 )
...
* Reset timestamp for slot and epoch-start on warp
* Fix genesis timestamp metric source
* Remove check that timestamp > unix_timestamp_from_genesis
Default to previous timestamp, not genesis timestamp
* Move timestamp metrics to report even on warp
* Initialize slot 0 timestamps correctly
* Add feature gate to warp testnet timestamp
* Review suggestion: simplify warp-timestamp slot check
2020-12-20 21:36:56 +00:00
e8cc0bef6c
Add CPI support for upgradeable loader ( #14193 )
2020-12-17 15:39:49 -08:00
d6d76219b6
caches staked nodes computed from vote-accounts ( #13929 )
2020-12-17 21:22:50 +00:00
efc091e28a
Add transactionCount field to GetEpochInfo
2020-12-17 12:29:50 -08:00
75e9e321de
Fix race between setting tick height and calculating accounts hash ( #14101 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-15 12:45:40 -08:00
1e977ac025
Bump version to v1.6.0
2020-12-15 18:28:04 +00:00
d513b0c4ca
Add Program loader/environment instruction errors ( #14120 )
2020-12-15 09:54:07 -08:00
53fe0bb5eb
Remove regular expressions, speeds up untar ( #14119 )
2020-12-15 08:48:21 -08:00
5294f70189
Remove some non-warnings ( #14115 )
2020-12-14 16:26:17 -08:00
9e90394583
Upgradeable loader ( #13689 )
2020-12-14 15:35:10 -08:00
e907510a1a
clippy
2020-12-14 13:11:50 -08:00
7143aaa89b
Clippy
2020-12-14 08:03:29 -08:00
de9ac43ebf
Simple cap by including sysvars and native programs ( #13884 )
...
* Simpler cap by including sysvars and native programs
* Fix tests
* Add comment
* revert some unrelated code
* Update test_bank_update_sysvar_account for cap.
* Test cap. for add_native_program using new helper
* Improve the cap adjustment with new tests
* Fix typo...
* Adjust test for improved code coverage
* Rename simpler_capitalization => simple_capitalization
* More rename and bonus commenting
2020-12-14 21:03:24 +09:00
5788e75188
Shrink slots by sparseness of written data size ( #14072 )
2020-12-14 02:58:38 +09:00
88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 ( #13691 )
...
* chore: bump log from 0.4.8 to 0.4.11
Bumps [log](https://github.com/rust-lang/log ) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases )
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
8df9bcc784
chore: bump libc from 0.2.80 to 0.2.81 ( #14092 )
...
* chore: bump libc from 0.2.80 to 0.2.81
Bumps [libc](https://github.com/rust-lang/libc ) from 0.2.80 to 0.2.81.
- [Release notes](https://github.com/rust-lang/libc/releases )
- [Commits](https://github.com/rust-lang/libc/compare/0.2.80...0.2.81 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 12:13:17 +09:00
55fc963595
Move slot cleanup to AccountsBackgroundService ( #13911 )
...
* Move bank drop to AccountsBackgroundService
* Send to ABS on drop instead, protects against other places banks are dropped
* Fix Abi
* test
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-13 01:22:34 +00:00
edff62af3d
Switch to memmap2 fork
2020-12-11 23:44:03 +00:00
d6eff3d62c
Filter out stake and vote accounts with incorrect owners ( #14062 )
...
* Add failing test
* Check stake/vote accounts for validity
* Feature gate change
* Add datapoint
* Add test realism
2020-12-11 12:21:25 -07:00
2d3a337200
Increase another snapshot limit: UNPACKED_COUNT ( #14071 )
2020-12-11 11:01:22 +00:00
7078a6ac61
Add more comments in Stakes::store for its subtlety ( #14065 )
...
* Add more comments in Stakes::store for its subtlety
* more comment tweak
2020-12-11 18:13:36 +09:00
09bd412b13
Wrong rpc inflation rate ( #14063 )
...
* Fix wrong inflation calculation in rpc
* Reorder
* Fix test
2020-12-11 07:42:39 +00:00
0a9ff1dc9d
Initial solana-test-validator command-line program
2020-12-11 04:17:38 +00:00
83fda2d972
Clean invoke in prep for native invoke ( #14055 )
2020-12-10 18:25:57 -08:00
164b7895b3
Tiny add_native_program bug fixes with cleanups ( #14042 )
...
* Tiny add_native_program bug fixes with cleanups
* Fix typo
2020-12-11 11:03:31 +09:00
7abd8084b6
Add get_minimum_balance_for_rent_excemption to Client ( #14048 )
2020-12-10 16:39:28 -08:00
5ea80e673f
Fix priv escalation test ( #14046 )
2020-12-10 22:36:33 +00:00
68109a46e0
Lower priority for hashing threads. ( #14043 )
2020-12-10 12:26:47 -08:00
14e241be35
Increase snapshot file size limits ( #14024 )
...
* Increase snapshot file size limits
* Fix test
* Reorder code a bit
2020-12-10 02:46:33 +09:00
28b014ccf4
Add tests for store_account_and_update_capitalization ( #14008 )
2020-12-09 00:46:28 +09:00
dca579851c
Cannot change owner if account is executable ( #13977 )
2020-12-07 11:37:07 -08:00
a706706572
Validator CLI option to enable just-in-time compilation of BPF ( #13789 )
...
* Adds a CLI option to the validator to enable just-in-time compilation of BPF.
* Refactoring to use bpf_loader_program instead of feature_set to pass JIT flag from the validator CLI to the executor.
2020-12-07 09:49:55 +01:00
882d033233
Don't use stale executor cache ( #13966 )
2020-12-05 05:54:42 +00:00
5dceddd21d
Remove associated functions on AccountsDb, replace with methods ( #13953 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-03 17:53:42 -08:00
17defbff13
Make `Accounts::is_non_loader_key()` a method on `Message`
2020-12-02 20:10:08 +00:00
404fc1570d
runtime: Replace `HashAgeKind` with `NonceRollbackInfo`
2020-12-02 20:10:08 +00:00
be7760caa1
accounts: Replace nonce_rollback unreachable block with descriptive panic
2020-12-02 20:10:08 +00:00
47af5933ca
accounts: Don't assume fee-payer is the first account
2020-12-02 20:10:08 +00:00
8df2a4bac0
runtime: Factor out `DurableNoncePartial` finishing logic and add tests
2020-12-02 20:10:08 +00:00
73111b005f
Reduce the number of snapshots
2020-12-01 11:13:37 -08:00
254790f8c8
Fully clean up temporary snapshot staging directories
2020-11-30 19:40:06 -08:00
e1793e5a13
caches vote-state de-serialized from vote accounts ( #13795 )
...
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.
This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.
Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
2020-11-30 17:18:33 +00:00
ce4304cc9a
Add extra checks to verify_and_update ( #13848 )
...
* Add extra checks to verify_and_update
* nudge
2020-11-29 10:06:43 +00:00
6048342c57
Enhance ledger-tool for multi-epoch rewards ( #13837 )
...
* Support ledger-tool for multi-epoch rewards
* nits
* Ensure not to skip some records in csv
2020-11-27 04:20:47 +00:00
955b99cf69
Fix roots_and_ref_count ( #13827 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-26 13:09:20 -08:00
43b82b31e5
More TestValidator cleanup
2020-11-26 08:56:25 +00:00
b5f7e39be8
TestValidator public interface cleanup
2020-11-25 17:04:37 -08:00
4ef2da0ff0
Add `solana logs` command
2020-11-25 11:44:41 -08:00
b70abdc645
Nonce updates ( #13799 )
...
* runtime: Add `FeeCalculator` resolution method to `HashAgeKind`
* runtime: Plumb fee-collected accounts for failed nonce tx rollback
* runtime: Use fee-collected nonce/fee account for nonced TX error rollback
* runtime: Add test for failed nonced TX accounts rollback
* Fee payer test
* fixup: replace nonce account when it pays the fee
* fixup: nonce fee-payer collect test
* fixup: fixup: clippy/fmt for replace...
* runtime: Test for `HashAgeKind::fee_calculator()`
* Clippy
Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-24 23:53:51 -08:00
c930f92411
Restore bank feature used to deprecate Rewards sysvar ( #13786 )
2020-11-24 10:22:31 -07:00
60ce6f1a9b
Truncate logs immediately upon collection. Also reduce limit from 100k to 10k
2020-11-23 17:20:02 -08:00
791fb17437
Prevent scans on unrooted slots from seeing partial clean ( #13628 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 13:01:04 -08:00
62fa8b0ed8
Demote bank-process_transactions-error counters to info!
2020-11-20 10:43:07 -08:00
c75d97e3f2
Add new inflation feature-ids ( #13671 )
...
* Add new inflation feature-ids, and full_inflation default values
* Compute inflation start from full_inflation activation
* Include pico_inflation in inflation start computation
* Add full-inflation constructor
* Align inflation taper with rewards accrual start and catch overflow edge case
2020-11-20 09:54:41 -07:00
11e92f0c9f
Snapshot archives are now cleaned up when `--snapshot-compression none` is used
2020-11-20 08:41:17 -08:00
b74d7b5758
Fix fragile tests in prep of stake rewrite pr ( #13654 )
...
* Fix fragile tests in prep of stake rewrite pr
* Restore BOOTSTRAP_VALIDATOR_LAMPORTS where appropriate
* Further clean up
* Further clean up
* Aligh with other call site change
* Remove false warn!
* fix ci!
2020-11-20 17:21:03 +09:00
a8c29505f0
sanitizes bloom filters to avoid division by zero ( #13714 )
...
Pull requests received over the wire can cause a validator to panic
because of division by zero in bloom filters:
https://github.com/solana-labs/solana/blob/af08ba93e/runtime/src/bloom.rs#L86-L88
2020-11-19 23:35:22 +00:00
397cf726fc
Allow GNUSparse for genesis.bin ( #13704 )
2020-11-19 20:29:19 +00:00
43d5e47ea9
Rewrite stake accounts for clear migration ( #13461 )
...
* Reduce overage stake by rewritng stake accounts
* Write tests and finish implemention
* Create and use new feature gate
* Clean up logging
* Fix typo
* Simplify enable_rewrite_stake
* Fix typo...
* Even simplify gating
* Add metrics
2020-11-20 05:15:06 +09:00
110acd20dc
Check for overflow in rent partition calculation ( #13569 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-19 04:17:00 -08:00
645598e615
Improve monitoring of timestamp correction and bounding ( #13656 )
...
* Include rejuvenated field in Clock parsing
* Expand timestamp-correction logging info
2020-11-17 21:51:43 +00:00
6276360468
Prevent scans from seeing root updates/clean ( #13464 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-16 17:23:11 -08:00
bcd303a447
ledger-tool cap: delegation owner and stake v2 flag ( #13602 )
...
* Output delegation owner as well
* Add --enable-stake-program-v2
* Small cleanup and add sanity assertion
* Fix typo...
2020-11-15 19:38:46 +00:00
baa6b3a261
Add stable program logging for BPF and native programs
2020-11-14 08:26:01 -08:00
c97a7d1105
Small cleaning update_epoch_stakes ( #13576 )
2020-11-13 11:16:38 +00:00
Jeff Washington (jwash)
carllin
Trent Nelson
François Garillot
steviez
Jack May
Justin Starry
behzad nouri
sakridge
Ryo Onodera
DimAn
Michael Vines
Tyera Eulberg
Tyera Eulberg
Greg Fitzgerald
Jon Cinque
carllin
dependabot[bot]
Alexander Meißner