f374b35944
Hash stored accounts in bg ( #16157 )
...
* lazy calculate account hash
* push to bg thread
* remove deadlock
* logs
* format
* some cleanup on aisle 9
* format, fix up some metrics
* fix test, remove legacy function only there for tests
* cleanup
* remove unused store_hasher
* Switch to crossbeam
* clippy
* format
* use iter()
* rework from feedback
* hash_slot -> slot
* hash(cluster_type)
Co-authored-by: Carl Lin <carl@solana.com>
2021-03-31 15:39:34 -05:00
a5dcee254d
get rid of unnecessary data copy ( #16214 )
...
* get rid of unnecessary data copy
* preserve rent_epoch
2021-03-31 09:11:39 -05:00
54c68ea83f
Drop write lock on sysvars ( #15497 )
...
* Drop write lock on sysvars
* adds env var for demoting sysvar write lock demotion
* moves demote logic to is_writable
* feature gates sysvar write lock demotion
* adds builtins to write lock demotion
* adds system program id to builtins
* adds Feature111...
* adds an abi-freeze test
* mvines set of builtin program keys
Co-authored-by: Michael Vines <mvines@gmail.com>
* update tests
* adds bpf loader keys
* Add test sysvar
* Plumb demote_sysvar to is_writable
* more plumbing of demote_sysvar_write_locks to is_writable
* patches test_program_bpf_instruction_introspection
* hard codes demote_sysvar_write_locks to false for serialization/encoding methods
* Revert "hard codes demote_sysvar_write_locks to false for serialization/encoding methods"
This reverts commit ae3e2d2e777437bddd753933097a210dcbc1b1fc.
* change the hardcoded ones to demote_sysvar_write_locks=true
* Use data_as_mut_slice
Co-authored-by: behzad nouri <behzadnouri@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-03-30 10:05:09 -07:00
5e5b63712b
Status cache improvements ( #16174 )
2021-03-28 19:10:14 -07:00
66c42f62d8
make AccountSharedData.data private to abstract storage ( #16091 )
...
* format
* 2 more tests
* use
2021-03-25 11:04:20 -05:00
6d5c6c17c5
Simplify account.rent_epoch handling for sysvar rent ( #16049 )
...
* Add some code for special local testing
* Add comment to store_account_and_update_capitalization
* Simplify account.rent_epoch handling for sysvar rent
* Introduce *_for_test functions
* Add deprecation messages to existing api
2021-03-25 15:23:20 +09:00
26af2d4c26
Up store limit for test_account_grow ( #16099 )
...
There can be more than 5 stores
2021-03-24 15:24:17 -07:00
2d24d13046
AccountsSharedData: data copy on write ( #15800 )
...
* Arc<AccountData>
* try custom serializer
* adapt test from Behzad's change
* clippy
* simplify serialization
* remove abi example derive
* refactor 'take'
* remove serialization
* remove serialize calls
* remove account_data
* remove intos
* remove left over file
2021-03-23 15:19:31 -05:00
fde43a906d
revert test to use Account for serialization ( #16063 )
2021-03-22 22:48:09 +00:00
4beb39f7a1
multi-pass bin scanning ( #15377 )
...
* multi-pass bin scanning
* pr feedback
* format
* fix typo
* adjust metrics for code changes
* merge errors
2021-03-18 10:32:07 -05:00
f548a04fae
Allow unbounded wallclock processing time in tests ( #15961 )
2021-03-17 15:48:50 -07:00
a5d144b00f
Revert to removing only tmp-
2021-03-17 10:31:01 -07:00
0209d334bd
Add option for separate snapshot location
...
(cherry picked from commit 6126878f509c69e23480a5ec22b3271e2b16e072)
2021-03-17 10:31:01 -07:00
efee8b62d7
a few missed set_data calls ( #15846 )
...
* a few missed set_data calls
* another set data call
2021-03-15 21:57:23 -05:00
e35d738906
rework BankClient.get_account_data ( #15853 )
2021-03-15 19:27:53 -05:00
c1ba265dd9
Wallclock BankingStage Throttle ( #15731 )
2021-03-15 17:11:15 -07:00
918d04e3f0
Add more slot update notifications ( #15734 )
...
* Add more slot update notifications
* fix merge
* Address feedback and add integration test
* switch to datapoint
* remove unused shred method
* fix clippy
* new thread for rpc completed slots
* remove extra constant
* fixes
* rely on channel closing
* fix check
2021-03-12 21:44:06 +08:00
4bbeb9c033
Remove old feature: simple_capitalization ( #15763 )
...
* Remove old feature: simple_capitalization
* Fix another failing test in core
* Finish up test cleanup
* Further clean up a bit
2021-03-12 11:12:40 +09:00
952c3bcbb7
AccountSharedData construction ( #15790 )
2021-03-11 18:09:04 -06:00
3419a5446e
AccountSharedData.set_data ( #15781 )
...
* account.set_data and resize_data
* remove data_resize
2021-03-11 16:40:45 -06:00
1061d021c9
Unpack snapshot AppendVecs directly into account paths
2021-03-11 08:25:24 -08:00
478518308d
Share RO and Executable accounts within invocations ( #15799 )
2021-03-10 23:04:00 -08:00
65dd177601
AppendVec cleanup: remove dead code, rename non-idiomatic "new" function
2021-03-10 14:50:59 -08:00
1135ffd595
mut data refs as slice ( #15782 )
2021-03-10 15:28:03 -06:00
9c1198c0c7
Improve load_largest_accounts more ( #15785 )
...
* Add load_largest_accounts bench
* Check lamports before address filter
* Use BinaryHeap, add Accounts test
* Use pubkey reference in the min-heap
Also, flatten code with early returns
Co-authored-by: Greg Fitzgerald <greg@solana.com>
2021-03-10 11:22:02 -07:00
369e13b111
cleanup old runtime features ( #15787 )
2021-03-10 09:48:41 -08:00
3355efbb5a
Fix test to use AccountSharedData
2021-03-09 20:47:54 -07:00
5991cef5f5
Improve load_largest_accounts and add Bank test ( #15775 )
2021-03-10 00:23:41 +00:00
52e54e1100
account.data -> data() ( #15778 )
2021-03-09 22:31:33 +00:00
8a3135d17b
Account->AccountSharedData ( #15691 )
2021-03-09 15:06:07 -06:00
e3e8179f2d
Sdk: start to appease clippy's integer arithmetic check ( #15736 )
...
* Move to module-level clippy allowance
* Fix stake_weighted_timestamp math
* Fix genesis_config
* Fix shred_version
* Fix hard_forks
* Fix process_instruction
* Add ArithmeticOverflow ix error
* Fix nonce_keyed_account
* Update BankSlotDelta frozen abi due to new ix error
2021-03-08 18:37:57 -07:00
beac2d6d3f
Remove stale comment
2021-03-08 10:18:32 +09:00
8b0c6db871
Remove old feature: cumulative_rent_related_fixes ( #15754 )
2021-03-08 09:58:50 +09:00
c5371fdc85
More debug on shrink and flush ( #15518 )
2021-03-05 14:10:34 -08:00
13b9bb6790
refactor function to store into account after data is finished ( #15710 )
2021-03-05 11:02:35 -06:00
edd159e7d7
Fix typos ( #15721 )
2021-03-05 07:16:53 +00:00
872f7117c3
Use OrderedIterator to produce TransactionLogInfo ( #15712 )
...
* Add failing test
* Fix iteration_order issue with stored logs
2021-03-04 15:26:19 -07:00
1e2f5a5f55
Permit the snapshots/status_cache file to be sparse
2021-03-04 12:36:34 -08:00
b53cb8eb2d
Deprecate Instruction::new ( #15695 )
2021-03-04 05:46:48 +00:00
be35c1c1b7
add execute detail timings ( #15638 )
2021-03-03 17:07:45 -06:00
9bbe015a05
Document and clean up AppendVec ( #15640 )
...
* Document AppendVec
* Remove the almost-duplicate state in AppendVec
AppendVec was maintaining two offsets, `current_len` and
`append_offset`. Despite the different looking names, the two
values have the same meaning, but were updated at slightly different
times. When appending a batch of accounts, `current_len` updates would
be immediately available to other threads after each append, whereas
`append_offset` would only be updated after its mutex was unlocked.
`append_offset` is redundant. By removing it, we eliminate potential
bugs and no longer need to suppress clippy warnings.
* Remove get_mut() from AppendVec design
Only the offset into the AppendVec memory is thread-safe. The memory
itself is only threadsafe because it is append-only and is otherwise
unprotected. Adding get_mut() would only be safe if the memory was
protected by a ReadWrite lock.
2021-03-03 12:51:48 -08:00
a9c8dbfd0c
Add error reporting to system program ( #15644 )
2021-03-02 21:44:25 -08:00
a2c47db920
extract expected cluster type ( #15615 )
...
* extract expected cluster type
* fix clippy warning
* more clippy
2021-03-02 13:55:34 -06:00
43663b1750
Fix clippy warnings ( #15632 )
2021-03-02 08:58:14 -08:00
da47ab7e50
fix indentation format problem ( #15618 )
...
* fix indentation format problem
* fix indentation format problem
2021-03-02 10:46:00 -06:00
aad79d9d5c
Remove unused Accounts state ( #15611 )
...
* Remove dead state from Accounts
* Refactor to remove reference to self.slot
Account::default() has zero lamports, so would return None anyway
2021-03-02 08:28:25 -08:00
2463cc1e6a
Fix typos ( #15610 )
2021-03-02 06:36:49 -08:00
97eaf3c334
Fix finalize_dead_slot_removal() of cached slots decrementing refcount ( #15534 )
2021-02-26 17:49:37 -08:00
8399851d11
check program owners ( #15495 )
...
* check program owners
* BankSlotDelta should change because InstructionError variant added
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2021-02-26 22:21:34 +00:00
21b43009f6
Introduce ttl eviction for RecycleStore ( #15513 )
2021-02-25 17:27:27 +09:00
eddb7f98f5
Check vote account initialization ( #15503 )
...
* Check account data_len on Vote account init
* Check account data populated on update_cached_accounts
2021-02-24 10:00:48 -07:00
2f46da346d
gracefully handle vote account without authorized voter ( #15501 )
2021-02-24 00:00:24 -08:00
9a7cd8885d
Remove read only locks when they hit ref count zero, cleanup accounts locking ( #15449 )
2021-02-22 23:56:43 -08:00
5df36aec7d
Pacify clippy
2021-02-19 20:08:41 -08:00
5d31763a3b
Update ABI digest
2021-02-19 10:54:39 -08:00
02432a548f
Cleanup old features ( #15391 )
2021-02-18 09:56:11 -08:00
e3006e3b46
correctly reserve space for pre_accounts vector ( #15378 )
2021-02-18 09:37:24 -06:00
0f6f6080f3
sdk: Add Borsh support for types and utilities ( #15290 )
...
* sdk: Add Borsh to Pubkey
* Add serialization error for easier borsh integration
* Add Borsh usage to banks-client and sdk
* Rename SerializationError -> IOError
* Add new errors to proto
* Update Cargo lock
* Update Cargo.lock based on CI
* Clippy
* Update ABI on bank
* Address review feedback
* Update sanity program instruction count test
2021-02-18 11:14:56 +01:00
7f7370c306
Re-allow clippy::integer_arithmetic at crate-level
2021-02-17 13:55:08 -07:00
9ba69a7381
Adapt to fs_extra 1.2.0
2021-02-17 12:44:58 -08:00
7fee0bc69c
move hashing functionality to accounts_hash ( #15353 )
2021-02-17 00:29:50 +00:00
8367740ff9
compute merkle root on chunks of fanout^3 ( #15344 )
...
* compute merkle root on chunks of fanout^3
* improve test_accountsdb_compute_merkle_root_large
2021-02-16 17:03:35 -06:00
ba02452d75
add validator flag no-accounts-db-index-hashing ( #15350 )
...
* add validator flag no_accounts_db_index_hashing
* add validator flag no_accounts_db_index_hashing
2021-02-16 21:13:48 +00:00
d83aeebcca
change 2d cumulative calculation ( #15346 )
2021-02-16 14:24:45 -06:00
708b8380b7
assert 2 items in stores don't have same slot and version with different values ( #15250 )
...
* assert 2 items in stores don't have same slot and version with different values
* add message to assert
2021-02-16 18:12:59 +00:00
53959b4bbc
divide account scan into bins for more efficient downstream processing ( #15303 )
2021-02-15 21:27:39 -06:00
78a176fc00
rename CumulativeOffset in preparation of 2d support ( #15338 )
2021-02-15 20:57:05 -06:00
30f18319f2
Track RecycleStore basic stats with needed refactor ( #15291 )
...
* Track RecycleStore basic stats with needed refactor
* Fix another wrong metrics def
2021-02-15 16:09:17 +09:00
866ce236b1
add another hash check ( #15278 )
...
* add another hash check
* update comment
2021-02-12 18:35:11 -06:00
b8448f4189
merkle root code no longer adds lamports ( #15298 )
2021-02-12 17:30:14 -06:00
2e7aebf0bb
Rework spl_token_v2_self_transfer_fix to avoid any SPL Token downtime
2021-02-12 22:46:39 +00:00
8944efddf7
combine flatten and hash ( #15269 )
2021-02-12 11:35:54 -06:00
eced3776b0
add metrics to hash calculation and consolidate ( #15165 )
...
* add metrics to hash calculation
* add metrics to hash calculation
* create HashStats struct to consolidate hash timing
* formatting
2021-02-11 20:06:04 -06:00
eb54e544c2
add div_ceil ( #15267 )
2021-02-11 20:05:41 -06:00
59e55f0371
remove hash accumulation with lamports ( #15275 )
2021-02-11 20:05:22 -06:00
47c60f8e98
Clean up mainnet-beta inflation candidate features
2021-02-10 17:46:22 -08:00
e59a24d9f9
pass expected capitalization to hash calculation to improve assert msg ( #15191 )
...
* cleanup if
* pass expected capitalization to hash calculation to improve assert message
* fix bank function
* one more level
* calculate_accounts_hash_helper
* add slot to error message
* success
2021-02-10 14:38:00 -06:00
da6753b8c0
Warp timestamp and extend max-allowable-drift for accommodate slow blocks ( #15204 )
...
* Remove timestamp_correction feature gating
* Remove timestamp_bounding feature gating
* Remove unused deprecated ledger code
* Remove unused deprecated unbounded-timestamp code
* Enable independent adjustment of fast/slow timestamp bounding
* Update timestamp bounds to 25% fast, 80% slow; warp timestamp
* Update bank hash test
* Add PR number to feature
Co-authored-by: Michael Vines <mvines@gmail.com>
Co-authored-by: Michael Vines <mvines@gmail.com>
2021-02-09 15:49:00 -07:00
8424fe2c12
use index version of calculating hash ( #15189 )
...
* use index version of calculating hash
* invert const
* formatting
2021-02-09 10:37:50 -06:00
600cea274d
change store account scan to not use dashmap ( #15104 )
...
* change store account scan to not use dashmap
* add test_accountsdb_de_dup_accounts_from_stores
* add tests
* add test_accountsdb_flatten_hash_intermediate
* add tests
* add sort test
* add test
* clippy
* first_slice -> is_first_slice
* comment
* use partial_cmp
2021-02-05 21:16:30 -06:00
fbf9dc47e9
Threadpool2 ( #15151 )
...
* rework thread pool for hash calculation
* rename
2021-02-05 18:48:16 -06:00
fabecdc86c
use thread pool for non-index hash calculations ( #15149 )
2021-02-05 19:48:55 +00:00
6fd5ec0e4c
caches descendants in bank forks ( #15107 )
2021-02-05 18:00:45 +00:00
f85be6259b
sentinel value for zero lamport accounts in hash scanning ( #15097 )
...
* sentinel value for zero lamport accounts in hash scanning
* fix test
2021-02-05 11:23:48 -06:00
7d9f5ad525
remove unused arg from function ( #15096 )
2021-02-04 20:46:30 -06:00
f0d58f5549
move timer end outside if ( #15087 )
2021-02-04 20:43:23 -06:00
e4d0d4bfae
Fix pubkey refcount for shrink + clean ( #14987 )
2021-02-04 12:44:19 -08:00
f49a70e626
borrow storages ( #15088 )
2021-02-04 11:29:03 -06:00
e5225b7e68
Add ref count from storage ( #15078 )
2021-02-04 09:11:05 -08:00
e7f53182e2
fix merge conflict ( #15085 )
2021-02-04 09:20:50 -06:00
600ff0d915
calculate hash from store instead of index ( #15034 )
...
* calculate hash from store instead of index
* restore update hash in abs
2021-02-04 09:00:33 -06:00
2970b59853
Don't load all accounts into memory for capitalization check ( #14957 )
...
* Don't load all accounts into memory for capitalization check
2021-02-03 15:00:42 -08:00
8e93a784f3
program-test: Add ability to warp to the future ( #14998 )
...
* program-test: Add ability to warp to the future
* Make `start_local_server` take by value
* Remove clear_invoke_context
2021-02-03 23:31:36 +01:00
2b9a0311d3
Don't reset accounts if the remove_account comes from a clean ( #15022 )
...
Store may be in-use with a snapshot creation, so don't disturb
it's state.
2021-02-03 13:06:28 -08:00
37aac5a12d
Cleanup v1 shrink path ( #15009 )
...
move legacy functions to another impl block
2021-02-02 13:29:46 -08:00
22800054e6
streamline calculate_accounts_hash ( #14980 )
2021-02-02 10:58:19 -06:00
31168fe343
Disable AppendVec warn! for now ( #14996 )
...
* Disable AppendVed warn! for now
* Fix version...
* Update append_vec.rs
2021-02-02 14:52:30 +00:00
1df93fa2be
chore: bump serde from 1.0.112 to 1.0.118 ( #14828 )
...
* chore: bump serde from 1.0.112 to 1.0.122
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.112 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.122 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Update frozen_abi digest following serde update
* Revert "chore: bump serde from 1.0.112 to 1.0.122"
This reverts commit a3ef4442a4c985144ae2bd7ceaf8899a7ab8d7c0.
* Revert "[auto-commit] Update all Cargo lock files"
This reverts commit c41c3b005fb1ccade55155302c52cd5736c4b55f.
* chore: bump serde from 1.0.112 to 1.0.118
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.112 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.118 )
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
* Remove serum-dex pinning
* blind commit!
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-02-02 23:28:16 +09:00
73d9186502
More rich runtime logging ( #14938 )
2021-02-01 11:40:49 -08:00
63c44bd690
Use helper for count() in accountsDB ( #14953 )
2021-01-31 09:32:02 -08:00
07cef5a557
Buffer authority must match upgrade authority for deploys and upgrades ( #14923 )
2021-01-29 12:43:42 -08:00
65315fa4c2
Use already-generated key set to populate dirty keys for clean ( #14905 )
...
Don't need to scan the stores again when we already found the key
set of updates per slot. Just insert it earlier.
2021-01-29 11:44:43 -08:00
85b5dbead6
Add feature for pending SPL Token self-transfer fix
2021-01-29 04:19:33 +00:00
0b1015f7d3
Richer runtime failure logging ( #14875 )
2021-01-28 10:04:54 -08:00
a53b8558cd
Only mmap file from snapshot once ( #14815 )
2021-01-28 08:15:33 -08:00
63429507b2
Add syscall feature activation test ( #14890 )
2021-01-27 17:21:25 -08:00
72f10f5f29
Aggregate purge and shrink metrics ( #14763 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-27 01:39:47 -08:00
daddcd361a
Snapshots missing slots from accounts cache clean optimization ( #14852 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-26 20:08:55 -08:00
ffa5c7dcc8
Deprecate commitment variants ( #14797 )
...
* Deprecate commitment variants
* Add new CommitmentConfig builders
* Add helpers to avoid allowing deprecated variants
* Remove deprecated transaction-status code
* Include new commitment variants in runtime commitment; allow deprecated as long as old variants persist
* Remove deprecated banks code
* Remove deprecated variants in core; allow deprecated in rpc/rpc-subscriptions for now
* Heavier hand with rpc/rpc-subscription commitment
* Remove deprecated variants from local-cluster
* Remove deprecated variants from various tools
* Remove deprecated variants from validator
* Update docs
* Remove deprecated client code
* Add new variants to cli; remove deprecated variants as possible
* Don't send new commitment variants to old clusters
* Retain deprecated method in test_validator_saves_tower
* Fix clippy matches! suggestion for BPF solana-sdk legacy compile test
* Refactor node version check to handle commitment variants and transaction encoding
* Hide deprecated variants from cli help
* Add cli App comments
2021-01-26 19:23:07 +00:00
2b50433099
Remove legacy_stake program
2021-01-25 19:34:40 -08:00
1b85114a9c
remove legacy merkle root ( #14772 )
...
* remove legacy merkle root
f78197a
* clippy
* compile error
* borrow error
* derministic results
* clippy
* borrow
2021-01-25 15:26:08 -06:00
5700e835ef
Refactors tuple of TransactionLoadResult into a struct. ( #14773 )
2021-01-25 10:35:08 +01:00
0d32a0e0f4
Partial clean ( #14800 )
...
* Revert "Revert "Partial accounts clean (#14652 )" (#14777 )"
This reverts commit ad2e10e17b
2021-01-24 09:50:19 -08:00
cbffab7850
Upgrade to Rust v1.49.0
2021-01-23 19:16:36 -08:00
96b27ca0cb
Remove zero_lamport_pubkeys reference ( #14808 )
2021-01-23 18:31:55 +00:00
424bb797a6
Speed up generate_index ( #14792 )
2021-01-23 08:05:05 -08:00
c77461e428
Remove unnecesary flushes in previous roots ( #14596 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-23 04:02:44 -08:00
bf1943e489
Add solana-test-validator --warp-slot argument
2021-01-22 21:17:02 -08:00
dd5a2ef05f
Add convenience function to create a snapshot archive out of any Bank
2021-01-22 21:17:02 -08:00
ad2e10e17b
Revert "Partial accounts clean ( #14652 )" ( #14777 )
...
This reverts commit ca7914aa3d
2021-01-22 18:05:49 -08:00
77572a7c53
Track account writable deescalation ( #14626 )
2021-01-22 15:28:01 -08:00
18bd0c9a5b
speed up merkle calculation ( #14710 )
2021-01-22 09:21:50 -06:00
ca7914aa3d
Partial accounts clean ( #14652 )
...
Clean less keys by tracking the two cases:
* Touched keys per slot in uncleaned_keys derived from
accounts delta hash operation.
* Set of keys with any zero-lamport updates.
2021-01-21 11:01:53 -08:00
072e5e54d8
Add generalized voting process to enable full inflation
2021-01-20 23:37:21 -08:00
2745b79b74
Parallel cache scan ( #14544 )
...
* Parallel cache scan
* PR comments
* PR comments
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-20 08:50:17 +00:00
e3bd9e5300
Prevent the invoke and upgrade of programs in the same tx batch ( #14653 )
...
* Prevent the invoke and upgrade of programs in the same tx batch
* Pass program address as writable in the upgrade instruction
2021-01-19 16:24:44 -08:00
5f14f45282
More generic accounts purge functions ( #14595 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2021-01-17 20:31:03 -08:00
66b54b852d
Update-executable flag in pre-accounts ( #14622 )
2021-01-15 17:48:29 -08:00
907f518f6d
Add load/execute/store timings ( #14561 )
2021-01-14 14:14:16 -08:00
6e8a1ba7de
Load executable accounts from invoke context ( #14574 )
2021-01-14 00:19:22 -08:00
6dfad0652f
Cache account stores, flush from AccountsBackgroundService ( #13140 )
2021-01-11 17:00:23 -08:00
4a66e3eddc
Use standard tmp-snapshot- file prefix for the "new_state" archive for better cleanup/consistency
2021-01-11 16:51:26 -08:00
b0e6e29527
Update timestamp max allowable drift to 50% of PoH ( #14531 )
...
* Repurpose warp-timestamp feature for general bump
* Change max_allowable_drift to 50%
* Fill in PR#
* Fix rpc test setup
2021-01-11 23:27:30 +00:00
8ad5931bfc
Check native account owner ( #14535 )
2021-01-11 22:36:52 +00:00
872a249e97
Restore snapshot hard linking
2021-01-11 12:11:25 -08:00
a95675a7ce
Avoid tmp snapshot backlog in SnapshotPackagerService under high load ( #14516 )
2021-01-11 10:21:15 -08:00
7be6770808
Rename CompressionType to ArchiveFormat
2021-01-09 09:07:49 -08:00
141e6706e6
Rename AccountsPackage::root to AccountsPackage::slot
2021-01-09 09:07:49 -08:00
9f70f7dc3e
Reduce TempDir exposure
2021-01-09 09:07:49 -08:00
f2a7f561a0
Replace incorrect symlink_dir usage with symlink_file
2021-01-09 09:07:49 -08:00
fe0ba4a429
Create account paths once
2021-01-09 09:07:49 -08:00
58487c6360
Add buffer authority to upgradeable loader ( #14482 )
2021-01-08 09:37:57 -08:00
c9df6134fa
Save 7G mem on mainnet fixing AccIndex overalloc. ( #14435 )
2021-01-06 09:48:59 +09:00
a8b5a32b50
Gate cpi program account passing ( #14443 )
2021-01-05 21:53:41 +00:00
53d65009a0
Lower recycle store count ( #14429 )
...
Too many stores can cause swap usage which
is detrimental to account store times.
2021-01-05 12:21:17 -08:00
0619805806
Upgradeable programs needs program account's address as program id ( #14417 )
2021-01-04 21:45:05 +00:00
db6ee289c9
Don't bother restoring snapshots, they're never used
2021-01-02 17:26:33 -08:00
b6dcdb90e8
Remove dead code
2021-01-02 17:26:33 -08:00
5affd8aa72
Add secondary indexes ( #14212 )
2020-12-31 18:06:03 -08:00
1c61d005b5
Re-enable shrink test ( #14331 )
2020-12-30 08:25:45 -08:00
7893e2e307
Remove Testnet-specific old code ( #14305 )
2020-12-29 05:11:46 +09:00
addffd7694
Log error from AppendVec removal & a panic clean ( #14302 )
2020-12-29 04:59:09 +09:00
c693ffaa08
Fix subtraction overflow in metrics ( #14290 )
2020-12-27 16:26:22 -08:00
49019c6613
obtains staked-nodes from the root-bank ( #14257 )
...
... as opposed to the working bank
2020-12-27 13:28:05 +00:00
ee0a80a092
Prevent bpf loader impersonators ( #14278 )
2020-12-24 03:04:48 +00:00
b1d702a618
Don't use caller passed executable account ( #14276 )
2020-12-23 14:34:14 -08:00
5945305b1d
Save cloning program account data ( #14251 )
2020-12-23 10:18:14 -08:00
5b903318b2
vote: Add helper for creating current-versioned states
2020-12-22 19:37:26 -07:00
3881ae10fb
Deinitialize nonce data upon zero balance
2020-12-22 19:37:26 -07:00
a14cfd660a
removes &Arc<Self> receivers ( #14234 )
2020-12-22 23:51:53 +00:00
c276513b84
Ignore long/hanging test ( #14261 )
2020-12-22 16:13:03 -07:00
7042f11791
Feature-gate stake-program-v3 ( #14232 )
...
* Remove deprecated legacy stake program
* Add legacy stake program
* Strip out duplicative legacy code
* Feature-deploy stake-program-v3
* Add ownership check in stake processor
2020-12-22 11:20:38 -07:00
ab205b682a
Upgradeable programs called same as non-upgradeable ( #14239 )
...
* Upgradeable programs called same as non-upgradeable
* nudge
2020-12-22 09:26:55 -08:00
baa9602411
Add shrink paths ( #14238 )
2020-12-21 21:33:37 -08:00
7b08cb1f0d
improves performance in replay-stage ( #14217 )
...
bank::vote_accounts returns a hash-map which is slow to iterate, but all uses
only require an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/runtime/src/bank.rs#L4300-L4306
Similarly, calculate_stake_weighted_timestamp takes a hash-map whereas it only
requires an iterator:
https://github.com/solana-labs/solana/blob/b3dc98856/sdk/src/stake_weighted_timestamp.rs#L21-L28
2020-12-21 19:18:19 +00:00
93ae177503
Do not delete ALL other snapshots before downloading a new snapshot
2020-12-21 08:58:52 +00:00
e15f95a36f
Fix timestamp handling on ledger warp ( #14210 )
...
* Reset timestamp for slot and epoch-start on warp
* Fix genesis timestamp metric source
* Remove check that timestamp > unix_timestamp_from_genesis
Default to previous timestamp, not genesis timestamp
* Move timestamp metrics to report even on warp
* Initialize slot 0 timestamps correctly
* Add feature gate to warp testnet timestamp
* Review suggestion: simplify warp-timestamp slot check
2020-12-20 21:36:56 +00:00
e8cc0bef6c
Add CPI support for upgradeable loader ( #14193 )
2020-12-17 15:39:49 -08:00
d6d76219b6
caches staked nodes computed from vote-accounts ( #13929 )
2020-12-17 21:22:50 +00:00
efc091e28a
Add transactionCount field to GetEpochInfo
2020-12-17 12:29:50 -08:00
75e9e321de
Fix race between setting tick height and calculating accounts hash ( #14101 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-15 12:45:40 -08:00
d513b0c4ca
Add Program loader/environment instruction errors ( #14120 )
2020-12-15 09:54:07 -08:00
53fe0bb5eb
Remove regular expressions, speeds up untar ( #14119 )
2020-12-15 08:48:21 -08:00
5294f70189
Remove some non-warnings ( #14115 )
2020-12-14 16:26:17 -08:00
9e90394583
Upgradeable loader ( #13689 )
2020-12-14 15:35:10 -08:00
e907510a1a
clippy
2020-12-14 13:11:50 -08:00
7143aaa89b
Clippy
2020-12-14 08:03:29 -08:00
de9ac43ebf
Simple cap by including sysvars and native programs ( #13884 )
...
* Simpler cap by including sysvars and native programs
* Fix tests
* Add comment
* revert some unrelated code
* Update test_bank_update_sysvar_account for cap.
* Test cap. for add_native_program using new helper
* Improve the cap adjustment with new tests
* Fix typo...
* Adjust test for improved code coverage
* Rename simpler_capitalization => simple_capitalization
* More rename and bonus commenting
2020-12-14 21:03:24 +09:00
5788e75188
Shrink slots by sparseness of written data size ( #14072 )
2020-12-14 02:58:38 +09:00
55fc963595
Move slot cleanup to AccountsBackgroundService ( #13911 )
...
* Move bank drop to AccountsBackgroundService
* Send to ABS on drop instead, protects against other places banks are dropped
* Fix Abi
* test
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-13 01:22:34 +00:00
edff62af3d
Switch to memmap2 fork
2020-12-11 23:44:03 +00:00
d6eff3d62c
Filter out stake and vote accounts with incorrect owners ( #14062 )
...
* Add failing test
* Check stake/vote accounts for validity
* Feature gate change
* Add datapoint
* Add test realism
2020-12-11 12:21:25 -07:00
2d3a337200
Increase another snapshot limit: UNPACKED_COUNT ( #14071 )
2020-12-11 11:01:22 +00:00
7078a6ac61
Add more comments in Stakes::store for its subtlety ( #14065 )
...
* Add more comments in Stakes::store for its subtlety
* more comment tweak
2020-12-11 18:13:36 +09:00
09bd412b13
Wrong rpc inflation rate ( #14063 )
...
* Fix wrong inflation calculation in rpc
* Reorder
* Fix test
2020-12-11 07:42:39 +00:00
0a9ff1dc9d
Initial solana-test-validator command-line program
2020-12-11 04:17:38 +00:00
83fda2d972
Clean invoke in prep for native invoke ( #14055 )
2020-12-10 18:25:57 -08:00
164b7895b3
Tiny add_native_program bug fixes with cleanups ( #14042 )
...
* Tiny add_native_program bug fixes with cleanups
* Fix typo
2020-12-11 11:03:31 +09:00
7abd8084b6
Add get_minimum_balance_for_rent_excemption to Client ( #14048 )
2020-12-10 16:39:28 -08:00
5ea80e673f
Fix priv escalation test ( #14046 )
2020-12-10 22:36:33 +00:00
68109a46e0
Lower priority for hashing threads. ( #14043 )
2020-12-10 12:26:47 -08:00
14e241be35
Increase snapshot file size limits ( #14024 )
...
* Increase snapshot file size limits
* Fix test
* Reorder code a bit
2020-12-10 02:46:33 +09:00
28b014ccf4
Add tests for store_account_and_update_capitalization ( #14008 )
2020-12-09 00:46:28 +09:00
dca579851c
Cannot change owner if account is executable ( #13977 )
2020-12-07 11:37:07 -08:00
a706706572
Validator CLI option to enable just-in-time compilation of BPF ( #13789 )
...
* Adds a CLI option to the validator to enable just-in-time compilation of BPF.
* Refactoring to use bpf_loader_program instead of feature_set to pass JIT flag from the validator CLI to the executor.
2020-12-07 09:49:55 +01:00
882d033233
Don't use stale executor cache ( #13966 )
2020-12-05 05:54:42 +00:00
5dceddd21d
Remove associated functions on AccountsDb, replace with methods ( #13953 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-12-03 17:53:42 -08:00
17defbff13
Make `Accounts::is_non_loader_key()` a method on `Message`
2020-12-02 20:10:08 +00:00
404fc1570d
runtime: Replace `HashAgeKind` with `NonceRollbackInfo`
2020-12-02 20:10:08 +00:00
be7760caa1
accounts: Replace nonce_rollback unreachable block with descriptive panic
2020-12-02 20:10:08 +00:00
47af5933ca
accounts: Don't assume fee-payer is the first account
2020-12-02 20:10:08 +00:00
8df2a4bac0
runtime: Factor out `DurableNoncePartial` finishing logic and add tests
2020-12-02 20:10:08 +00:00
73111b005f
Reduce the number of snapshots
2020-12-01 11:13:37 -08:00
254790f8c8
Fully clean up temporary snapshot staging directories
2020-11-30 19:40:06 -08:00
e1793e5a13
caches vote-state de-serialized from vote accounts ( #13795 )
...
Gossip and other places repeatedly de-serialize vote-state stored in
vote accounts. Ideally the first de-serialization should cache the
result.
This commit adds new VoteAccount type which lazily de-serializes
VoteState from Account data and caches the result internally.
Serialize and Deserialize traits are manually implemented to match
existing code. So, despite changes to frozen_abi, this commit should be
backward compatible.
2020-11-30 17:18:33 +00:00
ce4304cc9a
Add extra checks to verify_and_update ( #13848 )
...
* Add extra checks to verify_and_update
* nudge
2020-11-29 10:06:43 +00:00
6048342c57
Enhance ledger-tool for multi-epoch rewards ( #13837 )
...
* Support ledger-tool for multi-epoch rewards
* nits
* Ensure not to skip some records in csv
2020-11-27 04:20:47 +00:00
955b99cf69
Fix roots_and_ref_count ( #13827 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-26 13:09:20 -08:00
43b82b31e5
More TestValidator cleanup
2020-11-26 08:56:25 +00:00
b5f7e39be8
TestValidator public interface cleanup
2020-11-25 17:04:37 -08:00
4ef2da0ff0
Add `solana logs` command
2020-11-25 11:44:41 -08:00
b70abdc645
Nonce updates ( #13799 )
...
* runtime: Add `FeeCalculator` resolution method to `HashAgeKind`
* runtime: Plumb fee-collected accounts for failed nonce tx rollback
* runtime: Use fee-collected nonce/fee account for nonced TX error rollback
* runtime: Add test for failed nonced TX accounts rollback
* Fee payer test
* fixup: replace nonce account when it pays the fee
* fixup: nonce fee-payer collect test
* fixup: fixup: clippy/fmt for replace...
* runtime: Test for `HashAgeKind::fee_calculator()`
* Clippy
Co-authored-by: Trent Nelson <trent@solana.com>
2020-11-24 23:53:51 -08:00
c930f92411
Restore bank feature used to deprecate Rewards sysvar ( #13786 )
2020-11-24 10:22:31 -07:00
60ce6f1a9b
Truncate logs immediately upon collection. Also reduce limit from 100k to 10k
2020-11-23 17:20:02 -08:00
791fb17437
Prevent scans on unrooted slots from seeing partial clean ( #13628 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-20 13:01:04 -08:00
62fa8b0ed8
Demote bank-process_transactions-error counters to info!
2020-11-20 10:43:07 -08:00
c75d97e3f2
Add new inflation feature-ids ( #13671 )
...
* Add new inflation feature-ids, and full_inflation default values
* Compute inflation start from full_inflation activation
* Include pico_inflation in inflation start computation
* Add full-inflation constructor
* Align inflation taper with rewards accrual start and catch overflow edge case
2020-11-20 09:54:41 -07:00
11e92f0c9f
Snapshot archives are now cleaned up when `--snapshot-compression none` is used
2020-11-20 08:41:17 -08:00
b74d7b5758
Fix fragile tests in prep of stake rewrite pr ( #13654 )
...
* Fix fragile tests in prep of stake rewrite pr
* Restore BOOTSTRAP_VALIDATOR_LAMPORTS where appropriate
* Further clean up
* Further clean up
* Aligh with other call site change
* Remove false warn!
* fix ci!
2020-11-20 17:21:03 +09:00
a8c29505f0
sanitizes bloom filters to avoid division by zero ( #13714 )
...
Pull requests received over the wire can cause a validator to panic
because of division by zero in bloom filters:
https://github.com/solana-labs/solana/blob/af08ba93e/runtime/src/bloom.rs#L86-L88
2020-11-19 23:35:22 +00:00
397cf726fc
Allow GNUSparse for genesis.bin ( #13704 )
2020-11-19 20:29:19 +00:00
43d5e47ea9
Rewrite stake accounts for clear migration ( #13461 )
...
* Reduce overage stake by rewritng stake accounts
* Write tests and finish implemention
* Create and use new feature gate
* Clean up logging
* Fix typo
* Simplify enable_rewrite_stake
* Fix typo...
* Even simplify gating
* Add metrics
2020-11-20 05:15:06 +09:00
110acd20dc
Check for overflow in rent partition calculation ( #13569 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-19 04:17:00 -08:00
645598e615
Improve monitoring of timestamp correction and bounding ( #13656 )
...
* Include rejuvenated field in Clock parsing
* Expand timestamp-correction logging info
2020-11-17 21:51:43 +00:00
6276360468
Prevent scans from seeing root updates/clean ( #13464 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-16 17:23:11 -08:00
bcd303a447
ledger-tool cap: delegation owner and stake v2 flag ( #13602 )
...
* Output delegation owner as well
* Add --enable-stake-program-v2
* Small cleanup and add sanity assertion
* Fix typo...
2020-11-15 19:38:46 +00:00
baa6b3a261
Add stable program logging for BPF and native programs
2020-11-14 08:26:01 -08:00
c97a7d1105
Small cleaning update_epoch_stakes ( #13576 )
2020-11-13 11:16:38 +00:00
a8a77614fe
Fix potential undefined behavior ( #13555 )
...
* Switch to ouroboros 0.5.1
* Update other lock files
2020-11-13 18:12:41 +09:00
89b474e192
Fix slow/stuck unstaking due to toggling in epoch ( #13501 )
...
* Fix slow/stuck unstaking due to toggling in epoch
* nits
* nits
* Add stake_program_v2 feature status check to cli
Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-11-11 14:11:57 -07:00
2c2432fddc
Refactor function ( #13294 )
...
Co-authored-by: Carl Lin <carl@solana.com>
2020-11-10 17:20:35 -08:00
91f4e99b4c
Fix parsing CreateAccountWithSeed instructions ( #13513 )
...
* Reduce required num_system_accounts and handle 2-account instructions properly
* Update CreateAccountWithSeed account docs to be correct
* Add CreateAccountWithSeed test
2020-11-10 23:51:53 +00:00
c0e2ef06dc
Fix Bank accounts hash mismatch related to Clock::unix_timestamp ( #13477 )
...
* Test for different ancestors with mismatch bank hash
* Test cleanup
* Remove nondeterministic ancestor check
* Update timestamp bounding feature key
* Update design doc
* Filter recent_timestamps to nodes voting within the last epoch
Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-11-10 02:10:09 +00:00
a81e7e7749
Output more inflation calc details in ledger-tool ( #13345 )
...
* Output more inflation calc details in ledger-tool
* Fix broken ci...
* Rename confusing variables
* Fix panic by wrapping PointValue with Opiton...
* Minor modifications
* Remove explict needless flush; Drop already does
* Yet another csv field adjustments
* Add data_size and rename epochs to earned_epochs
* Introduce null_tracer
* Unwrap Option in new_from_parent_with_tracer
* Don't shorten identifiers
* Allow irrefutable_let_patterns temporalily
* More null_tracer
* More field adjustments
2020-11-08 16:43:50 +09:00
1b1d9f6b0c
Feature-gate stake program ( #13394 )
...
* Add legacy stake-program handling
* Strip out duplicative legacy code
* Add feature for stake-program-fix
* Feature-deploy new stake program
* Expand comment
2020-11-06 08:08:11 +00:00
8f0796436a
shares the lock on gossip when processing prune messages ( #13339 )
...
Processing prune messages acquires an exclusive lock on gossip:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/cluster_info.rs#L1824-L1825
This can be reduced to a shared lock if active-sets are changed to use
atomic bloom filters:
https://github.com/solana-labs/solana/blob/55b0428ff/core/src/crds_gossip_push.rs#L50
2020-11-05 15:42:00 +00:00
bc62313c66
Allow feature builtins to overwrite existing builtins ( #13403 )
...
* Allow feature builtins to overwrite existing builtins
* Add feature_builtin ActivationType
* Correctly retain idempotent for replacing case
* Fix test
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2020-11-05 15:37:07 +00:00
c24fbb6f8b
Fix duplicate records of inner instructions ( #13380 )
...
* Fix duplicate records of inner instructions
* fix tests
* fix clippy
* Remove bad_inner_instructions
2020-11-05 15:23:52 +08:00
43053dcc90
Re-use accounts stores ( #12885 )
...
* Re-use accounts_db stores
Creating files and dropping mmap areas can be expensive
* Add test for storage finder
Can encounter an infinite loop when the store is too small, but
smaller than the normal store size.
* Fix storage finding
* Check for strong_count == 1
* try_recycle helper
2020-11-04 09:17:05 -08:00
b0d1ae1d8b
Comment Stakes::clone_with_epoch ( #13388 )
2020-11-04 11:18:05 +00:00
0e4509c497
Small code cleanup and typo fixes ( #13325 )
...
* Small code cleanup and typo fixes
* Clean up calculate_points_and_credits
2020-11-02 10:28:24 +09:00
55b0428ff7
Add store-tool ( #13254 )
2020-11-01 09:41:28 -08:00
e60bda082d
Adapt to feature::create_account
2020-11-01 05:53:29 +00:00
4260b3b416
Sample votes from ancestors back to root
2020-10-31 21:30:42 -07:00
eb2560e782
Prevent block times from ever going backward
2020-10-31 21:30:42 -07:00
90778615f6
Use bounded timestamp-correction when feature enabled
2020-10-31 21:30:42 -07:00
80db6c0980
Add bounded timestamp-estimation method
2020-10-31 21:30:42 -07:00
0049ab69fb
Repurpose unused as Clock::epoch_start_timestamp; add gated update
2020-10-31 21:30:42 -07:00
4b65e32f22
Move Feature struct to solana-program
2020-10-30 17:57:51 -07:00
bc7133d752
Print the entry type as well when checking archive ( #13312 )
2020-10-31 01:36:12 +09:00
Jeff Washington (jwash)
sakridge
Ryo Onodera
carllin
DimAn
Justin Starry
Michael Vines
Jack May
Tyera Eulberg
Tyera Eulberg
Greg Fitzgerald
Jon Cinque
Trent Nelson
behzad nouri
carllin
dependabot[bot]
Alexander Meißner
joshua-maros