3f4731b37f
Standardize thread names
...
Tenets:
1. Limit thread names to 15 characters
2. Prefix all Solana-controlled threads with "sol"
3. Use Camel case. It's more character dense than Snake or Kebab case
2022-08-20 07:49:39 -07:00
7573000d87
Revert "Rust v1.63.0 ( #27148 )" ( #27245 )
...
This reverts commit a2e7bdf50a
2022-08-19 09:19:44 +01:00
a2e7bdf50a
Rust v1.63.0 ( #27148 )
...
* Upgrade to Rust v1.63.0
* Add nightly_clippy_allows
* Resolve some new clippy nightly lints
* Increase QUIC packets completion timeout
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-08-17 15:48:33 -07:00
225cddcffb
serialize incremental_snapshot_hash ( #26839 )
...
* serialize incremental_snapshot_hash
* pr feedback
2022-08-17 15:14:31 -05:00
69598ed4c0
Refactor: Add `RuntimeConfig` field to Bank ( #26946 )
...
* Refactor: Simplify arguments for bank constructor methods
* Refactor: Add RuntimeConfig to Bank fields
* Arc wrap runtime_config
* Arc wrap all runtime config usages
* Remove Copy trait derivation from RuntimeConfig
* Remove some arc wrapping
2022-08-05 20:49:00 +01:00
a7e4df5d6e
Separate remapping append_vecs into functions ( #26696 )
...
Separate remapping append_vecs into remap_and_reconstruct_storages and helpers
2022-07-25 14:55:30 -05:00
f2b81fe665
Separate field_from_stream to get bank/account fields from individual snapshot stream ( #26679 )
2022-07-21 12:27:31 -05:00
3e9bd6170c
use OnceCell instead of RwLock for rent_paying_accounts_by_partition ( #26601 )
2022-07-14 08:52:41 -05:00
6b0eb5a42b
handle rent paying accounts differently ( #26487 )
...
* handle rent paying accounts differently
* restore collection
* downgrade assert to metric and warn until we get more runtime
2022-07-12 17:51:33 -05:00
90ef2cd02a
Parse snapshot for bank fields ( #26016 )
2022-07-06 17:30:30 -05:00
17a99d98dd
Revert "avoid adding to 'uncleaned_roots' when generating index and c… ( #26441 )
...
Revert "avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink (#25936 )"
This reverts commit e24cc537a4
2022-07-06 11:32:45 -05:00
53b9420562
Remove `INTO!` macros in serde_snapshot ( #26409 )
2022-07-05 16:37:20 -05:00
e24cc537a4
avoid adding to 'uncleaned_roots' when generating index and caller passes accounts-db-skip-shrink ( #25936 )
2022-06-14 10:10:44 -05:00
8caced68ce
Serialize lamports per signature ( #25364 )
...
* Serialize lamports per signature
* Add full snapshot archive test, enable features in previous tests
2022-06-06 13:07:02 -07:00
dd0bf4ff4b
add filler accounts gradually with cache flush ( #25678 )
2022-06-01 15:47:21 -05:00
de03a3d3aa
filler accounts get a non-zero rent_epoch ( #25386 )
...
* fill accounts get a non-zero rent_epoch
* use snapshot_slot
* get rid of perf simulation code
2022-05-20 09:03:55 -05:00
60b2155bd3
Add accounts-filler-size command line option ( #23896 )
2022-04-11 13:10:09 -05:00
c27150b1a3
reserialize_bank_fields_with_hash ( #23916 )
...
* reserialize_bank_with_new_accounts_hash
* Update runtime/src/serde_snapshot.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/serde_snapshot/tests.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-07 14:05:57 -05:00
550ca7bf92
compare contents of serialized banks instead of exact file format ( #24141 )
...
* compare contents of serialized banks instead of exact file format
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* Update runtime/src/snapshot_utils.rs
Co-authored-by: Brooks Prumo <brooks@prumo.org>
* pr feedback
* get rid of clone
* pr feedback
Co-authored-by: Brooks Prumo <brooks@prumo.org>
2022-04-06 21:55:44 -05:00
fddd162645
reserialize bank in ahv by first writing to temp file in abs ( #23947 )
2022-04-06 21:39:26 -05:00
6a7f6585ce
persist historical_roots ( #24029 )
2022-04-04 13:13:11 -05:00
0ca5a0ec68
prior_roots -> historical_roots ( #24064 )
2022-04-02 12:01:43 -05:00
396b49a7c1
Start saving/loading prior_roots(_with_hash) to snapshot ( #23844 )
...
* Start saving/loading prior_roots(_with_hash) to snapshot
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* Update runtime/src/accounts_index.rs
Co-authored-by: Michael Vines <mvines@gmail.com>
* update comment
Co-authored-by: Michael Vines <mvines@gmail.com>
2022-03-24 10:06:24 -05:00
cc4d75a16f
fix typo ( #23535 )
2022-03-08 18:28:00 -06:00
fb0e71946f
Cleanup use declaration ( #22875 )
2022-02-01 23:34:11 +00:00
22f6db2e4f
Cleanup serde snapshot common.rs ( #22854 )
2022-02-01 05:39:53 -06:00
d3b95b9226
Cleanup AccountStorage struct ( #22463 )
...
* Revert "chore: bump dashmap from 4.0.2 to 5.0.0 (#21824 )"
This reverts commit 8aa3d690b5
2022-01-13 17:51:53 -06:00
2756abce39
More serde snapshot cleanup ( #22449 )
2022-01-13 09:20:20 -06:00
9c3144e286
Refactor serde snapshot's "future" to "newer" ( #22431 )
2022-01-13 07:12:09 -06:00
fb62407232
AcctIdx: appendvecid: u32 ( #21842 )
2022-01-03 10:35:17 -06:00
af53d2f692
simplify api on reconstruct_single_storage ( #21970 )
2021-12-17 09:06:23 -06:00
4c0373b72a
introduce AtomicAppendVecId ( #21793 )
2021-12-11 21:34:35 -06:00
eeb97fe7ce
Add accounts_data_len to Bank ( #21781 )
2021-12-11 09:34:38 -06:00
ec7e17787e
Compute accounts data len during generate_index() ( #21757 )
2021-12-10 13:27:59 -06:00
1528f85112
Add return types to generate_index() ( #21735 )
2021-12-09 17:15:13 -06:00
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
b78f5b6032
Refactor: Cleanup InstructionProcessor ( #21404 )
...
* Moves create_message(), native_invoke() and process_cross_program_instruction()
from the InstructionProcessor to the InvokeContext so that they can have a useful "self" parameter.
* Moves InstructionProcessor into InvokeContext and Bank.
* Moves ExecuteDetailsTimings into its own file.
* Moves Executor into invoke_context.rs
* Moves PreAccount into its own file.
* impl AbiExample for BuiltinPrograms
2021-12-01 08:54:42 +01:00
ba9dfa0d22
Remove frozen account support
2021-11-29 08:38:11 -08:00
c24e30f689
Constrain memory ordering on AccountsDb::next_id ( #21311 )
2021-11-18 20:34:37 -06:00
c3e5927d16
Constrain memory ordering on AccountsDb::write_version ( #21312 )
2021-11-17 07:11:16 -06:00
7200c5106e
Replaces MockInvokeContext by ThisInvokeContext in tests ( #20881 )
...
* Replaces MockInvokeContext by ThisInvokeContext in BpfLoader, SystemInstructionProcessor, CLIs, ConfigProcessor, StakeProcessor and VoteProcessor.
* Finally, removes MockInvokeContext, MockComputeMeter and MockLogger.
* Adjusts assert_instruction_count test.
* Moves ThisInvokeContext to the program-runtime crate.
2021-11-04 21:47:32 +01:00
b1e3a82c11
refactor rent collector and logs ( #21155 )
2021-11-04 06:28:41 -05:00
c59c56c0f8
log rent exempt accounts ( #21137 )
2021-11-03 14:44:52 -05:00
9d330fc638
FillerAccts: use variable cycle partitions ( #20963 )
2021-10-27 11:18:27 -05:00
f14365f4b4
Accountsdb plugin postgres -- bulk insertion at startup ( #20763 )
...
Use bulk insertion to Postgres at startup to reduce time taken for initial snapshot restore for postgres plugin. Avoid duplicate writes of accounts at startup. Doing account plugin notification and indexing in parallel.
Improved error handling for postgres plugin to show the real db issues for debug purpose
Added more metrics for postgres plugin.
Refactored plugin centric code out to a sub module from accounts_db and added unit tests
2021-10-24 12:43:33 -07:00
a8e000a2a6
add filler accounts to bloat validator and predict failure ( #20491 )
...
* add filler accounts to bloat validator and predict failure
* assert no accounts match filler
* cleanup magic numbers
* panic if can't load from snapshot with filler accounts specified
* some renames
* renames
* into_par_iter
* clean filler accts, too
2021-10-11 12:46:27 -05:00
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
1347b507da
Handle duplicate AppendVec IDs ( #20096 )
...
When reconstructing the AccountsDb, if the storages came from full and
incremental snapshots generated on different nodes, it's possible that
the AppendVec IDs could overlap/have duplicates, which would cause the
reconstruction to fail.
This commit handles this issue by unconditionally remapping the
AppendVec ID for every AppendVec.
Fixes #17088
2021-09-23 17:34:32 -05:00
456bf15012
AccountsIndexConfig -> AccountsDbConfig ( #19687 )
2021-09-08 04:30:38 +00:00
170927b7c4
nit: Move builtins types to builtins file ( #19597 )
2021-09-02 21:29:11 -07:00
Michael Vines
Brennan Watt
Jeff Washington (jwash)
Justin Starry
apfitzge
carllin
Brooks Prumo
Ashwin Sekar
Giorgio Gambino
Alexander Meißner
Jeff Washington (jwash)
Lijun Wang
Jack May