Commit Graph

1856 Commits

Author SHA1 Message Date
Josh 1d04c1db94
introduce RpcPerfSample and modify getPerformanceSamples output (#12434)
* introduce RpcPerfSample and modify getPerformanceSamples output

* camelCase test results
2020-09-24 14:22:22 -07:00
behzad nouri 42f1ef8acb
moves gossip-work thread pool cons to ClusterInfo::new (#12402) 2020-09-24 18:36:31 +00:00
Justin Starry 6601ec8f26
Record and store invoked instructions in transaction meta (#12311)
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions
2020-09-24 22:36:22 +08:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
sakridge 68e5a2ef56
Add RPC notify and banking keys debug (#12396) 2020-09-23 18:46:42 -07:00
Tyera Eulberg e1a212fb79
Bump spl-token (#12395) 2020-09-22 17:08:54 -06:00
Josh 65a6bfad09
Add blockstore column to store performance sampling data (#12251)
* Add blockstore column to store performance sampling data

* introduce timer and write performance metrics to blockstore

* introduce getRecentPerformanceSamples rpc

* only run on rpc nodes enabled with transaction history

* add unit tests for get_recent_performance_samples

* remove RpcResponse from rpc call

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* Add PerfSamples to purge/compaction

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* remove duplicate constants

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-09-22 12:26:32 -07:00
Michael Vines 208dd1de3a Move TestValidator into its own module 2020-09-19 08:35:26 -07:00
Michael Vines 1a03afccb1 validator/ cleanup 2020-09-19 08:35:26 -07:00
Ryo Onodera cb8661bd49
Persistent tower (#10718)
* Save/restore Tower

* Avoid unwrap()

* Rebase cleanups

* Forcibly pass test

* Correct reconcilation of votes after validator resume

* d b g

* Add more tests

* fsync and fix test

* Add test

* Fix fmt

* Debug

* Fix tests...

* save

* Clarify error message and code cleaning around it

* Move most of code out of tower save hot codepath

* Proper comment for the lack of fsync on tower

* Clean up

* Clean up

* Simpler type alias

* Manage tower-restored ancestor slots without banks

* Add comment

* Extract long code blocks...

* Add comment

* Simplify returned tuple...

* Tweak too aggresive log

* Fix typo...

* Add test

* Update comment

* Improve test to require non-empty stray restored slots

* Measure tower save and dump all tower contents

* Log adjust and add threshold related assertions

* cleanup adjust

* Properly lower stray restored slots priority...

* Rust fmt

* Fix test....

* Clarify comments a bit and add TowerError::TooNew

* Further clean-up arround TowerError

* Truly create ancestors by excluding last vote slot

* Add comment for stray_restored_slots

* Add comment for stray_restored_slots

* Use BTreeSet

* Consider root_slot into post-replay adjustment

* Tweak logging

* Add test for stray_restored_ancestors

* Reorder some code

* Better names for unit tests

* Add frozen_abi to SavedTower

* Fold long lines

* Tweak stray ancestors and too old slot history

* Re-adjust error conditon of too old slot history

* Test normal ancestors is checked before stray ones

* Fix conflict, update tests, adjust behavior a bit

* Fix test

* Address review comments

* Last touch!

* Immediately after creating cleaning pr

* Revert stray slots

* Revert comment...

* Report error as metrics

* Revert not to panic! and ignore unfixable test...

* Normalize lockouts.root_slot more strictly

* Add comments for panic! and more assertions

* Proper initialize root without vote account

* Clarify code and comments based on review feedback

* Fix rebase

* Further simplify based on assured tower root

* Reorder code for more readability

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-19 14:03:54 +09:00
Michael Vines c4913e3c9e SendTransactionServices now exit their thread on channel drop instead of by a flag 2020-09-18 17:29:10 +00:00
Michael Vines 75c3690ccd Give the duplicate send_transaction_service a different thread name 2020-09-18 17:29:10 +00:00
behzad nouri 9b866d79fb
shards crds values based on their hash prefix (#12187)
filter_crds_values checks every crds filter against every hash value:
https://github.com/solana-labs/solana/blob/ee646aa7/core/src/crds_gossip_pull.rs#L432
which can be inefficient if the filter's bit-mask only matches small
portion of the entire crds table.

This commit shards crds values into separate tables based on shard_bits
first bits of their hash prefix. Given a (mask, mask_bits) filter,
filtering crds can be done by inspecting only relevant shards.

If CrdsFilter.mask_bits <= shard_bits, then precisely only the crds
values which match (mask, mask_bits) bit pattern are traversed.
If CrdsFilter.mask_bits > shard_bits, then approximately only
1/2^shard_bits of crds values are inspected.

Benchmarking on a gce cluster of 20 nodes, I see ~10% improvement in
generate_pull_responses metric, but with larger clusters, crds table and
2^mask_bits are both larger, so the impact should be more significant.
2020-09-17 14:05:16 +00:00
Trent Nelson 32dcce0ac1 RPC: Limit request payload size to 50kB 2020-09-16 20:21:59 +00:00
Justin Starry f6cda2579f Fix off-by-one max payload checks 2020-09-16 12:46:06 -07:00
Michael Vines c6eea94edc Remove stale comment 2020-09-16 08:42:26 -07:00
Michael Vines 749208fa32 RPC sendTransaction now returns transaction logs on simulation failure 2020-09-16 08:42:26 -07:00
sakridge 3930cb865a
Add keccak-secp256k1 instruction (#11839)
* Implement keccak-secp256k1 instruction

Verifies eth addreses with ecrecover function

* Move secp256k1 test
2020-09-15 18:23:21 -07:00
Michael Vines daae638781 Add --gossip-validator argument 2020-09-14 20:18:27 -07:00
behzad nouri d6ec03f13c
patches default impl for crds filter (#12199)
In CrdsFilter.mask all bits after mask_bits are set to 1:
https://github.com/solana-labs/solana/blob/555252f4/core/src/crds_gossip_pull.rs#L65
However the default implementation, sets both mask and mask_bits to zero
which is inconsistent with CrdsFilter::compute_mask for a mask_bits of
zero.

This commit changes the default implementation by setting mask to
`!0u64` (i.e all bits set to one). As a result, for the default crds
filter, `test_mask` will always return true, whereas previously it was
always returning false.
https://github.com/solana-labs/solana/blob/555252f4/core/src/crds_gossip_pull.rs#L85

This is only used in tests and benchmarks, but causes some benchmarks to
be misleading by short circuiting in this line:
https://github.com/solana-labs/solana/blob/555252f4/core/src/crds_gossip_pull.rs#L429
2020-09-13 13:08:25 +00:00
Ryo Onodera de4a613610
Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
carllin 9c490e06b0
Fix propagation on startup from snapshot (#12177) 2020-09-11 02:03:11 -07:00
Michael Vines 6f325d4594 `solana-validator --rpc-bind-address` argument now works as expected 2020-09-10 13:36:13 -07:00
Michael Vines 10ce839ec0 Speed up `bigtable transaction-history` command 2020-09-10 05:22:10 +00:00
Tyera Eulberg 4431080066
Program subscriptions now properly check results len and token program id (#12139) 2020-09-09 21:07:29 +00:00
Tyera Eulberg 05db41fe9c
Cache block time in Blockstore (#11955)
* Add blockstore column to cache block times

* Add method to cache block time

* Add service to cache block time

* Update rpc getBlockTime to use new method, and refactor blockstore slightly

* Return block_time with confirmed block, if available

* Add measure and warning to cache-block-time
2020-09-09 09:33:14 -06:00
behzad nouri 28f2fa3fd5
uses rust intrinsics to convert hashes to u64 (#12097) 2020-09-09 15:28:17 +00:00
Michael Vines 9e96180ce4 getMinimumBalanceForRentExemption now only responds to valid account lengths 2020-09-08 17:48:09 +00:00
Ryo Onodera 53b8ea4464
Rename to ClusterType and restore devnet compat. (#12068)
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
anatoly yakovenko c67f8bd821
Forward transactions to the expected leader instead of your own TPU port (#12004)
* Use PoHRecorder to send to the right leader

* cleanup

* fmt

* clippy

* Cleanup, fix bug

Co-authored-by: Carl <carl@solana.com>
2020-09-08 17:00:49 +08:00
Justin Starry 9940870c89
Fix RPC transaction method configs serialization (#12100) 2020-09-08 13:08:09 +08:00
Justin Starry 369d376d10
Fix signaure subscription panic (#12077) 2020-09-06 14:33:12 +08:00
Ryo Onodera eabc63cdcd
Fix bad predicate with malformed gossip votes (#12072) 2020-09-06 13:27:50 +09:00
Michael Vines b64fb295a1 Add LARGEST_CONFIRMED_ROOT_UPLOAD_DELAY 2020-09-04 16:01:49 -07:00
Michael Vines 4ba43c29ce Request correct access 2020-09-04 16:01:49 -07:00
Michael Vines d3611f74c8 Add exit flag for bigtable upload operations 2020-09-04 16:01:49 -07:00
Michael Vines bafdcf24f5 Add BigTableUploadService 2020-09-04 16:01:49 -07:00
Michael Vines bc7731b969 Add BigTableUploadService 2020-09-04 16:01:49 -07:00
Michael Vines d8e2038dda Add --enable-bigtable-ledger-upload flag 2020-09-04 16:01:49 -07:00
behzad nouri 114c211b66
adds new CrdsFilterSet type for Vec<CrdsFilter> (#12029) 2020-09-04 13:04:47 +00:00
Tyera Eulberg 39246f9dd7
Revert signature-notification format change (#12032)
* Use untagged RpcSignatureResult enum to avoid breaking downstream consumers of current signature subscriptions

* Clean up client duplication

* Clippy
2020-09-03 18:14:45 -06:00
Greg Fitzgerald 2c091e4fca
Mark a withdraw authority as non-circulating (#12033) 2020-09-03 23:45:49 +00:00
behzad nouri bc7adb97ed
builds crds filters without looping over filters (#11998) 2020-09-03 20:32:23 +00:00
Tyera Eulberg 90fad36613
Use new_response for consistency 2020-09-03 12:02:26 -06:00
Tyera Eulberg b22de369b7
Rpc: add getMultipleAccounts endpoint (#12005)
* Add rpc endpoint to return the state of multiple accounts from the same bank

* Add docs

* Review comments: Dedupe account code, default to base64, add max const

* Add get_multiple_accounts to rpc-client
2020-09-03 11:35:06 -06:00
carllin 3f39ab1e04
Fix forwarding calculation (#12014)
Co-authored-by: Carl <carl@solana.com>
2020-09-03 02:31:28 -07:00
carllin 1c1a3f979d
Detect and notify when deserializable shreds are available (#11816)
* Add logic to check for complete data ranges

* Add RPC signature notification

Co-authored-by: Carl <carl@solana.com>
2020-09-01 22:06:06 -07:00
Michael Vines 7341e60043 Ensure that the spl-token 2 native mint account is owned by the spl-token 2 program.
Workaround for https://github.com/solana-labs/solana-program-library/issues/374 until spl-token 3 is shipped
2020-09-01 17:09:36 +00:00
carllin 7641b60a2b
Remove log (#11949)
Co-authored-by: Carl <carl@solana.com>
2020-08-31 16:40:22 -07:00
Tyera Eulberg 60c7ac6f95
Fix get_parsed_token_accounts (#11907) 2020-08-29 12:38:27 -06:00