Commit Graph

1910 Commits

Author SHA1 Message Date
Trent Nelson 1014781b87 Bump jsonrpc-* to 15.0.0 2020-09-25 19:30:47 +00:00
Michael Vines 7526bb96f3 Make test_process_rest_api less fragile 2020-09-25 11:40:36 -07:00
Michael Vines c10da16d7b Port instructions sysvar and secp256k1 program activation to FeatureSet 2020-09-25 11:40:36 -07:00
Michael Vines 35f5f9fc7b Add feature set identifier to gossiped version information 2020-09-25 11:40:36 -07:00
Josh 1d04c1db94
introduce RpcPerfSample and modify getPerformanceSamples output (#12434)
* introduce RpcPerfSample and modify getPerformanceSamples output

* camelCase test results
2020-09-24 14:22:22 -07:00
behzad nouri 42f1ef8acb
moves gossip-work thread pool cons to ClusterInfo::new (#12402) 2020-09-24 18:36:31 +00:00
Justin Starry 6601ec8f26
Record and store invoked instructions in transaction meta (#12311)
* Record invoked instructions and store in transaction meta

* Enable cpi recording if transaction sender is some

* Rename invoked to innerInstructions
2020-09-24 22:36:22 +08:00
Justin Starry 731a943239
Remove transaction encoding from storage layer (#12404) 2020-09-24 13:10:29 +08:00
sakridge 68e5a2ef56
Add RPC notify and banking keys debug (#12396) 2020-09-23 18:46:42 -07:00
Tyera Eulberg e1a212fb79
Bump spl-token (#12395) 2020-09-22 17:08:54 -06:00
Josh 65a6bfad09
Add blockstore column to store performance sampling data (#12251)
* Add blockstore column to store performance sampling data

* introduce timer and write performance metrics to blockstore

* introduce getRecentPerformanceSamples rpc

* only run on rpc nodes enabled with transaction history

* add unit tests for get_recent_performance_samples

* remove RpcResponse from rpc call

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* Add PerfSamples to purge/compaction

* refactor to use Instant::now and elapsed for timer

* switch to root bank and ensure not negative subraction

* remove duplicate constants

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-09-22 12:26:32 -07:00
Michael Vines 208dd1de3a Move TestValidator into its own module 2020-09-19 08:35:26 -07:00
Michael Vines 1a03afccb1 validator/ cleanup 2020-09-19 08:35:26 -07:00
Ryo Onodera cb8661bd49
Persistent tower (#10718)
* Save/restore Tower

* Avoid unwrap()

* Rebase cleanups

* Forcibly pass test

* Correct reconcilation of votes after validator resume

* d b g

* Add more tests

* fsync and fix test

* Add test

* Fix fmt

* Debug

* Fix tests...

* save

* Clarify error message and code cleaning around it

* Move most of code out of tower save hot codepath

* Proper comment for the lack of fsync on tower

* Clean up

* Clean up

* Simpler type alias

* Manage tower-restored ancestor slots without banks

* Add comment

* Extract long code blocks...

* Add comment

* Simplify returned tuple...

* Tweak too aggresive log

* Fix typo...

* Add test

* Update comment

* Improve test to require non-empty stray restored slots

* Measure tower save and dump all tower contents

* Log adjust and add threshold related assertions

* cleanup adjust

* Properly lower stray restored slots priority...

* Rust fmt

* Fix test....

* Clarify comments a bit and add TowerError::TooNew

* Further clean-up arround TowerError

* Truly create ancestors by excluding last vote slot

* Add comment for stray_restored_slots

* Add comment for stray_restored_slots

* Use BTreeSet

* Consider root_slot into post-replay adjustment

* Tweak logging

* Add test for stray_restored_ancestors

* Reorder some code

* Better names for unit tests

* Add frozen_abi to SavedTower

* Fold long lines

* Tweak stray ancestors and too old slot history

* Re-adjust error conditon of too old slot history

* Test normal ancestors is checked before stray ones

* Fix conflict, update tests, adjust behavior a bit

* Fix test

* Address review comments

* Last touch!

* Immediately after creating cleaning pr

* Revert stray slots

* Revert comment...

* Report error as metrics

* Revert not to panic! and ignore unfixable test...

* Normalize lockouts.root_slot more strictly

* Add comments for panic! and more assertions

* Proper initialize root without vote account

* Clarify code and comments based on review feedback

* Fix rebase

* Further simplify based on assured tower root

* Reorder code for more readability

Co-authored-by: Michael Vines <mvines@gmail.com>
2020-09-19 14:03:54 +09:00
Michael Vines c4913e3c9e SendTransactionServices now exit their thread on channel drop instead of by a flag 2020-09-18 17:29:10 +00:00
Michael Vines 75c3690ccd Give the duplicate send_transaction_service a different thread name 2020-09-18 17:29:10 +00:00
behzad nouri 9b866d79fb
shards crds values based on their hash prefix (#12187)
filter_crds_values checks every crds filter against every hash value:
https://github.com/solana-labs/solana/blob/ee646aa7/core/src/crds_gossip_pull.rs#L432
which can be inefficient if the filter's bit-mask only matches small
portion of the entire crds table.

This commit shards crds values into separate tables based on shard_bits
first bits of their hash prefix. Given a (mask, mask_bits) filter,
filtering crds can be done by inspecting only relevant shards.

If CrdsFilter.mask_bits <= shard_bits, then precisely only the crds
values which match (mask, mask_bits) bit pattern are traversed.
If CrdsFilter.mask_bits > shard_bits, then approximately only
1/2^shard_bits of crds values are inspected.

Benchmarking on a gce cluster of 20 nodes, I see ~10% improvement in
generate_pull_responses metric, but with larger clusters, crds table and
2^mask_bits are both larger, so the impact should be more significant.
2020-09-17 14:05:16 +00:00
Trent Nelson 32dcce0ac1 RPC: Limit request payload size to 50kB 2020-09-16 20:21:59 +00:00
Justin Starry f6cda2579f Fix off-by-one max payload checks 2020-09-16 12:46:06 -07:00
Michael Vines c6eea94edc Remove stale comment 2020-09-16 08:42:26 -07:00
Michael Vines 749208fa32 RPC sendTransaction now returns transaction logs on simulation failure 2020-09-16 08:42:26 -07:00
sakridge 3930cb865a
Add keccak-secp256k1 instruction (#11839)
* Implement keccak-secp256k1 instruction

Verifies eth addreses with ecrecover function

* Move secp256k1 test
2020-09-15 18:23:21 -07:00
Michael Vines daae638781 Add --gossip-validator argument 2020-09-14 20:18:27 -07:00
behzad nouri d6ec03f13c
patches default impl for crds filter (#12199)
In CrdsFilter.mask all bits after mask_bits are set to 1:
https://github.com/solana-labs/solana/blob/555252f4/core/src/crds_gossip_pull.rs#L65
However the default implementation, sets both mask and mask_bits to zero
which is inconsistent with CrdsFilter::compute_mask for a mask_bits of
zero.

This commit changes the default implementation by setting mask to
`!0u64` (i.e all bits set to one). As a result, for the default crds
filter, `test_mask` will always return true, whereas previously it was
always returning false.
https://github.com/solana-labs/solana/blob/555252f4/core/src/crds_gossip_pull.rs#L85

This is only used in tests and benchmarks, but causes some benchmarks to
be misleading by short circuiting in this line:
https://github.com/solana-labs/solana/blob/555252f4/core/src/crds_gossip_pull.rs#L429
2020-09-13 13:08:25 +00:00
Ryo Onodera de4a613610
Check bank capitalization (#11927)
* Check bank capitalization

* Simplify and unify capitalization calculation

* Improve and add tests

* Avoid overflow and inhibit automatic restart

* Fix test

* Tweak checked sum for cap. and add tests

* Fix broken build after merge conflicts..

* Rename to ClusterType

* Rename confusing method

* Clarify comment

* Verify cap. in rent and inflation tests

Co-authored-by: Stephen Akridge <sakridge@gmail.com>
2020-09-12 01:48:06 +09:00
carllin 9c490e06b0
Fix propagation on startup from snapshot (#12177) 2020-09-11 02:03:11 -07:00
Michael Vines 6f325d4594 `solana-validator --rpc-bind-address` argument now works as expected 2020-09-10 13:36:13 -07:00
Michael Vines 10ce839ec0 Speed up `bigtable transaction-history` command 2020-09-10 05:22:10 +00:00
Tyera Eulberg 4431080066
Program subscriptions now properly check results len and token program id (#12139) 2020-09-09 21:07:29 +00:00
Tyera Eulberg 05db41fe9c
Cache block time in Blockstore (#11955)
* Add blockstore column to cache block times

* Add method to cache block time

* Add service to cache block time

* Update rpc getBlockTime to use new method, and refactor blockstore slightly

* Return block_time with confirmed block, if available

* Add measure and warning to cache-block-time
2020-09-09 09:33:14 -06:00
behzad nouri 28f2fa3fd5
uses rust intrinsics to convert hashes to u64 (#12097) 2020-09-09 15:28:17 +00:00
Michael Vines 9e96180ce4 getMinimumBalanceForRentExemption now only responds to valid account lengths 2020-09-08 17:48:09 +00:00
Ryo Onodera 53b8ea4464
Rename to ClusterType and restore devnet compat. (#12068)
* Rename to ClusterType and restore devnet compat.

* De-duplicate parse code and add comments

* Adjust default Devnet genesis & reduce it in tests
2020-09-08 23:55:09 +09:00
anatoly yakovenko c67f8bd821
Forward transactions to the expected leader instead of your own TPU port (#12004)
* Use PoHRecorder to send to the right leader

* cleanup

* fmt

* clippy

* Cleanup, fix bug

Co-authored-by: Carl <carl@solana.com>
2020-09-08 17:00:49 +08:00
Justin Starry 9940870c89
Fix RPC transaction method configs serialization (#12100) 2020-09-08 13:08:09 +08:00
Justin Starry 369d376d10
Fix signaure subscription panic (#12077) 2020-09-06 14:33:12 +08:00
Ryo Onodera eabc63cdcd
Fix bad predicate with malformed gossip votes (#12072) 2020-09-06 13:27:50 +09:00
Michael Vines b64fb295a1 Add LARGEST_CONFIRMED_ROOT_UPLOAD_DELAY 2020-09-04 16:01:49 -07:00
Michael Vines 4ba43c29ce Request correct access 2020-09-04 16:01:49 -07:00
Michael Vines d3611f74c8 Add exit flag for bigtable upload operations 2020-09-04 16:01:49 -07:00
Michael Vines bafdcf24f5 Add BigTableUploadService 2020-09-04 16:01:49 -07:00
Michael Vines bc7731b969 Add BigTableUploadService 2020-09-04 16:01:49 -07:00
Michael Vines d8e2038dda Add --enable-bigtable-ledger-upload flag 2020-09-04 16:01:49 -07:00
behzad nouri 114c211b66
adds new CrdsFilterSet type for Vec<CrdsFilter> (#12029) 2020-09-04 13:04:47 +00:00
Tyera Eulberg 39246f9dd7
Revert signature-notification format change (#12032)
* Use untagged RpcSignatureResult enum to avoid breaking downstream consumers of current signature subscriptions

* Clean up client duplication

* Clippy
2020-09-03 18:14:45 -06:00
Greg Fitzgerald 2c091e4fca
Mark a withdraw authority as non-circulating (#12033) 2020-09-03 23:45:49 +00:00
behzad nouri bc7adb97ed
builds crds filters without looping over filters (#11998) 2020-09-03 20:32:23 +00:00
Tyera Eulberg 90fad36613
Use new_response for consistency 2020-09-03 12:02:26 -06:00
Tyera Eulberg b22de369b7
Rpc: add getMultipleAccounts endpoint (#12005)
* Add rpc endpoint to return the state of multiple accounts from the same bank

* Add docs

* Review comments: Dedupe account code, default to base64, add max const

* Add get_multiple_accounts to rpc-client
2020-09-03 11:35:06 -06:00
carllin 3f39ab1e04
Fix forwarding calculation (#12014)
Co-authored-by: Carl <carl@solana.com>
2020-09-03 02:31:28 -07:00
carllin 1c1a3f979d
Detect and notify when deserializable shreds are available (#11816)
* Add logic to check for complete data ranges

* Add RPC signature notification

Co-authored-by: Carl <carl@solana.com>
2020-09-01 22:06:06 -07:00
Michael Vines 7341e60043 Ensure that the spl-token 2 native mint account is owned by the spl-token 2 program.
Workaround for https://github.com/solana-labs/solana-program-library/issues/374 until spl-token 3 is shipped
2020-09-01 17:09:36 +00:00
carllin 7641b60a2b
Remove log (#11949)
Co-authored-by: Carl <carl@solana.com>
2020-08-31 16:40:22 -07:00
Tyera Eulberg 60c7ac6f95
Fix get_parsed_token_accounts (#11907) 2020-08-29 12:38:27 -06:00
Tyera Eulberg befd99edac
Bump spl-token version 2020-08-29 03:20:38 -06:00
Tyera Eulberg 2eff9a19c3
Update to token pack/unpack changes (#11900) 2020-08-28 22:06:30 -06:00
sakridge c4253dc0f9
Update to rayon 1.4.0 (#11898) 2020-08-28 21:00:11 -07:00
Tyera Eulberg 76be36c9ce
Update spl-token to v2.0 (#11884)
* Update account-decoder to spl-token v2.0

* Update transaction-status to spl-token v2.0

* Update rpc to spl-token v2.0

* Update getTokenSupply to pull from Mint directly

* Fixup to spl-token v2.0.1
2020-08-28 15:54:57 -06:00
Ryo Onodera d8c529a9b8
Small cleaning around consensus/bank_forks (#11873) 2020-08-28 08:56:53 +09:00
Tyera Eulberg 5c7d743883
Timestamp first vote (#11856) 2020-08-26 18:34:02 +00:00
Tyera Eulberg 1988ee9cd6
Rpc: Filter accounts with invalid mints from get_parsed_token_accounts (#11844)
* Filter out accounts with invalid mints from get_parsed_token_accounts

* Explicit docs
2020-08-26 11:39:42 -06:00
Michael Vines a1e2357d12 `solana-gossip spy` can now be given an identity keypair (`--identity` argument) 2020-08-22 17:00:50 -07:00
Michael Vines b660704faa Allow the sendTransaction preflight commitment level to be configured 2020-08-22 08:19:50 -07:00
behzad nouri 418b483af6
Fix filter_crds_values output alignment with the inputs (#11734) 2020-08-21 12:32:37 -07:00
Michael Vines 247f27af37
Submit a vote timestamp every vote (#10630)
* Submit a timestamp for every vote

* Submit at most one vote timestamp per second

* Submit a timestamp for every new vote

Co-authored-by: Tyera Eulberg <tyera@solana.com>
2020-08-21 11:10:51 -06:00
carllin c8d67aa8eb
Add option for repairing only from trusted validators (#11752)
Co-authored-by: Carl <carl@solana.com>
2020-08-21 00:35:11 -07:00
Tyera Eulberg 747f8d5877
Rpc: Return error if block does not exist (#11743)
* Return error if block does not exist

* Update docs
2020-08-20 15:05:30 -06:00
Tyera Eulberg 2fd2aceeb2
Bump spl-token to clean up magic number (#11726) 2020-08-20 09:51:12 -06:00
carllin b7ed06b17a
Cleanup test utilities (#11723)
* Add voting utility

* Add blockstore utility

Co-authored-by: Carl <carl@solana.com>
2020-08-20 05:04:38 +00:00
carllin 5f8d34feb3
Skip grace blocks if previous leader was on different fork (#11679)
* Start leader blocks if previous leader was on different fork

* Fix test

Co-authored-by: Carl <carl@solana.com>
2020-08-19 08:16:24 +00:00
carllin 0f0a2ddafe
Filter push/pulls from spies (#11620)
* Filter push/pulls from spies

* Don't pull from peers with shred version == 0, don't push to people with shred_version == 0

Co-authored-by: Carl <carl@solana.com>
2020-08-18 18:52:45 -07:00
Michael Vines d15173ad9d Address latest nightly clippy lints, but globally disable stable_sort_primitive 2020-08-17 22:36:10 -07:00
Michael Vines adc984a225 Rename Binary64 to Base64. Establish Base58 encoding 2020-08-17 17:26:29 -07:00
Michael Vines 757e147b3b Rework UiAccountData encode/decode such that it works from Rust 2020-08-17 17:26:29 -07:00
Ryo Onodera 23fa84b322
Re-do rent collection check on rent-exempt account (#11349)
* wip: re-do rent collection check on rent-exempt account

* Let's see how the ci goes

* Restore previous code

* Well, almost all new changes are revertable

* Update doc

* Add test and gating

* Fix tests

* Fix tests, especially avoid to change abi...

* Fix more tests...

* Fix snapshot restore

* Align to _new_ with better uninitialized detection
2020-08-17 14:22:16 +09:00
Tyera Eulberg 6c5b8f324a
Rpc: Add until parameter for getConfirmedSignaturesForAddress2 (#11644)
* Refactor bigtable apis to accept start and end keys

* Make helper fn to deserialize cell data

* Refactor get_confirmed_signatures_for_address to use get_row_data range

* Add until param to get_confirmed_signatures_for_address

* Add until param to blockstore api

* Plumb until through client/cli

* Simplify client params
2020-08-15 10:42:17 -06:00
Jack May 7c736f71fe
Make BPF Loader static (#11516) 2020-08-14 12:32:45 -07:00
Tyera Eulberg 4da1e9833c
Ensure highest_confirmed_root only grows (#11596)
* Split out commitment-cache update for unit testing

* Add failing test

* Ensure highest_confirmed_root only grows
2020-08-13 00:59:44 -06:00
sakridge f519fdecc2
generate_pull_response optimization (#11597) 2020-08-12 22:45:19 -07:00
Greg Fitzgerald 574c356863
Add CommitmentSlots::new_from_slot() (#11600) 2020-08-13 03:51:15 +00:00
Tyera Eulberg b1e452f876
Return blockstore signatures-for-address despite bigtable error (#11594) 2020-08-12 17:57:05 -06:00
sakridge 54137e3446
Add incoming pull response counter (#11591) 2020-08-12 14:07:05 -07:00
carllin 473b5249e3
Fix assertion failure (#11572)
Co-authored-by: Carl <carl@solana.com>
2020-08-12 12:10:24 -07:00
Jack May 9290e561e1
Align host addresses (#11384)
* Align host addresses

* support new program abi

* update epoch rollout

* Enforce aligned pointers in cross-program invocations
2020-08-11 16:11:52 -07:00
carllin 1b238dd63e
Gossip log (#11555)
Co-authored-by: Carl <carl@solana.com>
2020-08-11 21:03:54 +00:00
carllin 7ef50a9352
Move cluster slots update to separate thread (#11523)
* Add cluster_slots_service

Co-authored-by: Carl <carl@solana.com>
2020-08-11 12:48:13 -07:00
Ryo Onodera 2910fd467f
Fix rust fmt (#11537) 2020-08-11 22:53:55 +09:00
anatoly yakovenko 713851b68d
filter out old gossip pull requests (#11448)
* init

* builds

* stats

* revert

* tests

* clippy

* add some jitter

* shorter jitter timer

* update

* fixup! update

* use saturating_sub

* fix filters
2020-08-11 06:26:42 -07:00
dependabot[bot] 191a2eb088 chore: bump indexmap from 1.4.0 to 1.5.1
Bumps [indexmap](https://github.com/bluss/indexmap) from 1.4.0 to 1.5.1.
- [Release notes](https://github.com/bluss/indexmap/releases)
- [Commits](https://github.com/bluss/indexmap/compare/1.4.0...1.5.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-08-10 22:49:55 -07:00
Tyera Eulberg 88ca04dbdb
Add config param to specify offset/length for single and program account info (#11515)
* Add config param to specify dataSlice for account info and program accounts

* Use match instead of if
2020-08-10 22:35:29 +00:00
Tyera Eulberg da210ddd51
Fix parsing of spl-token Mint (#11512)
* Add failing test

* Fix jsonParsed mint
2020-08-10 20:35:36 +00:00
Tyera Eulberg a9f76862fb
Decode native-program and sysvar accounts (#11463)
* Pass pubkey in to account-decoder for sysvars

* Decode sysvar accounts

* Decode config accounts; move validator-info lower

* Decode stake accounts

* Review comments

* Stringify any account lamports and epochs that can be set to u64::MAX
2020-08-09 07:50:45 +00:00
sakridge 068d23f298
Add Binary64 option for account data (#11474)
* Add Binary64 option for account data

* Decode into binary64

* Reword docs
2020-08-08 22:40:13 -07:00
Greg Fitzgerald edadd5d6d5
Remove Budget from CLI (#11451)
* Remove support for Budget

Also:
* Make "pay" command a deprecated alias for the "transfer" command

* chore: remove budget from web3.js

* Drop Budget depedency from core

Validators no longer ship with builtin Budget
2020-08-07 16:01:51 -06:00
carllin 7e25130529
Send votes from banking stage to vote listener (#11434)
*  Send votes from banking stage to vote listener

Co-authored-by: Carl <carl@solana.com>
2020-08-07 11:21:35 -07:00
Tyera Eulberg b7c2681903
Token Accounts: return ui_amount, decimals with decoded account (#11407)
* Return ui_amount, decimals from token client methods

* Return ui_amount, decimals in RPC jsonParsed token accounts

* Fixup docs

* Return ui_amount, decimals in pubsub jsonParsed token accounts

* Remove unnecessary duplicate struct

* StringAmount rename
2020-08-07 11:37:39 -06:00
Greg Fitzgerald bad486823c
Add a client for BankForks (#10728)
Also:
* Use BanksClient in solana-tokens
2020-08-07 08:45:17 -06:00
Greg Fitzgerald 3fdd8ffdf0
Remove circular dep between InstructionError and SystemError (#11427) 2020-08-06 22:04:43 +00:00
Michael Vines 4222932e08 Plug getConfirmedSignaturesForAddress2 into bigtable storage 2020-08-05 23:04:16 -07:00
Michael Vines 02c0981ecf Rename startAfter to before 2020-08-05 14:33:14 -07:00