Commit Graph

369 Commits

Author SHA1 Message Date
Trent Nelson ef63ffce7d Bump ed25519-dalek to 1.0.1 2021-03-01 21:10:46 +00:00
Ivan Mironov 013daa8f47 RPC: Improve snapshot path sanitization 2021-02-20 13:06:07 -08:00
Justin Starry 15bbe6436d
Bump SPL token version to v3.1.0 (#15429)
* Bump SPL token version to v3.1.0

* Cargo.lock
2021-02-19 09:15:41 +00:00
Michael Vines 9ba69a7381 Adapt to fs_extra 1.2.0 2021-02-17 12:44:58 -08:00
Trent Nelson b3256aa722 Bump rand_core to 0.6.2
https://rustsec.org/advisories/RUSTSEC-2021-0023
2021-02-15 22:51:35 +00:00
carllin 629dcd0f39
Cleanup buffered packets (#15210) 2021-02-12 03:27:37 -08:00
Ryo Onodera c69027e5aa
chore: bump serde from 1.0.118 to 1.0.122 (#15126)
* chore: bump serde from 1.0.118 to 1.0.122

* Update programs/bpf/Cargo.lock
2021-02-05 19:23:24 +09:00
Tyera Eulberg d1563f0ccd
Bump tonic, prost, tarpc, tokio (#15013)
* Update tonic & prost, and regenerate proto

* Reignore doc code

* Revert pull #14367, but pin tokio to v0.2 for jsonrpc

* Bump backoff and goauth -> and therefore tokio

* Bump tokio in faucet, net-utils

* Bump remaining tokio, plus tarpc
2021-02-05 00:21:53 -07:00
Tyera Eulberg 98aa1fa4ea
Upgrade jsonrpc crates to v17.0.0 (#15018)
* Upgrade to jsonrpc 17.0.0

* Fix test

* tree

Co-authored-by: Michael Vines <mvines@gmail.com>
2021-02-02 19:53:08 -07:00
dependabot[bot] 1df93fa2be
chore: bump serde from 1.0.112 to 1.0.118 (#14828)
* chore: bump serde from 1.0.112 to 1.0.122

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.122.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.122)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Update frozen_abi digest following serde update

* Revert "chore: bump serde from 1.0.112 to 1.0.122"

This reverts commit a3ef4442a4c985144ae2bd7ceaf8899a7ab8d7c0.

* Revert "[auto-commit] Update all Cargo lock files"

This reverts commit c41c3b005fb1ccade55155302c52cd5736c4b55f.

* chore: bump serde from 1.0.112 to 1.0.118

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.112 to 1.0.118.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.112...v1.0.118)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

* Remove serum-dex pinning

* blind commit!

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
2021-02-02 23:28:16 +09:00
Tom Parker-Shemilt 01230a0105
Remove serial_test_derive dependency (#14891) 2021-01-28 22:35:31 -07:00
dependabot[bot] 4ecc0f4fa6
chore: bump rayon from 1.4.1 to 1.5.0 (#14177)
* chore: bump rayon from 1.4.1 to 1.5.0

Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/rayon-core-v1.4.1...rayon-core-v1.5.0)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-01-15 18:07:35 +09:00
Michael Vines 11daaadc93 Add --rpc-threads argument 2021-01-13 13:34:46 -08:00
behzad nouri 766195dded
limits number of crds values associated with a pubkey (#14467) 2021-01-08 18:54:40 +00:00
Michael Vines 3d077fb656 Revert "Upgrade in-tree tokio 0.2 usage to tokio 0.3"
This reverts commit 444ed768dc.
2020-12-31 06:26:53 +00:00
Michael Vines 444ed768dc Upgrade in-tree tokio 0.2 usage to tokio 0.3 2020-12-29 09:23:23 -08:00
Michael Vines b3dc988564 Stream RPC snapshot downloads 2020-12-19 15:54:23 -08:00
behzad nouri 6a3797e164
adds crds-value for broadcasting duplicate shreds through gossip (#14133)
In gossip, the header overhead we get from:
https://github.com/solana-labs/solana/blob/de9ac43eb/core/src/cluster_info.rs#L434-L435
https://github.com/solana-labs/solana/blob/de9ac43eb/core/src/crds_value.rs#L31-L36
https://github.com/solana-labs/solana/blob/de9ac43eb/core/src/crds_value.rs#L73
already exceeds SIZE_OF_NONCE in shreds. We also need aditional
meta-data (wallclock, source pubkey, ...). Which means that given the
SHRED_PAYLOAD_SIZE, we cannot fit all these in PACKET_DATA_SIZE:
https://github.com/solana-labs/solana/blob/de9ac43eb/ledger/src/shred.rs#L80

On top of that, we need 2 shred payloads as the proof of duplicate. So
each DuplicateShred crds value includes only a chunk of the payload,
along with the meta-data to reconstruct the full payload from the chunks
on the receiving end.
2020-12-18 14:32:43 +00:00
Trent Nelson 6875113d82 Bump SPL crates 2020-12-17 01:54:22 +00:00
Michael Vines 1e977ac025 Bump version to v1.6.0 2020-12-15 18:28:04 +00:00
Michael Vines cdd3e7d856
Remove solana-vote-signer (#14099) 2020-12-13 19:12:20 -08:00
dependabot[bot] 88d950e5a8
chore: bump log from 0.4.8 to 0.4.11 (#13691)
* chore: bump log from 0.4.8 to 0.4.11

Bumps [log](https://github.com/rust-lang/log) from 0.4.8 to 0.4.11.
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.8...0.4.11)

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2020-12-13 13:48:23 +09:00
Trent Nelson 546c6a4531 Override yanked miow crate to 0.2.2 2020-12-11 23:44:03 +00:00
Trent Nelson 277b6790c7 Override yanked net2 0.2.37 2020-12-11 23:44:03 +00:00
Michael Vines 0a9ff1dc9d Initial solana-test-validator command-line program 2020-12-11 04:17:38 +00:00
sakridge f6600810d7
Use LRU cache and blake3 hash of shreds to filter duplicates (#13976) 2020-12-07 16:42:39 -08:00
dependabot[bot] 856693ac1f chore: bump lru from 0.6.0 to 0.6.1
Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.6.0 to 0.6.1.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.6.0...0.6.1)

Signed-off-by: dependabot[bot] <support@github.com>
2020-11-19 14:28:50 -08:00
Michael Vines 8838a55a1a Bump spl-token and spl-memo crate versions 2020-11-04 21:44:33 +00:00
Greg Fitzgerald ca00197009
Upgrade tarpc and tokio (#13293) 2020-10-29 19:21:18 -06:00
behzad nouri ae91270961
implements ping-pong packets between nodes (#12794)
https://hackerone.com/reports/991106

> It’s possible to use UDP gossip protocol to amplify DDoS attacks. An attacker
> can spoof IP address in UDP packet when sending PullRequest to the node.
> There's no any validation if provided source IP address is not spoofed and
> the node can send much larger PullResponse to victim's IP. As I checked,
> PullRequest is about 290 bytes, while PullResponse is about 10 kB. It means
> that amplification is about 34x. This way an attacker can easily perform DDoS
> attack both on Solana node and third-party server.
>
> To prevent it, need for example to implement ping-pong mechanism similar as
> in Ethereum: Before accepting requests from remote client needs to validate
> his IP. Local node sends Ping packet to the remote node and it needs to reply
> with Pong packet that contains hash of matching Ping packet. Content of Ping
> packet is unpredictable. If hash from Pong packet matches, local node can
> remember IP where Ping packet was sent as correct and allow further
> communication.
>
> More info:
> https://github.com/ethereum/devp2p/blob/master/discv4.md#endpoint-proof
> https://github.com/ethereum/devp2p/blob/master/discv4.md#wire-protocol

The commit adds a PingCache, which maintains records of remote nodes
which have returned a valid response to a ping message, and on-the-fly
ping messages pending a pong response from the remote node.

When handling pull-requests, those from addresses which have not passed
the ping-pong check are filtered out, and additionally ping packets are
added for addresses which need to be (re)verified.
2020-10-28 17:03:02 +00:00
behzad nouri 37c8842bcb
scans crds table in parallel for finding old labels (#13073)
From runtime profiles, the majority time of ClusterInfo::handle_purge
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1605-L1626
is spent scanning crds table finding old labels:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/crds.rs#L175-L197

This can be done in parallel given that gossip thread-pool:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1637-L1641
is idle when handle_purge is invoked:
https://github.com/solana-labs/solana/blob/0776fa05c/core/src/cluster_info.rs#L1681
2020-10-23 14:17:37 +00:00
Michael Vines 6858950f76 Remove frozen ABI modules from solana-sdk 2020-10-20 16:11:30 -07:00
Tyera Eulberg d008dfb7ad
Bump spl-memo and spl-token versions (#12917) 2020-10-15 18:23:41 +00:00
Michael Vines 8a119c1483 Bump reqwest/rayon to get past `cargo audit` 2020-10-11 18:51:09 +00:00
Ryo Onodera 1f4bcf70b0
Fix various ledger-tool error due to no builtins (#12759)
* Fix various ledger-tool error due to no builtins

* Add missing file...
2020-10-09 12:19:36 -06:00
Michael Vines 11df2e2236 Bump version to v1.5.0 2020-10-08 04:51:36 +00:00
Jack May 41ad3dd8f0
Remove skip-no-mangle entirely (#12696) 2020-10-07 09:13:48 -07:00
Trent Nelson e35889542b RPC: Support base64 encoded transactions
Defaults to base58
2020-10-06 22:41:06 -06:00
behzad nouri 537bbde22e
builds crds filters in parallel (#12360)
Based on run-time profiles, the majority time of new_pull_requests is
spent building bloom filters, in hashing and bit-vec ops.

This commit builds crds filters in parallel using rayon constructs. The
added benchmark shows ~5x speedup (4-core machine, 8 threads).
2020-09-29 23:06:02 +00:00
Michael Vines 31696a1d72 Port BPFLoader2 activation to FeatureSet and rework built-in program activation 2020-09-28 12:50:19 -07:00
Trent Nelson 1014781b87 Bump jsonrpc-* to 15.0.0 2020-09-25 19:30:47 +00:00
Tyera Eulberg e1a212fb79
Bump spl-token (#12395) 2020-09-22 17:08:54 -06:00
behzad nouri 9b866d79fb
shards crds values based on their hash prefix (#12187)
filter_crds_values checks every crds filter against every hash value:
https://github.com/solana-labs/solana/blob/ee646aa7/core/src/crds_gossip_pull.rs#L432
which can be inefficient if the filter's bit-mask only matches small
portion of the entire crds table.

This commit shards crds values into separate tables based on shard_bits
first bits of their hash prefix. Given a (mask, mask_bits) filter,
filtering crds can be done by inspecting only relevant shards.

If CrdsFilter.mask_bits <= shard_bits, then precisely only the crds
values which match (mask, mask_bits) bit pattern are traversed.
If CrdsFilter.mask_bits > shard_bits, then approximately only
1/2^shard_bits of crds values are inspected.

Benchmarking on a gce cluster of 20 nodes, I see ~10% improvement in
generate_pull_responses metric, but with larger clusters, crds table and
2^mask_bits are both larger, so the impact should be more significant.
2020-09-17 14:05:16 +00:00
behzad nouri 28f2fa3fd5
uses rust intrinsics to convert hashes to u64 (#12097) 2020-09-09 15:28:17 +00:00
Tyera Eulberg befd99edac
Bump spl-token version 2020-08-29 03:20:38 -06:00
Tyera Eulberg 2eff9a19c3
Update to token pack/unpack changes (#11900) 2020-08-28 22:06:30 -06:00
sakridge c4253dc0f9
Update to rayon 1.4.0 (#11898) 2020-08-28 21:00:11 -07:00
Tyera Eulberg 76be36c9ce
Update spl-token to v2.0 (#11884)
* Update account-decoder to spl-token v2.0

* Update transaction-status to spl-token v2.0

* Update rpc to spl-token v2.0

* Update getTokenSupply to pull from Mint directly

* Fixup to spl-token v2.0.1
2020-08-28 15:54:57 -06:00
Tyera Eulberg 2fd2aceeb2
Bump spl-token to clean up magic number (#11726) 2020-08-20 09:51:12 -06:00
Jack May 7c736f71fe
Make BPF Loader static (#11516) 2020-08-14 12:32:45 -07:00