blockworks-foundation
/
solana
mirror of https://github.com/blockworks-foundation/solana.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
16,519 Commits 168 Branches 497 Tags 539 MiB
Commit Graph

688 Commits

Author SHA1 Message Date
behzad nouri 5fb0ab9d00
removes redundant args from Shredder::try_recovery (#21226) 
Shredder::try_recovery is already taking a Vec<Shred> as an argument. All the
other arguments are embedded in the shreds, and are so redundant.
 2021-11-10 21:19:03 +00:00
Kirill Fomichev b4d09388aa
Disable JIT compiler on unsupported platforms (#21215) 2021-11-09 11:31:21 +00:00
dependabot[bot] c2754b8a51
chore: bump libc from 0.2.106 to 0.2.107 (#21202) 
* chore: bump libc from 0.2.106 to 0.2.107

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.106 to 0.2.107.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.106...0.2.107)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-08 12:14:12 -07:00
behzad nouri 11a53de0e3
rewrites Shredder::try_recovery (#21082) 
For every missing data-shred, Shredder::try_recovery calls into
new_empty_data_shred which does a redundant serialization into payload
buffer which is then immediately overwritten by the erasure recovery:
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L372-L417

Additionally, the implementation is unnecessary complex hindering
upcoming changes to erasure coding generation
https://github.com/solana-labs/solana/blob/696501500/ledger/src/shred.rs#L814-L938

The commit simplifies the Shredder::try_recovery implementation.
 2021-11-04 16:15:51 +00:00
dependabot[bot] ca51ea3e04
chore: bump libc from 0.2.105 to 0.2.106 (#21093) 
* chore: bump libc from 0.2.105 to 0.2.106

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.105 to 0.2.106.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.105...0.2.106)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-11-01 14:17:45 -06:00
behzad nouri 484ead01ed
optimizes and simplifies SlotMeta::completed_data_indexes ops (#21059) 
SlotMeta::completed_data_indexes is defined as a Vec<u32>:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore_meta.rs#L31-L32

which results in inefficient updates:
https://github.com/solana-labs/solana/blob/a8d78e89d/ledger/src/blockstore.rs#L3245-L3326

This commit changes the type to BTreeSet<u32> for efficient and simpler
updates and lookups.

The change should be backward compatible because Vec<T> and BTreeSet<T>
are both serialized as seq:
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L207-L208
https://github.com/serde-rs/serde/blob/ce0844b9e/serde/src/ser/impls.rs#L216-L217
 2021-10-31 12:56:25 +00:00
sakridge 4642a2c856
Optimize test_recovery_with_expanded_coding_shreds (#20849) 2021-10-28 11:38:08 +02:00
Tyera Eulberg f0de3e9bf0
chore: bump tonic from 0.5.2 to 0.6.1 (#21035) 
* Bump tonic and prost crates

* connect_lazy no longer infallible
 2021-10-28 06:59:36 +00:00
Justin Starry 036d7fcc81
Clean up sanitized tx creation for tests (#21006) 2021-10-27 18:09:16 +01:00
dependabot[bot] cb1947bea2
chore: bump libc from 0.2.104 to 0.2.105 (#20972) 
* chore: bump libc from 0.2.104 to 0.2.105

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.104 to 0.2.105.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.104...0.2.105)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-26 00:14:58 -06:00
Michael Vines 350bb561eb Clippy 2021-10-23 08:21:20 +00:00
dependabot[bot] 66e431dad5
chore: bump libc from 0.2.103 to 0.2.104 (#20764) 
* chore: bump libc from 0.2.103 to 0.2.104

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.103 to 0.2.104.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.103...0.2.104)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-10-19 22:38:02 -06:00
carllin b5f21d5e34
Add ledger tool to remove dead slot (#20810) 2021-10-19 20:23:16 -07:00
behzad nouri 231b58b5f1
adds more metrics to blockstore insert shreds stats (#20701) 2021-10-17 11:02:34 -04:00
Jeff Washington (jwash) 70b2e5fef2
hash calc uses all threads during startup (#20735) 2021-10-15 17:41:23 -05:00
Tyera Eulberg e806fa6904
Include token owners in TransactionTokenBalances (#20642) 
* Cache owners in TransactionTokenBalances

* Light cleanup

* Use return struct, and remove pub

* Single-use statements

* Why not, just do the whole crate

* Add metrics

* Make datapoint_debug to prevent spam unless troubleshooting
 2021-10-13 21:46:52 -06:00
sakridge 09e7782d76
Refactor code to get block signatures in get_confirmed_signatures_for_address2 (#20575) 
* Refactor get_confirmed_signatures_for_address2

* Move blockstore benches to ledger where they belong
 2021-10-13 09:55:19 +02:00
Tao Zhu 005d6863fd
- move cost tracker into bank, so each bank has its own cost tracker; (#20527) 
- move related modules to runtime
 2021-10-12 08:51:33 -05:00
Jeff Washington (jwash) a8e000a2a6
add filler accounts to bloat validator and predict failure (#20491) 
* add filler accounts to bloat validator and predict failure

* assert no accounts match filler

* cleanup magic numbers

* panic if can't load from snapshot with filler accounts specified

* some renames

* renames

* into_par_iter

* clean filler accts, too
 2021-10-11 12:46:27 -05:00
Brooks Prumo 5440c1d2e1
SnapshotPackagerService pushes incremental snapshot hashes to CRDS (#20442) 
Now that CRDS supports incremental snapshot hashes,
SnapshotPackagerService needs to push 'em!

This commit does two main things:

1. SnapshotPackagerService now knows about incremental snapshot hashes,
   and will push SnapshotPackage::IncrementalSnapshot hashes to CRDS.
2. At startup, when loading from a full + incremental snapshot, the
   hashes need to be passed all the way to SnapshotPackagerService so it
   can push these starting hashes to CRDS.  Those values have been piped
   through.

Fixes #20441 and #20423
 2021-10-08 15:14:56 -05:00
Tao Zhu 675fa6993b
- update const cost values with data collected by #19627 (#20314) 
- update cost calculation to closely proposed fee schedule #16984
 2021-10-08 14:48:50 -05:00
Trent Nelson 767f740305 Bump version to 1.9.0 2021-10-06 17:57:41 -07:00
Michael Vines 7027d56064 Resolve nightly-2021-10-05 clippy complaints 2021-10-06 10:37:58 -07:00
steviez d8e4840b49
Follow up to blockstore test cleanup (#19656) (#20143) 2021-10-04 10:35:03 -05:00
Lijun Wang fe97cb2ddf
AccountsDb plugin framework (#20047) 
Summary of Changes

Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows

Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.

The code comprises 4 major parts:

accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
 2021-09-30 14:26:17 -07:00
dependabot[bot] 9393654af0
chore: bump libc from 0.2.102 to 0.2.103 (#20280) 
* chore: bump libc from 0.2.102 to 0.2.103

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.102 to 0.2.103.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.102...0.2.103)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <anatoly+githubjenkins@solana.io>
 2021-09-28 07:50:46 +00:00
Justin Starry c71fab6cb3
Add `delete` subcommand to `ledger-tool bigtable` (#19931) 
* Add `delete` subcommand to `ledger-tool bigtable` command

* feedback
 2021-09-16 23:37:45 +00:00
dependabot[bot] ea15c0b40b
chore: bump libc from 0.2.101 to 0.2.102 (#19924) 
* chore: bump libc from 0.2.101 to 0.2.102

Bumps [libc](https://github.com/rust-lang/libc) from 0.2.101 to 0.2.102.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.101...0.2.102)

---
updated-dependencies:
- dependency-name: libc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-09-16 09:45:01 -06:00
steviez 4fdd70c931
Properly clean temp files in Blockstore::create_new_ledger() (#19589) 
Use TempDir::path() instead of TempDir::into_path(); into_path() returns
the underlying PathBuf and voids the promise to delete the directory
when TempDir goes out of scope.
 2021-09-13 23:13:56 -04:00
Brooks Prumo 97fc64dd0c
Call exhaustively_free_unused_resource() instead (#19847) 2021-09-13 18:08:32 -05:00
Brooks Prumo ef749a2506
Flush accounts cache before calling clean_accounts() (#19839) 2021-09-13 14:56:13 -05:00
Brooks Prumo 3f6eb96d6e
Call `Bank::update_accounts_hash()` before taking bank snapshot (#19765) 
This is a bugfix.

When `load_frozen_forks()` called `snapshot_bank()`, the accounts hash
was not computed.  So then when a snapshot archive was eventually
created, its hash would be all 1s, which is clearly wrong.  The fix is
to update the bank's accounts hash before taking the bank snapshot.
 2021-09-13 08:27:50 -05:00
Brooks Prumo 62c8bcf565
Add default() to SnapshotConfig (#19776) 2021-09-12 13:44:27 -05:00
dependabot[bot] 857fce13f9
chore: bump sha2 from 0.9.6 to 0.9.8 (#19760) 
* chore: bump sha2 from 0.9.6 to 0.9.8

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.6 to 0.9.8.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.6...sha2-v0.9.8)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-09-11 16:22:40 -06:00
Giles Cope c3a610374a unused deps 2021-09-11 14:33:53 -07:00
steviez 2dee098b91
Blockstore test cleanup (#19656) 
* Use auto-deleting macro for ledger path
 2021-09-10 07:33:08 -05:00
Jeff Washington (jwash) 456bf15012
AccountsIndexConfig -> AccountsDbConfig (#19687) 2021-09-08 04:30:38 +00:00
Jeff Washington (jwash) d3f938f0cf
Remove Copy from AccountsIndexConfig. Not all types will support it (#19686) 2021-09-07 20:09:40 -05:00
Brooks Prumo 4a5f83d3a7
Add fn to check when to take snapshots (#19682) 2021-09-07 23:26:35 +00:00
Brooks Prumo a0552e5b46
Make startup aware of Incremental Snapshots (#19600) 2021-09-07 20:43:43 +00:00
Tyera Eulberg 5fa3e5744c
Populate memo in blockstore signatures-for-address (#19515) 
* Add TransactionMemos column family

* Traitify extract_memos

* Write TransactionMemos in TransactionStatusService

* Populate memos from column

* Dedupe and add unit test
 2021-09-03 00:28:52 -06:00
Jack May 170927b7c4
nit: Move builtins types to builtins file (#19597) 2021-09-02 21:29:11 -07:00
Brooks Prumo e9374d32a3
Revert "Make startup aware of Incremental Snapshots (#19550)" (#19599) 
This reverts commit d45ced0a5d.
 2021-09-02 19:14:41 -05:00
Brooks Prumo d45ced0a5d
Make startup aware of Incremental Snapshots (#19550) 2021-09-02 19:05:15 -05:00
Brooks Prumo 1d2f0f6641
Simplify process_blockstore tests (#19553) 
Some of the blockstore_processor tests weren't using
`test_process_blockstore()`, but could.  I updated those tests, and also
changed to using `..` for ignoring, instead of `_`, since I'll be adding
another return value here shortly (thus reducing the need to change
these again).
 2021-09-01 20:13:52 +00:00
behzad nouri 6d9818b8e4
skips retransmit for shreds with unknown slot leader (#19472) 
Shreds' signatures should be verified before they reach retransmit
stage, and if the leader is unknown they should fail signature check.
Therefore retransmit-stage can as well expect to know who the slot
leader is and otherwise just skip the shred.

Blockstore checking signature of recovered shreds before sending them to
retransmit stage:
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/blockstore.rs#L884-L930

Shred signature verifier:
https://github.com/solana-labs/solana/blob/4305d4b7b/core/src/sigverify_shreds.rs#L41-L57
https://github.com/solana-labs/solana/blob/4305d4b7b/ledger/src/sigverify_shreds.rs#L105
 2021-09-01 15:44:26 +00:00
dependabot[bot] 8e4360cf9c
chore: bump futures from 0.3.16 to 0.3.17 (#19513) 
* chore: bump futures from 0.3.16 to 0.3.17

Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
 2021-08-30 22:12:38 -06:00
dependabot[bot] dc3d7ad2bd
chore: bump sha2 from 0.9.5 to 0.9.6 (#19511) 
* chore: bump sha2 from 0.9.5 to 0.9.6

Bumps [sha2](https://github.com/RustCrypto/hashes) from 0.9.5 to 0.9.6.
- [Release notes](https://github.com/RustCrypto/hashes/releases)
- [Commits](https://github.com/RustCrypto/hashes/compare/sha2-v0.9.5...sha2-v0.9.6)

---
updated-dependencies:
- dependency-name: sha2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 19:58:26 -06:00
dependabot[bot] 6a53ec28e0
chore: bump serde from 1.0.129 to 1.0.130 (#19497) 
* chore: bump serde from 1.0.129 to 1.0.130

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.129 to 1.0.130.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.129...v1.0.130)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 18:19:34 +00:00
dependabot[bot] fbffdf9f3d
chore: bump futures-util from 0.3.16 to 0.3.17 (#19498) 
* chore: bump futures-util from 0.3.16 to 0.3.17

Bumps [futures-util](https://github.com/rust-lang/futures-rs) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.16...0.3.17)

---
updated-dependencies:
- dependency-name: futures-util
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [auto-commit] Update all Cargo lock files

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
 2021-08-30 17:45:47 +00:00