672fed04cb
Bump serde from 1.0.132 to 1.0.133 ( #22233 )
...
* Bump serde from 1.0.132 to 1.0.133
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.132 to 1.0.133.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.132...v1.0.133 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 22:19:39 +00:00
ecbfc70bfa
Bump serde_json from 1.0.73 to 1.0.74 ( #22231 )
...
* Bump serde_json from 1.0.73 to 1.0.74
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.73 to 1.0.74.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.73...v1.0.74 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2022-01-03 10:36:59 -07:00
f14928a970
Stream additional block metadata via plugin ( #22023 )
...
* Stream additional block metadata through plugin
blockhash, block_height, block_time, rewards are streamed
2021-12-29 15:12:01 -08:00
d6de4a2f4e
Fix transaction pk violation ( #22057 )
...
* Handle PK violation issue for transaction notification. The transaction might be replayed due to
validator restart.
2021-12-22 00:58:51 -08:00
8f547a6c98
chore: bump serde from 1.0.131 to 1.0.132 ( #21989 )
...
* chore: bump serde from 1.0.131 to 1.0.132
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.131 to 1.0.132.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.131...v1.0.132 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-18 07:45:20 +00:00
b610e5503e
Fixed a typo in the SQL statement ( #21872 )
...
* Fixed a typo in the SQL statement
* Fixed additional two errors in the postgres database objects
2021-12-14 08:26:59 -08:00
018b54dbd7
chore: bump serde_json from 1.0.72 to 1.0.73 ( #21856 )
...
* chore: bump serde_json from 1.0.72 to 1.0.73
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.72 to 1.0.73.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.72...v1.0.73 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-14 04:08:22 +00:00
90f41fd9b7
use cost model to limit new account creation ( #21369 )
...
* use cost model to limit new account creation
* handle every system instruction
* remove &
* simplify match
* simplify match
* add datapoint for account data size
* add postgres error handling
* handle accounts:unlock_accounts
2021-12-12 14:57:18 -06:00
6c108c8fc3
Migrate from address maps to address lookup tables ( #21634 )
...
* Migrate from address maps to address lookup tables
* update sanitize error
* cargo fmt
* update abi
2021-12-10 11:04:04 -05:00
d870f566ef
chore: bump serde from 1.0.130 to 1.0.131 ( #21758 )
...
* chore: bump serde from 1.0.130 to 1.0.131
Bumps [serde](https://github.com/serde-rs/serde ) from 1.0.130 to 1.0.131.
- [Release notes](https://github.com/serde-rs/serde/releases )
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.130...v1.0.131 )
---
updated-dependencies:
- dependency-name: serde
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-12-10 08:47:46 +00:00
f4ca87205f
If configured so, panic if there is an error saving transactions in the plugin ( #21602 )
2021-12-04 12:47:28 -08:00
d6f22433d0
Bump version to v1.10.0
2021-12-04 20:17:54 +00:00
b8837c04ec
Reformat imports to a consistent style for imports
...
rustfmt.toml configuration:
imports_granularity = "One"
group_imports = "One"
2021-12-03 09:19:13 -08:00
f5b0764795
Accountsdb plugin transaction part 4 -- postgres plugin implementations -- DB models ( #21407 )
...
AccountsDb plugin for transactions -- part 4 -- persisting to the Db.
1. DB models for transactions
2. Rust models for transactions
3. Transform from SDK models to rust db models
4. Unit tests
2021-12-01 09:23:26 -08:00
dd12d90eac
Upgrade to Rust 2021
2021-11-30 20:43:46 -08:00
a125e7b8c2
chore: bump serde_json from 1.0.71 to 1.0.72 ( #21426 )
...
* chore: bump serde_json from 1.0.71 to 1.0.72
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.71 to 1.0.72.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.71...v1.0.72 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-30 18:03:09 +00:00
c29838fce1
Accountsdb plugin transaction part 3: Transaction Notifier ( #21374 )
...
The TransactionNotifierInterface interface for notifying transactions.
Changes to transaction_status_service to notify the notifier of the transaction data.
Interface to query the plugin's interest in transaction data
2021-11-23 09:55:53 -08:00
4530a5980e
Accountsdb plugin transaction part 2: transaction selector ( #21331 )
...
Created the transaction_selector which allows the user to configure the transactions to stream:
all transactions
all vote transactions
transactions mentioning accounts
2021-11-19 14:26:22 -08:00
89102540b1
chore: bump serde_json from 1.0.70 to 1.0.71 ( #21338 )
...
* chore: bump serde_json from 1.0.70 to 1.0.71
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.70 to 1.0.71.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.70...v1.0.71 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-18 02:58:30 +00:00
89c45a57f8
Refactor slot status notification to decouple from accounts notifications ( #21308 )
...
Problem
Slot status can be used of in other scenarios in addition to account information such as transactions, blocks. The current implementation is too tightly coupled.
Summary of Changes
Decouple the slot status notification from accounts notification. Created a new slot status notification module.
2021-11-17 17:11:38 -08:00
55ea672041
chore: bump serde_json from 1.0.69 to 1.0.70 ( #21285 )
...
* chore: bump serde_json from 1.0.69 to 1.0.70
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.69 to 1.0.70.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.69...v1.0.70 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <you@example.com>
2021-11-15 15:27:47 -07:00
3c5f505d3e
chore: bump serde_json from 1.0.68 to 1.0.69 ( #21211 )
...
* chore: bump serde_json from 1.0.68 to 1.0.69
Bumps [serde_json](https://github.com/serde-rs/json ) from 1.0.68 to 1.0.69.
- [Release notes](https://github.com/serde-rs/json/releases )
- [Commits](https://github.com/serde-rs/json/compare/v1.0.68...v1.0.69 )
---
updated-dependencies:
- dependency-name: serde_json
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-11-09 11:04:40 -07:00
696501500f
Accountsdb plugin postgres improvement ( #21034 )
...
Summary of Changes
Added the reference postgresql.conf
Prepare slot update statement to reduce overhead in updating slot
Support custom connection string
Allow the plugin to panic on replication issues to ensure consistency
2021-10-30 20:18:11 -07:00
bbe3ce3db5
Accountsdb plugin write ordering ( #20948 )
...
Use the write_version in the Accounts's meta data so that account write with lower write_version would not overwrite the higher ones.
2021-10-25 14:07:56 -07:00
f4f281e7c7
chore: bump tokio-postgres from 0.7.3 to 0.7.4 ( #20930 )
...
Bumps [tokio-postgres](https://github.com/sfackler/rust-postgres ) from 0.7.3 to 0.7.4.
- [Release notes](https://github.com/sfackler/rust-postgres/releases )
- [Commits](https://github.com/sfackler/rust-postgres/compare/tokio-postgres-v0.7.3...tokio-postgres-v0.7.4 )
---
updated-dependencies:
- dependency-name: tokio-postgres
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-25 11:44:41 -06:00
f14365f4b4
Accountsdb plugin postgres -- bulk insertion at startup ( #20763 )
...
Use bulk insertion to Postgres at startup to reduce time taken for initial snapshot restore for postgres plugin. Avoid duplicate writes of accounts at startup. Doing account plugin notification and indexing in parallel.
Improved error handling for postgres plugin to show the real db issues for debug purpose
Added more metrics for postgres plugin.
Refactored plugin centric code out to a sub module from accounts_db and added unit tests
2021-10-24 12:43:33 -07:00
293a0d8797
Update to Rust v1.56
2021-10-23 08:21:20 +00:00
ad0a88f1f2
Support port number in postgres connection ( #20662 )
...
* Support port number in postgres connection
* Addressed a few comments from Trent
2021-10-14 11:55:10 -07:00
08e40bf32e
Accountsdb plugin metrics ( #20606 )
...
Added metrics for accountsdb plugin
Handle and log postgres db errors
Print account pubkeys nicely in logging
2021-10-13 08:36:39 -07:00
e9f459ef21
chore: bump thiserror from 1.0.29 to 1.0.30 ( #20583 )
...
* chore: bump thiserror from 1.0.29 to 1.0.30
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* [auto-commit] Update all Cargo lock files
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot-buildkite <dependabot-buildkite@noreply.solana.com>
2021-10-11 09:46:33 -06:00
d621994fee
Accountsdb stream plugin improvement ( #20419 )
...
Support using connection pooling and use multiple threads to do Postgres db operations. The performance is improved from 1500 RPS to 40,000 RPS measured during validator start.
Support multiple plugins at the same time.
2021-10-08 20:06:58 -07:00
767f740305
Bump version to 1.9.0
2021-10-06 17:57:41 -07:00
fb8a7cfa92
chore: bump postgres from 0.19.1 to 0.19.2 ( #20366 )
...
Bumps [postgres](https://github.com/sfackler/rust-postgres ) from 0.19.1 to 0.19.2.
- [Release notes](https://github.com/sfackler/rust-postgres/releases )
- [Commits](https://github.com/sfackler/rust-postgres/commits )
---
updated-dependencies:
- dependency-name: postgres
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-04 10:10:21 -06:00
fe97cb2ddf
AccountsDb plugin framework ( #20047 )
...
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
2021-09-30 14:26:17 -07:00
dependabot[bot]
Lijun Wang
Jeff Washington (jwash)
Justin Starry
Trent Nelson
Michael Vines