* upgrade spl-token-2022 to 0.6.0
* Add spl_token_2022-0.6.0.so to program-test
* cargo lock
* update account decoder tests to use Pubkey::new_from_array instead of Pubkey::new
* update rpc tests to use Pubkey::new_from_array instead of Pubkey::new
* update token-2022 .so file
* cargo lock
* increase the threshold for allowable rebuilds in ci
* update spl ATA to v1.1.3
* decrement back the threshold for allowable rebuilds
* cargo lock
* increment ci rebuild threshold
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
tpu-client/tpu_connection_cache is refactored out the module and moved to connection-cache/connection_cache and the logic in client/connection_cache is consolidated to connection-cache/connection_cache as well. client/connection_cache only has a thin wrapper which forward calls to connection-cache/connection_cache and deal with constructions of quic/udp connection cache for clients using them both.2.
The TpuConnection is refactored to ClientConnection to make it generic and functions renamed to be proper for other workflows. eg. tpu_addr -> server_addr, send_transaction --> send_data and etc...
The enum dispatch is removed so that we can make the bulk of code of quic and udp agnostic of each other. The client is possible to load quic or udp only into its runtime.
The generic type parameter in the tpu-client/tpu_connection_cache is removed in order to create both quic and udp connection cache and use the object to send transactions with multiple branching when sending data. The generic type parameters and associated types are dropped in other types in order to make the trait "object safe" for this purpose.
I have annotated the code explaining the reasoning and the refactoring source -> destination.
There is no functional changes
bench-tps has been performed for rpc-client, thin-client and tpu-client. And it is found the performance number largely match the ones before the refactoring.
* introduce workspace.package
* introduce workspace.dependencies
* read version from root cargo.toml
* pass check when version = { workspace = true }
* don't bump version when version = { workspace = true }
* including workspace Cargo.toml when bump version
* programs/sbf use workspace inheritance
* fix increasing cargo version ignore program/sbf/Cargo.toml
* Moves AlignedMemory from solana_rbpf into a new crate: memory-managment.
* Makes cargo clippy happy.
* deny clippy::integer_arithmetic
* Makes write_unchecked() take a generic parameter.
* Restricts AlignedMemory::write_unchecked() to Pod types.
* Adds DynamicLayoutArray.
* Moves is_memory_aligned() into crate root.
* Add new empty crates and tpu_connection_cache module
* Add BaseTpuConnection trait and impl for udp and quic
* Add ConnectionPool trait and impl for udp and quic (quic-client doesn't build)
* Add a couple quic-specific apis to QuicConfig (from ConnectionCache)
* Re-export quic- and udp-client modules to prep for move
* Move ConnectionCacheStats to new module
* Move consts
* Duplicate ConnectionCache into tpu_connection_cache (doesn't build)
* Move methods to QuicConfig and remove unneeded methods (doesn't build)
* Genericize new ConnectionCache
* Rename new struct
* Copy unit tests (tests don't build)
* Fixup unit tests
* Move quic-specific test to quic-client crate
* DRY by using ConnectionPool method
* Prevent evicting pool about to be used
* Allow extending upgradeable program data account length
* Add is_writable check
* Fix cargo version
* System program fix, comment, and test
* Switch to u32 for serialized bytes value in ix
Problem
We need a test that stress-tests the network using large transactions, including sending multiple large transactions that reference independent sets of accounts, so they can be executed in parallel.
Summary of Changes
Adds such a test. Also adds a version of the tuner program that runs for a configurable number of iterations.
* Move test-validator to own module to reduce core dependencies
* Fix a few TestValidator paths
* Use solana_test_validator crate for solana_test_validator bin
* Move client int tests to separate crate
Co-authored-by: Tyera Eulberg <tyera@solana.com>
Summary of Changes
Create a plugin mechanism in the accounts update path so that accounts data can be streamed out to external data stores (be it Kafka or Postgres). The plugin mechanism allows
Data stores of connection strings/credentials to be configured,
Accounts with patterns to be streamed
PostgreSQL implementation of the streaming for different destination stores to be plugged in.
The code comprises 4 major parts:
accountsdb-plugin-intf: defines the plugin interface which concrete plugin should implement.
accountsdb-plugin-manager: manages the load/unload of plugins and provide interfaces which the validator can notify of accounts update to plugins.
accountsdb-plugin-postgres: the concrete plugin implementation for PostgreSQL
The validator integrations: updated streamed right after snapshot restore and after account update from transaction processing or other real updates.
The plugin is optionally loaded on demand by new validator CLI argument -- there is no impact if the plugin is not loaded.
* introduce bucket map
* rename BucketMap bits to num_buckets_pow2
* use u64::BITS
* Store the number of buckets in BucketMapConfig as a regular number
* remove redundant type aliases
* use Slot from sdk
* use Arc::clone() instead
* fixup erase drives
* rename num_buckets to max_buckets
* add doc to BucketMapConfig::new()
* add more documentation
* rename to DEFAULT_CAPACITY_POW2
* doc
* add more traits while we can
* rename capacity to capacity_pow2
* fix a naming for max_buckets_pow2
* remove unused/incorrect DataBucket::bytes
* rework benches a bit
* fixup bench docs
* rename create_bucket_capacity_pow2 to bucket_capacity_when_created_pow2
* rename BucketMapKeyValue to BucketItem
* rename to items_in_range
* remove values()
* remove addref and unref
* remove more addref and unref
* resurect addref and unref since tests use 'em for now
* rename to BucketStorage
* move stats in bucket_stats
* remove specializations (i don't think they are needed)
* move MaxSearch and RefCount into lib.rs
* move BucketItem to bucket_item.rs
* add doc
* keys no longer returns an option
* Revert "remove specializations (i don't think they are needed)"
This reverts commit b22f78e072cf0f7107851b08e58c2e3fead3f64d.
Co-authored-by: Brooks Prumo <brooks@solana.com>
Solang requires a method for verify ed25519 signatures. Add a new
builtin program at address Ed25519SigVerify111111111111111111111111111
which takes any number of ed25519 signature, public key, and message.
If any of the signatures fails to verify, an error is returned.
The changes for the web3.js package will go into another commit, since
the tests test against a released solana node. Adding web3.js ed25519
testing will break CI.
This is the 2nd installment for the AccountsDb replication.
Summary of Changes
The basic google protocol buffer protocol for replicating updated slots and accounts. tonic/tokio is used for transporting the messages.
The basic framework of the client and server for replicating slots and accounts -- the persisting of accounts in the replica-side will be done at the next PR -- right now -- the accounts are streamed to the replica-node and dumped. Replication for information about Bank is also not done in this PR -- to be addressed in the next PR to limit the change size.
Functionality used by both the client and server side are encapsulated in the replica-lib crate.
There is no impact to the existing validator by default.
Tests:
Observe the confirmed slots replicated to the replica-node.
Observe the accounts for the confirmed slot are received at the replica-node side.
This is the first installment of the AccountsDb replication. Summary of Changes:
The rpc-node for AccountsDb replication executable cherry-picked from Steven's branch
Auto figuring out the snapshot to download via gossip
The replica now can download the snapshot, initialize the bank and AccountsDb, start the JsonRpcService
Integration test of a validator paired with a replica.
* Create solana-poh crate
* Move BigTableUploadService to solana-ledger
* Add solana-rpc to workspace
* Move dependencies to solana-rpc
* Move remaining rpc modules to solana-rpc
* Single use statement solana-poh
* Single use statement solana-rpc
* Add rbpf-cli tool
* Replace explicit file type specification by required positional arg
- the type of file is determined automatically. If ELF magic found,
the program file is considered to be an executable ELF file, and
an assembly input file otherwise.
* Add a note about tool's limitations and input format description
* Add Blockstore protobuf cf type
* Add Rewards message to proto and make generated pub
* Convert Rewards cf to ProtobufColumn
* Add bench
* Adjust tags
* Move solana proto definitions and conversion methods to new crate
* Fix comment and make less pub
* Add account-decoder crate and use to decode vote and system (nonce) accounts
* Update docs
* Rename RpcAccount struct
* s/Rpc/Display
* Call it jsonParsed and update docs
* Revert "s/Rpc/Display"
This reverts commit 6e7149f503f560f1e9237981058ff05642bb7db5.
* s/Rpc/Ui
* Add tests
* Ui more things
* Comments
* Initial commit
* Execute transfers
* Refactor for testing
* Cleanup readme
* Rewrite
* Cleanup
* Cleanup
* Cleanup client
* Use a Null Client to move prints closer to where messages are sent
* Upgrade Solana
* Move core functionality into its own module
* Handle transaction errors
* Merge allocations
* Fixes
* Cleanup readme
* Fix markdown
* Add example input
* Add integration test - currently fails
* Add integration test
* Add metrics
* Use RpcClient in dry-run, just don't send messages
* More metrics
* Fix dry run with no keys
* Only require one approval if fee-payer is the sender keypair
* Fix bugs
* Don't create the transaction log if nothing to put into it;
otherwise the next innvocation won't add the header
* Apply previous transactions to allocations with matching recipients
* Bail out of any account already has a balance
* Polish
* Add new 'balances' command
* 9 decimal places
* Add missing file
* Better dry-run; keypair options now optional
* Change field name from 'bid' to 'accepted'
Also, tolerate precision change from 2 decimal places to 4
* Write to transaction log immediately
* Rename allocations_csv to bids_csv
So that we can bypass bids_csv with an allocations CSV file
* Upgrade Solana
* Remove faucet from integration test
* Cleaner integration test
Won't work until this lands and is released:
https://github.com/solana-labs/solana/pull/9717
* Update README
* Add TravicCI script to build and test (#1)
* Add distribute-stake command (#2)
* Distribute -> DistributeTokens (#3)
* Cache cargo deps (#4)
* Add docs (#5)
* Switch to latest Solana 1.1 release (#7)
* distribute -> distribute-tokens (#9)
* Switch from CSV to a pickledb database (#8)
* Switch from CSV to a pickledb database
* Allow PickleDb errors to bubble up
* Dedup
* Hoist db
* Add finalized field to TransactionInfo
* Don't allow RPC client to resign transactions
* Remove dead code
* Use transport::Result
* Record unconfirmed transaction
* Fix: separate stake account per allocation
* Catch transport errors
* Panic if we attempt to replay a transaction that hasn't been finalized
* Attempt to fix CI
PickleDb isn't calling flush() or close() after writing to files.
No issue on MacOS, but looks racy in CI.
* Revert "Attempt to fix CI"
This reverts commit 1632394f636c54402b3578120e8817dd1660e19b.
* Poll for signature before returning
* Add --sol-for-fees option for stake distributions
* Add --allocations-csv option (#14)
* Add allocations-csv option
* Add tests or GTFO
* Apply review feedback
* apply feedback
* Add read_allocations function
* Update arg_parser.rs
* Fix balances command (#17)
* Fix balances command
* Fix readme
* Add --force to transfer to non-empty accounts (#18)
* Add --no-wait (#16)
* Add ThinClient methods to implement --no-wait
* Plumb --no-wait through
No tests yet
* Check transaction status on startup
* Easier to test
* Wait until transaction is finalized before checking if it failed with an error
It's possible that a minority fork thinks it failed.
* Add unit tests
* Remove dead code and rustfmt
* Don't flush database to file if doing a dry-run
* Continue when transactions not yet finalized (#20)
If those transactions are dropped, the next run will execute them.
* Return the number of confirmations (#21)
* Add read_allocations() unit-test (#22)
Delete the copy-pasted top-level test.
Fixes#19
* Add a CSV printer (#23)
* Remove all the copypasta (#24)
* Move resolve_distribute_stake_args into its own function
* Add stake args to token args
* Unify option names
* Move Command::DistributeStake into DistributeTokens
* Remove process_distribute_stake
* Only unique signers
* Use sender keypair to fund new fee-payer accounts
* Unify distribute_tokens and distribute_stake
* Rename print-database command to transaction-log (#25)
* Send all transactions as quickly as possible, then wait (#26)
* Send all transactions as quickly as possible, then wait
* Exit when finalized or blockhashes have expired
* Don't need blockhash in the CSV output
* Better types
CSV library was choking on Pubkey as a type. PickleDb doesn't have that problem.
* Resend if blockhash has not expired
* Attempt to fix CI
* Move log to stderr
* Add constructor, tuck away client (#30)
* Add constructor, tuck away client
* Fix unwrap() caught by CI
* Fix optional option flagged as required
* Bunch of cleanup (#31)
* Remove untested --no-wait feature
* Make --transactions-db an option, not an arg
So that in the future, we can make it optional
* Remove more untested features
Too many false positives in that santity check. Use --dry-run
instead.
* Add dry-run mode to ThinClient
* Cleaner dry-run
* Make key parameters required
Just don't use them in --dry-run
* Add option to write the transaction log
--dry-run doesn't write to the database. Use this option if you
want a copy of the transaction log before the final run.
* Revert --transaction-log addition
Implement #27 first
* Fix CI
* Update readme
* Fix CI in copypasta
* Sort transaction log by finalized date (#33)
* Make --transaction-db option implicit (#34)
* Move db functionality into its own module (#35)
* Move db functionality into its own module
* Rename tokens module to commands
* Version bump
* Upgrade Solana
* Add solana-tokens to build
* Remove Cargo.lock
* Remove vscode file
* Remove TravisCI build script
* Install solana-tokens
Co-authored-by: Dan Albert <dan@solana.com>
* Separate client types into own crate, so ledger does not need it
Removes about 50 crates of dependency from ledger
* Drop Rpc name from transaction-status types
* Use solana-cli config keypair in solana-keygen
* s/infile/keypair for consistency across modules and more generality across access methods
* Move config into separate crate
ryleung-solana
behzad nouri
samkim-crypto
Alessandro Decina
dependabot[bot]
Yihau Chen
Lijun Wang
Dmitri Makarov
Alexander Meißner
Tyera Eulberg
Ryo Onodera
Will Hickey
Justin Starry
kirill lykov
Brooks Prumo
anatoly yakovenko
Michael Vines
sakridge
Jack May
Jeff Washington (jwash)
Sean Young
Tyera Eulberg
Alfonso Subiotto Marques
Dmitri Makarov
Solana Maintainers
Trent Nelson
Greg Fitzgerald
sakridge
Rob Walker