Commit Graph

396 Commits

Author SHA1 Message Date
Pankaj Garg dc6c34da5d
Fast-track vote signature verification and processing (#3695) 2019-04-09 12:57:12 -07:00
carllin d4eebcc2aa
Check for frozen in confirm_forks (#3678) 2019-04-09 11:45:38 -07:00
carllin 483cc2fa4e
Support old repair strategy for reparing slots in a range for supporting replicators (#3665) 2019-04-08 12:46:23 -07:00
Jack May 44b391096d
Configurable local cluster native processors (#3676) 2019-04-08 11:15:58 -07:00
carllin 444e87f888
Fix metric (#3664) 2019-04-06 21:57:01 -07:00
carllin 20aa4434e2
Fix repair (#3581)
Add DetachedHeads repair protocol

Add DetachedHeads repair test

Repair starting from root
2019-04-06 19:41:22 -07:00
Greg Fitzgerald 878a842611
Move append_vec bench to the crate with append_vec (#3650)
* Move append_vec bench to the crate with append_vec

* Use black_box to tell the compiler not to optimize away test data

```
pub fn black_box<T>(dummy: T) -> T {
    unsafe {
        let ret = std::ptr::read_volatile(&dummy);
        std::mem::forget(dummy);
        ret
    }
}
```

* Revert "Use black_box to tell the compiler not to optimize away test data"

This reverts commit 5610b8ee957f1d8bf6e270e392859e8b23b1e472.

* Use black_box to tell the compiler not to optimize away test data

* Create bench directories
2019-04-06 07:18:56 -06:00
Tyera Eulberg 68e21911eb Remove redundant transfer_signed 2019-04-05 22:04:32 -07:00
Tyera Eulberg 95cc36af96 Impl SyncClient and AsyncClient for ThinClient 2019-04-05 22:04:32 -07:00
Tyera Eulberg f349c1f0dc Get everything off RpcSignatureStatus 2019-04-05 22:09:29 -06:00
Tyera Eulberg 90c1300bb6 Plumb TransactionError through Rpc 2019-04-05 22:09:29 -06:00
Sagar Dhawan f5f4434e0a Remove unnecessary lock in sigverify 2019-04-05 16:57:45 -07:00
Sagar Dhawan adcda3c715 Remove airdrop dependency from replicators 2019-04-05 16:11:39 -07:00
Rob Walker a5b5248a09
move vote_accounts up (#3647) 2019-04-05 14:23:00 -07:00
Tyera Eulberg ef5df6f3fe Add server specification 2019-04-05 11:44:57 -07:00
Tyera Eulberg 2f90f9fbd4 Version all jsonrpc crates, in core too 2019-04-05 11:44:57 -07:00
Greg Fitzgerald 46e6911ec1 Add get_signature_status() to SyncClient
And move bank::Result to transaction module.
2019-04-05 10:22:05 -07:00
carllin 4ea19b90a4
Fix update_ancestor_stakes in locktower (#3631)
* Fix update_ancestor_stakes in locktower

* Add test for vote threshold
2019-04-05 03:05:31 -07:00
Greg Fitzgerald 35298e01a8 Remove Instruction wrapper structs and name functions after enum fields 2019-04-03 13:34:27 -07:00
Greg Fitzgerald 867f6f107b Rename SystemInstruction::Move to SystemInstruction::Transfer 2019-04-03 08:35:57 -06:00
Greg Fitzgerald 43bb813cbe Rename 'new_account' to 'new_user_account'
And 'new_program_account' to 'new_account'
2019-04-02 21:24:42 -06:00
Greg Fitzgerald 978ff87b76 Fix potential storage bug
The previous code was assuming the instruction index and the
program_id index were the same. That's always true for
single-instruction transactions, but not for multiples.
2019-04-02 19:00:35 -06:00
Greg Fitzgerald 4c0bc1fd88 Add program_ids() methods
Added CompiledInstruction::program_id() so that we don't need to pass
around instruction indexes just for Message::program_id().

Also added Message.program_ids() that returns a slice so that we
can move those pubkeys into Message::account_keys.
2019-04-02 19:00:35 -06:00
Greg Fitzgerald 20189c5d45 Bump hashbrown to 0.2.0 2019-04-02 16:37:21 -06:00
Mark E. Sinclair d90b8c331d
Refactor blocktree storage abstraction (#3588) 2019-04-02 16:58:07 -05:00
Sagar Dhawan cd634801a2 Re-enable test but remove replicators from config 2019-04-02 10:38:30 -07:00
Sagar Dhawan 5f10a87dec Ignore Flaky Local Cluster test 2019-04-02 10:56:29 -06:00
Greg Fitzgerald fa1c1e3734 Rename native programs to native instruction processors 2019-04-02 10:36:19 -06:00
carllin d228b6467c
Implement finalizer so that all locked accounts are dropped (#3585)
* Implement finalizer so that all locked accounts are dropped when finalizer goes out of scope

* Add test for tx error with lock conflict

* Fix double unlock from destructor running after a call to unlock
2019-04-02 03:55:42 -07:00
Tyera Eulberg 6ff2a0a75e Rework discover to handle additional parameters, and be unit-testable 2019-04-01 23:05:25 -06:00
Greg Fitzgerald fcef54d062 Add a constructor to generate random pubkeys 2019-03-31 16:23:18 -06:00
carllin 8660c3581e
Add squashing metrics (#3573) 2019-03-29 21:21:59 -07:00
carllin f886b3b12b
Fix resetting PohRecorder to wrong bank (#3553)
* Check whether future slot already has transmission
2019-03-29 20:00:36 -07:00
Greg Fitzgerald 5646daa820 Delete lots of fee parameters
So many zeros!
2019-03-29 19:21:51 -06:00
Greg Fitzgerald 7896e8288d Replace Transaction::fee with a FeeCalculator 2019-03-29 19:21:51 -06:00
carllin 9369ea86ea
Track detached slots in blocktree (#3536)
* Add contains_all_parents flag to SlotMeta to prep for tracking detached heads

* Add new DetachedHeads column family

* Remove has_complete_parents

* Fix test
2019-03-29 16:07:24 -07:00
carllin dee5ede16d
Get rid of unnecessary frozen banks (#3572) 2019-03-29 16:06:48 -07:00
Stephen Akridge 0887832b00 Early exit if buffered packets is empty 2019-03-29 13:40:07 -07:00
Greg Fitzgerald 8e04fadb05 Cleanup magic numbers
Rename `num_signatures` to `num_required_signatures` to
disambiguate it from `tx.signatures.len()`.
2019-03-29 13:03:29 -07:00
Greg Fitzgerald 31f8b6d352 Integrate Message into Transaction 2019-03-29 13:03:29 -07:00
Greg Fitzgerald 98d60e6124 Expose a method for getting the Message from a Transaction
This currently constructs the message, but when message
is integrated, it can return a `&Message`.
2019-03-29 13:03:29 -07:00
Tyera Eulberg f482c9ab61 Functionalize tx serialization; make testing more explicit 2019-03-29 11:31:46 -06:00
Tyera Eulberg 75dcd97f5f Update test to deserialize txs 2019-03-29 11:31:46 -06:00
Tyera Eulberg 4776dc36ab Map entry txs to serialized txs in blockstream 2019-03-29 11:31:46 -06:00
Greg Fitzgerald 753d0dcabe Fix the cuda build
And add a test to check the condition that the cuda tests are
exercising.
2019-03-29 08:25:56 -06:00
Greg Fitzgerald b708998d9d Fix chacha build 2019-03-29 08:25:56 -06:00
Greg Fitzgerald c4bc710d3a Use Serde's `with` attribute to shorten length encodings in Transaction 2019-03-29 08:25:56 -06:00
Sathish Ambley 857dc2ba47 Remove custom serialization 2019-03-29 08:25:56 -06:00
Greg Fitzgerald 7a81f327ce Add sigverify tests 2019-03-28 19:42:11 -06:00
Greg Fitzgerald 2ab50cbae8 Move untested code out of SDK
verify_signature() was only used in a test that was testing
binary layout. It only worked because the test transaction only
had one signature.

from() was only used by verify_signature() and that's something
we'd typically called `pubkey()`.

hash() didn't return the hash of the Transaction, as you might
guess. It's only used for PoH, so move it into Entry.
2019-03-28 14:24:59 -06:00
Stephen Akridge 0482f153d0 Lower a bunch of debug
Can't afford to be printing on every transaction error, it will slow
the system down.
2019-03-28 12:24:47 -07:00
Mark E. Sinclair 50b0a5ae83
Blocktree+Erasure tests of basic erasure functionality (#3535)
* Remove WindowSlot; add Blocktree based tests to erasure
2019-03-28 01:55:51 -05:00
Sagar Dhawan 7d0ff8e713
Re-enable Replicator test (#3534) 2019-03-27 17:21:49 -07:00
Sagar Dhawan e8cc566b2b
Storage Account setup for replicators and validators (#3516)
* Setup Storage Accounts for replicators

* Setup Storage Accounts for validators

* Add Replicator Info to Local Cluster and Add test
2019-03-27 15:54:09 -07:00
Anatoly Yakovenko e45f7afd85 use the right id for delegate id 2019-03-27 15:04:09 -07:00
Sagar Dhawan 36ea088387
Fix Storage Stage not receiving entries when node is leader (#3528) 2019-03-27 13:10:33 -07:00
Anatoly Yakovenko 0346b9cb5c hang out on progress until fork is confirmed 2019-03-27 08:41:41 -07:00
Anatoly Yakovenko 6bfe497ab5 remove leader confirmaiton 2019-03-27 08:41:41 -07:00
Anatoly Yakovenko 6956bf635e validator confirmaiton 2019-03-27 08:41:41 -07:00
Anatoly Yakovenko e27d6d0988 validator confirmation 2019-03-27 08:41:41 -07:00
Greg Fitzgerald cecdb7061e Remove blockhash parameter from Bank::transfer
That parameter is an artifact from the Loom days, when I thought
Bank should implement the same interace as ThinClient.
2019-03-27 08:51:10 -06:00
Mark 16ff4ac1a8 Simplify storage interface in blocktree (#3522) 2019-03-27 01:36:39 -05:00
Greg Fitzgerald 8c69c40834 Make space for a new Transaction::new 2019-03-26 20:06:05 -06:00
Anatoly Yakovenko 9d73fbb84a also check the delegate_id 2019-03-26 12:03:22 -07:00
anatoly yakovenko 420cbc45cd
Record the current nodes locktower votes from the bank (#3502)
* observed_locktower_stats

* fixup! observed_locktower_stats
2019-03-26 11:06:31 -07:00
carllin 615472b52c
Initailize locktower with heaviest bank (#3489) 2019-03-25 20:00:11 -07:00
Greg Fitzgerald 4bc3f70150 Boot VoteTransaction 2019-03-25 17:11:57 -07:00
Anatoly Yakovenko 59b4f40f4e fixup! fixup! keep track of locktower slots and stakes 2019-03-25 16:05:28 -07:00
Anatoly Yakovenko 93c57934cb fixup! keep track of locktower slots and stakes 2019-03-25 16:05:28 -07:00
Anatoly Yakovenko e8e1d6b8ce keep track of locktower slots and stakes 2019-03-25 16:05:28 -07:00
Stephen Akridge 4916cd8da5 bench-tps in a cargo test 2019-03-25 15:05:56 -07:00
anatoly yakovenko 51004881f8
filter out banks that have an older epoch (#3472) 2019-03-25 11:09:39 -07:00
Greg Fitzgerald aefa9891c0 Delete unused code 2019-03-24 21:44:04 -07:00
Greg Fitzgerald 33972ef89e Boot BudgetTransaction 2019-03-24 14:52:06 -06:00
Greg Fitzgerald c49e84c75b Boot StorageTransaction 2019-03-24 13:51:02 -07:00
Greg Fitzgerald 8ff1987d2d Reorg Storage program to look more like the others 2019-03-24 13:51:02 -07:00
Greg Fitzgerald acedf4ca5a Move Instruction into its own module 2019-03-23 20:31:55 -07:00
carllin 68c35bfde6
Restart node test (#3459) (#3465)
* Restart node test (#3459)

* Add test to local_cluster for restarting a node

* fix so that we don't hit end of epoch - leader not found before trying to transfer

* Do not look for confirmations, b/c nobody is voting on empty transmissions in this single node test
2019-03-23 19:19:55 -07:00
Sathish e1a3708844
Ensure accounts are unlocked (#3458) 2019-03-23 13:30:56 -07:00
Pankaj Garg 46ecac3310
Update leader slot in poh recorder if we skipped it (#3452)
* reset poh recorder with the original start slot
2019-03-23 13:07:09 -07:00
Michael Vines f09b8d3921 Demote log level 2019-03-22 17:00:09 -07:00
anatoly yakovenko 52f6c33ff9
Make sure banking stage is recording with the same bank that it read (#3447)
* make sure banking stage is recording with the same bank that it read with
2019-03-22 14:17:39 -07:00
Sathish 06b0c98c75
Remove accounts when the fork is removed (#3384)
* Fix test

* Cleanup accounts when the fork is removed

* Update test to check for deleted accounts
2019-03-21 17:36:10 -07:00
Greg Fitzgerald d0761f57e8 Add _program suffix to directories of crates with _program suffix 2019-03-21 16:24:06 -06:00
Stephen Akridge 412ebfcaf2 ReplayStage::new is too long
break into more functions
2019-03-21 14:08:11 -07:00
Greg Fitzgerald 07d55d0092 Downgrade 'No next leader found' to warning 2019-03-21 11:18:49 -06:00
anatoly yakovenko 148e08a8a5
Enable cluster tests (#3372)
* Cluster tests

* stable!

* fixup! stable!

* fixup! fixup! stable!

* fixup! fixup! fixup! stable!

* fixup! fixup! fixup! fixup! stable!

* fixed space

* add getNumBlocksSinceSignatureConfirmation entry for the json rpc docs

* Check in upcoming epochs for potential leadership slots in next_leader_slot()
2019-03-21 07:43:21 -07:00
Pankaj Garg 7c8a4bf6a4 use ticks per slot to check if the current tick is in the leader slot 2019-03-20 16:55:01 -07:00
Pankaj Garg 71314d79a7 address review comments 2019-03-20 16:55:01 -07:00
Pankaj Garg d7ff6645a9 change pubkey to ref 2019-03-20 16:55:01 -07:00
Pankaj Garg 1824e09d0a find next leader slot before resetting working bank in Poh recorder 2019-03-20 16:55:01 -07:00
Pankaj Garg 205907d3d7 Check if poh recorder has over stepped the leader slot 2019-03-20 16:55:01 -07:00
Stephen Akridge 4fb0782892 Rename blocktree SlotMeta::is_rooted to is_connected
is_rooted is now is_connected and (still) indicates the set of connected
completed slots. 'rooted' slot terminology is used for a different
meaning in bank_forks and replay_stage.
2019-03-20 14:43:39 -07:00
Stephen Akridge 2694654a98 Change fixed 8050 port to one from bind_in_range. 2019-03-20 14:17:21 -07:00
Stephen Akridge 4126461f87 Fix dupe port on cluster_info
and remove unintended grow file
2019-03-20 14:17:21 -07:00
Rob Walker df9fd2bc0b
stop copying Blooms (#3379)
* stop copying Blooms

* fixup

* clippy
2019-03-20 11:06:39 -07:00
Michael Vines 0dc364c17a Relocate transaction reference verification to join the other validity checks 2019-03-20 07:46:01 -07:00
Sagar Dhawan 61f950a60c Sign Gossip Vote Messages 2019-03-19 19:56:17 -07:00
Anatoly Yakovenko 61af87972e allow empty ancestors 2019-03-19 17:51:01 -07:00
carllin fe9e771b9b
Clear progress map on squash (#3377) 2019-03-19 17:30:36 -07:00
Anatoly Yakovenko 7246d72f03 fix is_locked_out logic 2019-03-19 16:21:46 -07:00
Stephen Akridge 682b1b89b3 Adjust for vector of entries in blobs. 2019-03-19 13:49:48 -07:00
Stephen Akridge f1802e592a Review comments: node creation functions for replicators
And rework download loop.
2019-03-19 13:49:48 -07:00
Stephen Akridge ee58c1f960 Add test for replicator ledger download
Add an interface to query the storage slot a
  replicator is holding on storage_addr port.
Fix logic to poll blocktree for all slots
  replicated being filled.
Add test logic to ask replicator what slot it
  is replicating and then download an entry in
  the slot.
2019-03-19 13:49:48 -07:00
Stephen Akridge 07f4dd385d Cleanup replicator sockets
Add optional UdpSocket for storage interface.
Add new_localhost_replicator to create a new replicator local node.
2019-03-19 13:49:48 -07:00
Pankaj Garg 1be7ee51be Fix potential crash in banking stage 2019-03-19 12:06:42 -07:00
Pankaj Garg 56fcc93ef5
Schedule node for consecutive slots as leader (#3353)
* Also tweak epoch and slot duration

* new test for leader schedule
2019-03-19 06:36:45 -07:00
Rob Walker c70412d7bb
move core tests to core (#3355)
* move core tests to core

* remove window

* fix up flaky tests

* test_entryfication needs a singly-threaded banking_stage

* move core benches to core

* remove unnecessary dependencies

* remove core as a member for now, test it like runtime

* stop running tests twice

* remove duplicate runs of tests in perf
2019-03-18 22:08:21 -07:00
Carl 5e21268ca0 PR comments 2019-03-18 20:46:11 -07:00
Carl b38e3bef01 Modify bank_forks to support squashing/filtering new root and also don't remove parents from bank_forks when inserting, otherwise we lose potential fork points when querying blocktree for child slots 2019-03-18 20:46:11 -07:00
Stephen Akridge ef111dcbe1 Decendent is not a word 2019-03-18 15:58:27 -07:00
anatoly yakovenko 211c81f2a2
bank fork rpc (#3351) 2019-03-18 14:18:43 -07:00
Pankaj Garg efc39ffdde
Report how many grace ticks were afforded to previous leader (#3350) 2019-03-18 13:24:07 -07:00
anatoly yakovenko 61a4b998fa
Implement locktower voting (#3251)
* locktower components and tests

* integrate locktower into replay stage

* track locktower duration

* make sure threshold is checked after simulating the vote

* check vote lockouts using the VoteState program

* duplicate vote test

* epoch stakes

* disable impossible to verify tests
2019-03-18 12:12:33 -07:00
Stephen Akridge cedff2fca1 Cleanup sockets test 2019-03-18 11:56:18 -07:00
Greg Fitzgerald 8d032aba9d Merge InstructionError and ProgramError
From the user's perspective, it's just an instruction error.
For program-specific errors, we still have
InstructionError::CustomError.
2019-03-18 10:39:20 -06:00
carllin 60437a8dcb
Multiple entries per blob (#3337)
* Pack multiple entries into blob

* fix tests

* Add test for deserializing multi-entry blobs in blocktree

* more test fixes
2019-03-17 18:48:23 -07:00
Michael Vines c498775a3d Move generic rpc_client functions from wallet/ to client/ 2019-03-17 01:34:58 -07:00
Michael Vines 9632136cda Clean up stray retry_get_balance() function 2019-03-16 23:56:35 -07:00
Michael Vines 73ceaf07b1 client/: move RpcClient from rpc_request.rs to rpc_client.rs 2019-03-16 23:27:23 -07:00
Michael Vines 7b314f47f7 Factor RPC request mechanism out of RpcClient into *RpcClientRequest 2019-03-16 23:27:23 -07:00
Michael Vines 23337e08eb client/: Merge client.rs into thin_client.rs 2019-03-16 22:48:26 -07:00
Michael Vines e2c24481e4 wallet/ now only dev-depends on core/ 2019-03-16 21:40:39 -07:00
Michael Vines ad252fe4c5 Remove unnecessary Option from get_account_data 2019-03-16 11:32:01 -07:00
Michael Vines 4b04bc8612 Move thin_client RPC requests into rpc_request; de-mut thin_client 2019-03-16 11:32:01 -07:00
Michael Vines bcc34b906c Relieve the caller of having to care about the rpc request id 2019-03-16 11:32:01 -07:00
Michael Vines c2b1010f18 Clarify url vs addr 2019-03-16 11:32:01 -07:00
Michael Vines ad12b0efce Bump kvstore version to 0.13.0 to match all other solana crates (#3334) 2019-03-15 19:05:24 -05:00
Sagar Dhawan 00f005af25 Fix leader rotation counter 2019-03-15 17:01:18 -07:00
Mark 656fb173f9
Extract kvstore into separate crate (#3327)
* extract kvstore into new crate

* add kvstore crate to CI publishing list
2019-03-15 18:42:47 -05:00
Pankaj Garg 971da7325d Reduce log level for periodic debug messages 2019-03-15 15:41:26 -07:00
Greg Fitzgerald e091aa87ea More precise constructor names 2019-03-15 14:46:44 -06:00
Greg Fitzgerald 968022a1b0 Instruction name swap
* Instruction -> GenericInstruction
* Instruction<u8, u8> -> CompiledInstruction
* Instruction<Pubkey, (Pubkey, bool)> -> Instruction
2019-03-15 14:46:44 -06:00
Pankaj Garg 66fb1bbb2e
Give last leader some grace ticks to catch up (#3299)
* Wait for last leader for some ticks

* New tests and fixed existing tests
2019-03-15 13:22:16 -07:00
Mark fa3e1fa7c9
Add error correction to write-log (#3323) 2019-03-15 15:04:34 -05:00
carllin 36763d0802
Cleanup entry.rs packing code (#3303) 2019-03-15 12:48:32 -07:00
Sagar Dhawan be5f800390
Use the Mining Proof's Signature as storage keys (#3321) 2019-03-15 11:44:10 -07:00
Mark ca69b7b75b
Add CRC Reader and Writer I/O wrappers (#3322)
* add CRC Reader and Writer I/O wrappers

* typo fix and variable rename
2019-03-15 13:17:49 -05:00
Mark a15927f8d0
make KvStore Send+Sync (#3317) 2019-03-15 13:01:34 -05:00
Mark be1511a7ff
delete accidental file (#3316) 2019-03-15 11:28:08 -05:00
Greg Fitzgerald d72cac6e97 Fix chacha test 2019-03-15 09:06:54 -06:00
Greg Fitzgerald 7b4568b9bf Migrate to sign_checked() 2019-03-15 09:06:54 -06:00
Michael Vines 57606c6bf8
Bump log level for better CI logs 2019-03-15 07:48:23 -07:00
Rob Walker 5741400713
add support for finding the next slot a node will be leader (#3298) 2019-03-14 16:06:56 -07:00
Rob Walker 9f02a8d3d0
remove ticks_per_slot from blocktree (#3297) 2019-03-14 15:18:37 -07:00
Stephen Akridge c208f4dbb5 Add option of replicators to local cluster test 2019-03-14 13:55:11 -07:00
Greg Fitzgerald c1eec0290e
Rename userdata to data (#3282)
* Rename userdata to data

Instead of saying "userdata", which is ambiguous and imprecise,
say "instruction data" or "account data".

Also, add `ProgramError::InvalidInstructionData`

Fixes #2761
2019-03-14 10:48:27 -06:00
dependabot[bot] 77a7ffe543 Bump hex-literal from 0.1.3 to 0.1.4
Bumps [hex-literal](https://github.com/RustCrypto/utils) from 0.1.3 to 0.1.4.
- [Release notes](https://github.com/RustCrypto/utils/releases)
- [Commits](https://github.com/RustCrypto/utils/compare/hex-literal-v0.1.3...hex-literal-v0.1.4)

Signed-off-by: dependabot[bot] <support@dependabot.com>
2019-03-14 07:21:32 -06:00
Greg Fitzgerald e58220282a Move TransactionError into the SDK 2019-03-13 21:26:57 -06:00
Greg Fitzgerald 4ca4038d54 Rename BankError to TransactionError 2019-03-13 21:26:57 -06:00