diff --git a/docs/spec/light/api.md b/docs/spec/light/api.md new file mode 100644 index 000000000..4dc2619a3 --- /dev/null +++ b/docs/spec/light/api.md @@ -0,0 +1,391 @@ +# Cosmos Hub (Gaia) LCD API + +This document describes the API that is exposed by the specific LCD implementation of the Cosmos +Hub (Gaia). Those APIs are exposed by a REST server and can easily be accessed over HTTP/WS(websocket) +connections. + +The complete API is comprised of the sub-APIs of different modules. The modules in the Cosmos Hub +(Gaia) API are: + +* ICS0 (TendermintAPI) +* ICS1 (KeyAPI) +* ICS20 (TokenAPI) +* ICS21 (StakingAPI) - not yet implemented +* ICS22 (GovernanceAPI) - not yet implemented + +Error messages my change and should be only used for display purposes. Error messages should not be +used for determining the error type. + +## ICS0 - TendermintAPI - not yet implemented + +Exposes the same functionality as the Tendermint RPC from a full node. It aims to have a very +similar API. + +### /broadcast_tx_sync - POST + +url: /broadcast_tx_sync + +Functionality: Submit a signed transaction and wait for it to be committed. + +Parameters: +| Parameter | Type | Default | Required | Description | +| ----------- | ------ | ------- | -------- | --------------- | +| transaction | string | null | true | signed tx bytes | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "code": 0, + "hash": "0D33F2F03A5234F38706E43004489E061AC40A2E", + "data": "", + "log": "" + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not submit the transaction synchronously.", + "result": {} +} +``` + +### /broadcast_tx_async - POST + +url: /broadcast_tx_async + +Functionality: Submit a signed transaction asynchronously. + +Parameters: +| Parameter | Type | Default | Required | Description | +| ----------- | ------ | ------- | -------- | --------------- | +| transaction | string | null | true | signed tx bytes | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "code": 0, + "hash": "E39AAB7A537ABAA237831742DCE1117F187C3C52", + "data": "", + "log": "" + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not submit the transaction asynchronously.", + "result": {} +} +``` + +## ICS1 - KeyAPI + +This API exposes all functionality needed for key creation, signing and management. + +### /keys - GET + +url: /keys + +Functionality: Gets a list of all the keys. + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "keys": [ + { + "name": "monkey", + "address": "cosmosaccaddr1fedh326uxqlxs8ph9ej7cf854gz7fd5zlym5pd", + "pub_key": "cosmosaccpub1zcjduc3q8s8ha96ry4xc5xvjp9tr9w9p0e5lk5y0rpjs5epsfxs4wmf72x3shvus0t" + }, + { + "name": "test", + "address": "cosmosaccaddr1thlqhjqw78zvcy0ua4ldj9gnazqzavyw4eske2", + "pub_key": "cosmosaccpub1zcjduc3qyx6hlf825jcnj39adpkaxjer95q7yvy25yhfj3dmqy2ctev0rxmse9cuak" + } + ], + "block_height": 5241 + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error":"Could not retrieve the keys.", + "result":{} +} +``` + +### /keys/recover - POST + +url: /keys/recover + +Functionality: Recover your key from seed and persist it encrypted with the password. + +Parameter: +| Parameter | Type | Default | Required | Description | +| --------- | ------ | ------- | -------- | ---------------- | +| name | string | null | true | name of key | +| password | string | null | true | password of key | +| seed | string | null | true | seed of key | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "address":BD607C37147656A507A5A521AA9446EB72B2C907 + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not recover the key.", + "result": {} +} +``` + +### /keys/create - POST + +url: /keys/create + +Functionality: Create a new key. + +Parameter: +| Parameter | Type | Default | Required | Description | +| --------- | ------ | ------- | -------- | ---------------- | +| name | string | null | true | name of key | +| password | string | null | true | password of key | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "seed":crime carpet recycle erase simple prepare moral dentist fee cause pitch trigger when velvet animal abandon + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not create new key.", + "result": {} +} +``` + +### /keys/{name} - GET + +url: /keys/{name} + +Functionality: Get the information for the specified key. + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "name": "test", + "address": "cosmosaccaddr1thlqhjqw78zvcy0ua4ldj9gnazqzavyw4eske2", + "pub_key": "cosmosaccpub1zcjduc3qyx6hlf825jcnj39adpkaxjer95q7yvy25yhfj3dmqy2ctev0rxmse9cuak" + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not find information on the specified key.", + "result": {} +} +``` + +### /keys/{name} - PUT + +url: /keys/{name} + +Functionality: Change the encryption password for the specified key. + +Parameters: +| Parameter | Type | Default | Required | Description | +| --------------- | ------ | ------- | -------- | --------------- | +| old_password | string | null | true | old password | +| new_password | string | null | true | new password | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": {} +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not update the specified key.", + "result": {} +} +``` + +### /keys/{name} - DELETE + +url: /keys/{name} + +Functionality: Delete the specified key. + +Parameters: +| Parameter | Type | Default | Required | Description | +| --------- | ------ | ------- | -------- | ---------------- | +| password | string | null | true | password of key | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": {} +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not delete the specified key.", + "result": {} +} +``` + +## ICS20 - TokenAPI + +The TokenAPI exposes all functionality needed to query account balances and send transactions. + +### /bank/balance/{account} - GET + +url: /bank/balance/{account} + +Functionality: Query the specified account. + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "atom": 1000, + "photon": 500, + "ether": 20 + } +} +``` + +Returns on error: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not find any balance for the specified account.", + "result": {} +} +``` + +### /bank/create_transfer - POST + +url: /bank/create_transfer + +Functionality: Create a transfer in the bank module. + +Parameters: +| Parameter | Type | Default | Required | Description | +| ------------ | ------ | ------- | -------- | ------------------------- | +| sender | string | null | true | Address of sender | +| receiver | string | null | true | address of receiver | +| chain_id | string | null | true | chain id | +| amount | int | null | true | amount of the token | +| denomonation | string | null | true | denomonation of the token | + +Returns on success: + +```json +{ + "rest api": "2.0", + "code":200, + "error": "", + "result": { + "transaction": "TODO:" + } +} +``` + +Returns on failure: + +```json +{ + "rest api": "2.0", + "code":500, + "error": "Could not create the transaction.", + "result": {} +} +``` diff --git a/docs/spec/light/getting_started.md b/docs/spec/light/getting_started.md new file mode 100644 index 000000000..1856372c8 --- /dev/null +++ b/docs/spec/light/getting_started.md @@ -0,0 +1,40 @@ +# Getting Started + +To start a rest server, we need to specify the following parameters: +| Parameter | Type | Default | Required | Description | +| ----------- | --------- | ----------------------- | -------- | ---------------------------------------------------- | +| chain-id | string | null | true | chain id of the full node to connect | +| node | URL | "tcp://localhost:46657" | true | address of the full node to connect | +| laddr | URL | "tcp://localhost:1317" | true | address to run the rest server on | +| trust-node | bool | "false" | true | Whether this LCD is connected to a trusted full node | +| trust-store | DIRECTORY | "$HOME/.lcd" | false | directory for save checkpoints and validator sets | + +Sample command: + +```bash +gaiacli light-client --chain-id=test --laddr=tcp://localhost:1317 --node tcp://localhost:46657 --trust-node=false +``` + +## Gaia Light Use Cases + +LCD could be very helpful for related service providers. For a wallet service provider, LCD could +make transaction faster and more reliable in the following cases. + +### Create an account + +![deposit](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/create-account.png) + +First you need to get a new seed phrase :[get-seed](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/api.md#keysseed---get) + +After having new seed, you could generate a new account with it : [keys](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/api.md#keys---post) + +### Transfer a token + +![transfer](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/transfer-tokens.png) + +The first step is to build an asset transfer transaction. Here we can post all necessary parameters +to /create_transfer to get the unsigned transaction byte array. Refer to this link for detailed +operation: [build transaction](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/api.md#create_transfer---post) + +Then sign the returned transaction byte array with users' private key. Finally broadcast the signed +transaction. Refer to this link for how to broadcast the signed transaction: [broadcast transaction](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/api.md#create_transfer---post) diff --git a/docs/spec/light/load_balancer.md b/docs/spec/light/load_balancer.md new file mode 100644 index 000000000..b3ba212ed --- /dev/null +++ b/docs/spec/light/load_balancer.md @@ -0,0 +1,203 @@ +# Load Balancing Module + +The LCD will be an important bridge between service providers and cosmos blockchain network. Suppose +a service provider wants to monitor token information for millions of accounts. Then it has to keep +sending a large mount of requests to LCD to query token information. As a result, LCD will send huge +requests to full node to get token information and necessary proof which will cost full node much +computing and bandwidth resource. Too many requests to a single full node may result in some bad +situations: + +```text +1. The full node crash possibility increases. +2. The reply delay increases. +3. The system reliability will decrease. +4. As the full node may belong to other people or associates, they may deny too frequent access from a single client. +``` + +It is very urgent to solve this problems. Here we consider to import load balancing into LCD. By the +help of load balancing, LCD can distribute millions of requests to a set of full nodes. Thus the +load of each full node won't be too heavy and the unavailable full nodes will be wiped out of query +list. In addition, the system reliability will increase. + +## Design + +This module need combine with client to realize the real load balancing. It can embed the +[HTTP Client](https://github.com/tendermint/tendermint/rpc/lib/client/httpclient.go). In other +words,we realise the new httpclient based on `HTTP`. + +```go +type HTTPLoadBalancer struct { + rpcs map[string]*rpcclient.JSONRPCClient + *WSEvents +} +``` + +## The Diagram of LCD RPC WorkFlow with LoadBalance + +![The Diagram of LCD RPC WorkFlow](pics/loadbalanceDiagram.png) + +In the above sequence diagram, application calls the `Request()`, and LCD finally call the +`HTTP.Request()` through the SecureClient `Wrapper`. In every `HTTP.Request()`, `Getclient()` +selects the current working rpcclient by the load balancing algorithm,then run the +`JSONRPCClient.Call()` to request from the Full Node, finally `UpdateClient()` updates the weight of + the current rpcclient according to the status that is returned by the full node. The `GetAddr()` + and `UpdateAddrWeight()` are realized in the load balancing module. + +There are some abilities to do: + +* Add the Remote Address +* Delete the Remote Address +* Update the weights of the addresses + +## Load balancing Strategies + +We can design some strategies like nginx to combine the different load balancing algorithms to get +the final remote. We can also get the status of the remote server to add or delete the addresses and + update weights of the addresses. + +In a word,it can make the entire LCD work more effective in actual conditions. +We are working this module independently in this [Github Repository](https://github.com/MrXJC/GoLoadBalance). + +## Interface And Type + +### Balancer + +This interface `Balancer`is the core of the package. Every load balancing algorithm should realize +it,and it defined two interfaces. + +* `init` initialize the balancer, assigns the variables which `DoBalance` needs. +* `DoBalance` load balance the full node addresses according to the current situation. + +```go +package balance + +type Balancer interface { + init(NodeAddrs) + DoBalance(NodeAddrs) (*NodeAddr,int,error) +} +``` + +### NodeAddr + +* host: ip address +* port: the number of port +* weight: the weight of this full node address,default:1 + +This NodeAddr is the base struct of the address. + +```go +type NodeAddr struct{ + host string + port int + weight int +} + +func (p *NodeAddr) GetHost() string + +func (p *NodeAddr) GetPort() int + +func (p *NodeAddr) GetWeight() int + +func (p *NodeAddr) updateWeight(weight int) +``` + +The `weight` is the important factor that schedules which full node the LCD calls. The weight can be +changed by the information from the full node. So we have the function `updateWegiht`. + +### NodeAddrs + +>in `balance/types.go` + +`NodeAddrs` is the list of the full node address. This is the member variable in the +BalanceManager(`BalancerMgr`). + +```go +type NodeAddrs []*NodeAddr +``` + +## Load Balancing Algorithm + +### Random + +>in `balance/random.go` + +Random algorithm selects a remote address randomly to process the request. The probability of them +being selected is the same. + +### RandomWeight + +>in `balance/random.go` + +RandomWeight Algorithm also selects a remote address randomly to process the request. But the higher +the weight, the greater the probability. + +### RoundRobin + +>in `balance/roundrobin.go` + +RoundRobin Algorithm selects a remote address orderly. Every remote address have the same +probability to be selected. + +### RoundRobinWeight + +>in `balance/roundrobin.go` + +RoundRobinWeight Algorthm selects a remote address orderly. But every remote address have different +probability to be selected which are determined by their weight. + +### Hash + +//TODO + +## Load Balancing Manager + +### BalanceMgr + +>in `balance/manager.go` + +* addrs: the set of the remote full node addresses +* balancers: map the string of balancer name to the specific balancer +* change: record whether the machine reinitialize after the `addrs` changes + +`BalanceMgr` is the manager of many balancer. It is the access of load balancing. Its main function +is to maintain the `NodeAddrs` and to call the specific load balancing algorithm above. + +```go +type BalanceMgr struct{ + addrs NodeAddrs + balancers map[string]Balancer + change map[string]bool +} + +func (p *BalanceMgr) RegisterBalancer(name string,balancer Balancer) + +func (p *BalanceMgr) updateBalancer(name string) + +func (p *BalanceMgr) AddNodeAddr(addr *NodeAddr) + +func (p *BalanceMgr) DeleteNodeAddr(i int) + +func (p *BalanceMgr) UpdateWeightNodeAddr(i int,weight int) + +func (p *BalanceMgr) GetAddr(name string)(*NodeAddr,int,error) { + // if addrs change,update the balancer which we use. + if p.change[name]{ + p.updateBalancer(name) + } + + // get the balancer by name + balancer := p.balancers[name] + + // use the load balancing algorithm + addr,index,err := balancer.DoBalance(p.addrs) + + return addr,index,err +} +``` + +* `RegisterBalancer`: register the basic balancer implementing the `Balancer` interface and initialize them. +* `updateBalancer`: update the specific balancer after the `addrs` change. +* `AddNodeAddr`: add the remote address and set all the values of the `change` to true. +* `DeleteNodeAddr`: delete the remote address and set all the values of the `change` to true. +* `UpdateWeightNodeAddr`: update the weight of the remote address and set all the values of the `change` to true. +* `GetAddr`:select the address by the balancer the `name` decides. diff --git a/docs/spec/light/overview.md b/docs/spec/light/overview.md new file mode 100644 index 000000000..82cf542ea --- /dev/null +++ b/docs/spec/light/overview.md @@ -0,0 +1,55 @@ +# Overview + +## What is a Light Client? + + The LCD is split into two separate components. The first component is generic for any Tendermint based application. It handles the security and connectivity aspects of following the header chain and verify proofs from full nodes against locally trusted validator set. Furthermore it exposes exactly the same API as any Tendermint Core node. The second component is specific for the Cosmos Hub (Gaiad). It works as a query endpoint and exposes the application specific functionality, which can be arbitrary. All queries against the application state have to go through the query endpoint. The advantage of the query endpoint is that it can verify the proofs that the application returns. + +## High-Level Architecture + +An application developer that would like to build a third party integration can ship his application with the LCD for the Cosmos Hub (or any other zone) and only needs to initialise it. Afterwards his application can interact with the zone as if it was running against a full node. + +![high-level](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/high-level.png) + +An application developer that wants to build an third party application for the Cosmos Hub (or any other zone) should build it against it's canonical API. That API is a combination of multiple parts. All zones have to expose ICS0 (TendermintAPI). Beyond that any zone is free to choose any combination of module APIs, depending on which modules the state machine uses. The Cosmos Hub will initially support ICS0 (TendermintAPI), ICS1 (KeyAPI), ICS20 (TokenAPI), ICS21 (StakingAPI) and ICS22 (GovernanceAPI). + +All applications are expected to only run against the LCD. The LCD is the only piece of software that offers stability guarantees around the zone API. + +### Comparision + +A full node of ABCI is different from its light client in the following ways: + +|| Full Node | LCD | Description| +|-| ------------- | ----- | -------------- | +| Execute and verify transactions|Yes|No|Full node will execute and verify all transactions while LCD won't| +| Verify and save blocks|Yes|No|Full node will verify and save all blocks while LCD won't| +| Participate consensus| Yes|No|Only when the full node is a validtor, it will participate consensus. LCD nodes never participate consensus| +| Bandwidth cost|Huge|Little|Full node will receive all blocks. if the bandwidth is limited, it will fall behind the main network. What's more, if it happens to be a validator,it will slow down the consensus process. LCD requires little bandwidth. Only when serving local request, it will cost bandwidth| +| Computing resource|Huge|Little|Full node will execute all transactions and verify all blocks which require much computing resource| +| Storage resource|Huge|Little|Full node will save all blocks and ABCI states. LCD just saves validator sets and some checkpoints| +| Power consume|Huge|Little|Full nodes have to be deployed on machines which have high performance and will be running all the time. So power consume will be huge. LCD can be deployed on the same machines as users' applications, or on independent machines but with poor performance. Besides, LCD can be shutdown anytime when necessary. So LCD only consume very little power, even mobile devices can meet the power requirement| +| Provide APIs|All cosmos APIs|Modular APIs|Full node supports all cosmos APIs. LCD provides modular APIs according to users' configuration| +| Secuity level| High|High|Full node will verify all transactions and blocks by itself. LCD can't do this, but it can query any data from other full nodes and verify the data independently. So both full node and LCD don't need to trust any third nodes, they all can achieve high security| + +According to the above table, LCD can meet all users' functionality and security requirements, but only requires little resource on bandwidth, computing, storage and power. + +## How does LCD achieve high security? + +### Trusted validator set + +The base design philosophy of lcd follows the two rules: + +1. **Doesn't trust any blockchin nodes, including validator nodes and other full nodes** +2. **Only trusts the whole validator set** + +The original trusted validator set should be prepositioned into its trust store, usually this validator set comes from genesis file. During running time, if LCD detects different validator set, it will verify it and save new validated validator set to trust store. + +![validator-set-change](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/validatorSetChange.png) + + +### Trust propagation + +From the above section, we come to know how to get trusted validator set and how lcd keeps track of validator set evolution. Validator set is the foundation of trust, and the trust can propagate to other blockchain data, such as block and transaction. The propagate architecture is shown as follows: + +![change-process](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/trustPropagate.png) + +In general, by trusted validator set, LCD can verify each block commit which contains all pre-commit data and block header data. Then the block hash, data hash and appHash are trusted. Based on this and merkle proof, all transactions data and ABCI states can be verified too. Detailed implementation will be posted on technical specification. diff --git a/docs/spec/light/pics/C2H.png b/docs/spec/light/pics/C2H.png new file mode 100644 index 000000000..49f7e07f3 Binary files /dev/null and b/docs/spec/light/pics/C2H.png differ diff --git a/docs/spec/light/pics/H2C.png b/docs/spec/light/pics/H2C.png new file mode 100644 index 000000000..027eafcde Binary files /dev/null and b/docs/spec/light/pics/H2C.png differ diff --git a/docs/spec/light/pics/MA.png b/docs/spec/light/pics/MA.png new file mode 100644 index 000000000..ae3823962 Binary files /dev/null and b/docs/spec/light/pics/MA.png differ diff --git a/docs/spec/light/pics/absence1.png b/docs/spec/light/pics/absence1.png new file mode 100755 index 000000000..70b4c9e8d Binary files /dev/null and b/docs/spec/light/pics/absence1.png differ diff --git a/docs/spec/light/pics/absence2.png b/docs/spec/light/pics/absence2.png new file mode 100755 index 000000000..6ce5bcde3 Binary files /dev/null and b/docs/spec/light/pics/absence2.png differ diff --git a/docs/spec/light/pics/absence3.png b/docs/spec/light/pics/absence3.png new file mode 100755 index 000000000..d3afdc2c5 Binary files /dev/null and b/docs/spec/light/pics/absence3.png differ diff --git a/docs/spec/light/pics/architecture.png b/docs/spec/light/pics/architecture.png new file mode 100644 index 000000000..741a90c26 Binary files /dev/null and b/docs/spec/light/pics/architecture.png differ diff --git a/docs/spec/light/pics/changeProcess.png b/docs/spec/light/pics/changeProcess.png new file mode 100755 index 000000000..1771c239b Binary files /dev/null and b/docs/spec/light/pics/changeProcess.png differ diff --git a/docs/spec/light/pics/commitValidation.png b/docs/spec/light/pics/commitValidation.png new file mode 100755 index 000000000..89985bcc1 Binary files /dev/null and b/docs/spec/light/pics/commitValidation.png differ diff --git a/docs/spec/light/pics/create-account.png b/docs/spec/light/pics/create-account.png new file mode 100644 index 000000000..35a249b97 Binary files /dev/null and b/docs/spec/light/pics/create-account.png differ diff --git a/docs/spec/light/pics/deposit.png b/docs/spec/light/pics/deposit.png new file mode 100644 index 000000000..1fb3acdc5 Binary files /dev/null and b/docs/spec/light/pics/deposit.png differ diff --git a/docs/spec/light/pics/existProof.png b/docs/spec/light/pics/existProof.png new file mode 100755 index 000000000..ee5de0851 Binary files /dev/null and b/docs/spec/light/pics/existProof.png differ diff --git a/docs/spec/light/pics/high-level.png b/docs/spec/light/pics/high-level.png new file mode 100644 index 000000000..73a14e0d7 Binary files /dev/null and b/docs/spec/light/pics/high-level.png differ diff --git a/docs/spec/light/pics/light-client-architecture.png b/docs/spec/light/pics/light-client-architecture.png new file mode 100644 index 000000000..df44aeef5 Binary files /dev/null and b/docs/spec/light/pics/light-client-architecture.png differ diff --git a/docs/spec/light/pics/loadbalanceDiagram.png b/docs/spec/light/pics/loadbalanceDiagram.png new file mode 100644 index 000000000..56956ee9d Binary files /dev/null and b/docs/spec/light/pics/loadbalanceDiagram.png differ diff --git a/docs/spec/light/pics/simpleMerkleTree.png b/docs/spec/light/pics/simpleMerkleTree.png new file mode 100755 index 000000000..5c4e2b76e Binary files /dev/null and b/docs/spec/light/pics/simpleMerkleTree.png differ diff --git a/docs/spec/light/pics/substoreProof.png b/docs/spec/light/pics/substoreProof.png new file mode 100755 index 000000000..90dadaef3 Binary files /dev/null and b/docs/spec/light/pics/substoreProof.png differ diff --git a/docs/spec/light/pics/transfer-tokens.png b/docs/spec/light/pics/transfer-tokens.png new file mode 100644 index 000000000..c6635a30c Binary files /dev/null and b/docs/spec/light/pics/transfer-tokens.png differ diff --git a/docs/spec/light/pics/transfer.png b/docs/spec/light/pics/transfer.png new file mode 100644 index 000000000..7642e996c Binary files /dev/null and b/docs/spec/light/pics/transfer.png differ diff --git a/docs/spec/light/pics/trustPropagate.png b/docs/spec/light/pics/trustPropagate.png new file mode 100755 index 000000000..a743cfc72 Binary files /dev/null and b/docs/spec/light/pics/trustPropagate.png differ diff --git a/docs/spec/light/pics/updateValidatorToHeight.png b/docs/spec/light/pics/updateValidatorToHeight.png new file mode 100755 index 000000000..b2c9349b6 Binary files /dev/null and b/docs/spec/light/pics/updateValidatorToHeight.png differ diff --git a/docs/spec/light/pics/validatorSetChange.png b/docs/spec/light/pics/validatorSetChange.png new file mode 100755 index 000000000..16dbf39be Binary files /dev/null and b/docs/spec/light/pics/validatorSetChange.png differ diff --git a/docs/spec/light/pics/withdraw.png b/docs/spec/light/pics/withdraw.png new file mode 100644 index 000000000..2249b5e34 Binary files /dev/null and b/docs/spec/light/pics/withdraw.png differ diff --git a/docs/spec/light/readme.md b/docs/spec/light/readme.md new file mode 100644 index 000000000..d15901b47 --- /dev/null +++ b/docs/spec/light/readme.md @@ -0,0 +1,101 @@ +# Cosmos-Sdk Light Client + +## Introduction + +A light client allows clients, such as mobile phones, to receive proofs of the state of the +blockchain from any full node. Light clients do not have to trust any full node, since they are able +to verify any proof they receive and hence full nodes cannot lie about the state of the network. + +A light client can provide the same security as a full node with the minimal requirements on +bandwidth, computing and storage resource. Besides, it can also provide modular functionality +according to users' configuration. These fantastic features allow developers to build fully secure, +efficient and usable mobile apps, websites or any other applications without deploying or +maintaining any full blockchain nodes. + +LCD will be used in the Cosmos Hub, the first Hub in the Cosmos network. + +## Contents + +1. [**Overview**](#Overview) +2. [**Get Started**](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/getting_started.md) +3. [**API**](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/api.md) +4. [**Specifications**](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/specification.md) +5. [**Future Improvements**](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/Future%20Improvements.md) + +## Overview + +### What is a Light Client + +The LCD is split into two separate components. The first component is generic for any Tendermint +based application. It handles the security and connectivity aspects of following the header chain +and verify proofs from full nodes against locally trusted validator set. Furthermore it exposes +exactly the same API as any Tendermint Core node. The second component is specific for the Cosmos +Hub (Gaiad). It works as a query endpoint and exposes the application specific functionality, which +can be arbitrary. All queries against the application state have to go through the query endpoint. +The advantage of the query endpoint is that it can verify the proofs that the application returns. + +### High-Level Architecture + +An application developer that would like to build a third party integration can ship his application +with the LCD for the Cosmos Hub (or any other zone) and only needs to initialise it. Afterwards his +application can interact with the zone as if it was running against a full node. + +![high-level](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/high-level.png) + +An application developer that wants to build an third party application for the Cosmos Hub (or any +other zone) should build it against it's canonical API. That API is a combination of multiple parts. +All zones have to expose ICS0 (TendermintAPI). Beyond that any zone is free to choose any +combination of module APIs, depending on which modules the state machine uses. The Cosmos Hub will +initially support ICS0 (TendermintAPI), ICS1 (KeyAPI), ICS20 (TokenAPI), ICS21 (StakingAPI) and +ICS22 (GovernanceAPI). + +All applications are expected to only run against the LCD. The LCD is the only piece of software +that offers stability guarantees around the zone API. + +### Comparision + +A full node of ABCI is different from its light client in the following ways: + +|| Full Node | LCD | Description| +|-| ------------- | ----- | -------------- | +| Execute and verify transactions|Yes|No|Full node will execute and verify all transactions while LCD won't| +| Verify and save blocks|Yes|No|Full node will verify and save all blocks while LCD won't| +| Participate consensus| Yes|No|Only when the full node is a validator, it will participate consensus. LCD nodes never participate consensus| +| Bandwidth cost|Huge|Little|Full node will receive all blocks. if the bandwidth is limited, it will fall behind the main network. What's more, if it happens to be a validator,it will slow down the consensus process. LCD requires little bandwidth. Only when serving local request, it will cost bandwidth| +| Computing resource|Huge|Little|Full node will execute all transactions and verify all blocks which require much computing resource| +| Storage resource|Huge|Little|Full node will save all blocks and ABCI states. LCD just saves validator sets and some checkpoints| +| Power consume|Huge|Little|Full nodes have to be deployed on machines which have high performance and will be running all the time. So power consume will be huge. LCD can be deployed on the same machines as users' applications, or on independent machines but with poor performance. Besides, LCD can be shutdown anytime when necessary. So LCD only consume very little power, even mobile devices can meet the power requirement| +| Provide APIs|All cosmos APIs|Modular APIs|Full node supports all cosmos APIs. LCD provides modular APIs according to users' configuration| +| Security level| High|High|Full node will verify all transactions and blocks by itself. LCD can't do this, but it can query any data from other full nodes and verify the data independently. So both full node and LCD don't need to trust any third nodes, they all can achieve high security| + +According to the above table, LCD can meet all users' functionality and security requirements, but +only requires little resource on bandwidth, computing, storage and power. + +## How does LCD achieve high security? + +### Trusted validator set + +The base design philosophy of lcd follows the two rules: + +1. **Doesn't trust any blockchain nodes, including validator nodes and other full nodes** +2. **Only trusts the whole validator set** + +The original trusted validator set should be prepositioned into its trust store, usually this +validator set comes from genesis file. During running time, if LCD detects different validator set, +it will verify it and save new validated validator set to trust store. + +![validator-set-change](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/validatorSetChange.png) + +### Trust propagation + +From the above section, we come to know how to get trusted validator set and how lcd keeps track of +validator set evolution. Validator set is the foundation of trust, and the trust can propagate to +other blockchain data, such as block and transaction. The propagate architecture is shown as +follows: + +![change-process](https://github.com/irisnet/cosmos-sdk/raw/bianjie/lcd_spec/docs/spec/lcd/pics/trustPropagate.png) + +In general, by trusted validator set, LCD can verify each block commit which contains all pre-commit +data and block header data. Then the block hash, data hash and appHash are trusted. Based on this +and merkle proof, all transactions data and ABCI states can be verified too. Detailed implementation +will be posted on technical specification. diff --git a/docs/spec/light/specification.md b/docs/spec/light/specification.md new file mode 100644 index 000000000..48c87d047 --- /dev/null +++ b/docs/spec/light/specification.md @@ -0,0 +1,318 @@ +# Specifications + +This specification describes how to implement the LCD. LCD supports modular APIs. Currently, only +ICS0 (TendermintAPI), ICS1 (Key API) and ICS20 (Token API) are supported. Later, if necessary, more +APIs can be included. + +## Build and Verify Proof of ABCI States + +As we all know, storage of cosmos-sdk based application contains multi-substores. Each substore is +implemented by a IAVL store. These substores are organized by simple Merkle tree. To build the tree, +we need to extract name, height and store root hash from these substores to build a set of simple +Merkle leaf nodes, then calculate hash from leaf nodes to root. The root hash of the simple Merkle +tree is the AppHash which will be included in block header. + +![Simple Merkle Tree](pics/simpleMerkleTree.png) + +As we have discussed in [LCD trust-propagation](https://github.com/irisnet/cosmos-sdk/tree/bianjie/lcd_spec/docs/spec/lcd#trust-propagation), +the AppHash can be verified by checking voting power against a trusted validator set. Here we just +need to build proof from ABCI state to AppHash. The proof contains two parts: + +* IAVL proof +* Substore to AppHash proof + +### IAVL Proof + +The proof has two types: existance proof and absence proof. If the query key exists in the IAVL +store, then it returns key-value and its existance proof. On the other hand, if the key doesn't +exist, then it only returns absence proof which can demostrate the key definitely doesn't exist. + +### IAVL Existance Proof + +```go +type CommitID struct { + Version int64 + Hash []byte +} + +type storeCore struct { + CommitID CommitID +} + +type MultiStoreCommitID struct { + Name string + Core storeCore +} + +type proofInnerNode struct { + Height int8 + Size int64 + Version int64 + Left []byte + Right []byte +} + +type KeyExistsProof struct { + MultiStoreCommitInfo []MultiStoreCommitID //All substore commitIDs + StoreName string //Current substore name + Height int64 //The commit height of current substore + RootHash cmn.HexBytes //The root hash of this IAVL tree + Version int64 //The version of the key-value in this IAVL tree + InnerNodes []proofInnerNode //The path from to root node to key-value leaf node +} +``` + +The data structure of exist proof is shown as above. The process to build and verify existance proof +is shown as follows: + +![Exist Proof](pics/existProof.png) + +Steps to build proof: + +* Access the IAVL tree from the root node. +* Record the visited nodes in InnerNodes, +* Once the target leaf node is found, assign leaf node version to proof version +* Assign the current IAVL tree height to proof height +* Assign the current IAVL tree rootHash to proof rootHash +* Assign the current substore name to proof StoreName +* Read multistore commitInfo from db by height and assign it to proof StoreCommitInfo + +Steps to verify proof: + +* Build leaf node with key, value and proof version. +* Calculate leaf node hash +* Assign the hash to the first innerNode's rightHash, then calculate first innerNode hash +* Propagate the hash calculation process. If prior innerNode is the left child of next innerNode, then assign the prior innerNode hash to the left hash of next innerNode. Otherwise, assign the prior innerNode hash to the right hash of next innerNode. +* The hash of last innerNode should be equal to the rootHash of this proof. Otherwise, the proof is invalid. + +### IAVL Absence Proof + +As we all know, all IAVL leaf nodes are sorted by the key of each leaf nodes. So we can calculate +the postition of the target key in the whole key set of this IAVL tree. As shown below, we can find +out the left key and the right key. If we can demonstrate that both left key and right key +definitely exist, and they are adjacent nodes. Thus the target key definitely doesn't exist. + +![Absence Proof1](pics/absence1.png) + +If the target key is larger than the right most leaf node or less than the left most key, then the +target key definitely doesn't exist. + +![Absence Proof2](pics/absence2.png)![Absence Proof3](pics/absence3.png) + +```go +type proofLeafNode struct { + KeyBytes cmn.HexBytes + ValueBytes cmn.HexBytes + Version int64 +} + +type pathWithNode struct { + InnerNodes []proofInnerNode + Node proofLeafNode +} + +type KeyAbsentProof struct { + MultiStoreCommitInfo []MultiStoreCommitID + StoreName string + Height int64 + RootHash cmn.HexBytes + Left *pathWithNode // Proof the left key exist + Right *pathWithNode //Proof the right key exist +} +``` + +The above is the data structure of absence proof. Steps to build proof: + +* Access the IAVL tree from the root node. +* Get the deserved index(Marked as INDEX) of the key in whole key set. +* If the returned index equals to 0, the right index should be 0 and left node doesn't exist +* If the returned index equals to the size of the whole key set, the left node index should be INDEX-1 and the right node doesn't exist. +* Otherwise, the right node index should be INDEX and the left node index should be INDEX-1 +* Assign the current IAVL tree height to proof height +* Assign the current IAVL tree rootHash to proof rootHash +* Assign the current substore name to proof StoreName +* Read multistore commitInfo from db by height and assign it to proof StoreCommitInfo + +Steps to verify proof: + +* If only right node exist, verify its exist proof and verify if it is the left most node +* If only left node exist, verify its exist proof and verify if it is the right most node. +* If both right node and left node exist, verify if they are adjacent. + +### Substores to AppHash Proof + +After verify the IAVL proof, then we can start to verify substore proof against AppHash. Firstly, +iterate MultiStoreCommitInfo and find the substore commitID by proof StoreName. Verify if yhe Hash +in commitID equals to proof RootHash. If not, the proof is invalid. Then sort the substore +commitInfo array by the hash of substore name. Finally, build the simple Merkle tree with all +substore commitInfo array and verify if the Merkle root hash equal to appHash. + +![substore proof](pics/substoreProof.png) + +```go +func SimpleHashFromTwoHashes(left []byte, right []byte) []byte { + var hasher = ripemd160.New() + + err := encodeByteSlice(hasher, left) + if err != nil { + panic(err) + } + + err = encodeByteSlice(hasher, right) + if err != nil { + panic(err) + } + + return hasher.Sum(nil) +} + +func SimpleHashFromHashes(hashes [][]byte) []byte { + // Recursive impl. + switch len(hashes) { + case 0: + return nil + case 1: + return hashes[0] + default: + left := SimpleHashFromHashes(hashes[:(len(hashes)+1)/2]) + right := SimpleHashFromHashes(hashes[(len(hashes)+1)/2:]) + return SimpleHashFromTwoHashes(left, right) + } +} +``` + +## Verify block header against validator set + +Above sections refer appHash frequently. But where does the trusted appHash come from? Actually, +appHash exist in block header, so next we need to verify blocks header at specific height against +LCD trusted validator set. The validation flow is shown as follows: + +![commit verification](pics/commitValidation.png) + +When the trusted validator set doesn't match the block header, we need to try to update our +validator set to the height of this block. LCD have a rule that each validator set change should not +affact more than 1/3 voting power. Compare with the trusted validator set, if the voting power of +target validator set changes more than 1/3. We have to verify if there are hidden validator set +change before the target validator set. Only when all validator set changes obey this rule, can our +validator set update be accomplished. + +For instance: + +![Update validator set to height](pics/updateValidatorToHeight.png) + +* Update to 10000, tooMuchChangeErr +* Update to 5050, tooMuchChangeErr +* Update to 2575, Success +* Update to 5050, Success +* Update to 10000,tooMuchChangeErr +* Update to 7525, Success +* Update to 10000, Success + +## Load Balancing + +To improve LCD reliability and TPS, we recommend to connect LCD to more than one fullnode. But the +complexity will increase a lot. So load balancing module will be imported as the adapter. Please +refer to this link for detailed description: [load balancing](https://github.com/irisnet/cosmos-sdk/blob/bianjie/lcd_spec/docs/spec/lcd/loadbalance.md) + +## ICS1 (KeyAPI) + +### [/keys - GET](api.md#keys---get) + +Load the key store: + +```go +db, err := dbm.NewGoLevelDB(KeyDBName, filepath.Join(rootDir, "keys")) +if err != nil { + return nil, err +} + +keybase = client.GetKeyBase(db) +``` + +Iterate through the key store. + +```go +var res []Info +iter := kb.db.Iterator(nil, nil) +defer iter.Close() + +for ; iter.Valid(); iter.Next() { + // key := iter.Key() + info, err := readInfo(iter.Value()) + if err != nil { + return nil, err + } + res = append(res, info) +} + +return res, nil +``` + +Encode the addresses and public keys in bech32. + +```go +bechAccount, err := sdk.Bech32ifyAcc(sdk.Address(info.PubKey.Address().Bytes())) +if err != nil { + return KeyOutput{}, err +} + +bechPubKey, err := sdk.Bech32ifyAccPub(info.PubKey) +if err != nil { + return KeyOutput{}, err +} + +return KeyOutput{ + Name: info.Name, + Address: bechAccount, + PubKey: bechPubKey, +}, nil +``` + +### [/keys/recover - POST](api.md#keys/recover---get) + +1. Load the key store. +2. Parameter checking. Name, password and seed should not be empty. +3. Check for keys with the same name. +4. Build the key from the name, password and seed. +5. Persist the key to key store. + +### [/keys/create - GET](api.md#keys/create---get)** + +1. Load the key store. +2. Create a new key in the key store. +3. Save the key to disk. +4. Return the seed. + +### [/keys/{name} - GET](api.md#keysname---get) + +1. Load the key store. +2. Iterate the whole key store to find the key by name. +3. Encode address and public key in bech32. + +### [/keys/{name} - PUT](api.md#keysname---put) + +1. Load the key store. +2. Iterate the whole key store to find the key by name. +3. Verify if that the old-password matches the current key password. +4. Re-persist the key with the new password. + +### [/keys/{name} - DELETE](api.md#keysname---delete) + +1. Load the key store. +2. Iterate the whole key store to find the key by name. +3. Verify that the specified password matches the current key password. +4. Delete the key from the key store. + +## ICS20 (TokenAPI) + +### [/bank/balance/{account}](api.md#balanceaccount---get) + +1. Decode the address from bech32 to hex. +2. Send a query request to a full node. Ask for proof if required by Gaia Light. +3. Verify the proof against the root of trust. + +### [/bank/create_transfer](api.md#create_transfer---post) + +1. Check the parameters. +2. Build the transaction with the specified parameters. +3. Serialise the transaction and return the JSON encoded sign bytes. diff --git a/docs/spec/light/todo.md b/docs/spec/light/todo.md new file mode 100644 index 000000000..ce1f8508a --- /dev/null +++ b/docs/spec/light/todo.md @@ -0,0 +1,16 @@ +# TODO + +This document is a place to gather all points for future development. + +## API + +* finalise ICS0 - TendermintAPI + * make sure that the explorer and voyager can use it +* add ICS21 - StakingAPI +* add ICS22 - GovernanceAPI +* split Gaia Light into reusable components that other zones can leverage + * it should be possible to register extra standards on the light client + * the setup should be similar to how the app is currently started +* implement Gaia light and the general light client in Rust + * export the API as a C interface + * write thin wrappers around the C interface in JS, Swift and Kotlin/Java