poanetwork
/
quorum
mirror of https://github.com/poanetwork/quorum.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity

Quorum raft upgrade v3.3.13 (#887) 

* raft: upgrade etcd to version 3.3.13
* raft: added api for adding a learner node
This commit is contained in:
Sai V 2019-12-04 22:24:32 +08:00 committed by Samer Falah
parent 151cf6e18c
commit dfd93ff218
164 changed files with 29476 additions and 1341 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
cmd/geth/config.go
View File

@ -218,7 +218,6 @@ func RegisterRaftService(stack *node.Node, ctx *cli.Context, cfg gethConfig, eth
datadir := ctx.GlobalString(utils.DataDirFlag.Name)
joinExistingId := ctx.GlobalInt(utils.RaftJoinExistingFlag.Name)
useDns := ctx.GlobalBool(utils.RaftDNSEnabledFlag.Name)
raftPort := uint16(ctx.GlobalInt(utils.RaftPortFlag.Name))
if err := stack.Register(func(ctx *node.ServiceContext) (node.Service, error) {

1
cmd/utils/flags.go
View File

@ -594,6 +594,7 @@ var (
Usage: "The raft ID to assume when joining an pre-existing cluster",
Value: 0,
}
EmitCheckpointsFlag = cli.BoolFlag{
Name: "emitcheckpoints",
Usage: "If enabled, emit specially formatted logging checkpoints",

39
docs/Consensus/raft.md
View File

@ -13,9 +13,33 @@ When the `geth` binary is passed the `--raft` flag, the node will operate in "ra
Both Raft and Ethereum have their own notion of a "node":
In Raft, a node in normal operation is either a "leader" or a "follower." There is a single leader for the entire cluster, which all log entries must flow through. There's also the concept of a "candidate", but only during leader election. We won't go into more detail about Raft here, because by design these details are opaque to applications built on it.
In Raft, a node in normal operation can be a "leader", "follower" or "learner." There is a single leader for the entire cluster, through which all log entries must flow through. There's also the concept of a "candidate", but only during leader election. We won't go into more detail about Raft here, because by design these details are opaque to applications built on it. A Raft network can be started with a set of verifiers and one of them would get elected as a leader when the network starts. If the leader node dies, re-election is triggered and new leader is elected by the network. Once the network is up additional verifier nodes(peers) or learner nodes can be added to this network. Brief summary of each type of nodes is given below:
In vanilla Ethereum, there is no such thing as a "leader" or "follower." It's possible for any node in the network to mine a new block -- which is akin to being the leader for that round.
### Leader
- mints blocks and sends the blocks to the verifier and learner nodes
- takes part in voting during re-election and can become verifier if it does not win majority of votes
- the network triggers re-election if the leader node dies.
- can add/remove learner/verifier and promote learner to verifier
### Verifier
- follows the leader
- applies the blocks minted by the leader
- takes part in voting during re-election and can become leader if it wins majority of votes
- sends confirmation to leader
- can add/remove learner/verifier and promote learner to verifier
### Learner
- follows the leader
- applies the blocks minted by the leader
- cannot take part in voting during re-election
- cannot become a verifier on its own
- it needs to be promoted to be a verifier by a leader or verifier
- it cannot add learner/verifier or promote learner to verifier
- it cannot remove other learner/verifier but it can remove itself
It should be noted that when a node is added/removed as a verifier (peer), it impacts the Raft Quorum. However adding a node as learner does not change the Raft Quroum. Hence its recommended that when a new node is added to a long running network, the node is first added as a learner. Once the learner node syncs fully with the network, it can then be promoted to become a verifier.
In vanilla Ethereum, there is no such thing as a "leader", "learner" or "follower." It's possible for any node in the network to mine a new block -- which is akin to being the leader for that round.
In Raft-based consensus, we impose a one-to-one correspondence between Raft and Ethereum nodes: each Ethereum node is also a Raft node, and by convention, the leader of the Raft cluster is the only Ethereum node that should mine (or "mint") new blocks. A minter is responsible for bundling transactions into a block just like an Ethereum miner, but does not present a proof of work.
@ -24,6 +48,8 @@ Ethereum | Raft
minter | leader
verifier | follower
A learner node is passive node that just syncs blocks and can initiate transactions.
The main reasons we co-locate the leader and minter are (1) convenience, in that Raft ensures there is only one leader at a time, and (2) to avoid a network hop from a node minting blocks to the leader, through which all Raft writes must flow. Our implementation watches Raft leadership changes -- if a node becomes a leader it will start minting, and if a node loses its leadership, it will stop minting.
An observant reader might note that during raft leadership transitions, there could be a small period of time where more than one node might assume that it has minting duties; we detail how correctness is preserved in the [Chain extension, races, and correctness](#chain-extension-races-and-correctness) section.
@ -34,11 +60,12 @@ When the minter creates a block, unlike in vanilla Ethereum where the block is w
From the point of view of Ethereum, Raft is integrated via an implementation of the [`Service`](https://godoc.org/github.com/jpmorganchase/quorum/node#Service) interface in [`node/service.go`](https://github.com/jpmorganchase/quorum/blob/master/node/service.go): "an individual protocol that can be registered into a node". Other examples of services are [`Ethereum`](https://godoc.org/github.com/jpmorganchase/quorum/eth#Ethereum) and [`Whisper`](https://godoc.org/github.com/jpmorganchase/quorum/whisper/whisperv5#Whisper).
## The lifecycle of a transaction
Let's follow the lifecycle of a typical transaction:
#### on any node (whether minter or verifier):
#### on any node (whether minter, verifier or learner):
1. The transaction is submitted via an RPC call to geth.
2. Using the existing (p2p) transaction propagation mechanism in Ethereum, the transaction is announced to all peers and, because our cluster is currently configured to use "static nodes," every transaction is sent to all peers in the cluster.
@ -155,7 +182,11 @@ Currently Raft-based consensus requires that all _initial_ nodes in the cluster
To remove a node from the cluster, attach to a JS console and issue `raft.removePeer(raftId)`, where `raftId` is the number of the node you wish to remove. For initial nodes in the cluster, this number is the 1-indexed position of the node's enode ID in the static peers list. Once a node has been removed from the cluster, it is permanent; this raft ID can not ever re-connect to the cluster in the future, and the party must re-join the cluster with a new raft ID.
To add a node to the cluster, attach to a JS console and issue `raft.addPeer(enodeId)`. Note that like the enode IDs listed in the static peers JSON file, this enode ID should include a `raftport` querystring parameter. This call will allocate and return a raft ID that was not already in use. After `addPeer`, start the new geth node with the flag `--raftjoinexisting RAFTID` in addition to `--raft`.
* To add a verifier node to the cluster, attach to a JS console and issue `raft.addPeer(enodeId)`
* To add a learner node to the cluster, attach to a JS console and issue `raft.addLearner(enodeId)`
* To promote a learner to become verifier in the cluster, attach to a JS console of leader/verifier node and issue `raft.promoteToPeer(raftId)`.
Note that like the enode IDs listed in the static peers JSON file, this enode ID should include a `raftport` querystring parameter. This call will allocate and return a raft ID that was not already in use. After `addPeer`, start the new geth node with the flag `--raftjoinexisting RAFTID` in addition to `--raft`.
## FAQ

10
internal/web3ext/web3ext.go
View File

@ -725,6 +725,16 @@ web3._extend({
call: 'raft_addPeer',
params: 1
}),
new web3._extend.Method({
name: 'addLearner',
call: 'raft_addLearner',
params: 1
}),
new web3._extend.Method({
name: 'promoteToPeer',
call: 'raft_promoteToPeer',
params: 1
}),
new web3._extend.Method({
name: 'removePeer',
call: 'raft_removePeer',

1
log/emit_checkpoint.go
View File

@ -5,6 +5,7 @@ const (
TxAccepted = "TX-ACCEPTED"
BecameMinter = "BECAME-MINTER"
BecameVerifier = "BECAME-VERIFIER"
BecameLearner = "BECAME-LEARNER"
BlockCreated = "BLOCK-CREATED"
BlockVotingStarted = "BLOCK-VOTING-STARTED"
)

43
raft/api.go
View File

@ -23,25 +23,56 @@ func (s *PublicRaftAPI) Role() string {
}
func (s *PublicRaftAPI) AddPeer(enodeId string) (uint16, error) {
return s.raftService.raftProtocolManager.ProposeNewPeer(enodeId)
return s.raftService.raftProtocolManager.ProposeNewPeer(enodeId, false)
}
func (s *PublicRaftAPI) RemovePeer(raftId uint16) {
s.raftService.raftProtocolManager.ProposePeerRemoval(raftId)
func (s *PublicRaftAPI) AddLearner(enodeId string) (uint16, error) {
return s.raftService.raftProtocolManager.ProposeNewPeer(enodeId, true)
}
func (s *PublicRaftAPI) PromoteToPeer(raftId uint16) (bool, error) {
return s.raftService.raftProtocolManager.PromoteToPeer(raftId)
}
func (s *PublicRaftAPI) RemovePeer(raftId uint16) error {
return s.raftService.raftProtocolManager.ProposePeerRemoval(raftId)
}
func (s *PublicRaftAPI) Leader() (string, error) {
addr, err := s.raftService.raftProtocolManager.LeaderAddress()
if nil != err {
if err != nil {
return "", err
}
return addr.NodeId.String(), nil
}
func (s *PublicRaftAPI) Cluster() []*Address {
func (s *PublicRaftAPI) Cluster() ([]ClusterInfo, error) {
nodeInfo := s.raftService.raftProtocolManager.NodeInfo()
return append(nodeInfo.PeerAddresses, nodeInfo.Address)
if nodeInfo.Role == "" {
return []ClusterInfo{}, nil
}
leaderAddr, err := s.raftService.raftProtocolManager.LeaderAddress()
if err != nil {
if err == errNoLeaderElected && s.Role() == "" {
return []ClusterInfo{}, nil
}
return []ClusterInfo{}, err
}
peerAddresses := append(nodeInfo.PeerAddresses, nodeInfo.Address)
clustInfo := make([]ClusterInfo, len(peerAddresses))
for i, a := range peerAddresses {
role := ""
if a.RaftId == leaderAddr.RaftId {
role = "minter"
} else if s.raftService.raftProtocolManager.isLearner(a.RaftId) {
role = "learner"
} else if s.raftService.raftProtocolManager.isVerifier(a.RaftId) {
role = "verifier"
}
clustInfo[i] = ClusterInfo{*a, role}
}
return clustInfo, nil
}
func (s *PublicRaftAPI) GetRaftId(enodeId string) (uint16, error) {

127
raft/handler.go
View File

@ -1,6 +1,7 @@
package raft
import (
"context"
"errors"
"fmt"
"net"
@ -21,7 +22,6 @@ import (
"github.com/coreos/etcd/wal"
mapset "github.com/deckarep/golang-set"
"github.com/syndtr/goleveldb/leveldb"
"golang.org/x/net/context"
"github.com/ethereum/go-ethereum/core"
"github.com/ethereum/go-ethereum/core/types"
@ -58,7 +58,7 @@ type ProtocolManager struct {
// P2P transport
p2pServer *p2p.Server // Initialized in start()
useDns bool
useDns bool
// Blockchain services
blockchain *core.BlockChain
@ -93,6 +93,8 @@ type ProtocolManager struct {
raftStorage *etcdRaft.MemoryStorage // Volatile raft storage
}
var errNoLeaderElected = errors.New("no leader is currently elected")
//
// Public interface
//
@ -123,7 +125,7 @@ func NewProtocolManager(raftId uint16, raftPort uint16, blockchain *core.BlockCh
raftStorage: etcdRaft.NewMemoryStorage(),
minter: minter,
downloader: downloader,
useDns: useDns,
useDns: useDns,
}
if db, err := openQuorumRaftDb(quorumRaftDbLoc); err != nil {
@ -189,11 +191,13 @@ func (pm *ProtocolManager) NodeInfo() *RaftNodeInfo {
pm.mu.RLock() // as we read role and peers
defer pm.mu.RUnlock()
var roleDescription string
roleDescription := ""
if pm.role == minterRole {
roleDescription = "minter"
} else {
} else if pm.isVerifierNode() {
roleDescription = "verifier"
} else if pm.isLearnerNode() {
roleDescription = "learner"
}
peerAddresses := make([]*Address, len(pm.peers))
@ -317,7 +321,10 @@ func (pm *ProtocolManager) peerExist(node *enode.Node) bool {
return false
}
func (pm *ProtocolManager) ProposeNewPeer(enodeId string) (uint16, error) {
func (pm *ProtocolManager) ProposeNewPeer(enodeId string, isLearner bool) (uint16, error) {
if pm.isLearnerNode() {
return 0, errors.New("learner node can't add peer or learner")
}
parsedUrl, _ := url.Parse(enodeId)
node, err := enode.ParseV4(enodeId)
if err != nil {
@ -341,8 +348,14 @@ func (pm *ProtocolManager) ProposeNewPeer(enodeId string) (uint16, error) {
raftId := pm.nextRaftId()
address := newAddress(raftId, node.RaftPort(), node, pm.useDns)
confChangeType := raftpb.ConfChangeAddNode
if isLearner {
confChangeType = raftpb.ConfChangeAddLearnerNode
}
pm.confChangeProposalC <- raftpb.ConfChange{
Type: raftpb.ConfChangeAddNode,
Type: confChangeType,
NodeID: uint64(raftId),
Context: address.toBytes(pm.useDns),
}
@ -350,11 +363,31 @@ func (pm *ProtocolManager) ProposeNewPeer(enodeId string) (uint16, error) {
return raftId, nil
}
func (pm *ProtocolManager) ProposePeerRemoval(raftId uint16) {
func (pm *ProtocolManager) ProposePeerRemoval(raftId uint16) error {
if pm.isLearnerNode() && raftId != pm.raftId {
return errors.New("learner node can't remove other peer")
}
pm.confChangeProposalC <- raftpb.ConfChange{
Type: raftpb.ConfChangeRemoveNode,
NodeID: uint64(raftId),
}
return nil
}
func (pm *ProtocolManager) PromoteToPeer(raftId uint16) (bool, error) {
if pm.isLearnerNode() {
return false, errors.New("learner node can't promote to peer")
}
if !pm.isLearner(raftId) {
return false, fmt.Errorf("%d is not a learner. only learner can be promoted to peer", raftId)
}
pm.confChangeProposalC <- raftpb.ConfChange{
Type: raftpb.ConfChangeAddNode,
NodeID: uint64(raftId),
}
return true, nil
}
//
@ -410,8 +443,9 @@ func (pm *ProtocolManager) startRaft() {
walExisted := wal.Exist(pm.waldir)
lastAppliedIndex := pm.loadAppliedIndex()
ss := &stats.ServerStats{}
ss.Initialize()
id := raftTypes.ID(pm.raftId).String()
ss := stats.NewServerStats(id, id)
pm.transport = &rafthttp.Transport{
ID: raftTypes.ID(pm.raftId),
ClusterID: 0x1000,
@ -531,6 +565,7 @@ func (pm *ProtocolManager) startRaft() {
if walExisted {
log.Info("remounting an existing raft log; connecting to peers.")
pm.unsafeRawNode = etcdRaft.RestartNode(raftConfig)
} else if pm.joinExisting {
log.Info("newly joining an existing cluster; waiting for connections.")
@ -553,10 +588,9 @@ func (pm *ProtocolManager) startRaft() {
for _, peerAddress := range peerAddresses {
pm.addPeer(peerAddress)
}
pm.unsafeRawNode = etcdRaft.StartNode(raftConfig, raftPeers)
}
log.Info("raft node started")
go pm.serveRaft()
go pm.serveLocalProposals()
go pm.eventLoop()
@ -567,7 +601,6 @@ func (pm *ProtocolManager) setLocalAddress(addr *Address) {
pm.mu.Lock()
pm.address = addr
pm.mu.Unlock()
// By setting `URLs` on the raft transport, we advertise our URL (in an HTTP
// header) to any recipient. This is necessary for a newcomer to the cluster
// to be able to accept a snapshot from us to bootstrap them.
@ -598,6 +631,36 @@ func (pm *ProtocolManager) serveRaft() {
close(pm.httpdonec)
}
func (pm *ProtocolManager) isLearner(rid uint16) bool {
pm.mu.RLock()
defer pm.mu.RUnlock()
for _, n := range pm.confState.Learners {
if uint16(n) == rid {
return true
}
}
return false
}
func (pm *ProtocolManager) isLearnerNode() bool {
return pm.isLearner(pm.raftId)
}
func (pm *ProtocolManager) isVerifierNode() bool {
return pm.isVerifier(pm.raftId)
}
func (pm *ProtocolManager) isVerifier(rid uint16) bool {
pm.mu.RLock()
defer pm.mu.RUnlock()
for _, n := range pm.confState.Nodes {
if uint16(n) == rid {
return true
}
}
return false
}
func (pm *ProtocolManager) handleRoleChange(roleC <-chan interface{}) {
for {
select {
@ -612,7 +675,11 @@ func (pm *ProtocolManager) handleRoleChange(roleC <-chan interface{}) {
log.EmitCheckpoint(log.BecameMinter)
pm.minter.start()
} else { // verifier
log.EmitCheckpoint(log.BecameVerifier)
if pm.isVerifierNode() {
log.EmitCheckpoint(log.BecameVerifier)
} else {
log.EmitCheckpoint(log.BecameLearner)
}
pm.minter.stop()
}
@ -775,8 +842,8 @@ func (pm *ProtocolManager) eventLoop() {
case <-ticker.C:
pm.rawNode().Tick()
// when the node is first ready it gives us entries to commit and messages
// to immediately publish
// when the node is first ready it gives us entries to commit and messages
// to immediately publish
case rd := <-pm.rawNode().Ready():
pm.wal.Save(rd.HardState, rd.Entries)
@ -837,26 +904,30 @@ func (pm *ProtocolManager) eventLoop() {
raftId := uint16(cc.NodeID)
pm.confState = *pm.rawNode().ApplyConfChange(cc)
log.Info("confChange", "confState", pm.confState)
forceSnapshot := false
switch cc.Type {
case raftpb.ConfChangeAddNode:
case raftpb.ConfChangeAddNode, raftpb.ConfChangeAddLearnerNode:
confChangeTypeName := raftpb.ConfChangeType_name[int32(cc.Type)]
log.Info(confChangeTypeName, "raft id", raftId)
if pm.isRaftIdRemoved(raftId) {
log.Info("ignoring ConfChangeAddNode for permanently-removed peer", "raft id", raftId)
log.Info("ignoring "+confChangeTypeName+" for permanently-removed peer", "raft id", raftId)
} else if pm.isRaftIdUsed(raftId) && raftId <= uint16(len(pm.bootstrapNodes)) {
// See initial cluster logic in startRaft() for more information.
log.Info("ignoring expected ConfChangeAddNode for initial peer", "raft id", raftId)
log.Info("ignoring expected "+confChangeTypeName+" for initial peer", "raft id", raftId)
// We need a snapshot to exist to reconnect to peers on start-up after a crash.
forceSnapshot = true
} else if pm.isRaftIdUsed(raftId) {
log.Info("ignoring ConfChangeAddNode for already-used raft ID", "raft id", raftId)
} else {
log.Info("adding peer due to ConfChangeAddNode", "raft id", raftId)
} else { // add peer or add learner or promote learner to voter
forceSnapshot = true
pm.addPeer(bytesToAddress(cc.Context))
//if raft id exists as peer, you are promoting learner to peer
if pm.isRaftIdUsed(raftId) {
log.Info("promote learner node to voter node", "raft id", raftId)
} else {
//if raft id does not exist, you are adding peer/learner
log.Info("add peer/learner -> "+confChangeTypeName, "raft id", raftId)
pm.addPeer(bytesToAddress(cc.Context))
}
}
case raftpb.ConfChangeRemoveNode:
@ -1008,7 +1079,7 @@ func (pm *ProtocolManager) LeaderAddress() (*Address, error) {
return l.address, nil
}
// We expect to reach this if pm.leader is 0, which is how etcd denotes the lack of a leader.
return nil, errors.New("no leader is currently elected")
return nil, errNoLeaderElected
}
// Returns the raft id for a given enodeId

136
raft/minter_test.go
View File

@ -1,10 +1,17 @@
package raft
import (
"errors"
"fmt"
"math/big"
"strings"
"testing"
"time"
"github.com/coreos/etcd/raft/raftpb"
"github.com/deckarep/golang-set"
"github.com/ethereum/go-ethereum/p2p/enode"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/common/hexutil"
"github.com/ethereum/go-ethereum/core/types"
@ -13,6 +20,8 @@ import (
"github.com/ethereum/go-ethereum/rlp"
)
const TEST_URL = "enode://3d9ca5956b38557aba991e31cf510d4df641dce9cc26bfeb7de082f0c07abb6ede3a58410c8f249dabeecee4ad3979929ac4c7c496ad20b8cfdd061b7401b4f5@127.0.0.1:21003?discport=0&raftport=50404"
func TestSignHeader(t *testing.T) {
//create only what we need to test the seal
var testRaftId uint16 = 5
@ -67,3 +76,130 @@ func TestSignHeader(t *testing.T) {
}
}
func TestAddLearner_whenTypical(t *testing.T) {
raftService := newTestRaftService(t, 1, []uint64{1}, []uint64{})
propPeer := func() {
raftid, err := raftService.raftProtocolManager.ProposeNewPeer(TEST_URL, true)
if err != nil {
t.Errorf("propose new peer failed %v\n", err)
}
if raftid != raftService.raftProtocolManager.raftId+1 {
t.Errorf("1. wrong raft id. expected %d got %d\n", raftService.raftProtocolManager.raftId+1, raftid)
}
}
go propPeer()
select {
case confChange := <-raftService.raftProtocolManager.confChangeProposalC:
if confChange.Type != raftpb.ConfChangeAddLearnerNode {
t.Errorf("expected ConfChangeAddLearnerNode but got %s", confChange.Type.String())
}
if uint16(confChange.NodeID) != raftService.raftProtocolManager.raftId+1 {
t.Errorf("2. wrong raft id. expected %d got %d\n", raftService.raftProtocolManager.raftId+1, uint16(confChange.NodeID))
}
case <-time.After(time.Millisecond * 200):
t.Errorf("add learner conf change not recieved")
}
}
func TestPromoteLearnerToPeer_whenTypical(t *testing.T) {
learnerRaftId := uint16(3)
raftService := newTestRaftService(t, 2, []uint64{2}, []uint64{uint64(learnerRaftId)})
promoteToPeer := func() {
ok, err := raftService.raftProtocolManager.PromoteToPeer(learnerRaftId)
if err != nil || !ok {
t.Errorf("promote learner to peer failed %v\n", err)
}
}
go promoteToPeer()
select {
case confChange := <-raftService.raftProtocolManager.confChangeProposalC:
if confChange.Type != raftpb.ConfChangeAddNode {
t.Errorf("expected ConfChangeAddNode but got %s", confChange.Type.String())
}
if uint16(confChange.NodeID) != learnerRaftId {
t.Errorf("2. wrong raft id. expected %d got %d\n", learnerRaftId, uint16(confChange.NodeID))
}
case <-time.After(time.Millisecond * 200):
t.Errorf("add learner conf change not recieved")
}
}
func TestAddLearnerOrPeer_fromLearner(t *testing.T) {
raftService := newTestRaftService(t, 3, []uint64{2}, []uint64{3})
_, err := raftService.raftProtocolManager.ProposeNewPeer(TEST_URL, true)
if err == nil {
t.Errorf("learner should not be allowed to add learner or peer")
}
if err != nil && strings.Index(err.Error(), "learner node can't add peer or learner") == -1 {
t.Errorf("expect error message: propose new peer failed, got: %v\n", err)
}
_, err = raftService.raftProtocolManager.ProposeNewPeer(TEST_URL, false)
if err == nil {
t.Errorf("learner should not be allowed to add learner or peer")
}
if err != nil && strings.Index(err.Error(), "learner node can't add peer or learner") == -1 {
t.Errorf("expect error message: propose new peer failed, got: %v\n", err)
}
}
func TestPromoteLearnerToPeer_fromLearner(t *testing.T) {
learnerRaftId := uint16(3)
raftService := newTestRaftService(t, 2, []uint64{1}, []uint64{2, uint64(learnerRaftId)})
_, err := raftService.raftProtocolManager.PromoteToPeer(learnerRaftId)
if err == nil {
t.Errorf("learner should not be allowed to promote to peer")
}
if err != nil && strings.Index(err.Error(), "learner node can't promote to peer") == -1 {
t.Errorf("expect error message: propose new peer failed, got: %v\n", err)
}
}
func enodeId(id string, ip string, raftPort int) string {
return fmt.Sprintf("enode://%s@%s?discport=0&raftport=%d", id, ip, raftPort)
}
func peerList(url string) (error, []*enode.Node) {
var nodes []*enode.Node
node, err := enode.ParseV4(url)
if err != nil {
return errors.New(fmt.Sprintf("Node URL %s: %v\n", url, err)), nil
}
nodes = append(nodes, node)
return nil, nodes
}
func newTestRaftService(t *testing.T, raftId uint16, nodes []uint64, learners []uint64) *RaftService {
//create only what we need to test add learner node
config := &node.Config{Name: "unit-test", DataDir: ""}
nodeKey := config.NodeKey()
enodeIdStr := fmt.Sprintf("%x", crypto.FromECDSAPub(&nodeKey.PublicKey)[1:])
url := enodeId(enodeIdStr, "127.0.0.1:21001", 50401)
err, peers := peerList(url)
if err != nil {
t.Errorf("getting peers failed %v", err)
}
raftProtocolManager := &ProtocolManager{
raftId: raftId,
bootstrapNodes: peers,
confChangeProposalC: make(chan raftpb.ConfChange),
removedPeers: mapset.NewSet(),
confState: raftpb.ConfState{Nodes: nodes, Learners: learners},
}
raftService := &RaftService{nodeKey: nodeKey, raftProtocolManager: raftProtocolManager}
return raftService
}

6
raft/peer.go
View File

@ -4,7 +4,6 @@ import (
"bytes"
"fmt"
"log"
"net"
"github.com/ethereum/go-ethereum/p2p/enode"
@ -28,6 +27,11 @@ type Address struct {
Rest []rlp.RawValue `json:"-" rlp:"tail"`
}
type ClusterInfo struct {
Address
Role string `json:"role"`
}
func newAddress(raftId uint16, raftPort int, node *enode.Node, withHostname bool) *Address {
// derive 64 byte nodeID from 128 byte enodeID
id, err := enode.RaftHexID(node.EnodeID())

8
raft/snapshot.go
View File

@ -53,7 +53,7 @@ func (pm *ProtocolManager) buildSnapshot() *SnapshotWithHostnames {
pm.mu.RLock()
defer pm.mu.RUnlock()
numNodes := len(pm.confState.Nodes)
numNodes := len(pm.confState.Nodes) + len(pm.confState.Learners)
numRemovedNodes := pm.removedPeers.Cardinality()
snapshot := &SnapshotWithHostnames{
@ -64,7 +64,7 @@ func (pm *ProtocolManager) buildSnapshot() *SnapshotWithHostnames {
// Populate addresses
for i, rawRaftId := range pm.confState.Nodes {
for i, rawRaftId := range append(pm.confState.Nodes, pm.confState.Learners...) {
raftId := uint16(rawRaftId)
if raftId == pm.raftId {
@ -81,7 +81,6 @@ func (pm *ProtocolManager) buildSnapshot() *SnapshotWithHostnames {
snapshot.RemovedRaftIds[i] = removedIface.(uint16)
i++
}
return snapshot
}
@ -119,7 +118,7 @@ func (pm *ProtocolManager) triggerSnapshot(index uint64) {
func confStateIdSet(confState raftpb.ConfState) mapset.Set {
set := mapset.NewSet()
for _, rawRaftId := range confState.Nodes {
for _, rawRaftId := range append(confState.Nodes, confState.Learners...) {
set.Add(uint16(rawRaftId))
}
return set
@ -197,7 +196,6 @@ func (pm *ProtocolManager) maybeTriggerSnapshot() {
func (pm *ProtocolManager) loadSnapshot() *raftpb.Snapshot {
if raftSnapshot := pm.readRaftSnapshot(); raftSnapshot != nil {
log.Info("loading snapshot")
pm.applyRaftSnapshot(*raftSnapshot)
return raftSnapshot

9
vendor/github.com/coreos/etcd/.gitignore generated vendored
View File

@ -1,14 +1,21 @@
/agent-*
/coverage
/covdir
/docs
/vendor
/gopath
/gopath.proto
/go-bindata
/release
/machine*
/bin
.vagrant
*.etcd
*.log
/etcd
*.swp
/hack/insta-discovery/.env
*.test
tools/functional-tester/docker/bin
hack/tls-setup/certs
.idea
*.bak

84
vendor/github.com/coreos/etcd/.travis.yml generated vendored
View File

@ -1,61 +1,73 @@
dist: trusty
language: go
go_import_path: github.com/coreos/etcd
sudo: false
sudo: required
services: docker
go:
- 1.7.4
- tip
- 1.10.8
notifications:
on_success: never
on_failure: never
env:
matrix:
- TARGET=amd64
- TARGET=arm64
- TARGET=arm
- TARGET=386
- TARGET=ppc64le
- TARGET=linux-amd64-integration
- TARGET=linux-amd64-functional
- TARGET=linux-amd64-unit
- TARGET=all-build
- TARGET=linux-386-unit
matrix:
fast_finish: true
allow_failures:
- go: tip
- go: 1.10.8
env: TARGET=linux-386-unit
exclude:
- go: tip
env: TARGET=arm
- go: tip
env: TARGET=arm64
- go: tip
env: TARGET=386
- go: tip
env: TARGET=ppc64le
addons:
apt:
packages:
- libpcap-dev
- libaspell-dev
- libhunspell-dev
env: TARGET=linux-386-unit
before_install:
- go get -v github.com/chzchzchz/goword
- go get -v honnef.co/go/simple/cmd/gosimple
- go get -v honnef.co/go/unused/cmd/unused
- if [[ $TRAVIS_GO_VERSION == 1.* ]]; then docker pull gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION}; fi
# disable godep restore override
install:
- pushd cmd/etcd && go get -t -v ./... && popd
- pushd cmd/etcd && go get -t -v ./... && popd
script:
- echo "TRAVIS_GO_VERSION=${TRAVIS_GO_VERSION}"
- >
case "${TARGET}" in
amd64)
GOARCH=amd64 ./test
linux-amd64-integration)
docker run --rm \
--volume=`pwd`:/go/src/github.com/coreos/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "cd /go/src/github.com/coreos/etcd; GOARCH=amd64 PASSES='integration' ./test"
;;
386)
GOARCH=386 PASSES="build unit" ./test
linux-amd64-functional)
docker run --rm \
--volume=`pwd`:/go/src/github.com/coreos/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "cd /go/src/github.com/coreos/etcd; ./build && GOARCH=amd64 PASSES='functional' ./test"
;;
*)
# test building out of gopath
GO_BUILD_FLAGS="-a -v" GOPATH="" GOARCH="${TARGET}" ./build
linux-amd64-unit)
docker run --rm \
--volume=`pwd`:/go/src/github.com/coreos/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "cd /go/src/github.com/coreos/etcd; GOARCH=amd64 PASSES='unit' ./test"
;;
all-build)
docker run --rm \
--volume=`pwd`:/go/src/github.com/coreos/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "cd /go/src/github.com/coreos/etcd; GOARCH=amd64 PASSES='build' ./test \
&& GOARCH=386 PASSES='build' ./test \
&& GO_BUILD_FLAGS='-v' GOOS=darwin GOARCH=amd64 ./build \
&& GO_BUILD_FLAGS='-v' GOOS=windows GOARCH=amd64 ./build \
&& GO_BUILD_FLAGS='-v' GOARCH=arm ./build \
&& GO_BUILD_FLAGS='-v' GOARCH=arm64 ./build \
&& GO_BUILD_FLAGS='-v' GOARCH=ppc64le ./build"
;;
linux-386-unit)
docker run --rm \
--volume=`pwd`:/go/src/github.com/coreos/etcd gcr.io/etcd-development/etcd-test:go${TRAVIS_GO_VERSION} \
/bin/bash -c "cd /go/src/github.com/coreos/etcd; GOARCH=386 PASSES='unit' ./test"
;;
esac

44
vendor/github.com/coreos/etcd/.words generated vendored Normal file
View File

@ -0,0 +1,44 @@
DefaultMaxRequestBytes
ErrCodeEnhanceYourCalm
ErrTimeout
GoAway
KeepAlive
Keepalive
MiB
ResourceExhausted
RPC
RPCs
TODO
backoff
blackhole
blackholed
cancelable
cancelation
cluster_proxy
defragment
defragmenting
etcd
gRPC
goroutine
goroutines
healthcheck
iff
inflight
keepalive
keepalives
keyspace
linearization
localhost
mutex
prefetching
protobuf
prometheus
rafthttp
repin
serializable
teardown
too_many_pings
uncontended
unprefixed
unlisting

746
vendor/github.com/coreos/etcd/CHANGELOG.md generated vendored Normal file
View File

@ -0,0 +1,746 @@
## [v3.3.0](https://github.com/coreos/etcd/releases/tag/v3.3.0)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.0...v3.3.0) and [v3.3 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_3.md) for any breaking changes.
### Improved
- Use [`coreos/bbolt`](https://github.com/coreos/bbolt/releases) to replace [`boltdb/bolt`](https://github.com/boltdb/bolt#project-status).
- Fix [etcd database size grows until `mvcc: database space exceeded`](https://github.com/coreos/etcd/issues/8009).
- [Reduce memory allocation](https://github.com/coreos/etcd/pull/8428) on [Range operations](https://github.com/coreos/etcd/pull/8475).
- [Rate limit](https://github.com/coreos/etcd/pull/8099) and [randomize](https://github.com/coreos/etcd/pull/8101) lease revoke on restart or leader elections.
- Prevent [spikes in Raft proposal rate](https://github.com/coreos/etcd/issues/8096).
- Support `clientv3` balancer failover under [network faults/partitions](https://github.com/coreos/etcd/issues/8711).
- Better warning on [mismatched `--initial-cluster`](https://github.com/coreos/etcd/pull/8083) flag.
### Changed(Breaking Changes)
- Require [Go 1.9+](https://github.com/coreos/etcd/issues/6174).
- Compile with *Go 1.9.2*.
- Deprecate [`golang.org/x/net/context`](https://github.com/coreos/etcd/pull/8511).
- Require [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) [**`v1.7.4`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.4) or [**`v1.7.5+`**](https://github.com/grpc/grpc-go/releases/tag/v1.7.5):
- Deprecate [`metadata.Incoming/OutgoingContext`](https://github.com/coreos/etcd/pull/7896).
- Deprecate `grpclog.Logger`, upgrade to [`grpclog.LoggerV2`](https://github.com/coreos/etcd/pull/8533).
- Deprecate [`grpc.ErrClientConnTimeout`](https://github.com/coreos/etcd/pull/8505) errors in `clientv3`.
- Use [`MaxRecvMsgSize` and `MaxSendMsgSize`](https://github.com/coreos/etcd/pull/8437) to limit message size, in etcd server.
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) `v1.2.2` to `v1.3.0`.
- Translate [gRPC status error in v3 client `Snapshot` API](https://github.com/coreos/etcd/pull/9038).
- Upgrade [`github.com/ugorji/go/codec`](https://github.com/ugorji/go) for v2 `client`.
- [Regenerated](https://github.com/coreos/etcd/pull/8721) v2 `client` source code with latest `ugorji/go/codec`.
- Fix [`/health` endpoint JSON output](https://github.com/coreos/etcd/pull/8312).
- v3 `etcdctl` [`lease timetolive LEASE_ID`](https://github.com/coreos/etcd/issues/9028) on expired lease now prints [`lease LEASE_ID already expired`](https://github.com/coreos/etcd/pull/9047).
- <=3.2 prints `lease LEASE_ID granted with TTL(0s), remaining(-1s)`.
### Added(`etcd`)
- Add [`--experimental-enable-v2v3`](https://github.com/coreos/etcd/pull/8407) flag to [emulate v2 API with v3](https://github.com/coreos/etcd/issues/6925).
- Add [`--experimental-corrupt-check-time`](https://github.com/coreos/etcd/pull/8420) flag to [raise corrupt alarm monitoring](https://github.com/coreos/etcd/issues/7125).
- Add [`--experimental-initial-corrupt-check`](https://github.com/coreos/etcd/pull/8554) flag to [check database hash before serving client/peer traffic](https://github.com/coreos/etcd/issues/8313).
- Add [`--max-txn-ops`](https://github.com/coreos/etcd/pull/7976) flag to [configure maximum number operations in transaction](https://github.com/coreos/etcd/issues/7826).
- Add [`--max-request-bytes`](https://github.com/coreos/etcd/pull/7968) flag to [configure maximum client request size](https://github.com/coreos/etcd/issues/7923).
- If not configured, it defaults to 1.5 MiB.
- Add [`--client-crl-file`, `--peer-crl-file`](https://github.com/coreos/etcd/pull/8124) flags for [Certificate revocation list](https://github.com/coreos/etcd/issues/4034).
- Add [`--peer-require-cn`](https://github.com/coreos/etcd/pull/8616) flag to support [CN-based auth for inter-peer connection](https://github.com/coreos/etcd/issues/8262).
- Add [`--listen-metrics-urls`](https://github.com/coreos/etcd/pull/8242) flag for additional `/metrics` endpoints.
- Support [additional (non) TLS `/metrics` endpoints for a TLS-enabled cluster](https://github.com/coreos/etcd/pull/8282).
- e.g. `--listen-metrics-urls=https://localhost:2378,http://localhost:9379` to serve `/metrics` in secure port 2378 and insecure port 9379.
- Useful for [bypassing critical APIs when monitoring etcd](https://github.com/coreos/etcd/issues/8060).
- Add [`--auto-compaction-mode`](https://github.com/coreos/etcd/pull/8123) flag to [support revision-based compaction](https://github.com/coreos/etcd/issues/8098).
- Change `--auto-compaction-retention` flag to [accept string values](https://github.com/coreos/etcd/pull/8563) with [finer granularity](https://github.com/coreos/etcd/issues/8503).
- Add [`--grpc-keepalive-min-time`, `--grpc-keepalive-interval`, `--grpc-keepalive-timeout`](https://github.com/coreos/etcd/pull/8535) flags to configure server-side keepalive policies.
- Serve [`/health` endpoint as unhealthy](https://github.com/coreos/etcd/pull/8272) when [alarm is raised](https://github.com/coreos/etcd/issues/8207).
- Provide [error information in `/health`](https://github.com/coreos/etcd/pull/8312).
- e.g. `{"health":false,"errors":["NOSPACE"]}`.
- Move [logging setup to embed package](https://github.com/coreos/etcd/pull/8810)
- Disable gRPC server log by default.
- Use [monotonic time in Go 1.9](https://github.com/coreos/etcd/pull/8507) for `lease` package.
- Warn on [empty hosts in advertise URLs](https://github.com/coreos/etcd/pull/8384).
- Address [advertise client URLs accepts empty hosts](https://github.com/coreos/etcd/issues/8379).
- etcd `v3.4` will exit on this error.
- e.g. `--advertise-client-urls=http://:2379`.
- Warn on [shadowed environment variables](https://github.com/coreos/etcd/pull/8385).
- Address [error on shadowed environment variables](https://github.com/coreos/etcd/issues/8380).
- etcd `v3.4` will exit on this error.
### Added(API)
- Support [ranges in transaction comparisons](https://github.com/coreos/etcd/pull/8025) for [disconnected linearized reads](https://github.com/coreos/etcd/issues/7924).
- Add [nested transactions](https://github.com/coreos/etcd/pull/8102) to extend [proxy use cases](https://github.com/coreos/etcd/issues/7857).
- Add [lease comparison target in transaction](https://github.com/coreos/etcd/pull/8324).
- Add [lease list](https://github.com/coreos/etcd/pull/8358).
- Add [hash by revision](https://github.com/coreos/etcd/pull/8263) for [better corruption checking against boltdb](https://github.com/coreos/etcd/issues/8016).
### Added(`etcd/clientv3`)
- Add [health balancer](https://github.com/coreos/etcd/pull/8545) to fix [watch API hangs](https://github.com/coreos/etcd/issues/7247), improve [endpoint switch under network faults](https://github.com/coreos/etcd/issues/7941).
- [Refactor balancer](https://github.com/coreos/etcd/pull/8840) and add [client-side keepalive pings](https://github.com/coreos/etcd/pull/8199) to handle [network partitions](https://github.com/coreos/etcd/issues/8711).
- Add [`MaxCallSendMsgSize` and `MaxCallRecvMsgSize`](https://github.com/coreos/etcd/pull/9047) fields to [`clientv3.Config`](https://godoc.org/github.com/coreos/etcd/clientv3#Config).
- Fix [exceeded response size limit error in client-side](https://github.com/coreos/etcd/issues/9043).
- Address [kubernetes#51099](https://github.com/kubernetes/kubernetes/issues/51099).
- `MaxCallSendMsgSize` default value is 2 MiB, if not configured.
- `MaxCallRecvMsgSize` default value is `math.MaxInt32`, if not configured.
- Accept [`Compare_LEASE`](https://github.com/coreos/etcd/pull/8324) in [`clientv3.Compare`](https://godoc.org/github.com/coreos/etcd/clientv3#Compare).
- Add [`LeaseValue` helper](https://github.com/coreos/etcd/pull/8488) to `Cmp` `LeaseID` values in `Txn`.
- Add [`MoveLeader`](https://github.com/coreos/etcd/pull/8153) to `Maintenance`.
- Add [`HashKV`](https://github.com/coreos/etcd/pull/8351) to `Maintenance`.
- Add [`Leases`](https://github.com/coreos/etcd/pull/8358) to `Lease`.
- Add [`clientv3/ordering`](https://github.com/coreos/etcd/pull/8092) for enforce [ordering in serialized requests](https://github.com/coreos/etcd/issues/7623).
### Added(v2 `etcdctl`)
- Add [`backup --with-v3`](https://github.com/coreos/etcd/pull/8479) flag.
### Added(v3 `etcdctl`)
- Add [`--discovery-srv`](https://github.com/coreos/etcd/pull/8462) flag.
- Add [`--keepalive-time`, `--keepalive-timeout`](https://github.com/coreos/etcd/pull/8663) flags.
- Add [`lease list`](https://github.com/coreos/etcd/pull/8358) command.
- Add [`lease keep-alive --once`](https://github.com/coreos/etcd/pull/8775) flag.
- Make [`lease timetolive LEASE_ID`](https://github.com/coreos/etcd/issues/9028) on expired lease print [`lease LEASE_ID already expired`](https://github.com/coreos/etcd/pull/9047).
- <=3.2 prints `lease LEASE_ID granted with TTL(0s), remaining(-1s)`.
- Add [`defrag --data-dir`](https://github.com/coreos/etcd/pull/8367) flag.
- Add [`move-leader`](https://github.com/coreos/etcd/pull/8153) command.
- Add [`endpoint hashkv`](https://github.com/coreos/etcd/pull/8351) command.
- Add [`endpoint --cluster`](https://github.com/coreos/etcd/pull/8143) flag, equivalent to [v2 `etcdctl cluster-health`](https://github.com/coreos/etcd/issues/8117).
- Make `endpoint health` command terminate with [non-zero exit code on unhealthy status](https://github.com/coreos/etcd/pull/8342).
- Add [`lock --ttl`](https://github.com/coreos/etcd/pull/8370) flag.
- Support [`watch [key] [range_end] -- [exec-command…]`](https://github.com/coreos/etcd/pull/8919), equivalent to [v2 `etcdctl exec-watch`](https://github.com/coreos/etcd/issues/8814).
- Enable [`clientv3.WithRequireLeader(context.Context)` for `watch`](https://github.com/coreos/etcd/pull/8672) command.
- Print [`"del"` instead of `"delete"`](https://github.com/coreos/etcd/pull/8297) in `txn` interactive mode.
- Print [`ETCD_INITIAL_ADVERTISE_PEER_URLS` in `member add`](https://github.com/coreos/etcd/pull/8332).
### Added(metrics)
- Add [`etcd --listen-metrics-urls`](https://github.com/coreos/etcd/pull/8242) flag for additional `/metrics` endpoints.
- Useful for [bypassing critical APIs when monitoring etcd](https://github.com/coreos/etcd/issues/8060).
- Add [`etcd_server_version`](https://github.com/coreos/etcd/pull/8960) Prometheus metric.
- To replace [Kubernetes `etcd-version-monitor`](https://github.com/coreos/etcd/issues/8948).
- Add [`etcd_debugging_mvcc_db_compaction_keys_total`](https://github.com/coreos/etcd/pull/8280) Prometheus metric.
- Add [`etcd_debugging_server_lease_expired_total`](https://github.com/coreos/etcd/pull/8064) Prometheus metric.
- To improve [lease revoke monitoring](https://github.com/coreos/etcd/issues/8050).
- Document [Prometheus 2.0 rules](https://github.com/coreos/etcd/pull/8879).
- Initialize gRPC server [metrics with zero values](https://github.com/coreos/etcd/pull/8878).
### Added(`grpc-proxy`)
- Add [`grpc-proxy start --experimental-leasing-prefix`](https://github.com/coreos/etcd/pull/8341) flag:
- For disconnected linearized reads.
- Based on [V system leasing](https://github.com/coreos/etcd/issues/6065).
- See ["Disconnected consistent reads with etcd" blog post](https://coreos.com/blog/coreos-labs-disconnected-consistent-reads-with-etcd).
- Add [`grpc-proxy start --experimental-serializable-ordering`](https://github.com/coreos/etcd/pull/8315) flag.
- To ensure serializable reads have monotonically increasing store revisions across endpoints.
- Add [`grpc-proxy start --metrics-addr`](https://github.com/coreos/etcd/pull/8242) flag for an additional `/metrics` endpoint.
- Set `--metrics-addr=http://[HOST]:9379` to serve `/metrics` in insecure port 9379.
- Serve [`/health` endpoint in grpc-proxy](https://github.com/coreos/etcd/pull/8322).
- Add [`grpc-proxy start --debug`](https://github.com/coreos/etcd/pull/8994) flag.
### Added(gRPC gateway)
- Replace [gRPC gateway](https://github.com/grpc-ecosystem/grpc-gateway) endpoint with [`/v3beta`](https://github.com/coreos/etcd/pull/8880).
- To deprecate [`/v3alpha`](https://github.com/coreos/etcd/issues/8125) in `v3.4`.
- Support ["authorization" token](https://github.com/coreos/etcd/pull/7999).
- Support [websocket for bi-directional streams](https://github.com/coreos/etcd/pull/8257).
- Fix [`Watch` API with gRPC gateway](https://github.com/coreos/etcd/issues/8237).
- Upgrade gRPC gateway to [v1.3.0](https://github.com/coreos/etcd/issues/8838).
### Added(`etcd/raft`)
- Add [non-voting member](https://github.com/coreos/etcd/pull/8751).
- To implement [Raft thesis 4.2.1 Catching up new servers](https://github.com/coreos/etcd/issues/8568).
- `Learner` node does not vote or promote itself.
### Added/Fixed(Security/Auth)
- Add [CRL based connection rejection](https://github.com/coreos/etcd/pull/8124) to manage [revoked certs](https://github.com/coreos/etcd/issues/4034).
- Document [TLS authentication changes](https://github.com/coreos/etcd/pull/8895):
- [Server accepts connections if IP matches, without checking DNS entries](https://github.com/coreos/etcd/pull/8223). For instance, if peer cert contains IP addresses and DNS names in Subject Alternative Name (SAN) field, and the remote IP address matches one of those IP addresses, server just accepts connection without further checking the DNS names.
- [Server supports reverse-lookup on wildcard DNS `SAN`](https://github.com/coreos/etcd/pull/8281). For instance, if peer cert contains only DNS names (no IP addresses) in Subject Alternative Name (SAN) field, server first reverse-lookups the remote IP address to get a list of names mapping to that address (e.g. `nslookup IPADDR`). Then accepts the connection if those names have a matching name with peer cert's DNS names (either by exact or wildcard match). If none is matched, server forward-lookups each DNS entry in peer cert (e.g. look up `example.default.svc` when the entry is `*.example.default.svc`), and accepts connection only when the host's resolved addresses have the matching IP address with the peer's remote IP address.
- Add [`etcd --peer-require-cn`](https://github.com/coreos/etcd/pull/8616) flag.
- To support [CommonName(CN) based auth](https://github.com/coreos/etcd/issues/8262) for inter peer connection.
- [Swap priority](https://github.com/coreos/etcd/pull/8594) of cert CommonName(CN) and username + password.
- To address ["username and password specified in the request should take priority over CN in the cert"](https://github.com/coreos/etcd/issues/8584).
- Protect [lease revoke with auth](https://github.com/coreos/etcd/pull/8031).
- Provide user's role on [auth permission error](https://github.com/coreos/etcd/pull/8164).
- Fix [auth store panic with disabled token](https://github.com/coreos/etcd/pull/8695).
- Update `golang.org/x/crypto/bcrypt` (see [golang/crypto@6c586e1](https://github.com/golang/crypto/commit/6c586e17d90a7d08bbbc4069984180dce3b04117)).
### Fixed(v2)
- [Fail-over v2 client](https://github.com/coreos/etcd/pull/8519) to next endpoint on [oneshot failure](https://github.com/coreos/etcd/issues/8515).
- [Put back `/v2/machines`](https://github.com/coreos/etcd/pull/8062) endpoint for python-etcd wrapper.
### Fixed(v3)
- Fix [range/put/delete operation metrics](https://github.com/coreos/etcd/pull/8054) with transaction:
- `etcd_debugging_mvcc_range_total`
- `etcd_debugging_mvcc_put_total`
- `etcd_debugging_mvcc_delete_total`
- `etcd_debugging_mvcc_txn_total`
- Fix [`etcd_debugging_mvcc_keys_total`](https://github.com/coreos/etcd/pull/8390) on restore.
- Fix [`etcd_debugging_mvcc_db_total_size_in_bytes`](https://github.com/coreos/etcd/pull/8120) on restore.
- Also change to [`prometheus.NewGaugeFunc`](https://github.com/coreos/etcd/pull/8150).
- Fix [backend database in-memory index corruption](https://github.com/coreos/etcd/pull/8127) issue on restore (only 3.2.0 is affected).
- Fix [watch restore from snapshot](https://github.com/coreos/etcd/pull/8427).
- Fix ["put at-most-once" in `clientv3`](https://github.com/coreos/etcd/pull/8335).
- Handle [empty key permission](https://github.com/coreos/etcd/pull/8514) in `etcdctl`.
- [Fix server crash](https://github.com/coreos/etcd/pull/8010) on [invalid transaction request from gRPC gateway](https://github.com/coreos/etcd/issues/7889).
- Fix [`clientv3.WatchResponse.Canceled`](https://github.com/coreos/etcd/pull/8283) on [compacted watch request](https://github.com/coreos/etcd/issues/8231).
- Handle [WAL renaming failure on Windows](https://github.com/coreos/etcd/pull/8286).
- Make [peer dial timeout longer](https://github.com/coreos/etcd/pull/8599).
- See [coreos/etcd-operator#1300](https://github.com/coreos/etcd-operator/issues/1300) for more detail.
- Make server [wait up to request time-out](https://github.com/coreos/etcd/pull/8267) with [pending RPCs](https://github.com/coreos/etcd/issues/8224).
- Fix [`grpc.Server` panic on `GracefulStop`](https://github.com/coreos/etcd/pull/8987) with [TLS-enabled server](https://github.com/coreos/etcd/issues/8916).
- Fix ["multiple peer URLs cannot start" issue](https://github.com/coreos/etcd/issues/8383).
- Fix server-side auth so [concurrent auth operations do not return old revision error](https://github.com/coreos/etcd/pull/8442).
- Fix [`concurrency/stm` `Put` with serializable snapshot](https://github.com/coreos/etcd/pull/8439).
- Use store revision from first fetch to resolve write conflicts instead of modified revision.
- Fix [`grpc-proxy` Snapshot API error handling](https://github.com/coreos/etcd/commit/dbd16d52fbf81e5fd806d21ff5e9148d5bf203ab).
- Fix [`grpc-proxy` KV API `PrevKv` flag handling](https://github.com/coreos/etcd/pull/8366).
- Fix [`grpc-proxy` KV API `KeysOnly` flag handling](https://github.com/coreos/etcd/pull/8552).
- Upgrade [`coreos/go-systemd`](https://github.com/coreos/go-systemd/releases) to `v15` (see https://github.com/coreos/go-systemd/releases/tag/v15).
### Other
- Support previous two minor versions (see our [new release policy](https://github.com/coreos/etcd/pull/8805)).
- `v3.3.x` is the last release cycle that supports `ACI`:
- AppC was [officially suspended](https://github.com/appc/spec#-disclaimer-), as of late 2016.
- [`acbuild`](https://github.com/containers/build#this-project-is-currently-unmaintained) is not maintained anymore.
- `*.aci` files won't be available from etcd `v3.4` release.
- Add container registry [`gcr.io/etcd-development/etcd`](https://gcr.io/etcd-development/etcd).
- [quay.io/coreos/etcd](https://quay.io/coreos/etcd) is still supported as secondary.
## [v3.2.12](https://github.com/coreos/etcd/releases/tag/v3.2.12) (2017-12-20)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.11...v3.2.12) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Fix [error message of `Revision` compactor](https://github.com/coreos/etcd/pull/8999) in server-side.
### Added(`etcd/clientv3`,`etcdctl/v3`)
- Add [`MaxCallSendMsgSize` and `MaxCallRecvMsgSize`](https://github.com/coreos/etcd/pull/9047) fields to [`clientv3.Config`](https://godoc.org/github.com/coreos/etcd/clientv3#Config).
- Fix [exceeded response size limit error in client-side](https://github.com/coreos/etcd/issues/9043).
- Address [kubernetes#51099](https://github.com/kubernetes/kubernetes/issues/51099).
- `MaxCallSendMsgSize` default value is 2 MiB, if not configured.
- `MaxCallRecvMsgSize` default value is `math.MaxInt32`, if not configured.
### Other
- Pin [grpc v1.7.5](https://github.com/grpc/grpc-go/releases/tag/v1.7.5), [grpc-gateway v1.3.0](https://github.com/grpc-ecosystem/grpc-gateway/releases/tag/v1.3.0).
- No code change, just to be explicit about recommended versions.
## [v3.2.11](https://github.com/coreos/etcd/releases/tag/v3.2.11) (2017-12-05)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.10...v3.2.11) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Fix racey grpc-go's server handler transport `WriteStatus` call to prevent [TLS-enabled etcd server crash](https://github.com/coreos/etcd/issues/8904):
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) `v1.7.3` to `v1.7.4`.
- Add [gRPC RPC failure warnings](https://github.com/coreos/etcd/pull/8939) to help debug such issues in the future.
- Remove `--listen-metrics-urls` flag in monitoring document (non-released in `v3.2.x`, planned for `v3.3.x`).
### Added
- Provide [more cert details](https://github.com/coreos/etcd/pull/8952/files) on TLS handshake failures.
## [v3.1.11](https://github.com/coreos/etcd/releases/tag/v3.1.11) (2017-11-28)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.10...v3.1.11) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- [#8411](https://github.com/coreos/etcd/issues/8411),[#8806](https://github.com/coreos/etcd/pull/8806) mvcc: fix watch restore from snapshot
- [#8009](https://github.com/coreos/etcd/issues/8009),[#8902](https://github.com/coreos/etcd/pull/8902) backport coreos/bbolt v1.3.1-coreos.5
## [v3.2.10](https://github.com/coreos/etcd/releases/tag/v3.2.10) (2017-11-16)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.9...v3.2.10) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Replace backend key-value database `boltdb/bolt` with [`coreos/bbolt`](https://github.com/coreos/bbolt/releases) to address [backend database size issue](https://github.com/coreos/etcd/issues/8009).
- Fix `clientv3` balancer to handle [network partitions](https://github.com/coreos/etcd/issues/8711):
- Upgrade [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) `v1.2.1` to `v1.7.3`.
- Upgrade [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) `v1.2` to `v1.3`.
- Revert [discovery SRV auth `ServerName` with `*.{ROOT_DOMAIN}`](https://github.com/coreos/etcd/pull/8651) to support non-wildcard subject alternative names in the certs (see [issue #8445](https://github.com/coreos/etcd/issues/8445) for more contexts).
- For instance, `etcd --discovery-srv=etcd.local` will only authenticate peers/clients when the provided certs have root domain `etcd.local` (**not `*.etcd.local`**) as an entry in Subject Alternative Name (SAN) field.
## [v3.2.9](https://github.com/coreos/etcd/releases/tag/v3.2.9) (2017-10-06)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.8...v3.2.9) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed(Security)
- Compile with [Go 1.8.4](https://groups.google.com/d/msg/golang-nuts/sHfMg4gZNps/a-HDgDDDAAAJ).
- Update `golang.org/x/crypto/bcrypt` (see [golang/crypto@6c586e1](https://github.com/golang/crypto/commit/6c586e17d90a7d08bbbc4069984180dce3b04117)).
- Fix discovery SRV bootstrapping to [authenticate `ServerName` with `*.{ROOT_DOMAIN}`](https://github.com/coreos/etcd/pull/8651), in order to support sub-domain wildcard matching (see [issue #8445](https://github.com/coreos/etcd/issues/8445) for more contexts).
- For instance, `etcd --discovery-srv=etcd.local` will only authenticate peers/clients when the provided certs have root domain `*.etcd.local` as an entry in Subject Alternative Name (SAN) field.
## [v3.2.8](https://github.com/coreos/etcd/releases/tag/v3.2.8) (2017-09-29)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.7...v3.2.8) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Fix v2 client failover to next endpoint on mutable operation.
- Fix grpc-proxy to respect `KeysOnly` flag.
## [v3.2.7](https://github.com/coreos/etcd/releases/tag/v3.2.7) (2017-09-01)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.6...v3.2.7) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Fix server-side auth so concurrent auth operations do not return old revision error.
- Fix concurrency/stm Put with serializable snapshot
- Use store revision from first fetch to resolve write conflicts instead of modified revision.
## [v3.2.6](https://github.com/coreos/etcd/releases/tag/v3.2.6) (2017-08-21)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.5...v3.2.6).
### Fixed
- Fix watch restore from snapshot.
- Fix `etcd_debugging_mvcc_keys_total` inconsistency.
- Fix multiple URLs for `--listen-peer-urls` flag.
- Add `--enable-pprof` flag to etcd configuration file format.
## [v3.2.5](https://github.com/coreos/etcd/releases/tag/v3.2.5) (2017-08-04)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.4...v3.2.5) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Changed
- Use reverse lookup to match wildcard DNS SAN.
- Return non-zero exit code on unhealthy `endpoint health`.
### Fixed
- Fix unreachable /metrics endpoint when `--enable-v2=false`.
- Fix grpc-proxy to respect `PrevKv` flag.
### Added
- Add container registry `gcr.io/etcd-development/etcd`.
## [v3.2.4](https://github.com/coreos/etcd/releases/tag/v3.2.4) (2017-07-19)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.3...v3.2.4) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Do not block on active client stream when stopping server
- Fix gRPC proxy Snapshot RPC error handling
## [v3.2.3](https://github.com/coreos/etcd/releases/tag/v3.2.3) (2017-07-14)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.2...v3.2.3) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Let clients establish unlimited streams
### Added
- Tag docker images with minor versions
- e.g. `docker pull quay.io/coreos/etcd:v3.2` to fetch latest v3.2 versions
## [v3.1.10](https://github.com/coreos/etcd/releases/tag/v3.1.10) (2017-07-14)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.9...v3.1.10) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Changed
- Compile with Go 1.8.3 to fix panic on `net/http.CloseNotify`
### Added
- Tag docker images with minor versions.
- e.g. `docker pull quay.io/coreos/etcd:v3.1` to fetch latest v3.1 versions.
## [v3.2.2](https://github.com/coreos/etcd/releases/tag/v3.2.2) (2017-07-07)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.1...v3.2.2) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Improved
- Rate-limit lease revoke on expiration.
- Extend leases on promote to avoid queueing effect on lease expiration.
### Fixed
- Use user-provided listen address to connect to gRPC gateway:
- `net.Listener` rewrites IPv4 0.0.0.0 to IPv6 [::], breaking IPv6 disabled hosts.
- Only v3.2.0, v3.2.1 are affected.
- Accept connection with matched IP SAN but no DNS match.
- Don't check DNS entries in certs if there's a matching IP.
- Fix 'tools/benchmark' watch command.
## [v3.2.1](https://github.com/coreos/etcd/releases/tag/v3.2.1) (2017-06-23)
See [code changes](https://github.com/coreos/etcd/compare/v3.2.0...v3.2.1) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Fixed
- Fix backend database in-memory index corruption issue on restore (only 3.2.0 is affected).
- Fix gRPC gateway Txn marshaling issue.
- Fix backend database size debugging metrics.
## [v3.2.0](https://github.com/coreos/etcd/releases/tag/v3.2.0) (2017-06-09)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.0...v3.2.0) and [v3.2 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_2.md) for any breaking changes.
### Improved
- Improve backend read concurrency.
### Added
- Embedded etcd
- `Etcd.Peers` field is now `[]*peerListener`.
- RPCs
- Add Election, Lock service.
- Native client etcdserver/api/v3client
- client "embedded" in the server.
- gRPC proxy
- Proxy endpoint discovery.
- Namespaces.
- Coalesce lease requests.
- v3 client
- STM prefetching.
- Add namespace feature.
- Add `ErrOldCluster` with server version checking.
- Translate `WithPrefix()` into `WithFromKey()` for empty key.
- v3 etcdctl
- Add `check perf` command.
- Add `--from-key` flag to role grant-permission command.
- `lock` command takes an optional command to execute.
- etcd flags
- Add `--enable-v2` flag to configure v2 backend (enabled by default).
- Add `--auth-token` flag.
- `etcd gateway`
- Support DNS SRV priority.
- Auth
- Support Watch API.
- JWT tokens.
- Logging, monitoring
- Server warns large snapshot operations.
- Add `etcd_debugging_server_lease_expired_total` metrics.
- Security
- Deny incoming peer certs with wrong IP SAN.
- Resolve TLS `DNSNames` when SAN checking.
- Reload TLS certificates on every client connection.
- Release
- Annotate acbuild with supports-systemd-notify.
- Add `nsswitch.conf` to Docker container image.
- Add ppc64le, arm64(experimental) builds.
- Compile with `Go 1.8.3`.
### Changed
- v3 client
- `LeaseTimeToLive` returns TTL=-1 resp on lease not found.
- `clientv3.NewFromConfigFile` is moved to `clientv3/yaml.NewConfig`.
- concurrency package's elections updated to match RPC interfaces.
- let client dial endpoints not in the balancer.
- Dependencies
- Update [`google.golang.org/grpc`](https://github.com/grpc/grpc-go/releases) to `v1.2.1`.
- Update [`github.com/grpc-ecosystem/grpc-gateway`](https://github.com/grpc-ecosystem/grpc-gateway/releases) to `v1.2.0`.
### Fixed
- Allow v2 snapshot over 512MB.
## [v3.1.9](https://github.com/coreos/etcd/releases/tag/v3.1.9) (2017-06-09)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.8...v3.1.9) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Fixed
- Allow v2 snapshot over 512MB.
## [v3.1.8](https://github.com/coreos/etcd/releases/tag/v3.1.8) (2017-05-19)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.7...v3.1.8) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
## [v3.1.7](https://github.com/coreos/etcd/releases/tag/v3.1.7) (2017-04-28)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.6...v3.1.7) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
## [v3.1.6](https://github.com/coreos/etcd/releases/tag/v3.1.6) (2017-04-19)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.5...v3.1.6) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Changed
- Remove auth check in Status API.
### Fixed
- Fill in Auth API response header.
## [v3.1.5](https://github.com/coreos/etcd/releases/tag/v3.1.5) (2017-03-27)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.4...v3.1.5) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Added
- Add `/etc/nsswitch.conf` file to alpine-based Docker image.
### Fixed
- Fix raft memory leak issue.
- Fix Windows file path issues.
## [v3.1.4](https://github.com/coreos/etcd/releases/tag/v3.1.4) (2017-03-22)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.3...v3.1.4) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
## [v3.1.3](https://github.com/coreos/etcd/releases/tag/v3.1.3) (2017-03-10)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.2...v3.1.3) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Changed
- Use machine default host when advertise URLs are default values(`localhost:2379,2380`) AND if listen URL is `0.0.0.0`.
### Fixed
- Fix `etcd gateway` schema handling in DNS discovery.
- Fix sd_notify behaviors in `gateway`, `grpc-proxy`.
## [v3.1.2](https://github.com/coreos/etcd/releases/tag/v3.1.2) (2017-02-24)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.1...v3.1.2) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Changed
- Use IPv4 default host, by default (when IPv4 and IPv6 are available).
### Fixed
- Fix `etcd gateway` with multiple endpoints.
## [v3.1.1](https://github.com/coreos/etcd/releases/tag/v3.1.1) (2017-02-17)
See [code changes](https://github.com/coreos/etcd/compare/v3.1.0...v3.1.1) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Changed
- Compile with `Go 1.7.5`.
## [v2.3.8](https://github.com/coreos/etcd/releases/tag/v2.3.8) (2017-02-17)
See [code changes](https://github.com/coreos/etcd/compare/v2.3.7...v2.3.8).
### Changed
- Compile with `Go 1.7.5`.
## [v3.1.0](https://github.com/coreos/etcd/releases/tag/v3.1.0) (2017-01-20)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.0...v3.1.0) and [v3.1 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_1.md) for any breaking changes.
### Improved
- Faster linearizable reads (implements Raft read-index).
- v3 authentication API is now stable.
### Added
- Automatic leadership transfer when leader steps down.
- etcd flags
- `--strict-reconfig-check` flag is set by default.
- Add `--log-output` flag.
- Add `--metrics` flag.
- v3 client
- Add `SetEndpoints` method; update endpoints at runtime.
- Add `Sync` method; auto-update endpoints at runtime.
- Add `Lease TimeToLive` API; fetch lease information.
- replace Config.Logger field with global logger.
- Get API responses are sorted in ascending order by default.
- v3 etcdctl
- Add `lease timetolive` command.
- Add `--print-value-only` flag to get command.
- Add `--dest-prefix` flag to make-mirror command.
- `get` command responses are sorted in ascending order by default.
- `recipes` now conform to sessions defined in `clientv3/concurrency`.
- ACI has symlinks to `/usr/local/bin/etcd*`.
- Experimental gRPC proxy feature.
### Changed
- Deprecated following gRPC metrics in favor of [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus):
- `etcd_grpc_requests_total`
- `etcd_grpc_requests_failed_total`
- `etcd_grpc_active_streams`
- `etcd_grpc_unary_requests_duration_seconds`
- etcd uses default route IP if advertise URL is not given.
- Cluster rejects removing members if quorum will be lost.
- SRV records (e.g., infra1.example.com) must match the discovery domain (i.e., example.com) if no custom certificate authority is given.
- `TLSConfig.ServerName` is ignored with user-provided certificates for backwards compatibility; to be deprecated.
- For example, `etcd --discovery-srv=example.com` will only authenticate peers/clients when the provided certs have root domain `example.com` as an entry in Subject Alternative Name (SAN) field.
- Discovery now has upper limit for waiting on retries.
- Warn on binding listeners through domain names; to be deprecated.
## [v3.0.16](https://github.com/coreos/etcd/releases/tag/v3.0.16) (2016-11-13)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.15...v3.0.16) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
## [v3.0.15](https://github.com/coreos/etcd/releases/tag/v3.0.15) (2016-11-11)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.14...v3.0.15) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Fixed
- Fix cancel watch request with wrong range end.
## [v3.0.14](https://github.com/coreos/etcd/releases/tag/v3.0.14) (2016-11-04)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.13...v3.0.14) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Added
- v3 `etcdctl migrate` command now supports `--no-ttl` flag to discard keys on transform.
## [v3.0.13](https://github.com/coreos/etcd/releases/tag/v3.0.13) (2016-10-24)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.12...v3.0.13) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
## [v3.0.12](https://github.com/coreos/etcd/releases/tag/v3.0.12) (2016-10-07)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.11...v3.0.12) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
## [v3.0.11](https://github.com/coreos/etcd/releases/tag/v3.0.11) (2016-10-07)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.10...v3.0.11) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Added
- Server returns previous key-value (optional)
- `clientv3.WithPrevKV` option
- v3 etcdctl `put,watch,del --prev-kv` flag
## [v3.0.10](https://github.com/coreos/etcd/releases/tag/v3.0.10) (2016-09-23)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.9...v3.0.10) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
## [v3.0.9](https://github.com/coreos/etcd/releases/tag/v3.0.9) (2016-09-15)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.8...v3.0.9) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Added
- Warn on domain names on listen URLs (v3.2 will reject domain names).
## [v3.0.8](https://github.com/coreos/etcd/releases/tag/v3.0.8) (2016-09-09)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.7...v3.0.8) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Changed
- Allow only IP addresses in listen URLs (domain names are rejected).
## [v3.0.7](https://github.com/coreos/etcd/releases/tag/v3.0.7) (2016-08-31)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.6...v3.0.7) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Changed
- SRV records only allow A records (RFC 2052).
## [v3.0.6](https://github.com/coreos/etcd/releases/tag/v3.0.6) (2016-08-19)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.5...v3.0.6) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
## [v3.0.5](https://github.com/coreos/etcd/releases/tag/v3.0.5) (2016-08-19)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.4...v3.0.5) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Changed
- SRV records (e.g., infra1.example.com) must match the discovery domain (i.e., example.com) if no custom certificate authority is given.
## [v3.0.4](https://github.com/coreos/etcd/releases/tag/v3.0.4) (2016-07-27)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.3...v3.0.4) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Changed
- v2 auth can now use common name from TLS certificate when `--client-cert-auth` is enabled.
### Added
- v2 `etcdctl ls` command now supports `--output=json`.
- Add /var/lib/etcd directory to etcd official Docker image.
## [v3.0.3](https://github.com/coreos/etcd/releases/tag/v3.0.3) (2016-07-15)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.2...v3.0.3) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Changed
- Revert Dockerfile to use `CMD`, instead of `ENTRYPOINT`, to support `etcdctl` run.
- Docker commands for v3.0.2 won't work without specifying executable binary paths.
- v3 etcdctl default endpoints are now `127.0.0.1:2379`.
## [v3.0.2](https://github.com/coreos/etcd/releases/tag/v3.0.2) (2016-07-08)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.1...v3.0.2) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
### Changed
- Dockerfile uses `ENTRYPOINT`, instead of `CMD`, to run etcd without binary path specified.
## [v3.0.1](https://github.com/coreos/etcd/releases/tag/v3.0.1) (2016-07-01)
See [code changes](https://github.com/coreos/etcd/compare/v3.0.0...v3.0.1) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.
## [v3.0.0](https://github.com/coreos/etcd/releases/tag/v3.0.0) (2016-06-30)
See [code changes](https://github.com/coreos/etcd/compare/v2.3.0...v3.0.0) and [v3.0 upgrade guide](https://github.com/coreos/etcd/blob/master/Documentation/upgrades/upgrade_3_0.md) for any breaking changes.

63
vendor/github.com/coreos/etcd/CODE_OF_CONDUCT.md generated vendored Normal file
View File

@ -0,0 +1,63 @@
## CoreOS Community Code of Conduct
### Contributor Code of Conduct
As contributors and maintainers of this project, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating
documentation, submitting pull requests or patches, and other activities.
We are committed to making participation in this project a harassment-free
experience for everyone, regardless of level of experience, gender, gender
identity and expression, sexual orientation, disability, personal appearance,
body size, race, ethnicity, age, religion, or nationality.
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery
* Personal attacks
* Trolling or insulting/derogatory comments
* Public or private harassment
* Publishing others' private information, such as physical or electronic addresses, without explicit permission
* Other unethical or unprofessional conduct.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct. By adopting this Code of Conduct,
project maintainers commit themselves to fairly and consistently applying these
principles to every aspect of managing this project. Project maintainers who do
not follow or enforce the Code of Conduct may be permanently removed from the
project team.
This code of conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting a project maintainer, Brandon Philips
<brandon.philips@coreos.com>, and/or Meghan Schofield
<meghan.schofield@coreos.com>.
This Code of Conduct is adapted from the Contributor Covenant
(http://contributor-covenant.org), version 1.2.0, available at
http://contributor-covenant.org/version/1/2/0/
### CoreOS Events Code of Conduct
CoreOS events are working conferences intended for professional networking and
collaboration in the CoreOS community. Attendees are expected to behave
according to professional standards and in accordance with their employer’s
policies on appropriate workplace behavior.
While at CoreOS events or related social networking opportunities, attendees
should not engage in discriminatory or offensive speech or actions including
but not limited to gender, sexuality, race, age, disability, or religion.
Speakers should be especially aware of these concerns.
CoreOS does not condone any statements by speakers contrary to these standards.
CoreOS reserves the right to deny entrance and/or eject from an event (without
refund) any individual found to be engaging in discriminatory or offensive
speech or actions.
Please bring any concerns to the immediate attention of designated on-site
staff, Brandon Philips <brandon.philips@coreos.com>, and/or Meghan Schofield
<meghan.schofield@coreos.com>.

28
vendor/github.com/coreos/etcd/CONTRIBUTING.md generated vendored
View File

@ -1,11 +1,11 @@
# How to contribute
etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests. This document outlines some of the conventions on commit message formatting, contact points for developers and other resources to make getting your contribution into etcd easier.
etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests. This document outlines some of the conventions on commit message formatting, contact points for developers, and other resources to help get contributions into etcd.
# Email and chat
- Email: [etcd-dev](https://groups.google.com/forum/?hl=en#!forum/etcd-dev)
- IRC: #[coreos](irc://irc.freenode.org:6667/#coreos) IRC channel on freenode.org
- IRC: #[etcd](irc://irc.freenode.org:6667/#etcd) IRC channel on freenode.org
## Getting started
@ -14,24 +14,20 @@ etcd is Apache 2.0 licensed and accepts contributions via GitHub pull requests.
## Reporting bugs and creating issues
Reporting bugs is one of the best ways to contribute. However, a good bug report
has some very specific qualities, so please read over our short document on
[reporting bugs](https://github.com/coreos/etcd/blob/master/Documentation/reporting_bugs.md)
before you submit your bug report. This document might contain links known
issues, another good reason to take a look there, before reporting your bug.
Reporting bugs is one of the best ways to contribute. However, a good bug report has some very specific qualities, so please read over our short document on [reporting bugs](https://github.com/coreos/etcd/blob/master/Documentation/reporting_bugs.md) before submitting a bug report. This document might contain links to known issues, another good reason to take a look there before reporting a bug.
## Contribution flow
This is a rough outline of what a contributor's workflow looks like:
- Create a topic branch from where you want to base your work. This is usually master.
- Create a topic branch from where to base the contribution. This is usually master.
- Make commits of logical units.
- Make sure your commit messages are in the proper format (see below).
- Push your changes to a topic branch in your fork of the repository.
- Make sure commit messages are in the proper format (see below).
- Push changes in a topic branch to a personal fork of the repository.
- Submit a pull request to coreos/etcd.
- Your PR must receive a LGTM from two maintainers found in the MAINTAINERS file.
- The PR must receive a LGTM from two maintainers found in the MAINTAINERS file.
Thanks for your contributions!
Thanks for contributing!
### Code style
@ -48,8 +44,7 @@ the body of the commit should describe the why.
```
scripts: add the test-cluster command
this uses tmux to setup a test cluster that you can easily kill and
start for debugging.
this uses tmux to setup a test cluster that can easily be killed and started for debugging.
Fixes #38
```
@ -64,7 +59,4 @@ The format can be described more formally as follows:
<footer>
```
The first line is the subject and should be no longer than 70 characters, the
second line is always blank, and other lines should be wrapped at 80 characters.
This allows the message to be easier to read on GitHub as well as in various
git tools.
The first line is the subject and should be no longer than 70 characters, the second line is always blank, and other lines should be wrapped at 80 characters. This allows the message to be easier to read on GitHub as well as in various git tools.

53
vendor/github.com/coreos/etcd/Dockerfile-functional-tester generated vendored Normal file
View File

@ -0,0 +1,53 @@
FROM ubuntu:17.10
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get -y update \
&& apt-get -y install \
build-essential \
gcc \
apt-utils \
pkg-config \
software-properties-common \
apt-transport-https \
libssl-dev \
sudo \
bash \
curl \
wget \
tar \
git \
&& apt-get -y update \
&& apt-get -y upgrade \
&& apt-get -y autoremove \
&& apt-get -y autoclean
ENV GOROOT /usr/local/go
ENV GOPATH /go
ENV PATH ${GOPATH}/bin:${GOROOT}/bin:${PATH}
ENV GO_VERSION REPLACE_ME_GO_VERSION
ENV GO_DOWNLOAD_URL https://storage.googleapis.com/golang
RUN rm -rf ${GOROOT} \
&& curl -s ${GO_DOWNLOAD_URL}/go${GO_VERSION}.linux-amd64.tar.gz | tar -v -C /usr/local/ -xz \
&& mkdir -p ${GOPATH}/src ${GOPATH}/bin \
&& go version
RUN mkdir -p ${GOPATH}/src/github.com/coreos/etcd
ADD . ${GOPATH}/src/github.com/coreos/etcd
RUN go get -v github.com/coreos/gofail \
&& pushd ${GOPATH}/src/github.com/coreos/etcd \
&& GO_BUILD_FLAGS="-v" ./build \
&& cp ./bin/etcd /etcd \
&& cp ./bin/etcdctl /etcdctl \
&& GO_BUILD_FLAGS="-v" FAILPOINTS=1 ./build \
&& cp ./bin/etcd /etcd-failpoints \
&& ./tools/functional-tester/build \
&& cp ./bin/etcd-agent /etcd-agent \
&& cp ./bin/etcd-tester /etcd-tester \
&& cp ./bin/etcd-runner /etcd-runner \
&& go build -v -o /benchmark ./cmd/tools/benchmark \
&& go build -v -o /etcd-test-proxy ./cmd/tools/etcd-test-proxy \
&& popd \
&& rm -rf ${GOPATH}/src/github.com/coreos/etcd

6
vendor/github.com/coreos/etcd/Dockerfile-release generated vendored
View File

@ -5,6 +5,12 @@ ADD etcdctl /usr/local/bin/
RUN mkdir -p /var/etcd/
RUN mkdir -p /var/lib/etcd/
# Alpine Linux doesn't use pam, which means that there is no /etc/nsswitch.conf,
# but Golang relies on /etc/nsswitch.conf to check the order of DNS resolving
# (see https://github.com/golang/go/commit/9dee7771f561cf6aee081c0af6658cc81fac3918)
# To fix this we just create /etc/nsswitch.conf and add the following line:
RUN echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf
EXPOSE 2379 2380
# Define default command.

11
vendor/github.com/coreos/etcd/Dockerfile-release.arm64 generated vendored Normal file
View File

@ -0,0 +1,11 @@
FROM aarch64/ubuntu:16.04
ADD etcd /usr/local/bin/
ADD etcdctl /usr/local/bin/
ADD var/etcd /var/etcd
ADD var/lib/etcd /var/lib/etcd
EXPOSE 2379 2380
# Define default command.
CMD ["/usr/local/bin/etcd"]

11
vendor/github.com/coreos/etcd/Dockerfile-release.ppc64le generated vendored Normal file
View File

@ -0,0 +1,11 @@
FROM ppc64le/ubuntu:16.04
ADD etcd /usr/local/bin/
ADD etcdctl /usr/local/bin/
ADD var/etcd /var/etcd
ADD var/lib/etcd /var/lib/etcd
EXPOSE 2379 2380
# Define default command.
CMD ["/usr/local/bin/etcd"]

58
vendor/github.com/coreos/etcd/Dockerfile-test generated vendored Normal file
View File

@ -0,0 +1,58 @@
FROM ubuntu:16.10
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections
RUN apt-get -y update \
&& apt-get -y install \
build-essential \
gcc \
apt-utils \
pkg-config \
software-properties-common \
apt-transport-https \
libssl-dev \
sudo \
bash \
curl \
wget \
tar \
git \
netcat \
libaspell-dev \
libhunspell-dev \
hunspell-en-us \
aspell-en \
shellcheck \
&& apt-get -y update \
&& apt-get -y upgrade \
&& apt-get -y autoremove \
&& apt-get -y autoclean
ENV GOROOT /usr/local/go
ENV GOPATH /go
ENV PATH ${GOPATH}/bin:${GOROOT}/bin:${PATH}
ENV GO_VERSION REPLACE_ME_GO_VERSION
ENV GO_DOWNLOAD_URL https://storage.googleapis.com/golang
RUN rm -rf ${GOROOT} \
&& curl -s ${GO_DOWNLOAD_URL}/go${GO_VERSION}.linux-amd64.tar.gz | tar -v -C /usr/local/ -xz \
&& mkdir -p ${GOPATH}/src ${GOPATH}/bin \
&& go version
RUN mkdir -p ${GOPATH}/src/github.com/coreos/etcd
WORKDIR ${GOPATH}/src/github.com/coreos/etcd
ADD ./scripts/install-marker.sh /tmp/install-marker.sh
RUN go get -v -u -tags spell github.com/chzchzchz/goword \
&& go get -v -u github.com/coreos/license-bill-of-materials \
&& go get -v -u honnef.co/go/tools/cmd/gosimple \
&& go get -v -u honnef.co/go/tools/cmd/unused \
&& go get -v -u honnef.co/go/tools/cmd/staticcheck \
&& go get -v -u github.com/gyuho/gocovmerge \
&& go get -v -u github.com/gordonklaus/ineffassign \
&& go get -v -u github.com/alexkohler/nakedret \
&& /tmp/install-marker.sh amd64 \
&& rm -f /tmp/install-marker.sh \
&& curl -s https://codecov.io/bash >/codecov \
&& chmod 700 /codecov

1
vendor/github.com/coreos/etcd/MAINTAINERS generated vendored
View File

@ -1,5 +1,6 @@
Anthony Romano <anthony.romano@coreos.com> (@heyitsanthony) pkg:*
Brandon Philips <brandon.philips@coreos.com> (@philips) pkg:*
Fanmin Shi <fanmin.shi@coreos.com> (@fanminshi) pkg:*
Gyu-Ho Lee <gyu_ho.lee@coreos.com> (@gyuho) pkg:*
Xiang Li <xiang.li@coreos.com> (@xiang90) pkg:*

517
vendor/github.com/coreos/etcd/Makefile generated vendored Normal file
View File

@ -0,0 +1,517 @@
# run from repository root
# Example:
# make build
# make clean
# make docker-clean
# make docker-start
# make docker-kill
# make docker-remove
.PHONY: build
build:
GO_BUILD_FLAGS="-v" ./build
./bin/etcd --version
ETCDCTL_API=3 ./bin/etcdctl version
clean:
rm -f ./codecov
rm -rf ./agent-*
rm -rf ./covdir
rm -f ./*.coverprofile
rm -f ./*.log
rm -f ./bin/Dockerfile-release
rm -rf ./bin/*.etcd
rm -rf ./default.etcd
rm -rf ./tests/e2e/default.etcd
rm -rf ./gopath
rm -rf ./gopath.proto
rm -rf ./release
rm -f ./snapshot/localhost:*
rm -f ./integration/127.0.0.1:* ./integration/localhost:*
rm -f ./clientv3/integration/127.0.0.1:* ./clientv3/integration/localhost:*
rm -f ./clientv3/ordering/127.0.0.1:* ./clientv3/ordering/localhost:*
docker-clean:
docker images
docker image prune --force
docker-start:
service docker restart
docker-kill:
docker kill `docker ps -q` || true
docker-remove:
docker rm --force `docker ps -a -q` || true
docker rmi --force `docker images -q` || true
GO_VERSION ?= 1.10.3
ETCD_VERSION ?= $(shell git rev-parse --short HEAD || echo "GitNotFound")
TEST_SUFFIX = $(shell date +%s | base64 | head -c 15)
TEST_OPTS ?= PASSES='unit'
TMP_DIR_MOUNT_FLAG = --mount type=tmpfs,destination=/tmp
ifdef HOST_TMP_DIR
TMP_DIR_MOUNT_FLAG = --mount type=bind,source=$(HOST_TMP_DIR),destination=/tmp
endif
# Example:
# GO_VERSION=1.8.7 make build-docker-test
# make build-docker-test
#
# gcloud docker -- login -u _json_key -p "$(cat /etc/gcp-key-etcd-development.json)" https://gcr.io
# GO_VERSION=1.8.7 make push-docker-test
# make push-docker-test
#
# gsutil -m acl ch -u allUsers:R -r gs://artifacts.etcd-development.appspot.com
# make pull-docker-test
build-docker-test:
$(info GO_VERSION: $(GO_VERSION))
@sed -i.bak 's|REPLACE_ME_GO_VERSION|$(GO_VERSION)|g' ./tests/Dockerfile
docker build \
--tag gcr.io/etcd-development/etcd-test:go$(GO_VERSION) \
--file ./tests/Dockerfile .
@mv ./tests/Dockerfile.bak ./tests/Dockerfile
push-docker-test:
$(info GO_VERSION: $(GO_VERSION))
gcloud docker -- push gcr.io/etcd-development/etcd-test:go$(GO_VERSION)
pull-docker-test:
$(info GO_VERSION: $(GO_VERSION))
docker pull gcr.io/etcd-development/etcd-test:go$(GO_VERSION)
# Example:
# make build-docker-test
# make compile-with-docker-test
# make compile-setup-gopath-with-docker-test
compile-with-docker-test:
$(info GO_VERSION: $(GO_VERSION))
docker run \
--rm \
--mount type=bind,source=`pwd`,destination=/go/src/github.com/coreos/etcd \
gcr.io/etcd-development/etcd-test:go$(GO_VERSION) \
/bin/bash -c "GO_BUILD_FLAGS=-v ./build && ./bin/etcd --version"
compile-setup-gopath-with-docker-test:
$(info GO_VERSION: $(GO_VERSION))
docker run \
--rm \
--mount type=bind,source=`pwd`,destination=/etcd \
gcr.io/etcd-development/etcd-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && ETCD_SETUP_GOPATH=1 GO_BUILD_FLAGS=-v ./build && ./bin/etcd --version && rm -rf ./gopath"
# Example:
#
# Local machine:
# TEST_OPTS="PASSES='fmt'" make test
# TEST_OPTS="PASSES='fmt bom dep build unit'" make test
# TEST_OPTS="PASSES='build unit release integration_e2e functional'" make test
# TEST_OPTS="PASSES='build grpcproxy'" make test
#
# Example (test with docker):
# make pull-docker-test
# TEST_OPTS="PASSES='fmt'" make docker-test
# TEST_OPTS="VERBOSE=2 PASSES='unit'" make docker-test
#
# Travis CI (test with docker):
# TEST_OPTS="PASSES='fmt bom dep build unit'" make docker-test
#
# Semaphore CI (test with docker):
# TEST_OPTS="PASSES='build unit release integration_e2e functional'" make docker-test
# HOST_TMP_DIR=/tmp TEST_OPTS="PASSES='build unit release integration_e2e functional'" make docker-test
# TEST_OPTS="GOARCH=386 PASSES='build unit integration_e2e'" make docker-test
#
# grpc-proxy tests (test with docker):
# TEST_OPTS="PASSES='build grpcproxy'" make docker-test
# HOST_TMP_DIR=/tmp TEST_OPTS="PASSES='build grpcproxy'" make docker-test
.PHONY: test
test:
$(info TEST_OPTS: $(TEST_OPTS))
$(info log-file: test-$(TEST_SUFFIX).log)
$(TEST_OPTS) ./test 2>&1 | tee test-$(TEST_SUFFIX).log
! egrep "(--- FAIL:|panic: test timed out|appears to have leaked)" -B50 -A10 test-$(TEST_SUFFIX).log
docker-test:
$(info GO_VERSION: $(GO_VERSION))
$(info ETCD_VERSION: $(ETCD_VERSION))
$(info TEST_OPTS: $(TEST_OPTS))
$(info log-file: test-$(TEST_SUFFIX).log)
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`,destination=/go/src/github.com/coreos/etcd \
gcr.io/etcd-development/etcd-test:go$(GO_VERSION) \
/bin/bash -c "$(TEST_OPTS) ./test 2>&1 | tee test-$(TEST_SUFFIX).log"
! egrep "(--- FAIL:|panic: test timed out|appears to have leaked)" -B50 -A10 test-$(TEST_SUFFIX).log
docker-test-coverage:
$(info GO_VERSION: $(GO_VERSION))
$(info ETCD_VERSION: $(ETCD_VERSION))
$(info log-file: docker-test-coverage-$(TEST_SUFFIX).log)
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`,destination=/go/src/github.com/coreos/etcd \
gcr.io/etcd-development/etcd-test:go$(GO_VERSION) \
/bin/bash -c "COVERDIR=covdir PASSES='build build_cov cov' ./test 2>&1 | tee docker-test-coverage-$(TEST_SUFFIX).log && /codecov -t 6040de41-c073-4d6f-bbf8-d89256ef31e1"
! egrep "(--- FAIL:|panic: test timed out|appears to have leaked)" -B50 -A10 docker-test-coverage-$(TEST_SUFFIX).log
# Example:
# make compile-with-docker-test
# ETCD_VERSION=v3-test make build-docker-release-master
# ETCD_VERSION=v3-test make push-docker-release-master
# gsutil -m acl ch -u allUsers:R -r gs://artifacts.etcd-development.appspot.com
build-docker-release-master:
$(info ETCD_VERSION: $(ETCD_VERSION))
cp ./Dockerfile-release ./bin/Dockerfile-release
docker build \
--tag gcr.io/etcd-development/etcd:$(ETCD_VERSION) \
--file ./bin/Dockerfile-release \
./bin
rm -f ./bin/Dockerfile-release
docker run \
--rm \
gcr.io/etcd-development/etcd:$(ETCD_VERSION) \
/bin/sh -c "/usr/local/bin/etcd --version && ETCDCTL_API=3 /usr/local/bin/etcdctl version"
push-docker-release-master:
$(info ETCD_VERSION: $(ETCD_VERSION))
gcloud docker -- push gcr.io/etcd-development/etcd:$(ETCD_VERSION)
# Example:
# make build-docker-test
# make compile-with-docker-test
# make build-docker-static-ip-test
#
# gcloud docker -- login -u _json_key -p "$(cat /etc/gcp-key-etcd-development.json)" https://gcr.io
# make push-docker-static-ip-test
#
# gsutil -m acl ch -u allUsers:R -r gs://artifacts.etcd-development.appspot.com
# make pull-docker-static-ip-test
#
# make docker-static-ip-test-certs-run
# make docker-static-ip-test-certs-metrics-proxy-run
build-docker-static-ip-test:
$(info GO_VERSION: $(GO_VERSION))
@sed -i.bak 's|REPLACE_ME_GO_VERSION|$(GO_VERSION)|g' ./tests/docker-static-ip/Dockerfile
docker build \
--tag gcr.io/etcd-development/etcd-static-ip-test:go$(GO_VERSION) \
--file ./tests/docker-static-ip/Dockerfile \
./tests/docker-static-ip
@mv ./tests/docker-static-ip/Dockerfile.bak ./tests/docker-static-ip/Dockerfile
push-docker-static-ip-test:
$(info GO_VERSION: $(GO_VERSION))
gcloud docker -- push gcr.io/etcd-development/etcd-static-ip-test:go$(GO_VERSION)
pull-docker-static-ip-test:
$(info GO_VERSION: $(GO_VERSION))
docker pull gcr.io/etcd-development/etcd-static-ip-test:go$(GO_VERSION)
docker-static-ip-test-certs-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-static-ip/certs,destination=/certs \
gcr.io/etcd-development/etcd-static-ip-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs/run.sh && rm -rf m*.etcd"
docker-static-ip-test-certs-metrics-proxy-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-static-ip/certs-metrics-proxy,destination=/certs-metrics-proxy \
gcr.io/etcd-development/etcd-static-ip-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-metrics-proxy/run.sh && rm -rf m*.etcd"
# Example:
# make build-docker-test
# make compile-with-docker-test
# make build-docker-dns-test
#
# gcloud docker -- login -u _json_key -p "$(cat /etc/gcp-key-etcd-development.json)" https://gcr.io
# make push-docker-dns-test
#
# gsutil -m acl ch -u allUsers:R -r gs://artifacts.etcd-development.appspot.com
# make pull-docker-dns-test
#
# make docker-dns-test-insecure-run
# make docker-dns-test-certs-run
# make docker-dns-test-certs-gateway-run
# make docker-dns-test-certs-wildcard-run
# make docker-dns-test-certs-common-name-auth-run
# make docker-dns-test-certs-common-name-multi-run
build-docker-dns-test:
$(info GO_VERSION: $(GO_VERSION))
@sed -i.bak 's|REPLACE_ME_GO_VERSION|$(GO_VERSION)|g' ./tests/docker-dns/Dockerfile
docker build \
--tag gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
--file ./tests/docker-dns/Dockerfile \
./tests/docker-dns
@mv ./tests/docker-dns/Dockerfile.bak ./tests/docker-dns/Dockerfile
docker run \
--rm \
--dns 127.0.0.1 \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "/etc/init.d/bind9 start && cat /dev/null >/etc/hosts && dig etcd.local"
push-docker-dns-test:
$(info GO_VERSION: $(GO_VERSION))
gcloud docker -- push gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION)
pull-docker-dns-test:
$(info GO_VERSION: $(GO_VERSION))
docker pull gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION)
docker-dns-test-insecure-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns/insecure,destination=/insecure \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /insecure/run.sh && rm -rf m*.etcd"
docker-dns-test-certs-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns/certs,destination=/certs \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs/run.sh && rm -rf m*.etcd"
docker-dns-test-certs-gateway-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns/certs-gateway,destination=/certs-gateway \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-gateway/run.sh && rm -rf m*.etcd"
docker-dns-test-certs-wildcard-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns/certs-wildcard,destination=/certs-wildcard \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-wildcard/run.sh && rm -rf m*.etcd"
docker-dns-test-certs-common-name-auth-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns/certs-common-name-auth,destination=/certs-common-name-auth \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-common-name-auth/run.sh && rm -rf m*.etcd"
docker-dns-test-certs-common-name-multi-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns/certs-common-name-multi,destination=/certs-common-name-multi \
gcr.io/etcd-development/etcd-dns-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-common-name-multi/run.sh && rm -rf m*.etcd"
# Example:
# make build-docker-test
# make compile-with-docker-test
# make build-docker-dns-srv-test
# gcloud docker -- login -u _json_key -p "$(cat /etc/gcp-key-etcd-development.json)" https://gcr.io
# make push-docker-dns-srv-test
# gsutil -m acl ch -u allUsers:R -r gs://artifacts.etcd-development.appspot.com
# make pull-docker-dns-srv-test
# make docker-dns-srv-test-certs-run
# make docker-dns-srv-test-certs-gateway-run
# make docker-dns-srv-test-certs-wildcard-run
build-docker-dns-srv-test:
$(info GO_VERSION: $(GO_VERSION))
@sed -i.bak 's|REPLACE_ME_GO_VERSION|$(GO_VERSION)|g' ./tests/docker-dns-srv/Dockerfile
docker build \
--tag gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION) \
--file ./tests/docker-dns-srv/Dockerfile \
./tests/docker-dns-srv
@mv ./tests/docker-dns-srv/Dockerfile.bak ./tests/docker-dns-srv/Dockerfile
docker run \
--rm \
--dns 127.0.0.1 \
gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION) \
/bin/bash -c "/etc/init.d/bind9 start && cat /dev/null >/etc/hosts && dig +noall +answer SRV _etcd-client-ssl._tcp.etcd.local && dig +noall +answer SRV _etcd-server-ssl._tcp.etcd.local && dig +noall +answer m1.etcd.local m2.etcd.local m3.etcd.local"
push-docker-dns-srv-test:
$(info GO_VERSION: $(GO_VERSION))
gcloud docker -- push gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION)
pull-docker-dns-srv-test:
$(info GO_VERSION: $(GO_VERSION))
docker pull gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION)
docker-dns-srv-test-certs-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns-srv/certs,destination=/certs \
gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs/run.sh && rm -rf m*.etcd"
docker-dns-srv-test-certs-gateway-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns-srv/certs-gateway,destination=/certs-gateway \
gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-gateway/run.sh && rm -rf m*.etcd"
docker-dns-srv-test-certs-wildcard-run:
$(info GO_VERSION: $(GO_VERSION))
$(info HOST_TMP_DIR: $(HOST_TMP_DIR))
$(info TMP_DIR_MOUNT_FLAG: $(TMP_DIR_MOUNT_FLAG))
docker run \
--rm \
--tty \
--dns 127.0.0.1 \
$(TMP_DIR_MOUNT_FLAG) \
--mount type=bind,source=`pwd`/bin,destination=/etcd \
--mount type=bind,source=`pwd`/tests/docker-dns-srv/certs-wildcard,destination=/certs-wildcard \
gcr.io/etcd-development/etcd-dns-srv-test:go$(GO_VERSION) \
/bin/bash -c "cd /etcd && /certs-wildcard/run.sh && rm -rf m*.etcd"
# Example:
# make build-functional
# make build-docker-functional
# make push-docker-functional
# make pull-docker-functional
build-functional:
$(info GO_VERSION: $(GO_VERSION))
$(info ETCD_VERSION: $(ETCD_VERSION))
./functional/build
./bin/etcd-agent -help || true && \
./bin/etcd-proxy -help || true && \
./bin/etcd-runner --help || true && \
./bin/etcd-tester -help || true
build-docker-functional:
$(info GO_VERSION: $(GO_VERSION))
$(info ETCD_VERSION: $(ETCD_VERSION))
@sed -i.bak 's|REPLACE_ME_GO_VERSION|$(GO_VERSION)|g' ./functional/Dockerfile
docker build \
--tag gcr.io/etcd-development/etcd-functional:go$(GO_VERSION) \
--file ./functional/Dockerfile \
.
@mv ./functional/Dockerfile.bak ./functional/Dockerfile
docker run \
--rm \
gcr.io/etcd-development/etcd-functional:go$(GO_VERSION) \
/bin/bash -c "./bin/etcd --version && \
./bin/etcd-failpoints --version && \
ETCDCTL_API=3 ./bin/etcdctl version && \
./bin/etcd-agent -help || true && \
./bin/etcd-proxy -help || true && \
./bin/etcd-runner --help || true && \
./bin/etcd-tester -help || true && \
./bin/benchmark --help || true"
push-docker-functional:
$(info GO_VERSION: $(GO_VERSION))
$(info ETCD_VERSION: $(ETCD_VERSION))
gcloud docker -- push gcr.io/etcd-development/etcd-functional:go$(GO_VERSION)
pull-docker-functional:
$(info GO_VERSION: $(GO_VERSION))
$(info ETCD_VERSION: $(ETCD_VERSION))
docker pull gcr.io/etcd-development/etcd-functional:go$(GO_VERSION)

81
vendor/github.com/coreos/etcd/NEWS generated vendored
View File

@ -1,81 +0,0 @@
etcd v3.1.0 (2017-01-20)
- faster linearizable reads (implements Raft read-index)
- automatic leadership transfer when leader steps down
- etcd uses default route IP if advertise URL is not given
- cluster rejects removing members if quorum will be lost
- SRV records (e.g., infra1.example.com) must match the discovery domain
(i.e., example.com) if no custom certificate authority is given
- TLSConfig ServerName is ignored with user-provided certificates
for backwards compatibility; to be deprecated in 3.2
- discovery now has upper limit for waiting on retries
- etcd flags
- --strict-reconfig-check flag is set by default
- add --log-output flag
- add --metrics flag
- v3 authentication API is now stable
- v3 client
- add SetEndpoints method; update endpoints at runtime
- add Sync method; auto-update endpoints at runtime
- add Lease TimeToLive API; fetch lease information
- replace Config.Logger field with global logger
- Get API responses are sorted in ascending order by default
- v3 etcdctl
- add lease timetolive command
- add --print-value-only flag to get command
- add --dest-prefix flag to make-mirror command
- command get responses are sorted in ascending order by default
- recipes now conform to sessions defined in clientv3/concurrency
- ACI has symlinks to /usr/local/bin/etcd*
- warn on binding listeners through domain names; to be deprecated in 3.2
- experimental gRPC proxy feature
etcd v3.0.16 (2017-01-13)
etcd v3.0.15 (2016-11-11)
- fix cancel watch request with wrong range end
etcd v3.0.14 (2016-11-04)
- v3 etcdctl migrate command now supports --no-ttl flag to discard keys on transform
etcd v3.0.13 (2016-10-24)
etcd v3.0.12 (2016-10-07)
etcd v3.0.11 (2016-10-07)
- server returns previous key-value (optional)
- clientv3 WithPrevKV option
- v3 etcdctl put,watch,del --prev-kv flag
etcd v3.0.10 (2016-09-23)
etcd v3.0.9 (2016-09-15)
- warn on domain names on listen URLs (v3.2 will reject domain names)
etcd v3.0.8 (2016-09-09)
- allow only IP addresses in listen URLs (domain names are rejected)
etcd v3.0.7 (2016-08-31)
- SRV records only allow A records (RFC 2052)
etcd v3.0.6 (2016-08-19)
etcd v3.0.5 (2016-08-19)
- SRV records (e.g., infra1.example.com) must match the discovery domain
(i.e., example.com) if no custom certificate authority is given
etcd v3.0.4 (2016-07-27)
- v2 auth can now use common name from TLS certificate when --client-cert-auth is enabled
- v2 etcdctl ls command now supports --output=json
- Add /var/lib/etcd directory to etcd official Docker image
etcd v3.0.3 (2016-07-15)
- Revert Dockerfile to use CMD, instead of ENTRYPOINT, to support etcdctl run
- Docker commands for v3.0.2 won't work without specifying executable binary paths
- v3 etcdctl default endpoints are now 127.0.0.1:2379
etcd v3.0.2 (2016-07-08)
- Dockerfile uses ENTRYPOINT, instead of CMD, to run etcd without binary path specified
etcd v3.0.1 (2016-07-01)
etcd v3.0.0 (2016-06-30)

3
vendor/github.com/coreos/etcd/Procfile generated vendored
View File

@ -2,5 +2,4 @@
etcd1: bin/etcd --name infra1 --listen-client-urls http://127.0.0.1:2379 --advertise-client-urls http://127.0.0.1:2379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
etcd2: bin/etcd --name infra2 --listen-client-urls http://127.0.0.1:22379 --advertise-client-urls http://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1:22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
etcd3: bin/etcd --name infra3 --listen-client-urls http://127.0.0.1:32379 --advertise-client-urls http://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1:32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
# in future, use proxy to listen on 2379
#proxy: bin/etcd --name infra-proxy1 --proxy=on --listen-client-urls http://127.0.0.1:2378 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --enable-pprof
#proxy: bin/etcd grpc-proxy start --endpoints=127.0.0.1:2379,127.0.0.1:22379,127.0.0.1:32379 --listen-addr=127.0.0.1:23790 --advertise-client-url=127.0.0.1:23790 --enable-pprof

6
vendor/github.com/coreos/etcd/Procfile.v2 generated vendored Normal file
View File

@ -0,0 +1,6 @@
# Use goreman to run `go get github.com/mattn/goreman`
etcd1: bin/etcd --name infra1 --listen-client-urls http://127.0.0.1:2379 --advertise-client-urls http://127.0.0.1:2379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
etcd2: bin/etcd --name infra2 --listen-client-urls http://127.0.0.1:22379 --advertise-client-urls http://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1:22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
etcd3: bin/etcd --name infra3 --listen-client-urls http://127.0.0.1:32379 --advertise-client-urls http://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1:32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
# in future, use proxy to listen on 2379
#proxy: bin/etcd --name infra-proxy1 --proxy=on --listen-client-urls http://127.0.0.1:2378 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --enable-pprof

62
vendor/github.com/coreos/etcd/README.md generated vendored
View File

@ -1,9 +1,12 @@
# etcd
[![Go Report Card](https://goreportcard.com/badge/github.com/coreos/etcd)](https://goreportcard.com/report/github.com/coreos/etcd)
[![Build Status](https://travis-ci.org/coreos/etcd.svg?branch=master)](https://travis-ci.org/coreos/etcd)
[![Build Status](https://semaphoreci.com/api/v1/coreos/etcd/branches/master/shields_badge.svg)](https://semaphoreci.com/coreos/etcd)
[![Docker Repository on Quay.io](https://quay.io/repository/coreos/etcd-git/status "Docker Repository on Quay.io")](https://quay.io/repository/coreos/etcd-git)
[![Go Report Card](https://goreportcard.com/badge/github.com/coreos/etcd?style=flat-square)](https://goreportcard.com/report/github.com/coreos/etcd)
[![Coverage](https://codecov.io/gh/coreos/etcd/branch/master/graph/badge.svg)](https://codecov.io/gh/coreos/etcd)
[![Build Status Travis](https://img.shields.io/travis/coreos/etcdlabs.svg?style=flat-square&&branch=master)](https://travis-ci.org/coreos/etcd)
[![Build Status Semaphore](https://semaphoreci.com/api/v1/coreos/etcd/branches/master/shields_badge.svg)](https://semaphoreci.com/coreos/etcd)
[![Godoc](http://img.shields.io/badge/go-documentation-blue.svg?style=flat-square)](https://godoc.org/github.com/coreos/etcd)
[![Releases](https://img.shields.io/github/release/coreos/etcd/all.svg?style=flat-square)](https://github.com/coreos/etcd/releases)
[![LICENSE](https://img.shields.io/github/license/coreos/etcd.svg?style=flat-square)](https://github.com/coreos/etcd/blob/master/LICENSE)
**Note**: The `master` branch may be in an *unstable or even broken state* during development. Please use [releases][github-release] instead of the `master` branch in order to get stable binaries.
@ -11,7 +14,7 @@
![etcd Logo](logos/etcd-horizontal-color.png)
etcd is a distributed, consistent key-value store for shared configuration and service discovery, with a focus on being:
etcd is a distributed reliable key-value store for the most critical data of a distributed system, with a focus on being:
* *Simple*: well-defined, user-facing API (gRPC)
* *Secure*: automatic TLS with optional client cert authentication
@ -33,24 +36,45 @@ See [etcdctl][etcdctl] for a simple command line client.
[etcdctl]: https://github.com/coreos/etcd/tree/master/etcdctl
[etcd-tests]: http://dash.etcd.io
## Community meetings
etcd contributors and maintainers have bi-weekly meetings at 11:00 AM (USA Pacific) on Tuesdays. There is an [iCalendar][rfc5545] format for the meetings [here](meeting.ics). Anyone is welcome to join via [Zoom][zoom] or audio-only: +1 669 900 6833. An initial agenda will be posted to the [shared Google docs][shared-meeting-notes] a day before each meeting, and everyone is welcome to suggest additional topics or other agendas.
[rfc5545]: https://tools.ietf.org/html/rfc5545
[zoom]: https://coreos.zoom.us/j/854793406
[shared-meeting-notes]: https://docs.google.com/document/d/1DbVXOHvd9scFsSmL2oNg4YGOHJdXqtx583DmeVWrB_M/edit#
## Getting started
### Getting etcd
The easiest way to get etcd is to use one of the pre-built release binaries which are available for OSX, Linux, Windows, AppC (ACI), and Docker. Instructions for using these binaries are on the [GitHub releases page][github-release].
The easiest way to get etcd is to use one of the pre-built release binaries which are available for OSX, Linux, Windows, [rkt][rkt], and Docker. Instructions for using these binaries are on the [GitHub releases page][github-release].
For those wanting to try the very latest version, you can [build the latest version of etcd][dl-build] from the `master` branch.
You will first need [*Go*](https://golang.org/) installed on your machine (version 1.6+ is required).
All development occurs on `master`, including new features and bug fixes.
Bug fixes are first targeted at `master` and subsequently ported to release branches, as described in the [branch management][branch-management] guide.
For those wanting to try the very latest version, [build the latest version of etcd][dl-build] from the `master` branch. This first needs [*Go*](https://golang.org/) installed (version 1.9+ is required). All development occurs on `master`, including new features and bug fixes. Bug fixes are first targeted at `master` and subsequently ported to release branches, as described in the [branch management][branch-management] guide.
[rkt]: https://github.com/rkt/rkt/releases/
[github-release]: https://github.com/coreos/etcd/releases/
[branch-management]: ./Documentation/branch_management.md
[dl-build]: ./Documentation/dl_build.md#build-the-latest-version
### Running etcd
First start a single-member cluster of etcd:
First start a single-member cluster of etcd.
If etcd is installed using the [pre-built release binaries][github-release], run it from the installation location as below:
```sh
/tmp/etcd-download-test/etcd
```
The etcd command can be simply run as such if it is moved to the system path as below:
```sh
mv /tmp/etcd-download-test/etcd /usr/locale/bin/
etcd
```
If etcd is [build from the master branch][dl-build], run it as below:
```sh
./bin/etcd
@ -75,9 +99,9 @@ That's it! etcd is now running and serving client requests. For more
### etcd TCP ports
The [official etcd ports][iana-ports] are 2379 for client requests, and 2380 for peer communication.
The [official etcd ports][iana-ports] are 2379 for client requests, and 2380 for peer communication.
[iana-ports]: https://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml?search=etcd
[iana-ports]: http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt
### Running a local etcd cluster
@ -89,13 +113,13 @@ Our [Procfile script](./Procfile) will set up a local example cluster. Start it
goreman start
```
This will bring up 3 etcd members `infra1`, `infra2` and `infra3` and etcd proxy `proxy`, which runs locally and composes a cluster.
This will bring up 3 etcd members `infra1`, `infra2` and `infra3` and etcd `grpc-proxy`, which runs locally and composes a cluster.
Every cluster member and proxy accepts key value reads and key value writes.
### Running etcd on Kubernetes
If you want to run etcd cluster on Kubernetes, try [etcd operator](https://github.com/coreos/etcd-operator).
To run an etcd cluster on Kubernetes, try [etcd operator](https://github.com/coreos/etcd-operator).
### Next steps
@ -105,7 +129,7 @@ Now it's time to dig into the full etcd API and other guides.
- Explore the full gRPC [API][api].
- Set up a [multi-machine cluster][clustering].
- Learn the [config format, env variables and flags][configuration].
- Find [language bindings and tools][libraries-and-tools].
- Find [language bindings and tools][integrations].
- Use TLS to [secure an etcd cluster][security].
- [Tune etcd][tuning].
@ -113,7 +137,7 @@ Now it's time to dig into the full etcd API and other guides.
[api]: ./Documentation/dev-guide/api_reference_v3.md
[clustering]: ./Documentation/op-guide/clustering.md
[configuration]: ./Documentation/op-guide/configuration.md
[libraries-and-tools]: ./Documentation/libraries-and-tools.md
[integrations]: ./Documentation/integrations.md
[security]: ./Documentation/op-guide/security.md
[tuning]: ./Documentation/tuning.md
@ -130,10 +154,8 @@ See [CONTRIBUTING](CONTRIBUTING.md) for details on submitting patches and the co
## Reporting bugs
See [reporting bugs](Documentation/reporting_bugs.md) for details about reporting any issue you may encounter.
See [reporting bugs](Documentation/reporting_bugs.md) for details about reporting any issues.
### License
etcd is under the Apache 2.0 license. See the [LICENSE](LICENSE) file for details.

16
vendor/github.com/coreos/etcd/ROADMAP.md generated vendored
View File

@ -6,25 +6,17 @@ This document defines a high level roadmap for etcd development.
The dates below should not be considered authoritative, but rather indicative of the projected timeline of the project. The [milestones defined in GitHub](https://github.com/coreos/etcd/milestones) represent the most up-to-date and issue-for-issue plans.
etcd 3.0 is our current stable branch. The roadmap below outlines new features that will be added to etcd, and while subject to change, define what future stable will look like.
etcd 3.2 is our current stable branch. The roadmap below outlines new features that will be added to etcd, and while subject to change, define what future stable will look like.
### etcd 3.1 (2016-Oct)
- Stable L4 gateway
- Experimental support for scalable proxy
- Automatic leadership transfer for the rolling upgrade
- V3 API improvements
- Get previous key-value pair
- Get only keys (ignore values)
- Get only key count
### etcd 3.2 (2017-Apr)
### etcd 3.2 (2017-May)
- Stable scalable proxy
- Proxy-as-client interface passthrough
- Lock service
- Namespacing proxy
- JWT token based authentication
- TLS Command Name and JWT token based authentication
- Read-modify-write V3 Put
- Improved watch performance
- Support non-blocking concurrent read
### etcd 3.3 (?)
- TBD

469
vendor/github.com/coreos/etcd/bill-of-materials.json generated vendored Normal file
View File

@ -0,0 +1,469 @@
[
{
"project": "bitbucket.org/ww/goautoneg",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 1
}
]
},
{
"project": "github.com/beorn7/perks/quantile",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "github.com/bgentry/speakeasy",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9441624365482234
}
]
},
{
"project": "github.com/coreos/bbolt",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/coreos/etcd",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/coreos/go-semver/semver",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/coreos/go-systemd",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 0.9966703662597114
}
]
},
{
"project": "github.com/coreos/pkg",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/cpuguy83/go-md2man/md2man",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/dgrijalva/jwt-go",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "github.com/dustin/go-humanize",
"licenses": [
{
"type": "MIT License",
"confidence": 0.96875
}
]
},
{
"project": "github.com/ghodss/yaml",
"licenses": [
{
"type": "MIT License and BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 1
}
]
},
{
"project": "github.com/gogo/protobuf",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9090909090909091
}
]
},
{
"project": "github.com/golang/groupcache/lru",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 0.9966703662597114
}
]
},
{
"project": "github.com/golang/protobuf",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.92
}
]
},
{
"project": "github.com/google/btree",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/gorilla/websocket",
"licenses": [
{
"type": "BSD 2-clause \"Simplified\" License",
"confidence": 0.9852216748768473
}
]
},
{
"project": "github.com/grpc-ecosystem/go-grpc-prometheus",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/grpc-ecosystem/grpc-gateway",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.979253112033195
}
]
},
{
"project": "github.com/inconshreveable/mousetrap",
"licenses": [
{
"type": "MIT License and BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 1
},
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/jonboulle/clockwork",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/json-iterator/go",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/mattn/go-runewidth",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/matttproud/golang_protobuf_extensions/pbutil",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/modern-go/concurrent",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/modern-go/reflect2",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/olekukonko/tablewriter",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "github.com/prometheus/client_golang/prometheus",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/prometheus/client_model/go",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/prometheus/common",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/prometheus/procfs",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/russross/blackfriday",
"licenses": [
{
"type": "BSD 2-clause \"Simplified\" License",
"confidence": 0.9626168224299065
}
]
},
{
"project": "github.com/sirupsen/logrus",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/soheilhy/cmux",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "github.com/spf13/cobra",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 0.9573241061130334
}
]
},
{
"project": "github.com/spf13/pflag",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9663865546218487
}
]
},
{
"project": "github.com/tmc/grpc-websocket-proxy/wsproxy",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "github.com/urfave/cli",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "github.com/xiang90/probing",
"licenses": [
{
"type": "MIT License",
"confidence": 1
}
]
},
{
"project": "go.uber.org/atomic",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "go.uber.org/multierr",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "go.uber.org/zap",
"licenses": [
{
"type": "MIT License",
"confidence": 0.9891304347826086
}
]
},
{
"project": "golang.org/x/crypto",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9663865546218487
}
]
},
{
"project": "golang.org/x/net",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9663865546218487
}
]
},
{
"project": "golang.org/x/sys/unix",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9663865546218487
}
]
},
{
"project": "golang.org/x/text",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9663865546218487
}
]
},
{
"project": "golang.org/x/time/rate",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9663865546218487
}
]
},
{
"project": "google.golang.org/genproto/googleapis/rpc/status",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "google.golang.org/grpc",
"licenses": [
{
"type": "Apache License 2.0",
"confidence": 1
}
]
},
{
"project": "gopkg.in/cheggaaa/pb.v1",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License",
"confidence": 0.9916666666666667
}
]
},
{
"project": "gopkg.in/yaml.v2",
"licenses": [
{
"type": "The Unlicense",
"confidence": 0.35294117647058826
},
{
"type": "MIT License",
"confidence": 0.8975609756097561
}
]
}
]

26
vendor/github.com/coreos/etcd/bill-of-materials.override.json generated vendored Normal file
View File

@ -0,0 +1,26 @@
[
{
"project": "bitbucket.org/ww/goautoneg",
"licenses": [
{
"type": "BSD 3-clause \"New\" or \"Revised\" License"
}
]
},
{
"project": "github.com/ghodss/yaml",
"licenses": [
{
"type": "MIT License and BSD 3-clause \"New\" or \"Revised\" License"
}
]
},
{
"project": "github.com/inconshreveable/mousetrap",
"licenses": [
{
"type": "Apache License 2.0"
}
]
}
]

43
vendor/github.com/coreos/etcd/build generated vendored
View File

@ -3,10 +3,8 @@
# set some environment variables
ORG_PATH="github.com/coreos"
REPO_PATH="${ORG_PATH}/etcd"
export GO15VENDOREXPERIMENT="1"
eval $(go env)
GIT_SHA=`git rev-parse --short HEAD || echo "GitNotFound"`
GIT_SHA=$(git rev-parse --short HEAD || echo "GitNotFound")
if [ ! -z "$FAILPOINTS" ]; then
GIT_SHA="$GIT_SHA"-FAILPOINTS
fi
@ -16,44 +14,49 @@ GO_LDFLAGS="$GO_LDFLAGS -X ${REPO_PATH}/cmd/vendor/${REPO_PATH}/version.GitSHA=$
# enable/disable failpoints
toggle_failpoints() {
FAILPKGS="etcdserver/ mvcc/backend/"
mode="disable"
if [ ! -z "$FAILPOINTS" ]; then mode="enable"; fi
if [ ! -z "$1" ]; then mode="$1"; fi
mode="$1"
if which gofail >/dev/null 2>&1; then
gofail "$mode" $FAILPKGS
gofail "$mode" etcdserver/ mvcc/backend/
elif [ "$mode" != "disable" ]; then
echo "FAILPOINTS set but gofail not found"
exit 1
fi
}
toggle_failpoints_default() {
mode="disable"
if [ ! -z "$FAILPOINTS" ]; then mode="enable"; fi
toggle_failpoints "$mode"
}
etcd_build() {
out="bin"
if [ -n "${BINDIR}" ]; then out="${BINDIR}"; fi
toggle_failpoints
# Static compilation is useful when etcd is run in a container
CGO_ENABLED=0 go build $GO_BUILD_FLAGS -installsuffix cgo -ldflags "$GO_LDFLAGS" -o ${out}/etcd ${REPO_PATH}/cmd/etcd || return
CGO_ENABLED=0 go build $GO_BUILD_FLAGS -installsuffix cgo -ldflags "$GO_LDFLAGS" -o ${out}/etcdctl ${REPO_PATH}/cmd/etcdctl || return
toggle_failpoints_default
# Static compilation is useful when etcd is run in a container. $GO_BUILD_FLAGS is OK
# shellcheck disable=SC2086
CGO_ENABLED=0 go build $GO_BUILD_FLAGS -installsuffix cgo -ldflags "$GO_LDFLAGS" -o "${out}/etcd" ${REPO_PATH}/cmd/etcd || return
# shellcheck disable=SC2086
CGO_ENABLED=0 go build $GO_BUILD_FLAGS -installsuffix cgo -ldflags "$GO_LDFLAGS" -o "${out}/etcdctl" ${REPO_PATH}/cmd/etcdctl || return
}
etcd_setup_gopath() {
CDIR=$(cd `dirname "$0"` && pwd)
d=$(dirname "$0")
CDIR=$(cd "$d" && pwd)
cd "$CDIR"
etcdGOPATH=${CDIR}/gopath
etcdGOPATH="${CDIR}/gopath"
# preserve old gopath to support building with unvendored tooling deps (e.g., gofail)
if [ -n "$GOPATH" ]; then
GOPATH=":$GOPATH"
fi
export GOPATH=${etcdGOPATH}$GOPATH
rm -rf ${etcdGOPATH}/src
mkdir -p ${etcdGOPATH}
ln -s ${CDIR}/cmd/vendor ${etcdGOPATH}/src
rm -rf "${etcdGOPATH}/src"
mkdir -p "${etcdGOPATH}"
ln -s "${CDIR}/cmd/vendor" "${etcdGOPATH}/src"
}
toggle_failpoints
toggle_failpoints_default
# only build when called directly, not sourced
if echo "$0" | grep "build$" >/dev/null; then

43
vendor/github.com/coreos/etcd/etcd.conf.yml.sample generated vendored
View File

@ -35,7 +35,7 @@ max-snapshots: 5
max-wals: 5
# Comma-separated white list of origins for CORS (cross-origin resource sharing).
cors:
cors:
# List of this member's peer URLs to advertise to the rest of the cluster.
# The URLs needed to be a comma-separated list.
@ -46,16 +46,16 @@ initial-advertise-peer-urls: http://localhost:2380
advertise-client-urls: http://localhost:2379
# Discovery URL used to bootstrap the cluster.
discovery:
discovery:
# Valid values include 'exit', 'proxy'
discovery-fallback: 'proxy'
# HTTP proxy to use for traffic to discovery service.
discovery-proxy:
discovery-proxy:
# DNS domain used to bootstrap initial cluster.
discovery-srv:
discovery-srv:
# Initial cluster configuration for bootstrapping.
initial-cluster:
@ -69,6 +69,12 @@ initial-cluster-state: 'new'
# Reject reconfiguration requests that would cause quorum loss.
strict-reconfig-check: false
# Accept etcd V2 client requests
enable-v2: true
# Enable runtime profiling data via HTTP server
enable-pprof: true
# Valid values include 'on', 'readonly', 'off'
proxy: 'off'
@ -87,40 +93,40 @@ proxy-write-timeout: 5000
# Time (in milliseconds) for a read to timeout.
proxy-read-timeout: 0
client-transport-security:
client-transport-security:
# DEPRECATED: Path to the client server TLS CA file.
ca-file:
ca-file:
# Path to the client server TLS cert file.
cert-file:
cert-file:
# Path to the client server TLS key file.
key-file:
key-file:
# Enable client cert authentication.
client-cert-auth: false
# Path to the client server TLS trusted CA key file.
trusted-ca-file:
# Path to the client server TLS trusted CA cert file.
trusted-ca-file:
# Client TLS using generated certificates
auto-tls: false
peer-transport-security:
peer-transport-security:
# DEPRECATED: Path to the peer server TLS CA file.
ca-file:
# Path to the peer server TLS cert file.
cert-file:
cert-file:
# Path to the peer server TLS key file.
key-file:
key-file:
# Enable peer client cert authentication.
client-cert-auth: false
peer-client-cert-auth: false
# Path to the peer server TLS trusted CA key file.
trusted-ca-file:
# Path to the peer server TLS trusted CA cert file.
trusted-ca-file:
# Peer TLS using generated certificates.
auto-tls: false
@ -129,7 +135,10 @@ peer-transport-security:
debug: false
# Specify a particular log level for each etcd package (eg: 'etcdmain=CRITICAL,etcdserver=DEBUG'.
log-package-levels:
log-package-levels:
# Specify 'stdout' or 'stderr' to skip journald logging even when running under systemd.
log-output: default
# Force to create a new one member cluster.
force-new-cluster: false

15
vendor/github.com/coreos/etcd/etcdserver/stats/leader.go generated vendored
View File

@ -24,25 +24,30 @@ import (
// LeaderStats is used by the leader in an etcd cluster, and encapsulates
// statistics about communication with its followers
type LeaderStats struct {
leaderStats
sync.Mutex
}
type leaderStats struct {
// Leader is the ID of the leader in the etcd cluster.
// TODO(jonboulle): clarify that these are IDs, not names
Leader string `json:"leader"`
Followers map[string]*FollowerStats `json:"followers"`
sync.Mutex
}
// NewLeaderStats generates a new LeaderStats with the given id as leader
func NewLeaderStats(id string) *LeaderStats {
return &LeaderStats{
Leader: id,
Followers: make(map[string]*FollowerStats),
leaderStats: leaderStats{
Leader: id,
Followers: make(map[string]*FollowerStats),
},
}
}
func (ls *LeaderStats) JSON() []byte {
ls.Lock()
stats := *ls
stats := ls.leaderStats
ls.Unlock()
b, err := json.Marshal(stats)
// TODO(jonboulle): appropriate error handling?

56
vendor/github.com/coreos/etcd/etcdserver/stats/server.go generated vendored
View File

@ -26,6 +26,26 @@ import (
// ServerStats encapsulates various statistics about an EtcdServer and its
// communication with other members of the cluster
type ServerStats struct {
serverStats
sync.Mutex
}
func NewServerStats(name, id string) *ServerStats {
ss := &ServerStats{
serverStats: serverStats{
Name: name,
ID: id,
},
}
now := time.Now()
ss.StartTime = now
ss.LeaderInfo.StartTime = now
ss.sendRateQueue = &statsQueue{back: -1}
ss.recvRateQueue = &statsQueue{back: -1}
return ss
}
type serverStats struct {
Name string `json:"name"`
// ID is the raft ID of the node.
// TODO(jonboulle): use ID instead of name?
@ -49,17 +69,15 @@ type ServerStats struct {
sendRateQueue *statsQueue
recvRateQueue *statsQueue
sync.Mutex
}
func (ss *ServerStats) JSON() []byte {
ss.Lock()
stats := *ss
ss.Unlock()
stats := ss.serverStats
stats.SendingPkgRate, stats.SendingBandwidthRate = stats.sendRateQueue.Rate()
stats.RecvingPkgRate, stats.RecvingBandwidthRate = stats.recvRateQueue.Rate()
stats.LeaderInfo.Uptime = time.Since(stats.LeaderInfo.StartTime).String()
stats.SendingPkgRate, stats.SendingBandwidthRate = stats.SendRates()
stats.RecvingPkgRate, stats.RecvingBandwidthRate = stats.RecvRates()
ss.Unlock()
b, err := json.Marshal(stats)
// TODO(jonboulle): appropriate error handling?
if err != nil {
@ -68,32 +86,6 @@ func (ss *ServerStats) JSON() []byte {
return b
}
// Initialize clears the statistics of ServerStats and resets its start time
func (ss *ServerStats) Initialize() {
if ss == nil {
return
}
now := time.Now()
ss.StartTime = now
ss.LeaderInfo.StartTime = now
ss.sendRateQueue = &statsQueue{
back: -1,
}
ss.recvRateQueue = &statsQueue{
back: -1,
}
}
// RecvRates calculates and returns the rate of received append requests
func (ss *ServerStats) RecvRates() (float64, float64) {
return ss.recvRateQueue.Rate()
}
// SendRates calculates and returns the rate of sent append requests
func (ss *ServerStats) SendRates() (float64, float64) {
return ss.sendRateQueue.Rate()
}
// RecvAppendReq updates the ServerStats in response to an AppendRequest
// from the given leader being received
func (ss *ServerStats) RecvAppendReq(leader string, reqSize int) {

206
vendor/github.com/coreos/etcd/functional.yaml generated vendored Normal file
View File

@ -0,0 +1,206 @@
agent-configs:
- etcd-exec-path: ./bin/etcd
agent-addr: 127.0.0.1:19027
failpoint-http-addr: http://127.0.0.1:7381
base-dir: /tmp/etcd-functional-1
etcd-log-path: /tmp/etcd-functional-1/etcd.log
etcd-client-proxy: false
etcd-peer-proxy: true
etcd-client-endpoint: 127.0.0.1:1379
etcd:
name: s1
data-dir: /tmp/etcd-functional-1/etcd.data
wal-dir: /tmp/etcd-functional-1/etcd.data/member/wal
heartbeat-interval: 100
election-timeout: 1000
listen-client-urls: ["https://127.0.0.1:1379"]
advertise-client-urls: ["https://127.0.0.1:1379"]
auto-tls: true
client-cert-auth: false
cert-file: ""
key-file: ""
trusted-ca-file: ""
listen-peer-urls: ["https://127.0.0.1:1380"]
initial-advertise-peer-urls: ["https://127.0.0.1:1381"]
peer-auto-tls: true
peer-client-cert-auth: false
peer-cert-file: ""
peer-key-file: ""
peer-trusted-ca-file: ""
initial-cluster: s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381
initial-cluster-state: new
initial-cluster-token: tkn
snapshot-count: 10000
quota-backend-bytes: 10740000000 # 10 GiB
pre-vote: true
initial-corrupt-check: true
client-cert-data: ""
client-cert-path: ""
client-key-data: ""
client-key-path: ""
client-trusted-ca-data: ""
client-trusted-ca-path: ""
peer-cert-data: ""
peer-cert-path: ""
peer-key-data: ""
peer-key-path: ""
peer-trusted-ca-data: ""
peer-trusted-ca-path: ""
snapshot-path: /tmp/etcd-functional-1.snapshot.db
- etcd-exec-path: ./bin/etcd
agent-addr: 127.0.0.1:29027
failpoint-http-addr: http://127.0.0.1:7382
base-dir: /tmp/etcd-functional-2
etcd-log-path: /tmp/etcd-functional-2/etcd.log
etcd-client-proxy: false
etcd-peer-proxy: true
etcd-client-endpoint: 127.0.0.1:2379
etcd:
name: s2
data-dir: /tmp/etcd-functional-2/etcd.data
wal-dir: /tmp/etcd-functional-2/etcd.data/member/wal
heartbeat-interval: 100
election-timeout: 1000
listen-client-urls: ["https://127.0.0.1:2379"]
advertise-client-urls: ["https://127.0.0.1:2379"]
auto-tls: true
client-cert-auth: false
cert-file: ""
key-file: ""
trusted-ca-file: ""
listen-peer-urls: ["https://127.0.0.1:2380"]
initial-advertise-peer-urls: ["https://127.0.0.1:2381"]
peer-auto-tls: true
peer-client-cert-auth: false
peer-cert-file: ""
peer-key-file: ""
peer-trusted-ca-file: ""
initial-cluster: s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381
initial-cluster-state: new
initial-cluster-token: tkn
snapshot-count: 10000
quota-backend-bytes: 10740000000 # 10 GiB
pre-vote: true
initial-corrupt-check: true
client-cert-data: ""
client-cert-path: ""
client-key-data: ""
client-key-path: ""
client-trusted-ca-data: ""
client-trusted-ca-path: ""
peer-cert-data: ""
peer-cert-path: ""
peer-key-data: ""
peer-key-path: ""
peer-trusted-ca-data: ""
peer-trusted-ca-path: ""
snapshot-path: /tmp/etcd-functional-2.snapshot.db
- etcd-exec-path: ./bin/etcd
agent-addr: 127.0.0.1:39027
failpoint-http-addr: http://127.0.0.1:7383
base-dir: /tmp/etcd-functional-3
etcd-log-path: /tmp/etcd-functional-3/etcd.log
etcd-client-proxy: false
etcd-peer-proxy: true
etcd-client-endpoint: 127.0.0.1:3379
etcd:
name: s3
data-dir: /tmp/etcd-functional-3/etcd.data
wal-dir: /tmp/etcd-functional-3/etcd.data/member/wal
heartbeat-interval: 100
election-timeout: 1000
listen-client-urls: ["https://127.0.0.1:3379"]
advertise-client-urls: ["https://127.0.0.1:3379"]
auto-tls: true
client-cert-auth: false
cert-file: ""
key-file: ""
trusted-ca-file: ""
listen-peer-urls: ["https://127.0.0.1:3380"]
initial-advertise-peer-urls: ["https://127.0.0.1:3381"]
peer-auto-tls: true
peer-client-cert-auth: false
peer-cert-file: ""
peer-key-file: ""
peer-trusted-ca-file: ""
initial-cluster: s1=https://127.0.0.1:1381,s2=https://127.0.0.1:2381,s3=https://127.0.0.1:3381
initial-cluster-state: new
initial-cluster-token: tkn
snapshot-count: 10000
quota-backend-bytes: 10740000000 # 10 GiB
pre-vote: true
initial-corrupt-check: true
client-cert-data: ""
client-cert-path: ""
client-key-data: ""
client-key-path: ""
client-trusted-ca-data: ""
client-trusted-ca-path: ""
peer-cert-data: ""
peer-cert-path: ""
peer-key-data: ""
peer-key-path: ""
peer-trusted-ca-data: ""
peer-trusted-ca-path: ""
snapshot-path: /tmp/etcd-functional-3.snapshot.db
tester-config:
data-dir: /tmp/etcd-tester-data
network: tcp
addr: 127.0.0.1:9028
# slow enough to trigger election
delay-latency-ms: 5000
delay-latency-ms-rv: 500
round-limit: 1
exit-on-failure: true
enable-pprof: true
case-delay-ms: 7000
case-shuffle: true
# For full descriptions,
# https://godoc.org/github.com/coreos/etcd/functional/rpcpb#Case
cases:
- SIGTERM_ONE_FOLLOWER
- SIGTERM_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT
- SIGTERM_LEADER
- SIGTERM_LEADER_UNTIL_TRIGGER_SNAPSHOT
- SIGTERM_QUORUM
- SIGTERM_ALL
- SIGQUIT_AND_REMOVE_ONE_FOLLOWER
- SIGQUIT_AND_REMOVE_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT
- BLACKHOLE_PEER_PORT_TX_RX_ONE_FOLLOWER
- BLACKHOLE_PEER_PORT_TX_RX_ONE_FOLLOWER_UNTIL_TRIGGER_SNAPSHOT
- BLACKHOLE_PEER_PORT_TX_RX_LEADER
- BLACKHOLE_PEER_PORT_TX_RX_LEADER_UNTIL_TRIGGER_SNAPSHOT
- BLACKHOLE_PEER_PORT_TX_RX_QUORUM
- DELAY_PEER_PORT_TX_RX_ONE_FOLLOWER
- DELAY_PEER_PORT_TX_RX_LEADER
- DELAY_PEER_PORT_TX_RX_QUORUM
failpoint-commands:
- panic("etcd-tester")
runner-exec-path: ./bin/etcd-runner
external-exec-path: ""
stressers:
- KV
- LEASE
checkers:
- KV_HASH
- LEASE_EXPIRE
stress-key-size: 100
stress-key-size-large: 32769
stress-key-suffix-range: 250000
stress-key-suffix-range-txn: 100
stress-key-txn-ops: 10
stress-clients: 100
stress-qps: 2000

145
vendor/github.com/coreos/etcd/glide.lock generated vendored
View File

@ -1,22 +1,20 @@
hash: ca3c895fa60c9ca9f53408202fb7643705f9960212d342967ed0da8e93606cc4
updated: 2017-01-18T10:26:48.990115455-08:00
hash: f0697416d74e4c0fb9d6471c39c3e005ecdeccc8a864c1b0b65e0087b3242027
updated: 2018-04-10T23:45:04.40596807-07:00
imports:
- name: github.com/beorn7/perks
version: 4c0e84591b9aa9e6dcfdf3e020114cd81f89d5f9
version: 3a771d992973f24aa725d07868b467d1ddfceafb
subpackages:
- quantile
- name: github.com/bgentry/speakeasy
version: 36e9cfdd690967f4f690c6edcc9ffacd006014a0
- name: github.com/boltdb/bolt
version: 583e8937c61f1af6513608ccc75c97b6abdf4ff9
- name: github.com/cockroachdb/cmux
version: 112f0506e7743d64a6eb8fedbcff13d9979bbf92
version: 4aabc24848ce5fd31929f7d1e4ea74d3709c14cd
- name: github.com/coreos/bbolt
version: 48ea1b39c25fc1bab3506fbc712ecbaa842c4d2d
- name: github.com/coreos/go-semver
version: 568e959cd89871e61434c1143528d9162da89ef2
version: 8ab6407b697782a06568d4b7f1db25550ec2e4c6
subpackages:
- semver
- name: github.com/coreos/go-systemd
version: 48702e0da86bd25e76cfef347e2adeb434a0d0a6
version: d2196463941895ee908e13531a23a39feb9e1243
subpackages:
- daemon
- journal
@ -27,28 +25,43 @@ imports:
- capnslog
- dlopen
- name: github.com/cpuguy83/go-md2man
version: a65d4d2de4d5f7c74868dfa9b202a3c8be315aaa
version: 23709d0847197db6021a51fdb193e66e9222d4e7
subpackages:
- md2man
- name: github.com/dgrijalva/jwt-go
version: d2709f9f1f31ebcda9651b03077758c1f3a0018c
- name: github.com/dustin/go-humanize
version: 8929fe90cee4b2cb9deb468b51fb34eba64d1bf0
version: bb3d318650d48840a39aa21a027c6630e198e626
- name: github.com/ghodss/yaml
version: 73d445a93680fa1a78ae23a5839bad48f32ba1ee
version: 0ca9ea5df5451ffdf184b4428c902747c2c11cd7
- name: github.com/gogo/protobuf
version: 909568be09de550ed094403c2bf8a261b5bb730a
version: 342cbe0a04158f6dcb03ca0079991a51a4248c02
subpackages:
- gogoproto
- proto
- protoc-gen-gogo/descriptor
- name: github.com/golang/groupcache
version: 02826c3e79038b59d737d3b1c0a1d937f71a4433
subpackages:
- lru
- name: github.com/golang/protobuf
version: 4bd1920723d7b7c925de087aa32e2187708897f7
version: 1e59b77b52bf8e4b449a57e6f79f21226d571845
subpackages:
- jsonpb
- proto
- ptypes
- ptypes/any
- ptypes/duration
- ptypes/struct
- ptypes/timestamp
- name: github.com/google/btree
version: 925471ac9e2131377a91e1595defec898166fe49
- name: github.com/gorilla/websocket
version: 4201258b820c74ac8e6922fc9e6b52f71fe46f8d
- name: github.com/grpc-ecosystem/go-grpc-prometheus
version: 6b7015e65d366bf3f19b2b2a000a831940f0f7e0
version: 0dafe0d496ea71181bf2dd039e7e3f44b6bd11a7
- name: github.com/grpc-ecosystem/grpc-gateway
version: 84398b94e188ee336f307779b57b3aa91af7063c
version: 8cc3a55af3bcf171a1c23a90c4df9cf591706104
subpackages:
- runtime
- runtime/internal
@ -57,61 +70,79 @@ imports:
version: 76626ae9c91c4f2a10f34cad8ce83ea42c93bb75
- name: github.com/jonboulle/clockwork
version: 2eee05ed794112d45db504eb05aa693efd2b8b09
- name: github.com/karlseguin/ccache
version: a2d62155777b39595c825ed3824279e642a5db3c
- name: github.com/kr/pty
version: f7ee69f31298ecbe5d2b349c711e2547a617d398
version: 2c10821df3c3cf905230d078702dfbe9404c9b23
- name: github.com/mattn/go-runewidth
version: 737072b4e32b7a5018b4a7125da8d12de90e8045
version: 9e777a8366cce605130a531d2cd6363d07ad7317
subpackages:
- runewidth.go
- name: github.com/matttproud/golang_protobuf_extensions
version: c12348ce28de40eed0136aa2b644d0ee0650e56c
subpackages:
- pbutil
- name: github.com/olekukonko/tablewriter
version: cca8bbc0798408af109aaaa239cbd2634846b340
version: a0225b3f23b5ce0cbec6d7a66a968f8a59eca9c4
- name: github.com/prometheus/client_golang
version: c5b7fccd204277076155f10851dad72b76a49317
version: 5cec1d0429b02e4323e042eb04dafdb079ddf568
subpackages:
- prometheus
- prometheus/promhttp
- name: github.com/prometheus/client_model
version: fa8ad6fec33561be4280a8f0514318c79d7f6cb6
version: 6f3806018612930941127f2a7c6c453ba2c527d2
subpackages:
- go
- name: github.com/prometheus/common
version: 195bde7883f7c39ea62b0d92ab7359b5327065cb
version: e3fb1a1acd7605367a2b378bc2e2f893c05174b7
subpackages:
- expfmt
- internal/bitbucket.org/ww/goautoneg
- model
- name: github.com/prometheus/procfs
version: fcdb11ccb4389efb1b210b7ffb623ab71c5fdd60
version: a6e9df898b1336106c743392c48ee0b71f5c4efa
subpackages:
- xfs
- name: github.com/russross/blackfriday
version: 5f33e7b7878355cd2b7e6b8eefc48a5472c69f70
- name: github.com/shurcooL/sanitized_anchor_name
version: 1dba4b3954bc059efc3991ec364f9f9a35f597d2
version: 4048872b16cc0fc2c5fd9eacf0ed2c2fedaa0c8c
- name: github.com/sirupsen/logrus
version: f006c2ac4710855cf0f916dd6b77acf6b048dc6e
- name: github.com/soheilhy/cmux
version: bb79a83465015a27a175925ebd155e660f55e9f1
- name: github.com/spf13/cobra
version: 1c44ec8d3f1552cac48999f9306da23c4d8a288b
- name: github.com/spf13/pflag
version: 08b1a584251b5b62f458943640fc8ebd4d50aaa5
- name: github.com/stretchr/testify
version: 976c720a22c8eb4eb6a0b4348ad85ad12491a506
version: e57e3eeb33f795204c1ca35f56c44f83227c6e66
- name: github.com/tmc/grpc-websocket-proxy
version: 89b8d40f7ca833297db804fcb3be53a76d01c238
subpackages:
- assert
- wsproxy
- name: github.com/ugorji/go
version: ded73eae5db7e7a0ef6f55aace87a2873c5d2b74
version: bdcc60b419d136a85cdf2e7cbcac34b3f1cd6e57
subpackages:
- codec
- name: github.com/urfave/cli
version: 1efa31f08b9333f1bd4882d61f9d668a70cd902e
- name: github.com/xiang90/probing
version: 07dd2e8dfe18522e9c447ba95f2fe95262f63bb2
- name: go.uber.org/atomic
version: 8474b86a5a6f79c443ce4b2992817ff32cf208b8
- name: go.uber.org/multierr
version: 3c4937480c32f4c13a875a1829af76c98ca3d40a
- name: go.uber.org/zap
version: 35aad584952c3e7020db7b839f6b102de6271f89
subpackages:
- buffer
- internal/bufferpool
- internal/color
- internal/exit
- zapcore
- name: golang.org/x/crypto
version: 1351f936d976c60a0a48d728281922cf63eafb8d
version: 9419663f5a44be8b34ca85f08abc5fe1be11f8a3
subpackages:
- bcrypt
- blowfish
- ssh/terminal
- name: golang.org/x/net
version: f2499483f923065a842d38eb4c7f1927e6fc6e6d
version: 66aacef3dd8a676686c7ae3716979581e8b03c47
subpackages:
- context
- http2
@ -121,34 +152,48 @@ imports:
- lex/httplex
- trace
- name: golang.org/x/sys
version: 478fcf54317e52ab69f40bb4c7a1520288d7f7ea
version: ebfc5b4631820b793c9010c87fd8fef0f39eb082
subpackages:
- unix
- windows
- name: golang.org/x/text
version: b19bf474d317b857955b12035d2c5acb57ce8b01
subpackages:
- secure/bidirule
- transform
- unicode/bidi
- unicode/norm
- name: golang.org/x/time
version: a4bde12657593d5e90d0533a3e4fd95e635124cb
version: c06e80d9300e4443158a03817b8a8cb37d230320
subpackages:
- rate
- name: google.golang.org/grpc
version: 777daa17ff9b5daef1cfdf915088a2ada3332bf0
- name: google.golang.org/genproto
version: 09f6ed296fc66555a25fe4ce95173148778dfa85
subpackages:
- googleapis/rpc/status
- name: google.golang.org/grpc
version: 5b3c4e850e90a4cf6a20ebd46c8b32a0a3afcb9e
subpackages:
- balancer
- codes
- connectivity
- credentials
- grpclb/grpc_lb_v1/messages
- grpclog
- health
- health/grpc_health_v1
- internal
- keepalive
- metadata
- naming
- peer
- resolver
- stats
- status
- tap
- transport
- name: gopkg.in/cheggaaa/pb.v1
version: 226d21d43a305fac52b3a104ef83e721b15275e0
- name: gopkg.in/yaml.v2
version: 53feefa2559fb8dfa8d81baad31be332c97d6c77
testImports:
- name: github.com/davecgh/go-spew
version: 6d212800a42e8ab5c146b8ace3490ee17e5225f9
subpackages:
- spew
- name: github.com/pmezard/go-difflib
version: d8ed2627bdf02c080bf22230dbb337003b7aba2d
subpackages:
- difflib
version: cd8b52f8269e0feb286dfeef29f8fe4d5b397e0b
testImports: []

107
vendor/github.com/coreos/etcd/glide.yaml generated vendored
View File

@ -1,42 +1,53 @@
package: github.com/coreos/etcd
ignore:
- google.golang.org/appengine
import:
- package: github.com/bgentry/speakeasy
version: 36e9cfdd690967f4f690c6edcc9ffacd006014a0
- package: github.com/boltdb/bolt
version: v1.3.0
- package: github.com/cockroachdb/cmux
version: 112f0506e7743d64a6eb8fedbcff13d9979bbf92
version: v0.1.0
- package: github.com/coreos/bbolt
version: v1.3.1-coreos.6
- package: github.com/coreos/go-semver
version: 568e959cd89871e61434c1143528d9162da89ef2
version: v0.2.0
subpackages:
- semver
- package: github.com/coreos/go-systemd
version: v14
version: v15
subpackages:
- daemon
- journal
- util
- package: go.uber.org/zap
version: v1.7.1
- package: github.com/coreos/pkg
version: v3
subpackages:
- capnslog
- package: github.com/cpuguy83/go-md2man
version: 23709d0847197db6021a51fdb193e66e9222d4e7
- package: github.com/dustin/go-humanize
version: 8929fe90cee4b2cb9deb468b51fb34eba64d1bf0
version: bb3d318650d48840a39aa21a027c6630e198e626
- package: github.com/ghodss/yaml
version: 73d445a93680fa1a78ae23a5839bad48f32ba1ee
version: v1.0.0
- package: github.com/gogo/protobuf
version: v0.3
version: v0.5
subpackages:
- proto
- gogoproto
- package: github.com/gorilla/websocket
version: 4201258b820c74ac8e6922fc9e6b52f71fe46f8d
- package: github.com/golang/groupcache
version: 02826c3e79038b59d737d3b1c0a1d937f71a4433
subpackages:
- lru
- package: github.com/golang/protobuf
version: 4bd1920723d7b7c925de087aa32e2187708897f7
version: 1e59b77b52bf8e4b449a57e6f79f21226d571845
subpackages:
- jsonpb
- proto
- package: github.com/google/btree
version: 925471ac9e2131377a91e1595defec898166fe49
- package: github.com/grpc-ecosystem/grpc-gateway
version: 84398b94e188ee336f307779b57b3aa91af7063c
version: v1.3.0
subpackages:
- runtime
- runtime/internal
@ -44,46 +55,63 @@ import:
- package: github.com/jonboulle/clockwork
version: v0.1.0
- package: github.com/kr/pty
version: f7ee69f31298ecbe5d2b349c711e2547a617d398
version: v1.0.0
- package: github.com/olekukonko/tablewriter
version: cca8bbc0798408af109aaaa239cbd2634846b340
version: a0225b3f23b5ce0cbec6d7a66a968f8a59eca9c4
- package: github.com/mattn/go-runewidth
version: v0.0.2
subpackages:
- runewidth.go
- package: github.com/prometheus/client_golang
version: v0.8.0
version: 5cec1d0429b02e4323e042eb04dafdb079ddf568
subpackages:
- prometheus
- prometheus/promhttp
- package: github.com/prometheus/client_model
version: 6f3806018612930941127f2a7c6c453ba2c527d2
subpackages:
- go
- package: github.com/prometheus/common
version: e3fb1a1acd7605367a2b378bc2e2f893c05174b7
- package: github.com/prometheus/procfs
version: a6e9df898b1336106c743392c48ee0b71f5c4efa
subpackages:
- xfs
- package: github.com/grpc-ecosystem/go-grpc-prometheus
version: 0dafe0d496ea71181bf2dd039e7e3f44b6bd11a7
- package: github.com/spf13/cobra
version: 1c44ec8d3f1552cac48999f9306da23c4d8a288b
- package: github.com/spf13/pflag
version: 08b1a584251b5b62f458943640fc8ebd4d50aaa5
version: v1.0.0
- package: github.com/ugorji/go
version: ded73eae5db7e7a0ef6f55aace87a2873c5d2b74
version: bdcc60b419d136a85cdf2e7cbcac34b3f1cd6e57
subpackages:
- codec
- package: github.com/urfave/cli
version: v1.18.0
- package: github.com/xiang90/probing
version: 07dd2e8dfe18522e9c447ba95f2fe95262f63bb2
- package: github.com/grpc-ecosystem/go-grpc-prometheus
version: v1.1
version: 0.0.1
- package: golang.org/x/crypto
version: 1351f936d976c60a0a48d728281922cf63eafb8d
version: 9419663f5a44be8b34ca85f08abc5fe1be11f8a3
subpackages:
- bcrypt
- blowfish
- package: golang.org/x/net
version: f2499483f923065a842d38eb4c7f1927e6fc6e6d
version: 66aacef3dd8a676686c7ae3716979581e8b03c47
subpackages:
- context
- http2
- http2/hpack
- internal/timeseries
- trace
- package: golang.org/x/sys
version: ebfc5b4631820b793c9010c87fd8fef0f39eb082
- package: golang.org/x/time
version: a4bde12657593d5e90d0533a3e4fd95e635124cb
version: c06e80d9300e4443158a03817b8a8cb37d230320
subpackages:
- rate
- package: google.golang.org/grpc
version: v1.0.4
version: v1.7.5
subpackages:
- codes
- credentials
@ -93,13 +121,32 @@ import:
- naming
- peer
- transport
- health
- health/grpc_health_v1
- package: gopkg.in/cheggaaa/pb.v1
version: v1.0.2
- package: gopkg.in/yaml.v2
version: 53feefa2559fb8dfa8d81baad31be332c97d6c77
- package: github.com/stretchr/testify
version: 976c720a22c8eb4eb6a0b4348ad85ad12491a506
version: cd8b52f8269e0feb286dfeef29f8fe4d5b397e0b
- package: github.com/dgrijalva/jwt-go
version: v3.0.0
- package: google.golang.org/genproto
version: 09f6ed296fc66555a25fe4ce95173148778dfa85
subpackages:
- assert
- package: github.com/karlseguin/ccache
version: v2.0.2
- googleapis/rpc/status
- package: golang.org/x/text
version: b19bf474d317b857955b12035d2c5acb57ce8b01
subpackages:
- secure/bidirule
- transform
- unicode/bidi
- unicode/norm
- package: github.com/russross/blackfriday
version: 4048872b16cc0fc2c5fd9eacf0ed2c2fedaa0c8c
- package: github.com/sirupsen/logrus
version: v1.0.3
- package: github.com/soheilhy/cmux
version: v0.1.3
- package: github.com/tmc/grpc-websocket-proxy
version: 89b8d40f7ca833297db804fcb3be53a76d01c238
subpackages:
- wsproxy

49
vendor/github.com/coreos/etcd/meeting.ics generated vendored Normal file
View File

@ -0,0 +1,49 @@
BEGIN:VCALENDAR
PRODID:-//Google Inc//Google Calendar 70.9054//EN
VERSION:2.0
CALSCALE:GREGORIAN
METHOD:REPLY
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
X-LIC-LOCATION:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:19700308T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:19701101T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20180116T110000
DTEND;TZID=America/Los_Angeles:20180116T115000
RRULE:FREQ=WEEKLY;INTERVAL=2;BYDAY=TU
DTSTAMP:20171213T131221Z
ORGANIZER;CN=Gyuho Lee:mailto:gyu_ho.lee@coreos.com
UID:11ivec3kg2egsng3vrl8t5alar@google.com
CREATED:20171212T194217Z
DESCRIPTION:<br>Please add your discussion items to the meeting notes.<br><
br>Meeting notes<br><a href="https://docs.google.com/document/d/1DbVXOHvd9s
cFsSmL2oNg4YGOHJdXqtx583DmeVWrB_M/edit?usp=sharing">https://docs.google.com
/document/d/1DbVXOHvd9scFsSmL2oNg4YGOHJdXqtx583DmeVWrB_M/edit?usp=sharing</
a><br><br>Zoom meeting<br><a href="https://www.google.com/url?q=https%3A%2F
%2Fcoreos.zoom.us%2Fj%2F854793406&amp\;sa=D&amp\;ust=1509474820520000&amp\;
usg=AFQjCNFIOIfx1O_dgC-1N5YLyLOMa7D3Dg" target="_blank">https://coreos.zoom
.us/j/854793406</a><br><br>Slack<br><a href="https://www.google.com/url?q=h
ttps%3A%2F%2Fkubernetes.slack.com&amp\;sa=D&amp\;ust=1513114941738000&amp\;
usg=AFQjCNHbdDPJcyZ2tVATRqTQDuZDFzGoRQ" target="_blank">https://kubernetes.
slack.com</a> <i>#etcd</i><br><br><i><br></i>
LAST-MODIFIED:20171213T131220Z
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:etcd meeting
TRANSP:OPAQUE
END:VEVENT
END:VCALENDAR

11
vendor/github.com/coreos/etcd/pkg/fileutil/fileutil.go generated vendored
View File

@ -17,9 +17,10 @@ package fileutil
import (
"fmt"
"io"
"io/ioutil"
"os"
"path"
"path/filepath"
"sort"
"github.com/coreos/pkg/capnslog"
@ -39,7 +40,7 @@ var (
// IsDirWriteable checks if dir is writable by writing and removing a file
// to dir. It returns nil if dir is writable.
func IsDirWriteable(dir string) error {
f := path.Join(dir, ".touch")
f := filepath.Join(dir, ".touch")
if err := ioutil.WriteFile(f, []byte(""), PrivateFileMode); err != nil {
return err
}
@ -101,11 +102,11 @@ func Exist(name string) bool {
// shorten the length of the file.
func ZeroToEnd(f *os.File) error {
// TODO: support FALLOC_FL_ZERO_RANGE
off, err := f.Seek(0, os.SEEK_CUR)
off, err := f.Seek(0, io.SeekCurrent)
if err != nil {
return err
}
lenf, lerr := f.Seek(0, os.SEEK_END)
lenf, lerr := f.Seek(0, io.SeekEnd)
if lerr != nil {
return lerr
}
@ -116,6 +117,6 @@ func ZeroToEnd(f *os.File) error {
if err = Preallocate(f, lenf, true); err != nil {
return err
}
_, err = f.Seek(off, os.SEEK_SET)
_, err = f.Seek(off, io.SeekStart)
return err
}

3
vendor/github.com/coreos/etcd/pkg/fileutil/lock_linux.go generated vendored
View File

@ -17,6 +17,7 @@
package fileutil
import (
"io"
"os"
"syscall"
)
@ -36,7 +37,7 @@ const (
var (
wrlck = syscall.Flock_t{
Type: syscall.F_WRLCK,
Whence: int16(os.SEEK_SET),
Whence: int16(io.SeekStart),
Start: 0,
Len: 0,
}

2
vendor/github.com/coreos/etcd/pkg/fileutil/lock_windows.go generated vendored
View File

@ -121,5 +121,5 @@ func lockFileEx(h syscall.Handle, flags, locklow, lockhigh uint32, ol *syscall.O
err = syscall.EINVAL
}
}
return
return err
}

15
vendor/github.com/coreos/etcd/pkg/fileutil/preallocate.go generated vendored
View File

@ -14,7 +14,10 @@
package fileutil
import "os"
import (
"io"
"os"
)
// Preallocate tries to allocate the space for given
// file. This operation is only supported on linux by a
@ -22,6 +25,10 @@ import "os"
// If the operation is unsupported, no error will be returned.
// Otherwise, the error encountered will be returned.
func Preallocate(f *os.File, sizeInBytes int64, extendFile bool) error {
if sizeInBytes == 0 {
// fallocate will return EINVAL if length is 0; skip
return nil
}
if extendFile {
return preallocExtend(f, sizeInBytes)
}
@ -29,15 +36,15 @@ func Preallocate(f *os.File, sizeInBytes int64, extendFile bool) error {
}
func preallocExtendTrunc(f *os.File, sizeInBytes int64) error {
curOff, err := f.Seek(0, os.SEEK_CUR)
curOff, err := f.Seek(0, io.SeekCurrent)
if err != nil {
return err
}
size, err := f.Seek(sizeInBytes, os.SEEK_END)
size, err := f.Seek(sizeInBytes, io.SeekEnd)
if err != nil {
return err
}
if _, err = f.Seek(curOff, os.SEEK_SET); err != nil {
if _, err = f.Seek(curOff, io.SeekStart); err != nil {
return err
}
if sizeInBytes > size {

22
vendor/github.com/coreos/etcd/pkg/fileutil/preallocate_darwin.go generated vendored
View File

@ -30,6 +30,8 @@ func preallocExtend(f *os.File, sizeInBytes int64) error {
}
func preallocFixed(f *os.File, sizeInBytes int64) error {
// allocate all requested space or no space at all
// TODO: allocate contiguous space on disk with F_ALLOCATECONTIG flag
fstore := &syscall.Fstore_t{
Flags: syscall.F_ALLOCATEALL,
Posmode: syscall.F_PEOFPOSMODE,
@ -39,5 +41,25 @@ func preallocFixed(f *os.File, sizeInBytes int64) error {
if errno == 0 || errno == syscall.ENOTSUP {
return nil
}
// wrong argument to fallocate syscall
if errno == syscall.EINVAL {
// filesystem "st_blocks" are allocated in the units of
// "Allocation Block Size" (run "diskutil info /" command)
var stat syscall.Stat_t
syscall.Fstat(int(f.Fd()), &stat)
// syscall.Statfs_t.Bsize is "optimal transfer block size"
// and contains matching 4096 value when latest OS X kernel
// supports 4,096 KB filesystem block size
var statfs syscall.Statfs_t
syscall.Fstatfs(int(f.Fd()), &statfs)
blockSize := int64(statfs.Bsize)
if stat.Blocks*blockSize >= sizeInBytes {
// enough blocks are already allocated
return nil
}
}
return errno
}

4
vendor/github.com/coreos/etcd/pkg/fileutil/purge.go generated vendored
View File

@ -16,7 +16,7 @@ package fileutil
import (
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
@ -45,7 +45,7 @@ func purgeFile(dirname string, suffix string, max uint, interval time.Duration,
sort.Strings(newfnames)
fnames = newfnames
for len(newfnames) > int(max) {
f := path.Join(dirname, newfnames[0])
f := filepath.Join(dirname, newfnames[0])
l, err := TryLockFile(f, os.O_WRONLY, PrivateFileMode)
if err != nil {
break

9
vendor/github.com/coreos/etcd/pkg/httputil/httputil.go generated vendored
View File

@ -13,15 +13,6 @@ import (
"net/http"
)
func RequestCanceler(req *http.Request) func() {
ch := make(chan struct{})
req.Cancel = ch
return func() {
close(ch)
}
}
// GracefulClose drains http.Response.Body until it hits EOF
// and closes it. This prevents TCP/TLS connections from closing,
// therefore available for reuse.

29
vendor/github.com/coreos/etcd/wal/wal_windows.go → vendor/github.com/coreos/etcd/pkg/report/doc.go generated vendored
View File

@ -12,30 +12,5 @@
// See the License for the specific language governing permissions and
// limitations under the License.
package wal
import (
"os"
"github.com/coreos/etcd/wal/walpb"
)
func (w *WAL) renameWal(tmpdirpath string) (*WAL, error) {
// rename of directory with locked files doesn't work on
// windows; close the WAL to release the locks so the directory
// can be renamed
w.Close()
if err := os.Rename(tmpdirpath, w.dir); err != nil {
return nil, err
}
// reopen and relock
newWAL, oerr := Open(w.dir, walpb.Snapshot{})
if oerr != nil {
return nil, oerr
}
if _, _, _, err := newWAL.ReadAll(); err != nil {
newWAL.Close()
return nil, err
}
return newWAL, nil
}
// Package report generates human-readable benchmark reports.
package report

278
vendor/github.com/coreos/etcd/pkg/report/report.go generated vendored Normal file
View File

@ -0,0 +1,278 @@
// Copyright 2014 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// the file is borrowed from github.com/rakyll/boom/boomer/print.go
package report
import (
"fmt"
"math"
"sort"
"strings"
"time"
)
const (
barChar = "∎"
)
// Result describes the timings for an operation.
type Result struct {
Start time.Time
End time.Time
Err error
Weight float64
}
func (res *Result) Duration() time.Duration { return res.End.Sub(res.Start) }
type report struct {
results chan Result
precision string
stats Stats
sps *secondPoints
}
// Stats exposes results raw data.
type Stats struct {
AvgTotal float64
Fastest float64
Slowest float64
Average float64
Stddev float64
RPS float64
Total time.Duration
ErrorDist map[string]int
Lats []float64
TimeSeries TimeSeries
}
func (s *Stats) copy() Stats {
ss := *s
ss.ErrorDist = copyMap(ss.ErrorDist)
ss.Lats = copyFloats(ss.Lats)
return ss
}
// Report processes a result stream until it is closed, then produces a
// string with information about the consumed result data.
type Report interface {
Results() chan<- Result
// Run returns results in print-friendly format.
Run() <-chan string
// Stats returns results in raw data.
Stats() <-chan Stats
}
func NewReport(precision string) Report { return newReport(precision) }
func newReport(precision string) *report {
r := &report{
results: make(chan Result, 16),
precision: precision,
}
r.stats.ErrorDist = make(map[string]int)
return r
}
func NewReportSample(precision string) Report {
r := NewReport(precision).(*report)
r.sps = newSecondPoints()
return r
}
func (r *report) Results() chan<- Result { return r.results }
func (r *report) Run() <-chan string {
donec := make(chan string, 1)
go func() {
defer close(donec)
r.processResults()
donec <- r.String()
}()
return donec
}
func (r *report) Stats() <-chan Stats {
donec := make(chan Stats, 1)
go func() {
defer close(donec)
r.processResults()
s := r.stats.copy()
if r.sps != nil {
s.TimeSeries = r.sps.getTimeSeries()
}
donec <- s
}()
return donec
}
func copyMap(m map[string]int) (c map[string]int) {
c = make(map[string]int, len(m))
for k, v := range m {
c[k] = v
}
return c
}
func copyFloats(s []float64) (c []float64) {
c = make([]float64, len(s))
copy(c, s)
return c
}
func (r *report) String() (s string) {
if len(r.stats.Lats) > 0 {
s += fmt.Sprintf("\nSummary:\n")
s += fmt.Sprintf(" Total:\t%s.\n", r.sec2str(r.stats.Total.Seconds()))
s += fmt.Sprintf(" Slowest:\t%s.\n", r.sec2str(r.stats.Slowest))
s += fmt.Sprintf(" Fastest:\t%s.\n", r.sec2str(r.stats.Fastest))
s += fmt.Sprintf(" Average:\t%s.\n", r.sec2str(r.stats.Average))
s += fmt.Sprintf(" Stddev:\t%s.\n", r.sec2str(r.stats.Stddev))
s += fmt.Sprintf(" Requests/sec:\t"+r.precision+"\n", r.stats.RPS)
s += r.histogram()
s += r.sprintLatencies()
if r.sps != nil {
s += fmt.Sprintf("%v\n", r.sps.getTimeSeries())
}
}
if len(r.stats.ErrorDist) > 0 {
s += r.errors()
}
return s
}
func (r *report) sec2str(sec float64) string { return fmt.Sprintf(r.precision+" secs", sec) }
type reportRate struct{ *report }
func NewReportRate(precision string) Report {
return &reportRate{NewReport(precision).(*report)}
}
func (r *reportRate) String() string {
return fmt.Sprintf(" Requests/sec:\t"+r.precision+"\n", r.stats.RPS)
}
func (r *report) processResult(res *Result) {
if res.Err != nil {
r.stats.ErrorDist[res.Err.Error()]++
return
}
dur := res.Duration()
r.stats.Lats = append(r.stats.Lats, dur.Seconds())
r.stats.AvgTotal += dur.Seconds()
if r.sps != nil {
r.sps.Add(res.Start, dur)
}
}
func (r *report) processResults() {
st := time.Now()
for res := range r.results {
r.processResult(&res)
}
r.stats.Total = time.Since(st)
r.stats.RPS = float64(len(r.stats.Lats)) / r.stats.Total.Seconds()
r.stats.Average = r.stats.AvgTotal / float64(len(r.stats.Lats))
for i := range r.stats.Lats {
dev := r.stats.Lats[i] - r.stats.Average
r.stats.Stddev += dev * dev
}
r.stats.Stddev = math.Sqrt(r.stats.Stddev / float64(len(r.stats.Lats)))
sort.Float64s(r.stats.Lats)
if len(r.stats.Lats) > 0 {
r.stats.Fastest = r.stats.Lats[0]
r.stats.Slowest = r.stats.Lats[len(r.stats.Lats)-1]
}
}
var pctls = []float64{10, 25, 50, 75, 90, 95, 99, 99.9}
// Percentiles returns percentile distribution of float64 slice.
func Percentiles(nums []float64) (pcs []float64, data []float64) {
return pctls, percentiles(nums)
}
func percentiles(nums []float64) (data []float64) {
data = make([]float64, len(pctls))
j := 0
n := len(nums)
for i := 0; i < n && j < len(pctls); i++ {
current := float64(i) * 100.0 / float64(n)
if current >= pctls[j] {
data[j] = nums[i]
j++
}
}
return data
}
func (r *report) sprintLatencies() string {
data := percentiles(r.stats.Lats)
s := fmt.Sprintf("\nLatency distribution:\n")
for i := 0; i < len(pctls); i++ {
if data[i] > 0 {
s += fmt.Sprintf(" %v%% in %s.\n", pctls[i], r.sec2str(data[i]))
}
}
return s
}
func (r *report) histogram() string {
bc := 10
buckets := make([]float64, bc+1)
counts := make([]int, bc+1)
bs := (r.stats.Slowest - r.stats.Fastest) / float64(bc)
for i := 0; i < bc; i++ {
buckets[i] = r.stats.Fastest + bs*float64(i)
}
buckets[bc] = r.stats.Slowest
var bi int
var max int
for i := 0; i < len(r.stats.Lats); {
if r.stats.Lats[i] <= buckets[bi] {
i++
counts[bi]++
if max < counts[bi] {
max = counts[bi]
}
} else if bi < len(buckets)-1 {
bi++
}
}
s := fmt.Sprintf("\nResponse time histogram:\n")
for i := 0; i < len(buckets); i++ {
// Normalize bar lengths.
var barLen int
if max > 0 {
barLen = counts[i] * 40 / max
}
s += fmt.Sprintf(" "+r.precision+" [%v]\t|%v\n", buckets[i], counts[i], strings.Repeat(barChar, barLen))
}
return s
}
func (r *report) errors() string {
s := fmt.Sprintf("\nError distribution:\n")
for err, num := range r.stats.ErrorDist {
s += fmt.Sprintf(" [%d]\t%s\n", num, err)
}
return s
}

160
vendor/github.com/coreos/etcd/pkg/report/timeseries.go generated vendored Normal file
View File

@ -0,0 +1,160 @@
// Copyright 2016 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package report
import (
"bytes"
"encoding/csv"
"fmt"
"log"
"math"
"sort"
"sync"
"time"
)
type DataPoint struct {
Timestamp int64
MinLatency time.Duration
AvgLatency time.Duration
MaxLatency time.Duration
ThroughPut int64
}
type TimeSeries []DataPoint
func (t TimeSeries) Swap(i, j int) { t[i], t[j] = t[j], t[i] }
func (t TimeSeries) Len() int { return len(t) }
func (t TimeSeries) Less(i, j int) bool { return t[i].Timestamp < t[j].Timestamp }
type secondPoint struct {
minLatency time.Duration
maxLatency time.Duration
totalLatency time.Duration
count int64
}
type secondPoints struct {
mu sync.Mutex
tm map[int64]secondPoint
}
func newSecondPoints() *secondPoints {
return &secondPoints{tm: make(map[int64]secondPoint)}
}
func (sp *secondPoints) Add(ts time.Time, lat time.Duration) {
sp.mu.Lock()
defer sp.mu.Unlock()
tk := ts.Unix()
if v, ok := sp.tm[tk]; !ok {
sp.tm[tk] = secondPoint{minLatency: lat, maxLatency: lat, totalLatency: lat, count: 1}
} else {
if lat != time.Duration(0) {
v.minLatency = minDuration(v.minLatency, lat)
}
v.maxLatency = maxDuration(v.maxLatency, lat)
v.totalLatency += lat
v.count++
sp.tm[tk] = v
}
}
func (sp *secondPoints) getTimeSeries() TimeSeries {
sp.mu.Lock()
defer sp.mu.Unlock()
var (
minTs int64 = math.MaxInt64
maxTs int64 = -1
)
for k := range sp.tm {
if minTs > k {
minTs = k
}
if maxTs < k {
maxTs = k
}
}
for ti := minTs; ti < maxTs; ti++ {
if _, ok := sp.tm[ti]; !ok { // fill-in empties
sp.tm[ti] = secondPoint{totalLatency: 0, count: 0}
}
}
var (
tslice = make(TimeSeries, len(sp.tm))
i int
)
for k, v := range sp.tm {
var lat time.Duration
if v.count > 0 {
lat = time.Duration(v.totalLatency) / time.Duration(v.count)
}
tslice[i] = DataPoint{
Timestamp: k,
MinLatency: v.minLatency,
AvgLatency: lat,
MaxLatency: v.maxLatency,
ThroughPut: v.count,
}
i++
}
sort.Sort(tslice)
return tslice
}
func (t TimeSeries) String() string {
buf := new(bytes.Buffer)
wr := csv.NewWriter(buf)
if err := wr.Write([]string{"UNIX-SECOND", "MIN-LATENCY-MS", "AVG-LATENCY-MS", "MAX-LATENCY-MS", "AVG-THROUGHPUT"}); err != nil {
log.Fatal(err)
}
rows := [][]string{}
for i := range t {
row := []string{
fmt.Sprintf("%d", t[i].Timestamp),
t[i].MinLatency.String(),
t[i].AvgLatency.String(),
t[i].MaxLatency.String(),
fmt.Sprintf("%d", t[i].ThroughPut),
}
rows = append(rows, row)
}
if err := wr.WriteAll(rows); err != nil {
log.Fatal(err)
}
wr.Flush()
if err := wr.Error(); err != nil {
log.Fatal(err)
}
return fmt.Sprintf("\nSample in one second (unix latency throughput):\n%s", buf.String())
}
func minDuration(a, b time.Duration) time.Duration {
if a < b {
return a
}
return b
}
func maxDuration(a, b time.Duration) time.Duration {
if a > b {
return a
}
return b
}

101
vendor/github.com/coreos/etcd/pkg/report/weighted.go generated vendored Normal file
View File

@ -0,0 +1,101 @@
// Copyright 2017 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// the file is borrowed from github.com/rakyll/boom/boomer/print.go
package report
import (
"time"
)
type weightedReport struct {
baseReport Report
report *report
results chan Result
weightTotal float64
}
// NewWeightedReport returns a report that includes
// both weighted and unweighted statistics.
func NewWeightedReport(r Report, precision string) Report {
return &weightedReport{
baseReport: r,
report: newReport(precision),
results: make(chan Result, 16),
}
}
func (wr *weightedReport) Results() chan<- Result { return wr.results }
func (wr *weightedReport) Run() <-chan string {
donec := make(chan string, 2)
go func() {
defer close(donec)
basec, rc := make(chan string, 1), make(chan Stats, 1)
go func() { basec <- (<-wr.baseReport.Run()) }()
go func() { rc <- (<-wr.report.Stats()) }()
go wr.processResults()
wr.report.stats = wr.reweighStat(<-rc)
donec <- wr.report.String()
donec <- (<-basec)
}()
return donec
}
func (wr *weightedReport) Stats() <-chan Stats {
donec := make(chan Stats, 2)
go func() {
defer close(donec)
basec, rc := make(chan Stats, 1), make(chan Stats, 1)
go func() { basec <- (<-wr.baseReport.Stats()) }()
go func() { rc <- (<-wr.report.Stats()) }()
go wr.processResults()
donec <- wr.reweighStat(<-rc)
donec <- (<-basec)
}()
return donec
}
func (wr *weightedReport) processResults() {
defer close(wr.report.results)
defer close(wr.baseReport.Results())
for res := range wr.results {
wr.processResult(res)
wr.baseReport.Results() <- res
}
}
func (wr *weightedReport) processResult(res Result) {
if res.Err != nil {
wr.report.results <- res
return
}
if res.Weight == 0 {
res.Weight = 1.0
}
wr.weightTotal += res.Weight
res.End = res.Start.Add(time.Duration(float64(res.End.Sub(res.Start)) / res.Weight))
res.Weight = 1.0
wr.report.results <- res
}
func (wr *weightedReport) reweighStat(s Stats) Stats {
weightCoef := wr.weightTotal / float64(len(s.Lats))
// weight > 1 => processing more than one request
s.RPS *= weightCoef
s.AvgTotal *= weightCoef * weightCoef
return s
}

51
vendor/github.com/coreos/etcd/pkg/tlsutil/cipher_suites.go generated vendored Normal file
View File

@ -0,0 +1,51 @@
// Copyright 2018 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package tlsutil
import "crypto/tls"
// cipher suites implemented by Go
// https://github.com/golang/go/blob/dev.boringcrypto.go1.10/src/crypto/tls/cipher_suites.go
var cipherSuites = map[string]uint16{
"TLS_RSA_WITH_RC4_128_SHA": tls.TLS_RSA_WITH_RC4_128_SHA,
"TLS_RSA_WITH_3DES_EDE_CBC_SHA": tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,
"TLS_RSA_WITH_AES_128_CBC_SHA": tls.TLS_RSA_WITH_AES_128_CBC_SHA,
"TLS_RSA_WITH_AES_256_CBC_SHA": tls.TLS_RSA_WITH_AES_256_CBC_SHA,
"TLS_RSA_WITH_AES_128_CBC_SHA256": tls.TLS_RSA_WITH_AES_128_CBC_SHA256,
"TLS_RSA_WITH_AES_128_GCM_SHA256": tls.TLS_RSA_WITH_AES_128_GCM_SHA256,
"TLS_RSA_WITH_AES_256_GCM_SHA384": tls.TLS_RSA_WITH_AES_256_GCM_SHA384,
"TLS_ECDHE_ECDSA_WITH_RC4_128_SHA": tls.TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA": tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA": tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
"TLS_ECDHE_RSA_WITH_RC4_128_SHA": tls.TLS_ECDHE_RSA_WITH_RC4_128_SHA,
"TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA": tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA": tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA": tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256": tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256": tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256": tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256": tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384": tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384": tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305": tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305": tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
}
// GetCipherSuite returns the corresponding cipher suite,
// and boolean value if it is supported.
func GetCipherSuite(s string) (uint16, bool) {
v, ok := cipherSuites[s]
return v, ok
}

19
vendor/github.com/coreos/etcd/pkg/transport/fixtures/ca-csr.json generated vendored Normal file
View File

@ -0,0 +1,19 @@
{
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"O": "etcd",
"OU": "etcd Security",
"L": "San Francisco",
"ST": "California",
"C": "USA"
}
],
"CN": "ca",
"ca": {
"expiry": "87600h"
}
}

22
vendor/github.com/coreos/etcd/pkg/transport/fixtures/ca.crt generated vendored Normal file
View File

@ -0,0 +1,22 @@
-----BEGIN CERTIFICATE-----
MIIDsTCCApmgAwIBAgIUZzOo4zcHY/nEXY1PD8A7povXlWUwDQYJKoZIhvcNAQEL
BQAwbzEMMAoGA1UEBhMDVVNBMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH
Ew1TYW4gRnJhbmNpc2NvMQ0wCwYDVQQKEwRldGNkMRYwFAYDVQQLEw1ldGNkIFNl
Y3VyaXR5MQswCQYDVQQDEwJjYTAeFw0xODAxMDIxNjQxMDBaFw0yNzEyMzExNjQx
MDBaMG8xDDAKBgNVBAYTA1VTQTETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UE
BxMNU2FuIEZyYW5jaXNjbzENMAsGA1UEChMEZXRjZDEWMBQGA1UECxMNZXRjZCBT
ZWN1cml0eTELMAkGA1UEAxMCY2EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQDD4Ys48LDWGyojj3Rcr6fnESY+UycaaGoTXADWLPmm+sQR3KcsJxF4054S
d2G+NBfJHZvTHhVqOeqZxNtoqgje4paY2A5TbWBdV+xoGfbakwwngiX1yeF1I54k
KH19zb8rBKAm7xixO60hE2CIYzMuw9lDkwoHpI6/PJdy7jwtytbo2Oac512JiO9Y
dHp9dr3mrCzoKEBRtL1asRKfzp6gBC5rIw5T4jrq37feerV4pDEJX7fvexxVocVm
tT4bmMq3Ap6OFFAzmE/ITI8pXvFaOd9lyebNXQmrreKJLUfEIZa6JulLCYxfkJ8z
+CcNLyn6ZXNMaIZ8G9Hm6VRdRi8/AgMBAAGjRTBDMA4GA1UdDwEB/wQEAwIBBjAS
BgNVHRMBAf8ECDAGAQH/AgECMB0GA1UdDgQWBBRDLNYEX8XI7nM53k1rUR+mpTjQ
NTANBgkqhkiG9w0BAQsFAAOCAQEACDe3Fa1KE/rvVtyCLW/IBfKV01NShFTsb6x8
GrPEQ6NJLZQ2MzdyJgAF2a/nZ9KVgrhGXoyoZBCKP9Dd/JDzSSZcBztfNK8dRv2A
XHBBF6tZ19I+XY9c7/CfhJ2CEYJpeN9r3GKSqV+njkmg8n/On2BTlFsij88plK8H
ORyemc1nQI+ARPSu2r3rJbYa4yI2U6w4L4BTCVImg3bX50GImmXGlwvnJMFik1FX
+0hdfetRxxMZ1pm2Uy6099KkULnSKabZGwRiBUHQJYh0EeuAOQ4a6MG5DRkURWNs
dInjPOLY9/7S5DQKwz/NtqXA8EEymZosHxpiRp+zzKB4XaV9Ig==
-----END CERTIFICATE-----

13
vendor/github.com/coreos/etcd/pkg/transport/fixtures/gencert.json generated vendored Normal file
View File

@ -0,0 +1,13 @@
{
"signing": {
"default": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}

26
vendor/github.com/coreos/etcd/pkg/transport/fixtures/gencerts.sh generated vendored Executable file
View File

@ -0,0 +1,26 @@
#!/bin/bash
if ! [[ "$0" =~ "./gencerts.sh" ]]; then
echo "must be run from 'fixtures'"
exit 255
fi
if ! which cfssl; then
echo "cfssl is not installed"
exit 255
fi
cfssl gencert --initca=true ./ca-csr.json | cfssljson --bare ./ca
mv ca.pem ca.crt
openssl x509 -in ca.crt -noout -text
# generate DNS: localhost, IP: 127.0.0.1, CN: example.com certificates
cfssl gencert \
--ca ./ca.crt \
--ca-key ./ca-key.pem \
--config ./gencert.json \
./server-ca-csr.json | cfssljson --bare ./server
mv server.pem server.crt
mv server-key.pem server.key.insecure
rm -f *.csr *.pem *.stderr *.txt

20
vendor/github.com/coreos/etcd/pkg/transport/fixtures/server-ca-csr.json generated vendored Normal file
View File

@ -0,0 +1,20 @@
{
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"O": "etcd",
"OU": "etcd Security",
"L": "San Francisco",
"ST": "California",
"C": "USA"
}
],
"CN": "example.com",
"hosts": [
"127.0.0.1",
"localhost"
]
}

24
vendor/github.com/coreos/etcd/pkg/transport/fixtures/server.crt generated vendored Normal file
View File

@ -0,0 +1,24 @@
-----BEGIN CERTIFICATE-----
MIIEEjCCAvqgAwIBAgIUIYc+vmysep1pDc2ua/VQEeMFQVAwDQYJKoZIhvcNAQEL
BQAwbzEMMAoGA1UEBhMDVVNBMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQH
Ew1TYW4gRnJhbmNpc2NvMQ0wCwYDVQQKEwRldGNkMRYwFAYDVQQLEw1ldGNkIFNl
Y3VyaXR5MQswCQYDVQQDEwJjYTAeFw0xODAxMDIxNjQxMDBaFw0yNzEyMzExNjQx
MDBaMHgxDDAKBgNVBAYTA1VTQTETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UE
BxMNU2FuIEZyYW5jaXNjbzENMAsGA1UEChMEZXRjZDEWMBQGA1UECxMNZXRjZCBT
ZWN1cml0eTEUMBIGA1UEAxMLZXhhbXBsZS5jb20wggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDEq7aT2BQZfmJ2xpUm8xWJlN0c3cOLVZRH9mIrEutIHmip
BYq3ZIq3q52w+T3sMcaJNMGjCteE8Lu+G9YSmtfZMAWnkaM02KOjVMkkQcK7Z4vM
lOUjlO+dsvhfmw3CPghqSs6M1K2CTqhuEiXdOBofuEMmwKNRgkV/jT92PUs0h8kq
loc/I3/H+hx/ZJ1i0S0xkZKpaImc0oZ9ZDo07biMrsUIzjwbN69mEs+CtVkah4sy
k6UyRoU2k21lyRTK0LxNjWc9ylzDNUuf6DwduU7lPZsqTaJrFNAAPpOlI4k2EcjL
3zD8amKkJGDm+PQz97PbTA381ec4ZAtB8volxCebAgMBAAGjgZwwgZkwDgYDVR0P
AQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMB
Af8EAjAAMB0GA1UdDgQWBBTTZQnMn5tuUgVE+8c9W0hmbghGoDAfBgNVHSMEGDAW
gBRDLNYEX8XI7nM53k1rUR+mpTjQNTAaBgNVHREEEzARgglsb2NhbGhvc3SHBH8A
AAEwDQYJKoZIhvcNAQELBQADggEBAKUQVj0YDuxg4tinlOZhp4ge7tCA+gL7vV+Q
iDrkWfOlGjDgwYqWMYDXMHWKIW9ea8LzyI/bVEcaHlnBmNOYuS7g47EWNiU7WUA5
iTkm3CKA5zHFFPcXHW0GQeCQrX9y3SepKS3cP8TAyZFfC/FvV24Kn1oQhJbEe0ZV
In/vPHssW7jlVe0FGVUn7FutRQgiA1pTAtS6AP4LeZ9O41DTWkPqV4nBgcxlvkgD
KjEoXXSb5C0LoR5zwAo9zB3RtmqnmvkHAOv3G92YctdS2VbCmd8CNLj9H7gMmQiH
ThsStVOhb2uo6Ni4PgzUIYKGTd4ZjUXCYxFKck//ajDyCHlL8v4=
-----END CERTIFICATE-----

27
vendor/github.com/coreos/etcd/pkg/transport/fixtures/server.key.insecure generated vendored Normal file
View File

@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAxKu2k9gUGX5idsaVJvMViZTdHN3Di1WUR/ZiKxLrSB5oqQWK
t2SKt6udsPk97DHGiTTBowrXhPC7vhvWEprX2TAFp5GjNNijo1TJJEHCu2eLzJTl
I5TvnbL4X5sNwj4IakrOjNStgk6obhIl3TgaH7hDJsCjUYJFf40/dj1LNIfJKpaH
PyN/x/ocf2SdYtEtMZGSqWiJnNKGfWQ6NO24jK7FCM48GzevZhLPgrVZGoeLMpOl
MkaFNpNtZckUytC8TY1nPcpcwzVLn+g8HblO5T2bKk2iaxTQAD6TpSOJNhHIy98w
/GpipCRg5vj0M/ez20wN/NXnOGQLQfL6JcQnmwIDAQABAoIBAGTx1eaQk9B6BEP+
rXOudTGGzO8SDFop9M/y8HQ3Y7hCk2mdxJNY8bJQTcIWS+g9rC+kencbC3/aqCJt
2zT1cTCy61QU9nYbc/JThGIttqvF/AVnryzSNyL0R3Oa/Dbk7CDSgK3cQ6qMgPru
Ka0gLJh3VVBAtBMUEGPltdsUntM4sHTh5FAabP0ioBJ1QLG6Aak7LOQikjBEFJoc
Tea4uRsE7IreP5Mn7UW92nkt1ey5UGzBtNNtpHbVaHmfQojwlwkLtnV35sumbvK6
6KTMNREZv6xSIMwkYxm1zRE3Cus/1jGIc8MZF0BxgcCR+G37l+BKwL8CSymHPxhH
dvGxoPECgYEA3STp52CbI/KyVfvjxK2OIex/NV1jKh85wQsLtkaRv3/a/EEg7MV7
54dEvo5KKOZXfeOd9r9G9h1RffjSD9MhxfPhyGwuOcqa8IE1zNwlY/v7KL7HtDIf
2mrXWF5Klafh8aXYcaRH0ZSLnl/nXUXYht4/0NRGiXnttUgqs6hvY70CgYEA46tO
J5QkgF3YVY0gx10wRCAnnKLkAaHdtxtteXOJh79xsGXQ4LLngc+mz1hLt+TNJza+
BZhoWwY/ZgyiTH0pebGr/U0QUMoUHlGgjgj3Aa/XFpOhtyLU+IU/PYl0BUz9dqsN
TDtv6p/HQhfd98vUNsbACQda+YAo+oRdO5kLQjcCgYB3OAZNcXxRte5EgoY5KqN8
UGYH2++w7qKRGqZWvtamGYRyB557Zr+0gu0hmc4LHJrASGyJcHcOCaI8Ol7snxMP
B7qJ9SA6kapTzCS361rQ+zBct/UrhPY9JuovPq4Q3i/luVXldf4t01otqGAvnY7s
rnZS242nYa8v0tcKgdyDNQKBgB3Z60BzQyn1pBTrkT2ysU5tbOQz03OHVrvYg80l
4gWDi5OWdgHQU1yI7pVHPX5aKLAYlGfFaQFuW0e1Jl6jFpoXOrbWsOn25RZom4Wk
FUcKWEhkiRKrJYOEbRtTd3vucVlq6i5xqKX51zWKTZddCXE5NBq69Sm7rSPT0Sms
UnaXAoGAXYAE5slvjcylJpMV4lxTBmNtA9+pw1T7I379mIyqZ0OS25nmpskHU7FR
SQDSRHw7hHuyjEHyhMoHEGLfUMIltQoi+pcrieVQelJdSuX7VInzHPAR5RppUVFl
jOZZKlIiqs+UfCoOgsIblXuw7a/ATnAnXakutSFgHU1lN1gN02U=
-----END RSA PRIVATE KEY-----

2
vendor/github.com/coreos/etcd/pkg/transport/keepalive_listener.go generated vendored
View File

@ -79,7 +79,7 @@ func (l *tlsKeepaliveListener) Accept() (c net.Conn, err error) {
kac.SetKeepAlive(true)
kac.SetKeepAlivePeriod(30 * time.Second)
c = tls.Server(c, l.config)
return
return c, nil
}
// NewListener creates a Listener which accepts connections from an inner

125
vendor/github.com/coreos/etcd/pkg/transport/listener.go generated vendored
View File

@ -22,24 +22,23 @@ import (
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"errors"
"fmt"
"log"
"math/big"
"net"
"os"
"path"
"path/filepath"
"strings"
"time"
"github.com/coreos/etcd/pkg/fileutil"
"github.com/coreos/etcd/pkg/tlsutil"
)
func NewListener(addr, scheme string, tlscfg *tls.Config) (l net.Listener, err error) {
func NewListener(addr, scheme string, tlsinfo *TLSInfo) (l net.Listener, err error) {
if l, err = newListener(addr, scheme); err != nil {
return nil, err
}
return wrapTLS(addr, scheme, tlscfg, l)
return wrapTLS(addr, scheme, tlsinfo, l)
}
func newListener(addr string, scheme string) (net.Listener, error) {
@ -50,36 +49,46 @@ func newListener(addr string, scheme string) (net.Listener, error) {
return net.Listen("tcp", addr)
}
func wrapTLS(addr, scheme string, tlscfg *tls.Config, l net.Listener) (net.Listener, error) {
func wrapTLS(addr, scheme string, tlsinfo *TLSInfo, l net.Listener) (net.Listener, error) {
if scheme != "https" && scheme != "unixs" {
return l, nil
}
if tlscfg == nil {
l.Close()
return nil, fmt.Errorf("cannot listen on TLS for %s: KeyFile and CertFile are not presented", scheme+"://"+addr)
}
return tls.NewListener(l, tlscfg), nil
return newTLSListener(l, tlsinfo, checkSAN)
}
type TLSInfo struct {
CertFile string
KeyFile string
CAFile string
TrustedCAFile string
ClientCertAuth bool
CertFile string
KeyFile string
CAFile string // TODO: deprecate this in v4
TrustedCAFile string
ClientCertAuth bool
CRLFile string
InsecureSkipVerify bool
// ServerName ensures the cert matches the given host in case of discovery / virtual hosting
ServerName string
// HandshakeFailure is optionally called when a connection fails to handshake. The
// connection will be closed immediately afterwards.
HandshakeFailure func(*tls.Conn, error)
// CipherSuites is a list of supported cipher suites.
// If empty, Go auto-populates it by default.
// Note that cipher suites are prioritized in the given order.
CipherSuites []uint16
selfCert bool
// parseFunc exists to simplify testing. Typically, parseFunc
// should be left nil. In that case, tls.X509KeyPair will be used.
parseFunc func([]byte, []byte) (tls.Certificate, error)
// AllowedCN is a CN which must be provided by a client.
AllowedCN string
}
func (info TLSInfo) String() string {
return fmt.Sprintf("cert = %s, key = %s, ca = %s, trusted-ca = %s, client-cert-auth = %v", info.CertFile, info.KeyFile, info.CAFile, info.TrustedCAFile, info.ClientCertAuth)
return fmt.Sprintf("cert = %s, key = %s, ca = %s, trusted-ca = %s, client-cert-auth = %v, crl-file = %s", info.CertFile, info.KeyFile, info.CAFile, info.TrustedCAFile, info.ClientCertAuth, info.CRLFile)
}
func (info TLSInfo) Empty() bool {
@ -87,12 +96,12 @@ func (info TLSInfo) Empty() bool {
}
func SelfCert(dirpath string, hosts []string) (info TLSInfo, err error) {
if err = fileutil.TouchDirAll(dirpath); err != nil {
if err = os.MkdirAll(dirpath, 0700); err != nil {
return
}
certPath := path.Join(dirpath, "cert.pem")
keyPath := path.Join(dirpath, "key.pem")
certPath := filepath.Join(dirpath, "cert.pem")
keyPath := filepath.Join(dirpath, "key.pem")
_, errcert := os.Stat(certPath)
_, errkey := os.Stat(keyPath)
if errcert == nil && errkey == nil {
@ -120,10 +129,11 @@ func SelfCert(dirpath string, hosts []string) (info TLSInfo, err error) {
}
for _, host := range hosts {
if ip := net.ParseIP(host); ip != nil {
h, _, _ := net.SplitHostPort(host)
if ip := net.ParseIP(h); ip != nil {
tmpl.IPAddresses = append(tmpl.IPAddresses, ip)
} else {
tmpl.DNSNames = append(tmpl.DNSNames, strings.Split(host, ":")[0])
tmpl.DNSNames = append(tmpl.DNSNames, h)
}
}
@ -163,15 +173,40 @@ func (info TLSInfo) baseConfig() (*tls.Config, error) {
return nil, fmt.Errorf("KeyFile and CertFile must both be present[key: %v, cert: %v]", info.KeyFile, info.CertFile)
}
tlsCert, err := tlsutil.NewCert(info.CertFile, info.KeyFile, info.parseFunc)
_, err := tlsutil.NewCert(info.CertFile, info.KeyFile, info.parseFunc)
if err != nil {
return nil, err
}
cfg := &tls.Config{
Certificates: []tls.Certificate{*tlsCert},
MinVersion: tls.VersionTLS12,
ServerName: info.ServerName,
MinVersion: tls.VersionTLS12,
ServerName: info.ServerName,
}
if len(info.CipherSuites) > 0 {
cfg.CipherSuites = info.CipherSuites
}
if info.AllowedCN != "" {
cfg.VerifyPeerCertificate = func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
for _, chains := range verifiedChains {
if len(chains) != 0 {
if info.AllowedCN == chains[0].Subject.CommonName {
return nil
}
}
}
return errors.New("CommonName authentication failed")
}
}
// this only reloads certs when there's a client request
// TODO: support server-side refresh (e.g. inotify, SIGHUP), caching
cfg.GetCertificate = func(clientHello *tls.ClientHelloInfo) (*tls.Certificate, error) {
return tlsutil.NewCert(info.CertFile, info.KeyFile, info.parseFunc)
}
cfg.GetClientCertificate = func(unused *tls.CertificateRequestInfo) (*tls.Certificate, error) {
return tlsutil.NewCert(info.CertFile, info.KeyFile, info.parseFunc)
}
return cfg, nil
}
@ -228,6 +263,7 @@ func (info TLSInfo) ClientConfig() (*tls.Config, error) {
} else {
cfg = &tls.Config{ServerName: info.ServerName}
}
cfg.InsecureSkipVerify = info.InsecureSkipVerify
CAFiles := info.cafiles()
if len(CAFiles) > 0 {
@ -235,9 +271,6 @@ func (info TLSInfo) ClientConfig() (*tls.Config, error) {
if err != nil {
return nil, err
}
// if given a CA, trust any host with a cert signed by the CA
log.Println("warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated")
cfg.ServerName = ""
}
if info.selfCert {
@ -246,31 +279,11 @@ func (info TLSInfo) ClientConfig() (*tls.Config, error) {
return cfg, nil
}
// ShallowCopyTLSConfig copies *tls.Config. This is only
// work-around for go-vet tests, which complains
//
// assignment copies lock value to p: crypto/tls.Config contains sync.Once contains sync.Mutex
//
// Keep up-to-date with 'go/src/crypto/tls/common.go'
func ShallowCopyTLSConfig(cfg *tls.Config) *tls.Config {
ncfg := tls.Config{
Time: cfg.Time,
Certificates: cfg.Certificates,
NameToCertificate: cfg.NameToCertificate,
GetCertificate: cfg.GetCertificate,
RootCAs: cfg.RootCAs,
NextProtos: cfg.NextProtos,
ServerName: cfg.ServerName,
ClientAuth: cfg.ClientAuth,
ClientCAs: cfg.ClientCAs,
InsecureSkipVerify: cfg.InsecureSkipVerify,
CipherSuites: cfg.CipherSuites,
PreferServerCipherSuites: cfg.PreferServerCipherSuites,
SessionTicketKey: cfg.SessionTicketKey,
ClientSessionCache: cfg.ClientSessionCache,
MinVersion: cfg.MinVersion,
MaxVersion: cfg.MaxVersion,
CurvePreferences: cfg.CurvePreferences,
}
return &ncfg
// IsClosedConnError returns true if the error is from closing listener, cmux.
// copied from golang.org/x/net/http2/http2.go
func IsClosedConnError(err error) bool {
// 'use of closed network connection' (Go <=1.8)
// 'use of closed file or network connection' (Go >1.8, internal/poll.ErrClosing)
// 'mux: listener closed' (cmux.ErrListenerClosed)
return err != nil && strings.Contains(err.Error(), "closed")
}

272
vendor/github.com/coreos/etcd/pkg/transport/listener_tls.go generated vendored Normal file
View File

@ -0,0 +1,272 @@
// Copyright 2017 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package transport
import (
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"io/ioutil"
"net"
"strings"
"sync"
)
// tlsListener overrides a TLS listener so it will reject client
// certificates with insufficient SAN credentials or CRL revoked
// certificates.
type tlsListener struct {
net.Listener
connc chan net.Conn
donec chan struct{}
err error
handshakeFailure func(*tls.Conn, error)
check tlsCheckFunc
}
type tlsCheckFunc func(context.Context, *tls.Conn) error
// NewTLSListener handshakes TLS connections and performs optional CRL checking.
func NewTLSListener(l net.Listener, tlsinfo *TLSInfo) (net.Listener, error) {
check := func(context.Context, *tls.Conn) error { return nil }
return newTLSListener(l, tlsinfo, check)
}
func newTLSListener(l net.Listener, tlsinfo *TLSInfo, check tlsCheckFunc) (net.Listener, error) {
if tlsinfo == nil || tlsinfo.Empty() {
l.Close()
return nil, fmt.Errorf("cannot listen on TLS for %s: KeyFile and CertFile are not presented", l.Addr().String())
}
tlscfg, err := tlsinfo.ServerConfig()
if err != nil {
return nil, err
}
hf := tlsinfo.HandshakeFailure
if hf == nil {
hf = func(*tls.Conn, error) {}
}
if len(tlsinfo.CRLFile) > 0 {
prevCheck := check
check = func(ctx context.Context, tlsConn *tls.Conn) error {
if err := prevCheck(ctx, tlsConn); err != nil {
return err
}
st := tlsConn.ConnectionState()
if certs := st.PeerCertificates; len(certs) > 0 {
return checkCRL(tlsinfo.CRLFile, certs)
}
return nil
}
}
tlsl := &tlsListener{
Listener: tls.NewListener(l, tlscfg),
connc: make(chan net.Conn),
donec: make(chan struct{}),
handshakeFailure: hf,
check: check,
}
go tlsl.acceptLoop()
return tlsl, nil
}
func (l *tlsListener) Accept() (net.Conn, error) {
select {
case conn := <-l.connc:
return conn, nil
case <-l.donec:
return nil, l.err
}
}
func checkSAN(ctx context.Context, tlsConn *tls.Conn) error {
st := tlsConn.ConnectionState()
if certs := st.PeerCertificates; len(certs) > 0 {
addr := tlsConn.RemoteAddr().String()
return checkCertSAN(ctx, certs[0], addr)
}
return nil
}
// acceptLoop launches each TLS handshake in a separate goroutine
// to prevent a hanging TLS connection from blocking other connections.
func (l *tlsListener) acceptLoop() {
var wg sync.WaitGroup
var pendingMu sync.Mutex
pending := make(map[net.Conn]struct{})
ctx, cancel := context.WithCancel(context.Background())
defer func() {
cancel()
pendingMu.Lock()
for c := range pending {
c.Close()
}
pendingMu.Unlock()
wg.Wait()
close(l.donec)
}()
for {
conn, err := l.Listener.Accept()
if err != nil {
l.err = err
return
}
pendingMu.Lock()
pending[conn] = struct{}{}
pendingMu.Unlock()
wg.Add(1)
go func() {
defer func() {
if conn != nil {
conn.Close()
}
wg.Done()
}()
tlsConn := conn.(*tls.Conn)
herr := tlsConn.Handshake()
pendingMu.Lock()
delete(pending, conn)
pendingMu.Unlock()
if herr != nil {
l.handshakeFailure(tlsConn, herr)
return
}
if err := l.check(ctx, tlsConn); err != nil {
l.handshakeFailure(tlsConn, err)
return
}
select {
case l.connc <- tlsConn:
conn = nil
case <-ctx.Done():
}
}()
}
}
func checkCRL(crlPath string, cert []*x509.Certificate) error {
// TODO: cache
crlBytes, err := ioutil.ReadFile(crlPath)
if err != nil {
return err
}
certList, err := x509.ParseCRL(crlBytes)
if err != nil {
return err
}
revokedSerials := make(map[string]struct{})
for _, rc := range certList.TBSCertList.RevokedCertificates {
revokedSerials[string(rc.SerialNumber.Bytes())] = struct{}{}
}
for _, c := range cert {
serial := string(c.SerialNumber.Bytes())
if _, ok := revokedSerials[serial]; ok {
return fmt.Errorf("transport: certificate serial %x revoked", serial)
}
}
return nil
}
func checkCertSAN(ctx context.Context, cert *x509.Certificate, remoteAddr string) error {
if len(cert.IPAddresses) == 0 && len(cert.DNSNames) == 0 {
return nil
}
h, _, herr := net.SplitHostPort(remoteAddr)
if herr != nil {
return herr
}
if len(cert.IPAddresses) > 0 {
cerr := cert.VerifyHostname(h)
if cerr == nil {
return nil
}
if len(cert.DNSNames) == 0 {
return cerr
}
}
if len(cert.DNSNames) > 0 {
ok, err := isHostInDNS(ctx, h, cert.DNSNames)
if ok {
return nil
}
errStr := ""
if err != nil {
errStr = " (" + err.Error() + ")"
}
return fmt.Errorf("tls: %q does not match any of DNSNames %q"+errStr, h, cert.DNSNames)
}
return nil
}
func isHostInDNS(ctx context.Context, host string, dnsNames []string) (ok bool, err error) {
// reverse lookup
wildcards, names := []string{}, []string{}
for _, dns := range dnsNames {
if strings.HasPrefix(dns, "*.") {
wildcards = append(wildcards, dns[1:])
} else {
names = append(names, dns)
}
}
lnames, lerr := net.DefaultResolver.LookupAddr(ctx, host)
for _, name := range lnames {
// strip trailing '.' from PTR record
if name[len(name)-1] == '.' {
name = name[:len(name)-1]
}
for _, wc := range wildcards {
if strings.HasSuffix(name, wc) {
return true, nil
}
}
for _, n := range names {
if n == name {
return true, nil
}
}
}
err = lerr
// forward lookup
for _, dns := range names {
addrs, lerr := net.DefaultResolver.LookupHost(ctx, dns)
if lerr != nil {
err = lerr
continue
}
for _, addr := range addrs {
if addr == host {
return true, nil
}
}
}
return false, err
}
func (l *tlsListener) Close() error {
err := l.Listener.Close()
<-l.donec
return err
}

5
vendor/github.com/coreos/etcd/pkg/transport/timeout_listener.go generated vendored
View File

@ -15,7 +15,6 @@
package transport
import (
"crypto/tls"
"net"
"time"
)
@ -23,7 +22,7 @@ import (
// NewTimeoutListener returns a listener that listens on the given address.
// If read/write on the accepted connection blocks longer than its time limit,
// it will return timeout error.
func NewTimeoutListener(addr string, scheme string, tlscfg *tls.Config, rdtimeoutd, wtimeoutd time.Duration) (net.Listener, error) {
func NewTimeoutListener(addr string, scheme string, tlsinfo *TLSInfo, rdtimeoutd, wtimeoutd time.Duration) (net.Listener, error) {
ln, err := newListener(addr, scheme)
if err != nil {
return nil, err
@ -33,7 +32,7 @@ func NewTimeoutListener(addr string, scheme string, tlscfg *tls.Config, rdtimeou
rdtimeoutd: rdtimeoutd,
wtimeoutd: wtimeoutd,
}
if ln, err = wrapTLS(addr, scheme, tlscfg, ln); err != nil {
if ln, err = wrapTLS(addr, scheme, tlsinfo, ln); err != nil {
return nil, err
}
return ln, nil

4
vendor/github.com/coreos/etcd/pkg/transport/unix_listener.go generated vendored
View File

@ -22,7 +22,7 @@ import (
type unixListener struct{ net.Listener }
func NewUnixListener(addr string) (net.Listener, error) {
if err := os.RemoveAll(addr); err != nil {
if err := os.Remove(addr); err != nil && !os.IsNotExist(err) {
return nil, err
}
l, err := net.Listen("unix", addr)
@ -33,7 +33,7 @@ func NewUnixListener(addr string) (net.Listener, error) {
}
func (ul *unixListener) Close() error {
if err := os.RemoveAll(ul.Addr().String()); err != nil {
if err := os.Remove(ul.Addr().String()); err != nil && !os.IsNotExist(err) {
return err
}
return ul.Listener.Close()

4
vendor/github.com/coreos/etcd/pkg/types/set.go generated vendored
View File

@ -61,7 +61,7 @@ func (us *unsafeSet) Remove(value string) {
// Contains returns whether the set contains the given value
func (us *unsafeSet) Contains(value string) (exists bool) {
_, exists = us.d[value]
return
return exists
}
// ContainsAll returns whether the set contains all given values
@ -94,7 +94,7 @@ func (us *unsafeSet) Values() (values []string) {
for val := range us.d {
values = append(values, val)
}
return
return values
}
// Copy creates a new Set containing the values of the first

101
vendor/github.com/coreos/etcd/raft/README.md generated vendored
View File

@ -13,9 +13,7 @@ To keep the codebase small as well as provide flexibility, the library only impl
In order to easily test the Raft library, its behavior should be deterministic. To achieve this determinism, the library models Raft as a state machine. The state machine takes a `Message` as input. A message can either be a local timer update or a network message sent from a remote peer. The state machine's output is a 3-tuple `{[]Messages, []LogEntries, NextState}` consisting of an array of `Messages`, `log entries`, and `Raft state changes`. For state machines with the same state, the same state machine input should always generate the same state machine output.
A simple example application, _raftexample_, is also available to help illustrate
how to use this package in practice:
https://github.com/coreos/etcd/tree/master/contrib/raftexample
A simple example application, _raftexample_, is also available to help illustrate how to use this package in practice: https://github.com/coreos/etcd/tree/master/contrib/raftexample
# Features
@ -27,12 +25,12 @@ This raft implementation is a full feature implementation of Raft protocol. Feat
- Membership changes
- Leadership transfer extension
- Efficient linearizable read-only queries served by both the leader and followers
- leader checks with quorum and bypasses Raft log before processing read-only queries
- followers asks leader to get a safe read index before processing read-only queries
- leader checks with quorum and bypasses Raft log before processing read-only queries
- followers asks leader to get a safe read index before processing read-only queries
- More efficient lease-based linearizable read-only queries served by both the leader and followers
- leader bypasses Raft log and processing read-only queries locally
- followers asks leader to get a safe read index before processing read-only queries
- this approach relies on the clock of the all the machines in raft group
- leader bypasses Raft log and processing read-only queries locally
- followers asks leader to get a safe read index before processing read-only queries
- this approach relies on the clock of the all the machines in raft group
This raft implementation also includes a few optional enhancements:
@ -51,11 +49,11 @@ This raft implementation also includes a few optional enhancements:
- [etcd](https://github.com/coreos/etcd) A distributed reliable key-value store
- [tikv](https://github.com/pingcap/tikv) A Distributed transactional key value database powered by Rust and Raft
- [swarmkit](https://github.com/docker/swarmkit) A toolkit for orchestrating distributed systems at any scale.
- [chain core](https://github.com/chain/chain) Software for operating permissioned, multi-asset blockchain networks
## Usage
The primary object in raft is a Node. You either start a Node from scratch
using raft.StartNode or start a Node from some initial state using raft.RestartNode.
The primary object in raft is a Node. Either start a Node from scratch using raft.StartNode or start a Node from some initial state using raft.RestartNode.
To start a three-node cluster
```go
@ -73,7 +71,7 @@ To start a three-node cluster
n := raft.StartNode(c, []raft.Peer{{ID: 0x02}, {ID: 0x03}})
```
You can start a single node cluster, like so:
Start a single node cluster, like so:
```go
// Create storage and config as shown above.
// Set peer list to itself, so this node can become the leader of this single-node cluster.
@ -81,7 +79,7 @@ You can start a single node cluster, like so:
n := raft.StartNode(c, peers)
```
To allow a new node to join this cluster, do not pass in any peers. First, you need add the node to the existing cluster by calling `ProposeConfChange` on any existing node inside the cluster. Then, you can start the node with empty peer list, like so:
To allow a new node to join this cluster, do not pass in any peers. First, add the node to the existing cluster by calling `ProposeConfChange` on any existing node inside the cluster. Then, start the node with an empty peer list, like so:
```go
// Create storage and config as shown above.
n := raft.StartNode(c, nil)
@ -110,46 +108,21 @@ To restart a node from previous state:
n := raft.RestartNode(c)
```
Now that you are holding onto a Node you have a few responsibilities:
After creating a Node, the user has a few responsibilities:
First, you must read from the Node.Ready() channel and process the updates
it contains. These steps may be performed in parallel, except as noted in step
2.
First, read from the Node.Ready() channel and process the updates it contains. These steps may be performed in parallel, except as noted in step 2.
1. Write HardState, Entries, and Snapshot to persistent storage if they are
not empty. Note that when writing an Entry with Index i, any
previously-persisted entries with Index >= i must be discarded.
1. Write Entries, HardState and Snapshot to persistent storage in order, i.e. Entries first, then HardState and Snapshot if they are not empty. If persistent storage supports atomic writes then all of them can be written together. Note that when writing an Entry with Index i, any previously-persisted entries with Index >= i must be discarded.
2. Send all Messages to the nodes named in the To field. It is important that
no messages be sent until the latest HardState has been persisted to disk,
and all Entries written by any previous Ready batch (Messages may be sent while
entries from the same batch are being persisted). To reduce the I/O latency, an
optimization can be applied to make leader write to disk in parallel with its
followers (as explained at section 10.2.1 in Raft thesis). If any Message has type
MsgSnap, call Node.ReportSnapshot() after it has been sent (these messages may be
large). Note: Marshalling messages is not thread-safe; it is important that you
make sure that no new entries are persisted while marshalling.
The easiest way to achieve this is to serialise the messages directly inside
your main raft loop.
2. Send all Messages to the nodes named in the To field. It is important that no messages be sent until the latest HardState has been persisted to disk, and all Entries written by any previous Ready batch (Messages may be sent while entries from the same batch are being persisted). To reduce the I/O latency, an optimization can be applied to make leader write to disk in parallel with its followers (as explained at section 10.2.1 in Raft thesis). If any Message has type MsgSnap, call Node.ReportSnapshot() after it has been sent (these messages may be large). Note: Marshalling messages is not thread-safe; it is important to make sure that no new entries are persisted while marshalling. The easiest way to achieve this is to serialise the messages directly inside the main raft loop.
3. Apply Snapshot (if any) and CommittedEntries to the state machine.
If any committed Entry has Type EntryConfChange, call Node.ApplyConfChange()
to apply it to the node. The configuration change may be cancelled at this point
by setting the NodeID field to zero before calling ApplyConfChange
(but ApplyConfChange must be called one way or the other, and the decision to cancel
must be based solely on the state machine and not external information such as
the observed health of the node).
3. Apply Snapshot (if any) and CommittedEntries to the state machine. If any committed Entry has Type EntryConfChange, call Node.ApplyConfChange() to apply it to the node. The configuration change may be cancelled at this point by setting the NodeID field to zero before calling ApplyConfChange (but ApplyConfChange must be called one way or the other, and the decision to cancel must be based solely on the state machine and not external information such as the observed health of the node).
4. Call Node.Advance() to signal readiness for the next batch of updates.
This may be done at any time after step 1, although all updates must be processed
in the order they were returned by Ready.
4. Call Node.Advance() to signal readiness for the next batch of updates. This may be done at any time after step 1, although all updates must be processed in the order they were returned by Ready.
Second, all persisted log entries must be made available via an
implementation of the Storage interface. The provided MemoryStorage
type can be used for this (if you repopulate its state upon a
restart), or you can supply your own disk-backed implementation.
Second, all persisted log entries must be made available via an implementation of the Storage interface. The provided MemoryStorage type can be used for this (if repopulating its state upon a restart), or a custom disk-backed implementation can be supplied.
Third, when you receive a message from another node, pass it to Node.Step:
Third, after receiving a message from another node, pass it to Node.Step:
```go
func recvRaftRPC(ctx context.Context, m raftpb.Message) {
@ -157,10 +130,7 @@ Third, when you receive a message from another node, pass it to Node.Step:
}
```
Finally, you need to call `Node.Tick()` at regular intervals (probably
via a `time.Ticker`). Raft has two important timeouts: heartbeat and the
election timeout. However, internally to the raft package time is
represented by an abstract "tick".
Finally, call `Node.Tick()` at regular intervals (probably via a `time.Ticker`). Raft has two important timeouts: heartbeat and the election timeout. However, internally to the raft package time is represented by an abstract "tick".
The total state machine handling loop will look something like this:
@ -190,16 +160,13 @@ The total state machine handling loop will look something like this:
}
```
To propose changes to the state machine from your node take your application
data, serialize it into a byte slice and call:
To propose changes to the state machine from the node to take application data, serialize it into a byte slice and call:
```go
n.Propose(ctx, data)
```
If the proposal is committed, data will appear in committed entries with type
raftpb.EntryNormal. There is no guarantee that a proposed command will be
committed; you may have to re-propose after a timeout.
If the proposal is committed, data will appear in committed entries with type raftpb.EntryNormal. There is no guarantee that a proposed command will be committed; the command may have to be reproposed after a timeout.
To add or remove node in a cluster, build ConfChange struct 'cc' and call:
@ -207,8 +174,7 @@ To add or remove node in a cluster, build ConfChange struct 'cc' and call:
n.ProposeConfChange(ctx, cc)
```
After config change is committed, some committed entry with type
raftpb.EntryConfChange will be returned. You must apply it to node through:
After config change is committed, some committed entry with type raftpb.EntryConfChange will be returned. This must be applied to node through:
```go
var cc raftpb.ConfChange
@ -223,25 +189,8 @@ may be reused. Node IDs must be non-zero.
## Implementation notes
This implementation is up to date with the final Raft thesis
(https://ramcloud.stanford.edu/~ongaro/thesis.pdf), although our
implementation of the membership change protocol differs somewhat from
that described in chapter 4. The key invariant that membership changes
happen one node at a time is preserved, but in our implementation the
membership change takes effect when its entry is applied, not when it
is added to the log (so the entry is committed under the old
membership instead of the new). This is equivalent in terms of safety,
since the old and new configurations are guaranteed to overlap.
This implementation is up to date with the final Raft thesis (https://ramcloud.stanford.edu/~ongaro/thesis.pdf), although this implementation of the membership change protocol differs somewhat from that described in chapter 4. The key invariant that membership changes happen one node at a time is preserved, but in our implementation the membership change takes effect when its entry is applied, not when it is added to the log (so the entry is committed under the old membership instead of the new). This is equivalent in terms of safety, since the old and new configurations are guaranteed to overlap.
To ensure that we do not attempt to commit two membership changes at
once by matching log positions (which would be unsafe since they
should have different quorum requirements), we simply disallow any
proposed membership change while any uncommitted change appears in
the leader's log.
To ensure there is no attempt to commit two membership changes at once by matching log positions (which would be unsafe since they should have different quorum requirements), any proposed membership change is simply disallowed while any uncommitted change appears in the leader's log.
This approach introduces a problem when you try to remove a member
from a two-member cluster: If one of the members dies before the
other one receives the commit of the confchange entry, then the member
cannot be removed any more since the cluster cannot make progress.
For this reason it is highly recommended to use three or more nodes in
every cluster.
This approach introduces a problem when removing a member from a two-member cluster: If one of the members dies before the other one receives the commit of the confchange entry, then the member cannot be removed any more since the cluster cannot make progress. For this reason it is highly recommended to use three or more nodes in every cluster.

20
vendor/github.com/coreos/etcd/raft/log_unstable.go generated vendored
View File

@ -85,6 +85,26 @@ func (u *unstable) stableTo(i, t uint64) {
if gt == t && i >= u.offset {
u.entries = u.entries[i+1-u.offset:]
u.offset = i + 1
u.shrinkEntriesArray()
}
}
// shrinkEntriesArray discards the underlying array used by the entries slice
// if most of it isn't being used. This avoids holding references to a bunch of
// potentially large entries that aren't needed anymore. Simply clearing the
// entries wouldn't be safe because clients might still be using them.
func (u *unstable) shrinkEntriesArray() {
// We replace the array if we're using less than half of the space in
// it. This number is fairly arbitrary, chosen as an attempt to balance
// memory usage vs number of allocations. It could probably be improved
// with some focused tuning.
const lenMultiple = 2
if len(u.entries) == 0 {
u.entries = nil
} else if len(u.entries)*lenMultiple < cap(u.entries) {
newEntries := make([]pb.Entry, len(u.entries))
copy(newEntries, u.entries)
u.entries = newEntries
}
}

103
vendor/github.com/coreos/etcd/raft/node.go generated vendored
View File

@ -15,10 +15,10 @@
package raft
import (
"context"
"errors"
pb "github.com/coreos/etcd/raft/raftpb"
"golang.org/x/net/context"
"github.com/eapache/channels"
)
@ -88,6 +88,10 @@ type Ready struct {
// If it contains a MsgSnap message, the application MUST report back to raft
// when the snapshot has been received or has failed by calling ReportSnapshot.
Messages []pb.Message
// MustSync indicates whether the HardState and Entries must be synchronously
// written to disk or if an asynchronous write is permissible.
MustSync bool
}
func isHardStateEqual(a, b pb.HardState) bool {
@ -177,10 +181,13 @@ type Peer struct {
Context []byte
}
// StartNode returns a new Node given configuration and a list of raft peers.
// It appends a ConfChangeAddNode entry for each given peer to the initial log.
func StartNode(c *Config, peers []Peer) Node {
r := newRaft(c)
var r *raft
r = newRaft(c)
// become the follower at term 1 and apply initial configuration
// entries of term 1
r.becomeFollower(1, None)
@ -229,9 +236,14 @@ func RestartNode(c *Config) Node {
return &n
}
type msgWithResult struct {
m pb.Message
result chan error
}
// node is the canonical implementation of the Node interface
type node struct {
propc chan pb.Message
propc chan msgWithResult
recvc chan pb.Message
confc chan pb.ConfChange
confstatec chan pb.ConfState
@ -251,7 +263,7 @@ type node struct {
func newNode() node {
return node{
propc: make(chan pb.Message),
propc: make(chan msgWithResult),
recvc: make(chan pb.Message),
confc: make(chan pb.ConfChange),
confstatec: make(chan pb.ConfState),
@ -285,7 +297,7 @@ func (n *node) RoleChan() *channels.RingChannel {
}
func (n *node) run(r *raft) {
var propc chan pb.Message
var propc chan msgWithResult
var readyc chan Ready
var advancec chan struct{}
var prevLastUnstablei, prevLastUnstablet uint64
@ -337,19 +349,24 @@ func (n *node) run(r *raft) {
// TODO: maybe buffer the config propose if there exists one (the way
// described in raft dissertation)
// Currently it is dropped in Step silently.
case m := <-propc:
case pm := <-propc:
m := pm.m
m.From = r.id
r.Step(m)
err := r.Step(m)
if pm.result != nil {
pm.result <- err
close(pm.result)
}
case m := <-n.recvc:
// filter out response message from unknown From.
if _, ok := r.prs[m.From]; ok || !IsResponseMsg(m.Type) {
r.Step(m) // raft never returns an error
if pr := r.getProgress(m.From); pr != nil || !IsResponseMsg(m.Type) {
r.Step(m)
}
case cc := <-n.confc:
if cc.NodeID == None {
r.resetPendingConf()
select {
case n.confstatec <- pb.ConfState{Nodes: r.nodes()}:
case n.confstatec <- pb.ConfState{Nodes: r.voters(), Learners: r.learners()}:
case <-n.done:
}
break
@ -357,6 +374,8 @@ func (n *node) run(r *raft) {
switch cc.Type {
case pb.ConfChangeAddNode:
r.addNode(cc.NodeID)
case pb.ConfChangeAddLearnerNode:
r.addLearner(cc.NodeID)
case pb.ConfChangeRemoveNode:
// block incoming proposal when local node is
// removed
@ -370,7 +389,7 @@ func (n *node) run(r *raft) {
panic("unexpected conf type")
}
select {
case n.confstatec <- pb.ConfState{Nodes: r.nodes()}:
case n.confstatec <- pb.ConfState{Nodes: r.voters(), Learners: r.learners()}:
case <-n.done:
}
case <-n.tickc:
@ -427,7 +446,7 @@ func (n *node) Tick() {
func (n *node) Campaign(ctx context.Context) error { return n.step(ctx, pb.Message{Type: pb.MsgHup}) }
func (n *node) Propose(ctx context.Context, data []byte) error {
return n.step(ctx, pb.Message{Type: pb.MsgProp, Entries: []pb.Entry{{Data: data}}})
return n.stepWait(ctx, pb.Message{Type: pb.MsgProp, Entries: []pb.Entry{{Data: data}}})
}
func (n *node) Step(ctx context.Context, m pb.Message) error {
@ -447,22 +466,56 @@ func (n *node) ProposeConfChange(ctx context.Context, cc pb.ConfChange) error {
return n.Step(ctx, pb.Message{Type: pb.MsgProp, Entries: []pb.Entry{{Type: pb.EntryConfChange, Data: data}}})
}
func (n *node) step(ctx context.Context, m pb.Message) error {
return n.stepWithWaitOption(ctx, m, false)
}
func (n *node) stepWait(ctx context.Context, m pb.Message) error {
return n.stepWithWaitOption(ctx, m, true)
}
// Step advances the state machine using msgs. The ctx.Err() will be returned,
// if any.
func (n *node) step(ctx context.Context, m pb.Message) error {
ch := n.recvc
if m.Type == pb.MsgProp {
ch = n.propc
// Step advances the state machine using msgs. The ctx.Err() will be returned,
// if any.
func (n *node) stepWithWaitOption(ctx context.Context, m pb.Message, wait bool) error {
if m.Type != pb.MsgProp {
select {
case n.recvc <- m:
return nil
case <-ctx.Done():
return ctx.Err()
case <-n.done:
return ErrStopped
}
}
ch := n.propc
pm := msgWithResult{m: m}
if wait {
pm.result = make(chan error, 1)
}
select {
case ch <- m:
return nil
case ch <- pm:
if !wait {
return nil
}
case <-ctx.Done():
return ctx.Err()
case <-n.done:
return ErrStopped
}
select {
case rsp := <-pm.result:
if rsp != nil {
return rsp
}
case <-ctx.Done():
return ctx.Err()
case <-n.done:
return ErrStopped
}
return nil
}
func (n *node) Ready() <-chan Ready { return n.readyc }
@ -544,5 +597,17 @@ func newReady(r *raft, prevSoftSt *SoftState, prevHardSt pb.HardState) Ready {
if len(r.readStates) != 0 {
rd.ReadStates = r.readStates
}
rd.MustSync = MustSync(rd.HardState, prevHardSt, len(rd.Entries))
return rd
}
// MustSync returns true if the hard state and count of Raft entries indicate
// that a synchronous write to persistent storage is required.
func MustSync(st, prevst pb.HardState, entsnum int) bool {
// Persistent state on all servers:
// (Updated on stable storage before responding to RPCs)
// currentTerm
// votedFor
// log entries[]
return entsnum != 0 || st.Vote != prevst.Vote || st.Term != prevst.Term
}

7
vendor/github.com/coreos/etcd/raft/progress.go generated vendored
View File

@ -48,6 +48,7 @@ type Progress struct {
// When in ProgressStateSnapshot, leader should have sent out snapshot
// before and stops sending any replication message.
State ProgressStateType
// Paused is used in ProgressStateProbe.
// When Paused is true, raft should pause sending replication message to this peer.
Paused bool
@ -76,6 +77,9 @@ type Progress struct {
// be freed by calling inflights.freeTo with the index of the last
// received entry.
ins *inflights
// IsLearner is true if this progress is tracked for a learner.
IsLearner bool
}
func (pr *Progress) resetState(state ProgressStateType) {
@ -243,7 +247,8 @@ func (in *inflights) freeTo(to uint64) {
return
}
i, idx := 0, in.start
idx := in.start
var i int
for i = 0; i < in.count; i++ {
if to < in.buffer[idx] { // found the first large inflight
break

364
vendor/github.com/coreos/etcd/raft/raft.go generated vendored
View File

@ -67,6 +67,10 @@ const (
campaignTransfer CampaignType = "CampaignTransfer"
)
// ErrProposalDropped is returned when the proposal is ignored by some cases,
// so that the proposer can be notified and fail fast.
var ErrProposalDropped = errors.New("raft proposal dropped")
// lockedRand is a small wrapper around rand.Rand to provide
// synchronization. Only the methods needed by the code are exposed
// (e.g. Intn).
@ -116,6 +120,10 @@ type Config struct {
// used for testing right now.
peers []uint64
// learners contains the IDs of all leaner nodes (including self if the local node is a leaner) in the raft cluster.
// learners only receives entries from the leader node. It does not vote or promote itself.
learners []uint64
// ElectionTick is the number of Node.Tick invocations that must pass between
// elections. That is, if a follower does not receive any message from the
// leader of current term before ElectionTick has elapsed, it will become
@ -171,11 +179,22 @@ type Config struct {
// If the clock drift is unbounded, leader might keep the lease longer than it
// should (clock can move backward/pause without any bound). ReadIndex is not safe
// in that case.
// CheckQuorum MUST be enabled if ReadOnlyOption is ReadOnlyLeaseBased.
ReadOnlyOption ReadOnlyOption
// Logger is the logger used for raft log. For multinode which can host
// multiple raft group, each raft group can have its own logger
Logger Logger
// DisableProposalForwarding set to true means that followers will drop
// proposals, rather than forwarding them to the leader. One use case for
// this feature would be in a situation where the Raft leader is used to
// compute the data of a proposal, for example, adding a timestamp from a
// hybrid logical clock to data in a monotonically increasing way. Forwarding
// should be disabled to prevent a follower with an innaccurate hybrid
// logical clock from assigning the timestamp and then forwarding the data
// to the leader.
DisableProposalForwarding bool
}
func (c *Config) validate() error {
@ -203,6 +222,10 @@ func (c *Config) validate() error {
c.Logger = raftLogger
}
if c.ReadOnlyOption == ReadOnlyLeaseBased && !c.CheckQuorum {
return errors.New("CheckQuorum must be enabled when ReadOnlyOption is ReadOnlyLeaseBased")
}
return nil
}
@ -220,9 +243,13 @@ type raft struct {
maxInflight int
maxMsgSize uint64
prs map[uint64]*Progress
learnerPrs map[uint64]*Progress
state StateType
// isLearner is true if the local raft node is a learner.
isLearner bool
votes map[uint64]bool
msgs []pb.Message
@ -256,6 +283,7 @@ type raft struct {
// [electiontimeout, 2 * electiontimeout - 1]. It gets reset
// when raft changes its state to follower or candidate.
randomizedElectionTimeout int
disableProposalForwarding bool
tick func()
step stepFunc
@ -263,42 +291,63 @@ type raft struct {
logger Logger
}
func newRaft(c *Config) *raft {
if err := c.validate(); err != nil {
panic(err.Error())
}
raftlog := newLog(c.Storage, c.Logger)
hs, cs, err := c.Storage.InitialState()
c.Logger.Info("newRaft storage", "hardState", hs, " confState", cs)
if err != nil {
panic(err) // TODO(bdarnell)
}
peers := c.peers
if len(cs.Nodes) > 0 {
if len(peers) > 0 {
learners := c.learners
c.Logger.Info("newRaft ", "config.peers", cs.Nodes, " config.learners", cs.Learners)
if len(cs.Nodes) > 0 || len(cs.Learners) > 0 {
if len(peers) > 0 || len(learners) > 0 {
// TODO(bdarnell): the peers argument is always nil except in
// tests; the argument should be removed and these tests should be
// updated to specify their nodes through a snapshot.
panic("cannot specify both newRaft(peers) and ConfState.Nodes)")
panic("cannot specify both newRaft(peers, learners) and ConfState.(Nodes, Learners)")
}
peers = cs.Nodes
learners = cs.Learners
}
r := &raft{
id: c.ID,
lead: None,
raftLog: raftlog,
maxMsgSize: c.MaxSizePerMsg,
maxInflight: c.MaxInflightMsgs,
prs: make(map[uint64]*Progress),
electionTimeout: c.ElectionTick,
heartbeatTimeout: c.HeartbeatTick,
logger: c.Logger,
checkQuorum: c.CheckQuorum,
preVote: c.PreVote,
readOnly: newReadOnly(c.ReadOnlyOption),
id: c.ID,
lead: None,
isLearner: false,
raftLog: raftlog,
maxMsgSize: c.MaxSizePerMsg,
maxInflight: c.MaxInflightMsgs,
prs: make(map[uint64]*Progress),
learnerPrs: make(map[uint64]*Progress),
electionTimeout: c.ElectionTick,
heartbeatTimeout: c.HeartbeatTick,
logger: c.Logger,
checkQuorum: c.CheckQuorum,
preVote: c.PreVote,
readOnly: newReadOnly(c.ReadOnlyOption),
disableProposalForwarding: c.DisableProposalForwarding,
}
for _, p := range peers {
r.prs[p] = &Progress{Next: 1, ins: newInflights(r.maxInflight)}
}
for _, p := range learners {
if _, ok := r.prs[p]; ok {
panic(fmt.Sprintf("node %x is in both learner and peer list", p))
}
r.learnerPrs[p] = &Progress{Next: 1, ins: newInflights(r.maxInflight), IsLearner: true}
if r.id == p {
r.isLearner = true
r.logger.Infof("raft %d is a learner", r.id)
}
}
if !isHardStateEqual(hs, emptyState) {
r.loadState(hs)
}
@ -312,8 +361,8 @@ func newRaft(c *Config) *raft {
nodesStrs = append(nodesStrs, fmt.Sprintf("%x", n))
}
r.logger.Infof("newRaft %x [peers: [%s], term: %d, commit: %d, applied: %d, lastindex: %d, lastterm: %d]",
r.id, strings.Join(nodesStrs, ","), r.Term, r.raftLog.committed, r.raftLog.applied, r.raftLog.lastIndex(), r.raftLog.lastTerm())
r.logger.Infof("newRaft %x learner: %v [peers: [%s], term: %d, commit: %d, applied: %d, lastindex: %d, lastterm: %d]",
r.id, r.isLearner, strings.Join(nodesStrs, ","), r.Term, r.raftLog.committed, r.raftLog.applied, r.raftLog.lastIndex(), r.raftLog.lastTerm())
return r
}
@ -332,10 +381,34 @@ func (r *raft) hardState() pb.HardState {
func (r *raft) quorum() int { return len(r.prs)/2 + 1 }
func (r *raft) nodes() []uint64 {
nodes := make([]uint64, 0, len(r.prs)+len(r.learnerPrs))
for id := range r.prs {
nodes = append(nodes, id)
}
for id := range r.learnerPrs {
nodes = append(nodes, id)
}
sort.Sort(uint64Slice(nodes))
return nodes
}
func (r *raft) voters() []uint64 {
nodes := make([]uint64, 0, len(r.prs))
for id := range r.prs {
nodes = append(nodes, id)
}
sort.Sort(uint64Slice(nodes))
return nodes
}
//TODO (Amal): review
func (r *raft) learners() []uint64 {
nodes := make([]uint64, 0, len(r.learnerPrs))
for id := range r.learnerPrs {
nodes = append(nodes, id)
}
sort.Sort(uint64Slice(nodes))
return nodes
}
@ -343,10 +416,20 @@ func (r *raft) nodes() []uint64 {
// send persists state to stable storage and then sends to its mailbox.
func (r *raft) send(m pb.Message) {
m.From = r.id
if m.Type == pb.MsgVote || m.Type == pb.MsgPreVote {
if m.Type == pb.MsgVote || m.Type == pb.MsgVoteResp || m.Type == pb.MsgPreVote || m.Type == pb.MsgPreVoteResp {
if m.Term == 0 {
// PreVote RPCs are sent at a term other than our actual term, so the code
// that sends these messages is responsible for setting the term.
// All {pre-,}campaign messages need to have the term set when
// sending.
// - MsgVote: m.Term is the term the node is campaigning for,
// non-zero as we increment the term when campaigning.
// - MsgVoteResp: m.Term is the new r.Term if the MsgVote was
// granted, non-zero for the same reason MsgVote is
// - MsgPreVote: m.Term is the term the node will campaign,
// non-zero as we use m.Term to indicate the next term we'll be
// campaigning for
// - MsgPreVoteResp: m.Term is the term received in the original
// MsgPreVote if the pre-vote was granted, non-zero for the
// same reasons MsgPreVote is
panic(fmt.Sprintf("term should be set when sending %s", m.Type))
}
} else {
@ -364,9 +447,16 @@ func (r *raft) send(m pb.Message) {
r.msgs = append(r.msgs, m)
}
func (r *raft) getProgress(id uint64) *Progress {
if pr, ok := r.prs[id]; ok {
return pr
}
return r.learnerPrs[id]
}
// sendAppend sends RPC, with entries to the given peer.
func (r *raft) sendAppend(to uint64) {
pr := r.prs[to]
pr := r.getProgress(to)
if pr.IsPaused() {
return
}
@ -431,7 +521,7 @@ func (r *raft) sendHeartbeat(to uint64, ctx []byte) {
// or it might not have all the committed entries.
// The leader MUST NOT forward the follower's commit to
// an unmatched index.
commit := min(r.prs[to].Match, r.raftLog.committed)
commit := min(r.getProgress(to).Match, r.raftLog.committed)
m := pb.Message{
To: to,
Type: pb.MsgHeartbeat,
@ -442,15 +532,26 @@ func (r *raft) sendHeartbeat(to uint64, ctx []byte) {
r.send(m)
}
func (r *raft) forEachProgress(f func(id uint64, pr *Progress)) {
for id, pr := range r.prs {
f(id, pr)
}
for id, pr := range r.learnerPrs {
f(id, pr)
}
}
// bcastAppend sends RPC, with entries to all peers that are not up-to-date
// according to the progress recorded in r.prs.
func (r *raft) bcastAppend() {
for id := range r.prs {
r.forEachProgress(func(id uint64, _ *Progress) {
if id == r.id {
continue
return
}
r.sendAppend(id)
}
})
}
// bcastHeartbeat sends RPC, without entries to all the peers.
@ -464,12 +565,12 @@ func (r *raft) bcastHeartbeat() {
}
func (r *raft) bcastHeartbeatWithCtx(ctx []byte) {
for id := range r.prs {
r.forEachProgress(func(id uint64, _ *Progress) {
if id == r.id {
continue
return
}
r.sendHeartbeat(id, ctx)
}
})
}
// maybeCommit attempts to advance the commit index. Returns true if
@ -478,8 +579,8 @@ func (r *raft) bcastHeartbeatWithCtx(ctx []byte) {
func (r *raft) maybeCommit() bool {
// TODO(bmizerany): optimize.. Currently naive
mis := make(uint64Slice, 0, len(r.prs))
for id := range r.prs {
mis = append(mis, r.prs[id].Match)
for _, p := range r.prs {
mis = append(mis, p.Match)
}
sort.Sort(sort.Reverse(mis))
mci := mis[r.quorum()-1]
@ -500,12 +601,13 @@ func (r *raft) reset(term uint64) {
r.abortLeaderTransfer()
r.votes = make(map[uint64]bool)
for id := range r.prs {
r.prs[id] = &Progress{Next: r.raftLog.lastIndex() + 1, ins: newInflights(r.maxInflight)}
r.forEachProgress(func(id uint64, pr *Progress) {
*pr = Progress{Next: r.raftLog.lastIndex() + 1, ins: newInflights(r.maxInflight), IsLearner: pr.IsLearner}
if id == r.id {
r.prs[id].Match = r.raftLog.lastIndex()
pr.Match = r.raftLog.lastIndex()
}
}
})
r.pendingConf = false
r.readOnly = newReadOnly(r.readOnly.option)
}
@ -517,7 +619,7 @@ func (r *raft) appendEntry(es ...pb.Entry) {
es[i].Index = li + 1 + uint64(i)
}
r.raftLog.append(es...)
r.prs[r.id].maybeUpdate(r.raftLog.lastIndex())
r.getProgress(r.id).maybeUpdate(r.raftLog.lastIndex())
// Regardless of maybeCommit's return, our caller will call bcastAppend.
r.maybeCommit()
}
@ -589,6 +691,7 @@ func (r *raft) becomePreCandidate() {
// but doesn't change anything else. In particular it does not increase
// r.Term or change r.Vote.
r.step = stepCandidate
r.votes = make(map[uint64]bool)
r.tick = r.tickElection
r.state = StatePreCandidate
r.logger.Infof("%x became pre-candidate at term %d", r.id, r.Term)
@ -682,7 +785,6 @@ func (r *raft) Step(m pb.Message) error {
case m.Term == 0:
// local message
case m.Term > r.Term:
lead := m.From
if m.Type == pb.MsgVote || m.Type == pb.MsgPreVote {
force := bytes.Equal(m.Context, []byte(campaignTransfer))
inLease := r.checkQuorum && r.lead != None && r.electionElapsed < r.electionTimeout
@ -693,7 +795,6 @@ func (r *raft) Step(m pb.Message) error {
r.id, r.raftLog.lastTerm(), r.raftLog.lastIndex(), r.Vote, m.Type, m.From, m.LogTerm, m.Index, r.Term, r.electionTimeout-r.electionElapsed)
return nil
}
lead = None
}
switch {
case m.Type == pb.MsgPreVote:
@ -707,7 +808,11 @@ func (r *raft) Step(m pb.Message) error {
default:
r.logger.Infof("%x [term: %d] received a %s message with higher term from %x [term: %d]",
r.id, r.Term, m.Type, m.From, m.Term)
r.becomeFollower(m.Term, lead)
if m.Type == pb.MsgApp || m.Type == pb.MsgHeartbeat || m.Type == pb.MsgSnap {
r.becomeFollower(m.Term, m.From)
} else {
r.becomeFollower(m.Term, None)
}
}
case m.Term < r.Term:
@ -757,12 +862,27 @@ func (r *raft) Step(m pb.Message) error {
}
case pb.MsgVote, pb.MsgPreVote:
if r.isLearner {
// TODO: learner may need to vote, in case of node down when confchange.
r.logger.Infof("%x [logterm: %d, index: %d, vote: %x] ignored %s from %x [logterm: %d, index: %d] at term %d: learner can not vote",
r.id, r.raftLog.lastTerm(), r.raftLog.lastIndex(), r.Vote, m.Type, m.From, m.LogTerm, m.Index, r.Term)
return nil
}
// The m.Term > r.Term clause is for MsgPreVote. For MsgVote m.Term should
// always equal r.Term.
if (r.Vote == None || m.Term > r.Term || r.Vote == m.From) && r.raftLog.isUpToDate(m.Index, m.LogTerm) {
r.logger.Infof("%x [logterm: %d, index: %d, vote: %x] cast %s for %x [logterm: %d, index: %d] at term %d",
r.id, r.raftLog.lastTerm(), r.raftLog.lastIndex(), r.Vote, m.Type, m.From, m.LogTerm, m.Index, r.Term)
r.send(pb.Message{To: m.From, Type: voteRespMsgType(m.Type)})
// When responding to Msg{Pre,}Vote messages we include the term
// from the message, not the local term. To see why consider the
// case where a single node was previously partitioned away and
// it's local term is now of date. If we include the local term
// (recall that for pre-votes we don't update the local term), the
// (pre-)campaigning node on the other end will proceed to ignore
// the message (it ignores all out of date messages).
// The term in the original message and current local term are the
// same in the case of regular votes, but different for pre-votes.
r.send(pb.Message{To: m.From, Term: m.Term, Type: voteRespMsgType(m.Type)})
if m.Type == pb.MsgVote {
// Only record real votes.
r.electionElapsed = 0
@ -771,29 +891,32 @@ func (r *raft) Step(m pb.Message) error {
} else {
r.logger.Infof("%x [logterm: %d, index: %d, vote: %x] rejected %s from %x [logterm: %d, index: %d] at term %d",
r.id, r.raftLog.lastTerm(), r.raftLog.lastIndex(), r.Vote, m.Type, m.From, m.LogTerm, m.Index, r.Term)
r.send(pb.Message{To: m.From, Type: voteRespMsgType(m.Type), Reject: true})
r.send(pb.Message{To: m.From, Term: r.Term, Type: voteRespMsgType(m.Type), Reject: true})
}
default:
r.step(r, m)
err := r.step(r, m)
if err != nil {
return err
}
}
return nil
}
type stepFunc func(r *raft, m pb.Message)
type stepFunc func(r *raft, m pb.Message) error
func stepLeader(r *raft, m pb.Message) {
func stepLeader(r *raft, m pb.Message) error {
// These message types do not require any progress for m.From.
switch m.Type {
case pb.MsgBeat:
r.bcastHeartbeat()
return
return nil
case pb.MsgCheckQuorum:
if !r.checkQuorumActive() {
r.logger.Warningf("%x stepped down to follower since quorum is not active", r.id)
r.becomeFollower(r.Term, None)
}
return
return nil
case pb.MsgProp:
if len(m.Entries) == 0 {
r.logger.Panicf("%x stepped empty MsgProp", r.id)
@ -802,11 +925,11 @@ func stepLeader(r *raft, m pb.Message) {
// If we are not currently a member of the range (i.e. this node
// was removed from the configuration while serving as leader),
// drop any new proposals.
return
return ErrProposalDropped
}
if r.leadTransferee != None {
r.logger.Debugf("%x [term %d] transfer leadership to %x is in progress; dropping proposal", r.id, r.Term, r.leadTransferee)
return
return ErrProposalDropped
}
for i, e := range m.Entries {
@ -820,9 +943,14 @@ func stepLeader(r *raft, m pb.Message) {
}
r.appendEntry(m.Entries...)
r.bcastAppend()
return
return nil
case pb.MsgReadIndex:
if r.quorum() > 1 {
if r.raftLog.zeroTermOnErrCompacted(r.raftLog.term(r.raftLog.committed)) != r.Term {
// Reject read only request when this leader has not committed any log entry at its term.
return nil
}
// thinking: use an interally defined context instead of the user given context.
// We can express this in terms of the term and index instead of a user-supplied value.
// This would allow multiple reads to piggyback on the same message.
@ -831,10 +959,7 @@ func stepLeader(r *raft, m pb.Message) {
r.readOnly.addRequest(r.raftLog.committed, m)
r.bcastHeartbeatWithCtx(m.Entries[0].Data)
case ReadOnlyLeaseBased:
var ri uint64
if r.checkQuorum {
ri = r.raftLog.committed
}
ri := r.raftLog.committed
if m.From == None || m.From == r.id { // from local member
r.readStates = append(r.readStates, ReadState{Index: r.raftLog.committed, RequestCtx: m.Entries[0].Data})
} else {
@ -845,14 +970,14 @@ func stepLeader(r *raft, m pb.Message) {
r.readStates = append(r.readStates, ReadState{Index: r.raftLog.committed, RequestCtx: m.Entries[0].Data})
}
return
return nil
}
// All other message types require a progress for m.From (pr).
pr, prOk := r.prs[m.From]
if !prOk {
pr := r.getProgress(m.From)
if pr == nil {
r.logger.Debugf("%x no progress available for %x", r.id, m.From)
return
return nil
}
switch m.Type {
case pb.MsgAppResp:
@ -908,12 +1033,12 @@ func stepLeader(r *raft, m pb.Message) {
}
if r.readOnly.option != ReadOnlySafe || len(m.Context) == 0 {
return
return nil
}
ackCount := r.readOnly.recvAck(m)
if ackCount < r.quorum() {
return
return nil
}
rss := r.readOnly.advance(m)
@ -927,7 +1052,7 @@ func stepLeader(r *raft, m pb.Message) {
}
case pb.MsgSnapStatus:
if pr.State != ProgressStateSnapshot {
return
return nil
}
if !m.Reject {
pr.becomeProbe()
@ -949,20 +1074,24 @@ func stepLeader(r *raft, m pb.Message) {
}
r.logger.Debugf("%x failed to send message to %x because it is unreachable [%s]", r.id, m.From, pr)
case pb.MsgTransferLeader:
if pr.IsLearner {
r.logger.Debugf("%x is learner. Ignored transferring leadership", r.id)
return nil
}
leadTransferee := m.From
lastLeadTransferee := r.leadTransferee
if lastLeadTransferee != None {
if lastLeadTransferee == leadTransferee {
r.logger.Infof("%x [term %d] transfer leadership to %x is in progress, ignores request to same node %x",
r.id, r.Term, leadTransferee, leadTransferee)
return
return nil
}
r.abortLeaderTransfer()
r.logger.Infof("%x [term %d] abort previous transferring leadership to %x", r.id, r.Term, lastLeadTransferee)
}
if leadTransferee == r.id {
r.logger.Debugf("%x is already leader. Ignored transferring leadership to self", r.id)
return
return nil
}
// Transfer leadership to third party.
r.logger.Infof("%x [term %d] starts to transfer leadership to %x", r.id, r.Term, leadTransferee)
@ -976,11 +1105,13 @@ func stepLeader(r *raft, m pb.Message) {
r.sendAppend(leadTransferee)
}
}
return nil
}
// stepCandidate is shared by StateCandidate and StatePreCandidate; the difference is
// whether they respond to MsgVoteResp or MsgPreVoteResp.
func stepCandidate(r *raft, m pb.Message) {
func stepCandidate(r *raft, m pb.Message) error {
// Only handle vote responses corresponding to our candidacy (while in
// StateCandidate, we may get stale MsgPreVoteResp messages in this term from
// our pre-candidate state).
@ -993,7 +1124,7 @@ func stepCandidate(r *raft, m pb.Message) {
switch m.Type {
case pb.MsgProp:
r.logger.Infof("%x no leader at term %d; dropping proposal", r.id, r.Term)
return
return ErrProposalDropped
case pb.MsgApp:
r.becomeFollower(r.Term, m.From)
r.handleAppendEntries(m)
@ -1020,14 +1151,18 @@ func stepCandidate(r *raft, m pb.Message) {
case pb.MsgTimeoutNow:
r.logger.Debugf("%x [term %d state %v] ignored MsgTimeoutNow from %x", r.id, r.Term, r.state, m.From)
}
return nil
}
func stepFollower(r *raft, m pb.Message) {
func stepFollower(r *raft, m pb.Message) error {
switch m.Type {
case pb.MsgProp:
if r.lead == None {
r.logger.Infof("%x no leader at term %d; dropping proposal", r.id, r.Term)
return
return ErrProposalDropped
} else if r.disableProposalForwarding {
r.logger.Infof("%x not forwarding to leader %x at term %d; dropping proposal", r.id, r.lead, r.Term)
return ErrProposalDropped
}
m.To = r.lead
r.send(m)
@ -1046,7 +1181,7 @@ func stepFollower(r *raft, m pb.Message) {
case pb.MsgTransferLeader:
if r.lead == None {
r.logger.Infof("%x no leader at term %d; dropping leader transfer msg", r.id, r.Term)
return
return nil
}
m.To = r.lead
r.send(m)
@ -1063,17 +1198,19 @@ func stepFollower(r *raft, m pb.Message) {
case pb.MsgReadIndex:
if r.lead == None {
r.logger.Infof("%x no leader at term %d; dropping index reading msg", r.id, r.Term)
return
return nil
}
m.To = r.lead
r.send(m)
case pb.MsgReadIndexResp:
if len(m.Entries) != 1 {
r.logger.Errorf("%x invalid format of MsgReadIndexResp from %x, entries count: %d", r.id, m.From, len(m.Entries))
return
return nil
}
r.readStates = append(r.readStates, ReadState{Index: m.Index, RequestCtx: m.Entries[0].Data})
}
return nil
}
func (r *raft) handleAppendEntries(m pb.Message) {
@ -1098,6 +1235,7 @@ func (r *raft) handleHeartbeat(m pb.Message) {
func (r *raft) handleSnapshot(m pb.Message) {
sindex, sterm := m.Snapshot.Metadata.Index, m.Snapshot.Metadata.Term
if r.restore(m.Snapshot) {
r.logger.Infof("%x [commit: %d] restored snapshot [index: %d, term: %d]",
r.id, r.raftLog.committed, sindex, sterm)
@ -1109,6 +1247,12 @@ func (r *raft) handleSnapshot(m pb.Message) {
}
}
func (r *raft) initialized() bool {
// empty 'prs' and 'learnerPrs' means that
// this peer is newly created by conf change
return len(r.prs) > 0 || len(r.learnerPrs) > 0
}
// restore recovers the state machine from a snapshot. It restores the log and the
// configuration of state machine.
func (r *raft) restore(s pb.Snapshot) bool {
@ -1127,15 +1271,22 @@ func (r *raft) restore(s pb.Snapshot) bool {
r.raftLog.restore(s)
r.prs = make(map[uint64]*Progress)
for _, n := range s.Metadata.ConfState.Nodes {
r.learnerPrs = make(map[uint64]*Progress)
r.restoreNode(s.Metadata.ConfState.Nodes, false)
r.restoreNode(s.Metadata.ConfState.Learners, true)
return true
}
func (r *raft) restoreNode(nodes []uint64, isLearner bool) {
for _, n := range nodes {
match, next := uint64(0), r.raftLog.lastIndex()+1
if n == r.id {
match = next - 1
r.isLearner = isLearner
}
r.setProgress(n, match, next)
r.logger.Infof("%x restored progress of %x [%s]", r.id, n, r.prs[n])
r.setProgress(n, match, next, isLearner)
r.logger.Infof("%x restored progress of %x [%s]", r.id, n, r.getProgress(n))
}
return true
}
// promotable indicates whether state machine can be promoted to leader,
@ -1146,14 +1297,46 @@ func (r *raft) promotable() bool {
}
func (r *raft) addNode(id uint64) {
r.addNodeOrLearnerNode(id, false)
}
func (r *raft) addLearner(id uint64) {
r.addNodeOrLearnerNode(id, true)
}
func (r *raft) addNodeOrLearnerNode(id uint64, isLearner bool) {
r.pendingConf = false
if _, ok := r.prs[id]; ok {
// Ignore any redundant addNode calls (which can happen because the
// initial bootstrapping entries are applied twice).
return
pr := r.getProgress(id)
if pr == nil {
r.setProgress(id, 0, r.raftLog.lastIndex()+1, isLearner)
} else {
if isLearner && !pr.IsLearner {
// can only change Learner to Voter
r.logger.Infof("%x ignored addLeaner: do not support changing %x from raft peer to learner.", r.id, id)
return
}
if isLearner == pr.IsLearner {
// Ignore any redundant addNode calls (which can happen because the
// initial bootstrapping entries are applied twice).
return
}
// change Learner to Voter, use origin Learner progress
delete(r.learnerPrs, id)
pr.IsLearner = false
r.prs[id] = pr
}
r.setProgress(id, 0, r.raftLog.lastIndex()+1)
if r.id == id {
r.isLearner = isLearner
}
// When a node is first added, we should mark it as recently active.
// Otherwise, CheckQuorum may cause us to step down if it is invoked
// before the added node has a chance to communicate with us.
pr = r.getProgress(id)
pr.RecentActive = true
}
func (r *raft) removeNode(id uint64) {
@ -1161,7 +1344,7 @@ func (r *raft) removeNode(id uint64) {
r.pendingConf = false
// do not try to commit or abort transferring if there is no nodes in the cluster.
if len(r.prs) == 0 {
if len(r.prs) == 0 && len(r.learnerPrs) == 0 {
return
}
@ -1178,12 +1361,23 @@ func (r *raft) removeNode(id uint64) {
func (r *raft) resetPendingConf() { r.pendingConf = false }
func (r *raft) setProgress(id, match, next uint64) {
r.prs[id] = &Progress{Next: next, Match: match, ins: newInflights(r.maxInflight)}
func (r *raft) setProgress(id, match, next uint64, isLearner bool) {
if !isLearner {
delete(r.learnerPrs, id)
r.prs[id] = &Progress{Next: next, Match: match, ins: newInflights(r.maxInflight)}
return
}
if _, ok := r.prs[id]; ok {
panic(fmt.Sprintf("%x unexpected changing from voter to learner for %x", r.id, id))
}
r.learnerPrs[id] = &Progress{Next: next, Match: match, ins: newInflights(r.maxInflight), IsLearner: true}
}
func (r *raft) delProgress(id uint64) {
delete(r.prs, id)
delete(r.learnerPrs, id)
}
func (r *raft) loadState(state pb.HardState) {
@ -1213,18 +1407,18 @@ func (r *raft) resetRandomizedElectionTimeout() {
func (r *raft) checkQuorumActive() bool {
var act int
for id := range r.prs {
r.forEachProgress(func(id uint64, pr *Progress) {
if id == r.id { // self is always active
act++
continue
return
}
if r.prs[id].RecentActive {
if pr.RecentActive && !pr.IsLearner {
act++
}
r.prs[id].RecentActive = false
}
pr.RecentActive = false
})
return act >= r.quorum()
}

286
vendor/github.com/coreos/etcd/raft/raftpb/raft.pb.go generated vendored
View File

@ -1,6 +1,5 @@
// Code generated by protoc-gen-gogo.
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: raft.proto
// DO NOT EDIT!
/*
Package raftpb is a generated protocol buffer package.
@ -26,6 +25,8 @@ import (
math "math"
_ "github.com/gogo/protobuf/gogoproto"
io "io"
)
@ -162,20 +163,23 @@ func (MessageType) EnumDescriptor() ([]byte, []int) { return fileDescriptorRaft,
type ConfChangeType int32
const (
ConfChangeAddNode ConfChangeType = 0
ConfChangeRemoveNode ConfChangeType = 1
ConfChangeUpdateNode ConfChangeType = 2
ConfChangeAddNode ConfChangeType = 0
ConfChangeRemoveNode ConfChangeType = 1
ConfChangeUpdateNode ConfChangeType = 2
ConfChangeAddLearnerNode ConfChangeType = 3
)
var ConfChangeType_name = map[int32]string{
0: "ConfChangeAddNode",
1: "ConfChangeRemoveNode",
2: "ConfChangeUpdateNode",
3: "ConfChangeAddLearnerNode",
}
var ConfChangeType_value = map[string]int32{
"ConfChangeAddNode": 0,
"ConfChangeRemoveNode": 1,
"ConfChangeUpdateNode": 2,
"ConfChangeAddNode": 0,
"ConfChangeRemoveNode": 1,
"ConfChangeUpdateNode": 2,
"ConfChangeAddLearnerNode": 3,
}
func (x ConfChangeType) Enum() *ConfChangeType {
@ -267,6 +271,7 @@ func (*HardState) Descriptor() ([]byte, []int) { return fileDescriptorRaft, []in
type ConfState struct {
Nodes []uint64 `protobuf:"varint,1,rep,name=nodes" json:"nodes,omitempty"`
Learners []uint64 `protobuf:"varint,2,rep,name=learners" json:"learners,omitempty"`
XXX_unrecognized []byte `json:"-"`
}
@ -537,6 +542,13 @@ func (m *ConfState) MarshalTo(dAtA []byte) (int, error) {
i = encodeVarintRaft(dAtA, i, uint64(num))
}
}
if len(m.Learners) > 0 {
for _, num := range m.Learners {
dAtA[i] = 0x10
i++
i = encodeVarintRaft(dAtA, i, uint64(num))
}
}
if m.XXX_unrecognized != nil {
i += copy(dAtA[i:], m.XXX_unrecognized)
}
@ -579,24 +591,6 @@ func (m *ConfChange) MarshalTo(dAtA []byte) (int, error) {
return i, nil
}
func encodeFixed64Raft(dAtA []byte, offset int, v uint64) int {
dAtA[offset] = uint8(v)
dAtA[offset+1] = uint8(v >> 8)
dAtA[offset+2] = uint8(v >> 16)
dAtA[offset+3] = uint8(v >> 24)
dAtA[offset+4] = uint8(v >> 32)
dAtA[offset+5] = uint8(v >> 40)
dAtA[offset+6] = uint8(v >> 48)
dAtA[offset+7] = uint8(v >> 56)
return offset + 8
}
func encodeFixed32Raft(dAtA []byte, offset int, v uint32) int {
dAtA[offset] = uint8(v)
dAtA[offset+1] = uint8(v >> 8)
dAtA[offset+2] = uint8(v >> 16)
dAtA[offset+3] = uint8(v >> 24)
return offset + 4
}
func encodeVarintRaft(dAtA []byte, offset int, v uint64) int {
for v >= 1<<7 {
dAtA[offset] = uint8(v&0x7f | 0x80)
@ -700,6 +694,11 @@ func (m *ConfState) Size() (n int) {
n += 1 + sovRaft(uint64(e))
}
}
if len(m.Learners) > 0 {
for _, e := range m.Learners {
n += 1 + sovRaft(uint64(e))
}
}
if m.XXX_unrecognized != nil {
n += len(m.XXX_unrecognized)
}
@ -1558,25 +1557,129 @@ func (m *ConfState) Unmarshal(dAtA []byte) error {
}
switch fieldNum {
case 1:
if wireType != 0 {
return fmt.Errorf("proto: wrong wireType = %d for field Nodes", wireType)
}
var v uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
if wireType == 0 {
var v uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
v |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
if iNdEx >= l {
m.Nodes = append(m.Nodes, v)
} else if wireType == 2 {
var packedLen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
packedLen |= (int(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
if packedLen < 0 {
return ErrInvalidLengthRaft
}
postIndex := iNdEx + packedLen
if postIndex > l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
v |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
for iNdEx < postIndex {
var v uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
v |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
m.Nodes = append(m.Nodes, v)
}
} else {
return fmt.Errorf("proto: wrong wireType = %d for field Nodes", wireType)
}
case 2:
if wireType == 0 {
var v uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
v |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
m.Learners = append(m.Learners, v)
} else if wireType == 2 {
var packedLen int
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
packedLen |= (int(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
if packedLen < 0 {
return ErrInvalidLengthRaft
}
postIndex := iNdEx + packedLen
if postIndex > l {
return io.ErrUnexpectedEOF
}
for iNdEx < postIndex {
var v uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return ErrIntOverflowRaft
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
v |= (uint64(b) & 0x7F) << shift
if b < 0x80 {
break
}
}
m.Learners = append(m.Learners, v)
}
} else {
return fmt.Errorf("proto: wrong wireType = %d for field Learners", wireType)
}
m.Nodes = append(m.Nodes, v)
default:
iNdEx = preIndex
skippy, err := skipRaft(dAtA[iNdEx:])
@ -1846,55 +1949,56 @@ var (
func init() { proto.RegisterFile("raft.proto", fileDescriptorRaft) }
var fileDescriptorRaft = []byte{
// 790 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0x64, 0x54, 0xcd, 0x6e, 0xdb, 0x46,
0x10, 0x16, 0x29, 0xea, 0x6f, 0x28, 0xcb, 0xab, 0xb5, 0x5a, 0x2c, 0x0c, 0x43, 0x55, 0x85, 0x1e,
0x04, 0x17, 0x76, 0x5b, 0x1d, 0x7a, 0xe8, 0xcd, 0x96, 0x0a, 0x58, 0x40, 0x65, 0xb8, 0xb2, 0xdc,
0x43, 0x83, 0x20, 0x58, 0x8b, 0x2b, 0x4a, 0x89, 0xc9, 0x25, 0x96, 0x2b, 0xc7, 0xbe, 0x04, 0x79,
0x80, 0x3c, 0x40, 0x2e, 0x79, 0x1f, 0x1f, 0x0d, 0xe4, 0x1e, 0xc4, 0xce, 0x8b, 0x04, 0xbb, 0x5c,
0x4a, 0x94, 0x74, 0xdb, 0xf9, 0xbe, 0xe1, 0xcc, 0x37, 0xdf, 0xce, 0x12, 0x40, 0xd0, 0xa9, 0x3c,
0x8e, 0x04, 0x97, 0x1c, 0x17, 0xd5, 0x39, 0xba, 0xde, 0x6f, 0xf8, 0xdc, 0xe7, 0x1a, 0xfa, 0x4d,
0x9d, 0x12, 0xb6, 0xfd, 0x0e, 0x0a, 0x7f, 0x87, 0x52, 0xdc, 0xe3, 0x5f, 0xc1, 0x19, 0xdf, 0x47,
0x8c, 0x58, 0x2d, 0xab, 0x53, 0xeb, 0xd6, 0x8f, 0x93, 0xaf, 0x8e, 0x35, 0xa9, 0x88, 0x53, 0xe7,
0xe1, 0xcb, 0x4f, 0xb9, 0x91, 0x4e, 0xc2, 0x04, 0x9c, 0x31, 0x13, 0x01, 0xb1, 0x5b, 0x56, 0xc7,
0x59, 0x32, 0x4c, 0x04, 0x78, 0x1f, 0x0a, 0x83, 0xd0, 0x63, 0x77, 0x24, 0x9f, 0xa1, 0x12, 0x08,
0x63, 0x70, 0xfa, 0x54, 0x52, 0xe2, 0xb4, 0xac, 0x4e, 0x75, 0xa4, 0xcf, 0xed, 0xf7, 0x16, 0xa0,
0xcb, 0x90, 0x46, 0xf1, 0x8c, 0xcb, 0x21, 0x93, 0xd4, 0xa3, 0x92, 0xe2, 0x3f, 0x01, 0x26, 0x3c,
0x9c, 0xbe, 0x8a, 0x25, 0x95, 0x89, 0x22, 0x77, 0xa5, 0xa8, 0xc7, 0xc3, 0xe9, 0xa5, 0x22, 0x4c,
0xf1, 0xca, 0x24, 0x05, 0x54, 0xf3, 0xb9, 0x6e, 0x9e, 0xd5, 0x95, 0x40, 0x4a, 0xb2, 0x54, 0x92,
0xb3, 0xba, 0x34, 0xd2, 0xfe, 0x1f, 0xca, 0xa9, 0x02, 0x25, 0x51, 0x29, 0xd0, 0x3d, 0xab, 0x23,
0x7d, 0xc6, 0x7f, 0x41, 0x39, 0x30, 0xca, 0x74, 0x61, 0xb7, 0x4b, 0x52, 0x2d, 0x9b, 0xca, 0x4d,
0xdd, 0x65, 0x7e, 0xfb, 0x53, 0x1e, 0x4a, 0x43, 0x16, 0xc7, 0xd4, 0x67, 0xf8, 0x08, 0x1c, 0xb9,
0x72, 0x78, 0x2f, 0xad, 0x61, 0xe8, 0xac, 0xc7, 0x2a, 0x0d, 0x37, 0xc0, 0x96, 0x7c, 0x6d, 0x12,
0x5b, 0x72, 0x35, 0xc6, 0x54, 0xf0, 0x8d, 0x31, 0x14, 0xb2, 0x1c, 0xd0, 0xd9, 0x1c, 0x10, 0x37,
0xa1, 0x74, 0xc3, 0x7d, 0x7d, 0x61, 0x85, 0x0c, 0x99, 0x82, 0x2b, 0xdb, 0x8a, 0xdb, 0xb6, 0x1d,
0x41, 0x89, 0x85, 0x52, 0xcc, 0x59, 0x4c, 0x4a, 0xad, 0x7c, 0xc7, 0xed, 0xee, 0xac, 0x6d, 0x46,
0x5a, 0xca, 0xe4, 0xe0, 0x03, 0x28, 0x4e, 0x78, 0x10, 0xcc, 0x25, 0x29, 0x67, 0x6a, 0x19, 0x0c,
0x77, 0xa1, 0x1c, 0x1b, 0xc7, 0x48, 0x45, 0x3b, 0x89, 0x36, 0x9d, 0x4c, 0x1d, 0x4c, 0xf3, 0x54,
0x45, 0xc1, 0x5e, 0xb3, 0x89, 0x24, 0xd0, 0xb2, 0x3a, 0xe5, 0xb4, 0x62, 0x82, 0xe1, 0x5f, 0x00,
0x92, 0xd3, 0xd9, 0x3c, 0x94, 0xc4, 0xcd, 0xf4, 0xcc, 0xe0, 0x98, 0x40, 0x69, 0xc2, 0x43, 0xc9,
0xee, 0x24, 0xa9, 0xea, 0x8b, 0x4d, 0xc3, 0xf6, 0x4b, 0xa8, 0x9c, 0x51, 0xe1, 0x25, 0xeb, 0x93,
0x3a, 0x68, 0x6d, 0x39, 0x48, 0xc0, 0xb9, 0xe5, 0x92, 0xad, 0xef, 0xbb, 0x42, 0x32, 0x03, 0xe7,
0xb7, 0x07, 0x6e, 0xff, 0x0c, 0x95, 0xe5, 0xba, 0xe2, 0x06, 0x14, 0x42, 0xee, 0xb1, 0x98, 0x58,
0xad, 0x7c, 0xc7, 0x19, 0x25, 0x41, 0xfb, 0x83, 0x05, 0xa0, 0x72, 0x7a, 0x33, 0x1a, 0xfa, 0xfa,
0xd6, 0x07, 0xfd, 0x35, 0x05, 0xf6, 0xa0, 0x8f, 0x7f, 0x37, 0x8f, 0xd3, 0xd6, 0xab, 0xf3, 0x63,
0xf6, 0x29, 0x24, 0xdf, 0x6d, 0xbd, 0xd0, 0x03, 0x28, 0x9e, 0x73, 0x8f, 0x0d, 0xfa, 0xeb, 0xba,
0x12, 0x4c, 0x19, 0xd2, 0x33, 0x86, 0x24, 0x8f, 0x31, 0x0d, 0x0f, 0xff, 0x80, 0xca, 0xf2, 0xc9,
0xe3, 0x5d, 0x70, 0x75, 0x70, 0xce, 0x45, 0x40, 0x6f, 0x50, 0x0e, 0xef, 0xc1, 0xae, 0x06, 0x56,
0x8d, 0x91, 0x75, 0xf8, 0xd9, 0x06, 0x37, 0xb3, 0xc4, 0x18, 0xa0, 0x38, 0x8c, 0xfd, 0xb3, 0x45,
0x84, 0x72, 0xd8, 0x85, 0xd2, 0x30, 0xf6, 0x4f, 0x19, 0x95, 0xc8, 0x32, 0xc1, 0x85, 0xe0, 0x11,
0xb2, 0x4d, 0xd6, 0x49, 0x14, 0xa1, 0x3c, 0xae, 0x01, 0x24, 0xe7, 0x11, 0x8b, 0x23, 0xe4, 0x98,
0xc4, 0xff, 0xb8, 0x64, 0xa8, 0xa0, 0x44, 0x98, 0x40, 0xb3, 0x45, 0xc3, 0xaa, 0x85, 0x41, 0x25,
0x8c, 0xa0, 0xaa, 0x9a, 0x31, 0x2a, 0xe4, 0xb5, 0xea, 0x52, 0xc6, 0x0d, 0x40, 0x59, 0x44, 0x7f,
0x54, 0xc1, 0x18, 0x6a, 0xc3, 0xd8, 0xbf, 0x0a, 0x05, 0xa3, 0x93, 0x19, 0xbd, 0xbe, 0x61, 0x08,
0x70, 0x1d, 0x76, 0x4c, 0x21, 0x75, 0x41, 0x8b, 0x18, 0xb9, 0x26, 0xad, 0x37, 0x63, 0x93, 0x37,
0xff, 0x2e, 0xb8, 0x58, 0x04, 0xa8, 0x8a, 0x7f, 0x80, 0xfa, 0x30, 0xf6, 0xc7, 0x82, 0x86, 0xf1,
0x94, 0x89, 0x7f, 0x18, 0xf5, 0x98, 0x40, 0x3b, 0xe6, 0xeb, 0xf1, 0x3c, 0x60, 0x7c, 0x21, 0xcf,
0xf9, 0x5b, 0x54, 0x33, 0x62, 0x46, 0x8c, 0x7a, 0xfa, 0x87, 0x87, 0x76, 0x8d, 0x98, 0x25, 0xa2,
0xc5, 0x20, 0x33, 0xef, 0x85, 0x60, 0x7a, 0xc4, 0xba, 0xe9, 0x6a, 0x62, 0x9d, 0x83, 0x0f, 0x5f,
0x40, 0x6d, 0xfd, 0x7a, 0x95, 0x8e, 0x15, 0x72, 0xe2, 0x79, 0xea, 0x2e, 0x51, 0x0e, 0x13, 0x68,
0xac, 0xe0, 0x11, 0x0b, 0xf8, 0x2d, 0xd3, 0x8c, 0xb5, 0xce, 0x5c, 0x45, 0x1e, 0x95, 0x09, 0x63,
0x9f, 0x92, 0x87, 0xa7, 0x66, 0xee, 0xf1, 0xa9, 0x99, 0x7b, 0x78, 0x6e, 0x5a, 0x8f, 0xcf, 0x4d,
0xeb, 0xeb, 0x73, 0xd3, 0xfa, 0xf8, 0xad, 0x99, 0xfb, 0x1e, 0x00, 0x00, 0xff, 0xff, 0xcf, 0x30,
0x01, 0x41, 0x3a, 0x06, 0x00, 0x00,
// 815 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x64, 0x54, 0xcd, 0x6e, 0x23, 0x45,
0x10, 0xf6, 0x8c, 0xc7, 0x7f, 0x35, 0x8e, 0xd3, 0xa9, 0x35, 0xa8, 0x15, 0x45, 0xc6, 0xb2, 0x38,
0x58, 0x41, 0x1b, 0x20, 0x07, 0x0e, 0x48, 0x1c, 0x36, 0x09, 0x52, 0x22, 0xad, 0xa3, 0xc5, 0x9b,
0xe5, 0x80, 0x84, 0x50, 0xc7, 0x53, 0x9e, 0x18, 0x32, 0xd3, 0xa3, 0x9e, 0xf6, 0xb2, 0xb9, 0x20,
0x1e, 0x80, 0x07, 0xe0, 0xc2, 0xfb, 0xe4, 0xb8, 0x12, 0x77, 0xc4, 0x86, 0x17, 0x41, 0xdd, 0xd3,
0x63, 0xcf, 0x24, 0xb7, 0xae, 0xef, 0xab, 0xae, 0xfa, 0xea, 0xeb, 0x9a, 0x01, 0x50, 0x62, 0xa9,
0x8f, 0x32, 0x25, 0xb5, 0xc4, 0xb6, 0x39, 0x67, 0xd7, 0xfb, 0xc3, 0x58, 0xc6, 0xd2, 0x42, 0x9f,
0x9b, 0x53, 0xc1, 0x4e, 0x7e, 0x83, 0xd6, 0xb7, 0xa9, 0x56, 0x77, 0xf8, 0x19, 0x04, 0x57, 0x77,
0x19, 0x71, 0x6f, 0xec, 0x4d, 0x07, 0xc7, 0x7b, 0x47, 0xc5, 0xad, 0x23, 0x4b, 0x1a, 0xe2, 0x24,
0xb8, 0xff, 0xe7, 0x93, 0xc6, 0xdc, 0x26, 0x21, 0x87, 0xe0, 0x8a, 0x54, 0xc2, 0xfd, 0xb1, 0x37,
0x0d, 0x36, 0x0c, 0xa9, 0x04, 0xf7, 0xa1, 0x75, 0x91, 0x46, 0xf4, 0x8e, 0x37, 0x2b, 0x54, 0x01,
0x21, 0x42, 0x70, 0x26, 0xb4, 0xe0, 0xc1, 0xd8, 0x9b, 0xf6, 0xe7, 0xf6, 0x3c, 0xf9, 0xdd, 0x03,
0xf6, 0x3a, 0x15, 0x59, 0x7e, 0x23, 0xf5, 0x8c, 0xb4, 0x88, 0x84, 0x16, 0xf8, 0x15, 0xc0, 0x42,
0xa6, 0xcb, 0x9f, 0x72, 0x2d, 0x74, 0xa1, 0x28, 0xdc, 0x2a, 0x3a, 0x95, 0xe9, 0xf2, 0xb5, 0x21,
0x5c, 0xf1, 0xde, 0xa2, 0x04, 0x4c, 0xf3, 0x95, 0x6d, 0x5e, 0xd5, 0x55, 0x40, 0x46, 0xb2, 0x36,
0x92, 0xab, 0xba, 0x2c, 0x32, 0xf9, 0x01, 0xba, 0xa5, 0x02, 0x23, 0xd1, 0x28, 0xb0, 0x3d, 0xfb,
0x73, 0x7b, 0xc6, 0xaf, 0xa1, 0x9b, 0x38, 0x65, 0xb6, 0x70, 0x78, 0xcc, 0x4b, 0x2d, 0x8f, 0x95,
0xbb, 0xba, 0x9b, 0xfc, 0xc9, 0x5f, 0x4d, 0xe8, 0xcc, 0x28, 0xcf, 0x45, 0x4c, 0xf8, 0x1c, 0x02,
0xbd, 0x75, 0xf8, 0x59, 0x59, 0xc3, 0xd1, 0x55, 0x8f, 0x4d, 0x1a, 0x0e, 0xc1, 0xd7, 0xb2, 0x36,
0x89, 0xaf, 0xa5, 0x19, 0x63, 0xa9, 0xe4, 0xa3, 0x31, 0x0c, 0xb2, 0x19, 0x30, 0x78, 0x3c, 0x20,
0x8e, 0xa0, 0x73, 0x2b, 0x63, 0xfb, 0x60, 0xad, 0x0a, 0x59, 0x82, 0x5b, 0xdb, 0xda, 0x4f, 0x6d,
0x7b, 0x0e, 0x1d, 0x4a, 0xb5, 0x5a, 0x51, 0xce, 0x3b, 0xe3, 0xe6, 0x34, 0x3c, 0xde, 0xa9, 0x6d,
0x46, 0x59, 0xca, 0xe5, 0xe0, 0x01, 0xb4, 0x17, 0x32, 0x49, 0x56, 0x9a, 0x77, 0x2b, 0xb5, 0x1c,
0x86, 0xc7, 0xd0, 0xcd, 0x9d, 0x63, 0xbc, 0x67, 0x9d, 0x64, 0x8f, 0x9d, 0x2c, 0x1d, 0x2c, 0xf3,
0x4c, 0x45, 0x45, 0x3f, 0xd3, 0x42, 0x73, 0x18, 0x7b, 0xd3, 0x6e, 0x59, 0xb1, 0xc0, 0xf0, 0x53,
0x80, 0xe2, 0x74, 0xbe, 0x4a, 0x35, 0x0f, 0x2b, 0x3d, 0x2b, 0x38, 0x72, 0xe8, 0x2c, 0x64, 0xaa,
0xe9, 0x9d, 0xe6, 0x7d, 0xfb, 0xb0, 0x65, 0x38, 0xf9, 0x11, 0x7a, 0xe7, 0x42, 0x45, 0xc5, 0xfa,
0x94, 0x0e, 0x7a, 0x4f, 0x1c, 0xe4, 0x10, 0xbc, 0x95, 0x9a, 0xea, 0xfb, 0x6e, 0x90, 0xca, 0xc0,
0xcd, 0xa7, 0x03, 0x4f, 0xbe, 0x81, 0xde, 0x66, 0x5d, 0x71, 0x08, 0xad, 0x54, 0x46, 0x94, 0x73,
0x6f, 0xdc, 0x9c, 0x06, 0xf3, 0x22, 0xc0, 0x7d, 0xe8, 0xde, 0x92, 0x50, 0x29, 0xa9, 0x9c, 0xfb,
0x96, 0xd8, 0xc4, 0x93, 0x3f, 0x3c, 0x00, 0x73, 0xff, 0xf4, 0x46, 0xa4, 0xb1, 0xdd, 0x88, 0x8b,
0xb3, 0x9a, 0x3a, 0xff, 0xe2, 0x0c, 0xbf, 0x70, 0x1f, 0xae, 0x6f, 0xd7, 0xea, 0xe3, 0xea, 0x67,
0x52, 0xdc, 0x7b, 0xf2, 0xf5, 0x1e, 0x40, 0xfb, 0x52, 0x46, 0x74, 0x71, 0x56, 0xd7, 0x5c, 0x60,
0xc6, 0xac, 0x53, 0x67, 0x56, 0xf1, 0xa1, 0x96, 0xe1, 0xe1, 0x97, 0xd0, 0xdb, 0xfc, 0x0e, 0x70,
0x17, 0x42, 0x1b, 0x5c, 0x4a, 0x95, 0x88, 0x5b, 0xd6, 0xc0, 0x67, 0xb0, 0x6b, 0x81, 0x6d, 0x63,
0xe6, 0x1d, 0xfe, 0xed, 0x43, 0x58, 0x59, 0x70, 0x04, 0x68, 0xcf, 0xf2, 0xf8, 0x7c, 0x9d, 0xb1,
0x06, 0x86, 0xd0, 0x99, 0xe5, 0xf1, 0x09, 0x09, 0xcd, 0x3c, 0x17, 0xbc, 0x52, 0x32, 0x63, 0xbe,
0xcb, 0x7a, 0x91, 0x65, 0xac, 0x89, 0x03, 0x80, 0xe2, 0x3c, 0xa7, 0x3c, 0x63, 0x81, 0x4b, 0xfc,
0x5e, 0x6a, 0x62, 0x2d, 0x23, 0xc2, 0x05, 0x96, 0x6d, 0x3b, 0xd6, 0x2c, 0x13, 0xeb, 0x20, 0x83,
0xbe, 0x69, 0x46, 0x42, 0xe9, 0x6b, 0xd3, 0xa5, 0x8b, 0x43, 0x60, 0x55, 0xc4, 0x5e, 0xea, 0x21,
0xc2, 0x60, 0x96, 0xc7, 0x6f, 0x52, 0x45, 0x62, 0x71, 0x23, 0xae, 0x6f, 0x89, 0x01, 0xee, 0xc1,
0x8e, 0x2b, 0x64, 0x1e, 0x6f, 0x9d, 0xb3, 0xd0, 0xa5, 0x9d, 0xde, 0xd0, 0xe2, 0x97, 0xef, 0xd6,
0x52, 0xad, 0x13, 0xd6, 0xc7, 0x8f, 0x60, 0x6f, 0x96, 0xc7, 0x57, 0x4a, 0xa4, 0xf9, 0x92, 0xd4,
0x4b, 0x12, 0x11, 0x29, 0xb6, 0xe3, 0x6e, 0x5f, 0xad, 0x12, 0x92, 0x6b, 0x7d, 0x29, 0x7f, 0x65,
0x03, 0x27, 0x66, 0x4e, 0x22, 0xb2, 0x3f, 0x43, 0xb6, 0xeb, 0xc4, 0x6c, 0x10, 0x2b, 0x86, 0xb9,
0x79, 0x5f, 0x29, 0xb2, 0x23, 0xee, 0xb9, 0xae, 0x2e, 0xb6, 0x39, 0x78, 0x78, 0x07, 0x83, 0xfa,
0xf3, 0x1a, 0x1d, 0x5b, 0xe4, 0x45, 0x14, 0x99, 0xb7, 0x64, 0x0d, 0xe4, 0x30, 0xdc, 0xc2, 0x73,
0x4a, 0xe4, 0x5b, 0xb2, 0x8c, 0x57, 0x67, 0xde, 0x64, 0x91, 0xd0, 0x05, 0xe3, 0xe3, 0x01, 0xf0,
0x5a, 0xa9, 0x97, 0xc5, 0x36, 0x5a, 0xb6, 0x79, 0xc2, 0xef, 0x3f, 0x8c, 0x1a, 0xef, 0x3f, 0x8c,
0x1a, 0xf7, 0x0f, 0x23, 0xef, 0xfd, 0xc3, 0xc8, 0xfb, 0xf7, 0x61, 0xe4, 0xfd, 0xf9, 0xdf, 0xa8,
0xf1, 0x7f, 0x00, 0x00, 0x00, 0xff, 0xff, 0x86, 0x52, 0x5b, 0xe0, 0x74, 0x06, 0x00, 0x00,
}

10
vendor/github.com/coreos/etcd/raft/raftpb/raft.proto generated vendored
View File

@ -76,13 +76,15 @@ message HardState {
}
message ConfState {
repeated uint64 nodes = 1;
repeated uint64 nodes = 1;
repeated uint64 learners = 2;
}
enum ConfChangeType {
ConfChangeAddNode = 0;
ConfChangeRemoveNode = 1;
ConfChangeUpdateNode = 2;
ConfChangeAddNode = 0;
ConfChangeRemoveNode = 1;
ConfChangeUpdateNode = 2;
ConfChangeAddLearnerNode = 3;
}
message ConfChange {

16
vendor/github.com/coreos/etcd/raft/rafttest/doc.go generated vendored Normal file
View File

@ -0,0 +1,16 @@
// Copyright 2015 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package rafttest provides functional tests for etcd's raft implementation.
package rafttest

183
vendor/github.com/coreos/etcd/raft/rafttest/network.go generated vendored Normal file
View File

@ -0,0 +1,183 @@
// Copyright 2015 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package rafttest
import (
"math/rand"
"sync"
"time"
"github.com/coreos/etcd/raft/raftpb"
)
// a network interface
type iface interface {
send(m raftpb.Message)
recv() chan raftpb.Message
disconnect()
connect()
}
// a network
type network interface {
// drop message at given rate (1.0 drops all messages)
drop(from, to uint64, rate float64)
// delay message for (0, d] randomly at given rate (1.0 delay all messages)
// do we need rate here?
delay(from, to uint64, d time.Duration, rate float64)
disconnect(id uint64)
connect(id uint64)
// heal heals the network
heal()
}
type raftNetwork struct {
mu sync.Mutex
disconnected map[uint64]bool
dropmap map[conn]float64
delaymap map[conn]delay
recvQueues map[uint64]chan raftpb.Message
}
type conn struct {
from, to uint64
}
type delay struct {
d time.Duration
rate float64
}
func newRaftNetwork(nodes ...uint64) *raftNetwork {
pn := &raftNetwork{
recvQueues: make(map[uint64]chan raftpb.Message),
dropmap: make(map[conn]float64),
delaymap: make(map[conn]delay),
disconnected: make(map[uint64]bool),
}
for _, n := range nodes {
pn.recvQueues[n] = make(chan raftpb.Message, 1024)
}
return pn
}
func (rn *raftNetwork) nodeNetwork(id uint64) iface {
return &nodeNetwork{id: id, raftNetwork: rn}
}
func (rn *raftNetwork) send(m raftpb.Message) {
rn.mu.Lock()
to := rn.recvQueues[m.To]
if rn.disconnected[m.To] {
to = nil
}
drop := rn.dropmap[conn{m.From, m.To}]
dl := rn.delaymap[conn{m.From, m.To}]
rn.mu.Unlock()
if to == nil {
return
}
if drop != 0 && rand.Float64() < drop {
return
}
// TODO: shall we dl without blocking the send call?
if dl.d != 0 && rand.Float64() < dl.rate {
rd := rand.Int63n(int64(dl.d))
time.Sleep(time.Duration(rd))
}
// use marshal/unmarshal to copy message to avoid data race.
b, err := m.Marshal()
if err != nil {
panic(err)
}
var cm raftpb.Message
err = cm.Unmarshal(b)
if err != nil {
panic(err)
}
select {
case to <- cm:
default:
// drop messages when the receiver queue is full.
}
}
func (rn *raftNetwork) recvFrom(from uint64) chan raftpb.Message {
rn.mu.Lock()
fromc := rn.recvQueues[from]
if rn.disconnected[from] {
fromc = nil
}
rn.mu.Unlock()
return fromc
}
func (rn *raftNetwork) drop(from, to uint64, rate float64) {
rn.mu.Lock()
defer rn.mu.Unlock()
rn.dropmap[conn{from, to}] = rate
}
func (rn *raftNetwork) delay(from, to uint64, d time.Duration, rate float64) {
rn.mu.Lock()
defer rn.mu.Unlock()
rn.delaymap[conn{from, to}] = delay{d, rate}
}
func (rn *raftNetwork) heal() {
rn.mu.Lock()
defer rn.mu.Unlock()
rn.dropmap = make(map[conn]float64)
rn.delaymap = make(map[conn]delay)
}
func (rn *raftNetwork) disconnect(id uint64) {
rn.mu.Lock()
defer rn.mu.Unlock()
rn.disconnected[id] = true
}
func (rn *raftNetwork) connect(id uint64) {
rn.mu.Lock()
defer rn.mu.Unlock()
rn.disconnected[id] = false
}
type nodeNetwork struct {
id uint64
*raftNetwork
}
func (nt *nodeNetwork) connect() {
nt.raftNetwork.connect(nt.id)
}
func (nt *nodeNetwork) disconnect() {
nt.raftNetwork.disconnect(nt.id)
}
func (nt *nodeNetwork) send(m raftpb.Message) {
nt.raftNetwork.send(m)
}
func (nt *nodeNetwork) recv() chan raftpb.Message {
return nt.recvFrom(nt.id)
}

150
vendor/github.com/coreos/etcd/raft/rafttest/node.go generated vendored Normal file
View File

@ -0,0 +1,150 @@
// Copyright 2015 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package rafttest
import (
"context"
"log"
"sync"
"time"
"github.com/coreos/etcd/raft"
"github.com/coreos/etcd/raft/raftpb"
)
type node struct {
raft.Node
id uint64
iface iface
stopc chan struct{}
pausec chan bool
// stable
storage *raft.MemoryStorage
mu sync.Mutex // guards state
state raftpb.HardState
}
func startNode(id uint64, peers []raft.Peer, iface iface) *node {
st := raft.NewMemoryStorage()
c := &raft.Config{
ID: id,
ElectionTick: 10,
HeartbeatTick: 1,
Storage: st,
MaxSizePerMsg: 1024 * 1024,
MaxInflightMsgs: 256,
}
rn := raft.StartNode(c, peers)
n := &node{
Node: rn,
id: id,
storage: st,
iface: iface,
pausec: make(chan bool),
}
n.start()
return n
}
func (n *node) start() {
n.stopc = make(chan struct{})
ticker := time.Tick(5 * time.Millisecond)
go func() {
for {
select {
case <-ticker:
n.Tick()
case rd := <-n.Ready():
if !raft.IsEmptyHardState(rd.HardState) {
n.mu.Lock()
n.state = rd.HardState
n.mu.Unlock()
n.storage.SetHardState(n.state)
}
n.storage.Append(rd.Entries)
time.Sleep(time.Millisecond)
// TODO: make send async, more like real world...
for _, m := range rd.Messages {
n.iface.send(m)
}
n.Advance()
case m := <-n.iface.recv():
go n.Step(context.TODO(), m)
case <-n.stopc:
n.Stop()
log.Printf("raft.%d: stop", n.id)
n.Node = nil
close(n.stopc)
return
case p := <-n.pausec:
recvms := make([]raftpb.Message, 0)
for p {
select {
case m := <-n.iface.recv():
recvms = append(recvms, m)
case p = <-n.pausec:
}
}
// step all pending messages
for _, m := range recvms {
n.Step(context.TODO(), m)
}
}
}
}()
}
// stop stops the node. stop a stopped node might panic.
// All in memory state of node is discarded.
// All stable MUST be unchanged.
func (n *node) stop() {
n.iface.disconnect()
n.stopc <- struct{}{}
// wait for the shutdown
<-n.stopc
}
// restart restarts the node. restart a started node
// blocks and might affect the future stop operation.
func (n *node) restart() {
// wait for the shutdown
<-n.stopc
c := &raft.Config{
ID: n.id,
ElectionTick: 10,
HeartbeatTick: 1,
Storage: n.storage,
MaxSizePerMsg: 1024 * 1024,
MaxInflightMsgs: 256,
}
n.Node = raft.RestartNode(c)
n.start()
n.iface.connect()
}
// pause pauses the node.
// The paused node buffers the received messages and replies
// all of them when it resumes.
func (n *node) pause() {
n.pausec <- true
}
// resume resumes the paused node.
func (n *node) resume() {
n.pausec <- false
}

8
vendor/github.com/coreos/etcd/raft/rawnode.go generated vendored
View File

@ -170,11 +170,13 @@ func (rn *RawNode) ProposeConfChange(cc pb.ConfChange) error {
func (rn *RawNode) ApplyConfChange(cc pb.ConfChange) *pb.ConfState {
if cc.NodeID == None {
rn.raft.resetPendingConf()
return &pb.ConfState{Nodes: rn.raft.nodes()}
return &pb.ConfState{Nodes: rn.raft.voters(), Learners: rn.raft.learners()}
}
switch cc.Type {
case pb.ConfChangeAddNode:
rn.raft.addNode(cc.NodeID)
case pb.ConfChangeAddLearnerNode:
rn.raft.addLearner(cc.NodeID)
case pb.ConfChangeRemoveNode:
rn.raft.removeNode(cc.NodeID)
case pb.ConfChangeUpdateNode:
@ -182,7 +184,7 @@ func (rn *RawNode) ApplyConfChange(cc pb.ConfChange) *pb.ConfState {
default:
panic("unexpected conf type")
}
return &pb.ConfState{Nodes: rn.raft.nodes()}
return &pb.ConfState{Nodes: rn.raft.voters(), Learners: rn.raft.learners()}
}
// Step advances the state machine using the given message.
@ -191,7 +193,7 @@ func (rn *RawNode) Step(m pb.Message) error {
if IsLocalMsg(m.Type) {
return ErrStepLocalMsg
}
if _, ok := rn.raft.prs[m.From]; ok || !IsResponseMsg(m.Type) {
if pr := rn.raft.getProgress(m.From); pr != nil || !IsResponseMsg(m.Type) {
return rn.raft.Step(m)
}
return ErrStepPeerNotFound

4
vendor/github.com/coreos/etcd/raft/read_only.go generated vendored
View File

@ -18,7 +18,7 @@ import pb "github.com/coreos/etcd/raft/raftpb"
// ReadState provides state for read only query.
// It's caller's responsibility to call ReadIndex first before getting
// this state from ready, It's also caller's duty to differentiate if this
// this state from ready, it's also caller's duty to differentiate if this
// state is what it requests through RequestCtx, eg. given a unique id as
// RequestCtx
type ReadState struct {
@ -100,7 +100,7 @@ func (ro *readOnly) advance(m pb.Message) []*readIndexStatus {
if found {
ro.readIndexQueue = ro.readIndexQueue[i:]
for _, rs := range rss {
delete(ro.pendingReadIndex, string(rs.req.Context))
delete(ro.pendingReadIndex, string(rs.req.Entries[0].Data))
}
return rss
}

22
vendor/github.com/coreos/etcd/raft/status.go generated vendored
View File

@ -28,11 +28,17 @@ type Status struct {
Applied uint64
Progress map[uint64]Progress
LeadTransferee uint64
}
// getStatus gets a copy of the current raft status.
func getStatus(r *raft) Status {
s := Status{ID: r.id}
s := Status{
ID: r.id,
LeadTransferee: r.leadTransferee,
}
s.HardState = r.hardState()
s.SoftState = *r.softState()
@ -43,6 +49,10 @@ func getStatus(r *raft) Status {
for id, p := range r.prs {
s.Progress[id] = *p
}
for id, p := range r.learnerPrs {
s.Progress[id] = *p
}
}
return s
@ -51,19 +61,21 @@ func getStatus(r *raft) Status {
// MarshalJSON translates the raft status into JSON.
// TODO: try to simplify this by introducing ID type into raft
func (s Status) MarshalJSON() ([]byte, error) {
j := fmt.Sprintf(`{"id":"%x","term":%d,"vote":"%x","commit":%d,"lead":"%x","raftState":%q,"progress":{`,
s.ID, s.Term, s.Vote, s.Commit, s.Lead, s.RaftState)
j := fmt.Sprintf(`{"id":"%x","term":%d,"vote":"%x","commit":%d,"lead":"%x","raftState":%q,"applied":%d,"progress":{`,
s.ID, s.Term, s.Vote, s.Commit, s.Lead, s.RaftState, s.Applied)
if len(s.Progress) == 0 {
j += "}}"
j += "},"
} else {
for k, v := range s.Progress {
subj := fmt.Sprintf(`"%x":{"match":%d,"next":%d,"state":%q},`, k, v.Match, v.Next, v.State)
j += subj
}
// remove the trailing ","
j = j[:len(j)-1] + "}}"
j = j[:len(j)-1] + "},"
}
j += fmt.Sprintf(`"leadtransferee":"%x"}`, s.LeadTransferee)
return []byte(j), nil
}

36
vendor/github.com/coreos/etcd/rafthttp/http.go generated vendored
View File

@ -15,19 +15,20 @@
package rafthttp
import (
"context"
"errors"
"fmt"
"io/ioutil"
"net/http"
"path"
"strings"
"time"
pioutil "github.com/coreos/etcd/pkg/ioutil"
"github.com/coreos/etcd/pkg/types"
"github.com/coreos/etcd/raft/raftpb"
"github.com/coreos/etcd/snap"
"github.com/coreos/etcd/version"
"golang.org/x/net/context"
)
const (
@ -91,11 +92,7 @@ func (h *pipelineHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
return
}
if from, err := types.IDFromString(r.Header.Get("X-Server-From")); err != nil {
if urls := r.Header.Get("X-PeerURLs"); urls != "" {
h.tr.AddRemote(from, strings.Split(urls, ","))
}
}
addRemoteFromRequest(h.tr, r)
// Limit the data size that could be read from the request body, which ensures that read from
// connection will not time out accidentally due to possible blocking in underlying implementation.
@ -153,6 +150,8 @@ func newSnapshotHandler(tr Transporter, r Raft, snapshotter *snap.Snapshotter, c
}
}
const unknownSnapshotSender = "UNKNOWN_SNAPSHOT_SENDER"
// ServeHTTP serves HTTP request to receive and process snapshot message.
//
// If request sender dies without closing underlying TCP connection,
@ -163,9 +162,12 @@ func newSnapshotHandler(tr Transporter, r Raft, snapshotter *snap.Snapshotter, c
// received and processed.
// 2. this case should happen rarely, so no further optimization is done.
func (h *snapshotHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
start := time.Now()
if r.Method != "POST" {
w.Header().Set("Allow", "POST")
http.Error(w, "Method Not Allowed", http.StatusMethodNotAllowed)
snapshotReceiveFailures.WithLabelValues(unknownSnapshotSender).Inc()
return
}
@ -173,30 +175,31 @@ func (h *snapshotHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if err := checkClusterCompatibilityFromHeader(r.Header, h.cid); err != nil {
http.Error(w, err.Error(), http.StatusPreconditionFailed)
snapshotReceiveFailures.WithLabelValues(unknownSnapshotSender).Inc()
return
}
if from, err := types.IDFromString(r.Header.Get("X-Server-From")); err != nil {
if urls := r.Header.Get("X-PeerURLs"); urls != "" {
h.tr.AddRemote(from, strings.Split(urls, ","))
}
}
addRemoteFromRequest(h.tr, r)
dec := &messageDecoder{r: r.Body}
m, err := dec.decode()
// let snapshots be very large since they can exceed 512MB for large installations
m, err := dec.decodeLimit(uint64(1 << 63))
from := types.ID(m.From).String()
if err != nil {
msg := fmt.Sprintf("failed to decode raft message (%v)", err)
plog.Errorf(msg)
http.Error(w, msg, http.StatusBadRequest)
recvFailures.WithLabelValues(r.RemoteAddr).Inc()
snapshotReceiveFailures.WithLabelValues(from).Inc()
return
}
receivedBytes.WithLabelValues(types.ID(m.From).String()).Add(float64(m.Size()))
receivedBytes.WithLabelValues(from).Add(float64(m.Size()))
if m.Type != raftpb.MsgSnap {
plog.Errorf("unexpected raft message type %s on snapshot path", m.Type)
http.Error(w, "wrong raft message type", http.StatusBadRequest)
snapshotReceiveFailures.WithLabelValues(from).Inc()
return
}
@ -207,9 +210,10 @@ func (h *snapshotHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
msg := fmt.Sprintf("failed to save KV snapshot (%v)", err)
plog.Error(msg)
http.Error(w, msg, http.StatusInternalServerError)
snapshotReceiveFailures.WithLabelValues(from).Inc()
return
}
receivedBytes.WithLabelValues(types.ID(m.From).String()).Add(float64(n))
receivedBytes.WithLabelValues(from).Add(float64(n))
plog.Infof("received and saved database snapshot [index: %d, from: %s] successfully", m.Snapshot.Metadata.Index, types.ID(m.From))
if err := h.r.Process(context.TODO(), m); err != nil {
@ -222,12 +226,16 @@ func (h *snapshotHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
msg := fmt.Sprintf("failed to process raft message (%v)", err)
plog.Warningf(msg)
http.Error(w, msg, http.StatusInternalServerError)
snapshotReceiveFailures.WithLabelValues(from).Inc()
}
return
}
// Write StatusNoContent header after the message has been processed by
// raft, which facilitates the client to report MsgSnap status.
w.WriteHeader(http.StatusNoContent)
snapshotReceive.WithLabelValues(from).Inc()
snapshotReceiveSeconds.WithLabelValues(from).Observe(time.Since(start).Seconds())
}
type streamHandler struct {

70
vendor/github.com/coreos/etcd/rafthttp/metrics.go generated vendored
View File

@ -53,6 +53,68 @@ var (
[]string{"From"},
)
snapshotSend = prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "etcd",
Subsystem: "network",
Name: "snapshot_send_success",
Help: "Total number of successful snapshot sends",
},
[]string{"To"},
)
snapshotSendFailures = prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "etcd",
Subsystem: "network",
Name: "snapshot_send_failures",
Help: "Total number of snapshot send failures",
},
[]string{"To"},
)
snapshotSendSeconds = prometheus.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "etcd",
Subsystem: "network",
Name: "snapshot_send_total_duration_seconds",
Help: "Total latency distributions of v3 snapshot sends",
// lowest bucket start of upper bound 0.1 sec (100 ms) with factor 2
// highest bucket start of 0.1 sec * 2^9 == 51.2 sec
Buckets: prometheus.ExponentialBuckets(0.1, 2, 10),
},
[]string{"To"},
)
snapshotReceive = prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "etcd",
Subsystem: "network",
Name: "snapshot_receive_success",
Help: "Total number of successful snapshot receives",
},
[]string{"From"},
)
snapshotReceiveFailures = prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: "etcd",
Subsystem: "network",
Name: "snapshot_receive_failures",
Help: "Total number of snapshot receive failures",
},
[]string{"From"},
)
snapshotReceiveSeconds = prometheus.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "etcd",
Subsystem: "network",
Name: "snapshot_receive_total_duration_seconds",
Help: "Total latency distributions of v3 snapshot receives",
// lowest bucket start of upper bound 0.1 sec (100 ms) with factor 2
// highest bucket start of 0.1 sec * 2^9 == 51.2 sec
Buckets: prometheus.ExponentialBuckets(0.1, 2, 10),
},
[]string{"From"},
)
rtts = prometheus.NewHistogramVec(prometheus.HistogramOpts{
Namespace: "etcd",
Subsystem: "network",
@ -69,5 +131,13 @@ func init() {
prometheus.MustRegister(receivedBytes)
prometheus.MustRegister(sentFailures)
prometheus.MustRegister(recvFailures)
prometheus.MustRegister(snapshotSend)
prometheus.MustRegister(snapshotSendFailures)
prometheus.MustRegister(snapshotSendSeconds)
prometheus.MustRegister(snapshotReceive)
prometheus.MustRegister(snapshotReceiveFailures)
prometheus.MustRegister(snapshotReceiveSeconds)
prometheus.MustRegister(rtts)
}

6
vendor/github.com/coreos/etcd/rafthttp/msg_codec.go generated vendored
View File

@ -48,12 +48,16 @@ var (
)
func (dec *messageDecoder) decode() (raftpb.Message, error) {
return dec.decodeLimit(readBytesLimit)
}
func (dec *messageDecoder) decodeLimit(numBytes uint64) (raftpb.Message, error) {
var m raftpb.Message
var l uint64
if err := binary.Read(dec.r, binary.BigEndian, &l); err != nil {
return m, err
}
if l > readBytesLimit {
if l > numBytes {
return m, ErrExceedSizeLimit
}
buf := make([]byte, int(l))

8
vendor/github.com/coreos/etcd/rafthttp/peer.go generated vendored
View File

@ -15,6 +15,7 @@
package rafthttp
import (
"context"
"sync"
"time"
@ -23,7 +24,8 @@ import (
"github.com/coreos/etcd/raft"
"github.com/coreos/etcd/raft/raftpb"
"github.com/coreos/etcd/snap"
"golang.org/x/net/context"
"golang.org/x/time/rate"
)
const (
@ -188,6 +190,7 @@ func startPeer(transport *Transport, urls types.URLs, peerID types.ID, fs *stats
status: status,
recvc: p.recvc,
propc: p.propc,
rl: rate.NewLimiter(transport.DialRetryFrequency, 1),
}
p.msgAppReader = &streamReader{
peerID: peerID,
@ -197,7 +200,9 @@ func startPeer(transport *Transport, urls types.URLs, peerID types.ID, fs *stats
status: status,
recvc: p.recvc,
propc: p.propc,
rl: rate.NewLimiter(transport.DialRetryFrequency, 1),
}
p.msgAppV2Reader.start()
p.msgAppReader.start()
@ -225,6 +230,7 @@ func (p *peer) send(m raftpb.Message) {
plog.MergeWarningf("dropped internal raft message to %s since %s's sending buffer is full (bad/overloaded network)", p.id, name)
}
plog.Debugf("dropped %s to %s since %s's sending buffer is full", m.Type, p.id, name)
sentFailures.WithLabelValues(types.ID(m.To).String()).Inc()
}
}

2
vendor/github.com/coreos/etcd/rafthttp/peer_status.go generated vendored
View File

@ -56,7 +56,7 @@ func (s *peerStatus) deactivate(failure failureType, reason string) {
msg := fmt.Sprintf("failed to %s %s on %s (%s)", failure.action, s.id, failure.source, reason)
if s.active {
plog.Errorf(msg)
plog.Infof("peer %s became inactive", s.id)
plog.Infof("peer %s became inactive (message send to peer failed)", s.id)
s.active = false
s.since = time.Time{}
return

5
vendor/github.com/coreos/etcd/rafthttp/pipeline.go generated vendored
View File

@ -16,13 +16,13 @@ package rafthttp
import (
"bytes"
"context"
"errors"
"io/ioutil"
"sync"
"time"
"github.com/coreos/etcd/etcdserver/stats"
"github.com/coreos/etcd/pkg/httputil"
"github.com/coreos/etcd/pkg/pbutil"
"github.com/coreos/etcd/pkg/types"
"github.com/coreos/etcd/raft"
@ -118,7 +118,8 @@ func (p *pipeline) post(data []byte) (err error) {
req := createPostRequest(u, RaftPrefix, bytes.NewBuffer(data), "application/protobuf", p.tr.URLs, p.tr.ID, p.tr.ClusterID)
done := make(chan struct{}, 1)
cancel := httputil.RequestCanceler(req)
ctx, cancel := context.WithCancel(context.Background())
req = req.WithContext(ctx)
go func() {
select {
case <-done:

21
vendor/github.com/coreos/etcd/rafthttp/probing_status.go generated vendored
View File

@ -17,6 +17,7 @@ package rafthttp
import (
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/xiang90/probing"
)
@ -28,7 +29,15 @@ var (
statusErrorInterval = 5 * time.Second
)
func addPeerToProber(p probing.Prober, id string, us []string) {
const (
// RoundTripperNameRaftMessage is the name of round-tripper that sends
// all other Raft messages, other than "snap.Message".
RoundTripperNameRaftMessage = "ROUND_TRIPPER_RAFT_MESSAGE"
// RoundTripperNameSnapshot is the name of round-tripper that sends merged snapshot message.
RoundTripperNameSnapshot = "ROUND_TRIPPER_SNAPSHOT"
)
func addPeerToProber(p probing.Prober, id string, us []string, roundTripperName string, rttSecProm *prometheus.HistogramVec) {
hus := make([]string, len(us))
for i := range us {
hus[i] = us[i] + ProbingPrefix
@ -40,26 +49,26 @@ func addPeerToProber(p probing.Prober, id string, us []string) {
if err != nil {
plog.Errorf("failed to add peer %s into prober", id)
} else {
go monitorProbingStatus(s, id)
go monitorProbingStatus(s, id, roundTripperName, rttSecProm)
}
}
func monitorProbingStatus(s probing.Status, id string) {
func monitorProbingStatus(s probing.Status, id string, roundTripperName string, rttSecProm *prometheus.HistogramVec) {
// set the first interval short to log error early.
interval := statusErrorInterval
for {
select {
case <-time.After(interval):
if !s.Health() {
plog.Warningf("health check for peer %s could not connect: %v", id, s.Err())
plog.Warningf("health check for peer %s could not connect: %v (prober %q)", id, s.Err(), roundTripperName)
interval = statusErrorInterval
} else {
interval = statusMonitoringInterval
}
if s.ClockDiff() > time.Second {
plog.Warningf("the clock difference against peer %s is too high [%v > %v]", id, s.ClockDiff(), time.Second)
plog.Warningf("the clock difference against peer %s is too high [%v > %v] (prober %q)", id, s.ClockDiff(), time.Second, roundTripperName)
}
rtts.WithLabelValues(id).Observe(s.SRTT().Seconds())
rttSecProm.WithLabelValues(id).Observe(s.SRTT().Seconds())
case <-s.StopNotify():
return
}

1
vendor/github.com/coreos/etcd/rafthttp/remote.go generated vendored
View File

@ -53,6 +53,7 @@ func (g *remote) send(m raftpb.Message) {
plog.MergeWarningf("dropped internal raft message to %s since sending buffer is full (bad/overloaded network)", g.id)
}
plog.Debugf("dropped %s to %s since sending buffer is full", m.Type, g.id)
sentFailures.WithLabelValues(types.ID(m.To).String()).Inc()
}
}

17
vendor/github.com/coreos/etcd/rafthttp/snapshot_sender.go generated vendored
View File

@ -16,6 +16,7 @@ package rafthttp
import (
"bytes"
"context"
"io"
"io/ioutil"
"net/http"
@ -63,7 +64,10 @@ func newSnapshotSender(tr *Transport, picker *urlPicker, to types.ID, status *pe
func (s *snapshotSender) stop() { close(s.stopc) }
func (s *snapshotSender) send(merged snap.Message) {
start := time.Now()
m := merged.Message
to := types.ID(m.To).String()
body := createSnapBody(merged)
defer body.Close()
@ -91,20 +95,26 @@ func (s *snapshotSender) send(merged snap.Message) {
// machine knows about it, it would pause a while and retry sending
// new snapshot message.
s.r.ReportSnapshot(m.To, raft.SnapshotFailure)
sentFailures.WithLabelValues(types.ID(m.To).String()).Inc()
sentFailures.WithLabelValues(to).Inc()
snapshotSendFailures.WithLabelValues(to).Inc()
return
}
s.status.activate()
s.r.ReportSnapshot(m.To, raft.SnapshotFinish)
plog.Infof("database snapshot [index: %d, to: %s] sent out successfully", m.Snapshot.Metadata.Index, types.ID(m.To))
sentBytes.WithLabelValues(types.ID(m.To).String()).Add(float64(merged.TotalSize))
sentBytes.WithLabelValues(to).Add(float64(merged.TotalSize))
snapshotSend.WithLabelValues(to).Inc()
snapshotSendSeconds.WithLabelValues(to).Observe(time.Since(start).Seconds())
}
// post posts the given request.
// It returns nil when request is sent out and processed successfully.
func (s *snapshotSender) post(req *http.Request) (err error) {
cancel := httputil.RequestCanceler(req)
ctx, cancel := context.WithCancel(context.Background())
req = req.WithContext(ctx)
defer cancel()
type responseAndError struct {
resp *http.Response
@ -130,7 +140,6 @@ func (s *snapshotSender) post(req *http.Request) (err error) {
select {
case <-s.stopc:
cancel()
return errStopped
case r := <-result:
if r.err != nil {

75
vendor/github.com/coreos/etcd/rafthttp/stream.go generated vendored
View File

@ -15,18 +15,21 @@
package rafthttp
import (
"context"
"fmt"
"io"
"io/ioutil"
"net"
"net/http"
"path"
"strings"
"sync"
"time"
"golang.org/x/time/rate"
"github.com/coreos/etcd/etcdserver/stats"
"github.com/coreos/etcd/pkg/httputil"
"github.com/coreos/etcd/pkg/transport"
"github.com/coreos/etcd/pkg/types"
"github.com/coreos/etcd/raft/raftpb"
"github.com/coreos/etcd/version"
@ -51,6 +54,8 @@ var (
"2.3.0": {streamTypeMsgAppV2, streamTypeMessage},
"3.0.0": {streamTypeMsgAppV2, streamTypeMessage},
"3.1.0": {streamTypeMsgAppV2, streamTypeMessage},
"3.2.0": {streamTypeMsgAppV2, streamTypeMessage},
"3.3.0": {streamTypeMsgAppV2, streamTypeMessage},
}
)
@ -140,7 +145,8 @@ func (cw *streamWriter) run() {
flusher http.Flusher
batched int
)
tickc := time.Tick(ConnReadTimeout / 3)
tickc := time.NewTicker(ConnReadTimeout / 3)
defer tickc.Stop()
unflushed := 0
plog.Infof("started streaming with peer %s (writer)", cw.peerID)
@ -212,7 +218,7 @@ func (cw *streamWriter) run() {
plog.Warningf("closed an existing TCP streaming connection with peer %s (%s writer)", cw.peerID, t)
}
plog.Infof("established a TCP streaming connection with peer %s (%s writer)", cw.peerID, t)
heartbeatc, msgc = tickc, cw.msgc
heartbeatc, msgc = tickc.C, cw.msgc
case <-cw.stopc:
if cw.close() {
plog.Infof("closed the TCP streaming connection with peer %s (%s writer)", cw.peerID, t)
@ -240,7 +246,9 @@ func (cw *streamWriter) closeUnlocked() bool {
if !cw.working {
return false
}
cw.closer.Close()
if err := cw.closer.Close(); err != nil {
plog.Errorf("peer %s (writer) connection close error: %v", cw.peerID, err)
}
if len(cw.msgc) > 0 {
cw.r.ReportUnreachable(uint64(cw.peerID))
}
@ -275,25 +283,28 @@ type streamReader struct {
recvc chan<- raftpb.Message
propc chan<- raftpb.Message
rl *rate.Limiter // alters the frequency of dial retrial attempts
errorc chan<- error
mu sync.Mutex
paused bool
cancel func()
closer io.Closer
stopc chan struct{}
done chan struct{}
ctx context.Context
cancel context.CancelFunc
done chan struct{}
}
func (r *streamReader) start() {
r.stopc = make(chan struct{})
r.done = make(chan struct{})
if r.errorc == nil {
r.errorc = r.tr.ErrorC
func (cr *streamReader) start() {
cr.done = make(chan struct{})
if cr.errorc == nil {
cr.errorc = cr.tr.ErrorC
}
go r.run()
if cr.ctx == nil {
cr.ctx, cr.cancel = context.WithCancel(context.Background())
}
go cr.run()
}
func (cr *streamReader) run() {
@ -308,26 +319,27 @@ func (cr *streamReader) run() {
} else {
cr.status.activate()
plog.Infof("established a TCP streaming connection with peer %s (%s reader)", cr.peerID, cr.typ)
err := cr.decodeLoop(rc, t)
err = cr.decodeLoop(rc, t)
plog.Warningf("lost the TCP streaming connection with peer %s (%s reader)", cr.peerID, cr.typ)
switch {
// all data is read out
case err == io.EOF:
// connection is closed by the remote
case isClosedConnectionError(err):
case transport.IsClosedConnError(err):
default:
cr.status.deactivate(failureType{source: t.String(), action: "read"}, err.Error())
}
}
select {
// Wait 100ms to create a new stream, so it doesn't bring too much
// overhead when retry.
case <-time.After(100 * time.Millisecond):
case <-cr.stopc:
// Wait for a while before new dial attempt
err = cr.rl.Wait(cr.ctx)
if cr.ctx.Err() != nil {
plog.Infof("stopped streaming with peer %s (%s reader)", cr.peerID, t)
close(cr.done)
return
}
if err != nil {
plog.Errorf("streaming with peer %s (%s reader) rate limiter error: %v", cr.peerID, t, err)
}
}
}
@ -343,7 +355,7 @@ func (cr *streamReader) decodeLoop(rc io.ReadCloser, t streamType) error {
plog.Panicf("unhandled stream type %s", t)
}
select {
case <-cr.stopc:
case <-cr.ctx.Done():
cr.mu.Unlock()
if err := rc.Close(); err != nil {
return err
@ -398,11 +410,8 @@ func (cr *streamReader) decodeLoop(rc io.ReadCloser, t streamType) error {
}
func (cr *streamReader) stop() {
close(cr.stopc)
cr.mu.Lock()
if cr.cancel != nil {
cr.cancel()
}
cr.cancel()
cr.close()
cr.mu.Unlock()
<-cr.done
@ -426,14 +435,15 @@ func (cr *streamReader) dial(t streamType) (io.ReadCloser, error) {
setPeerURLsHeader(req, cr.tr.URLs)
req = req.WithContext(cr.ctx)
cr.mu.Lock()
select {
case <-cr.stopc:
case <-cr.ctx.Done():
cr.mu.Unlock()
return nil, fmt.Errorf("stream reader is stopped")
default:
}
cr.cancel = httputil.RequestCanceler(req)
cr.mu.Unlock()
resp, err := cr.tr.streamRt.RoundTrip(req)
@ -491,7 +501,9 @@ func (cr *streamReader) dial(t streamType) (io.ReadCloser, error) {
func (cr *streamReader) close() {
if cr.closer != nil {
cr.closer.Close()
if err := cr.closer.Close(); err != nil {
plog.Errorf("peer %s (reader) connection close error: %v", cr.peerID, err)
}
}
cr.closer = nil
}
@ -508,11 +520,6 @@ func (cr *streamReader) resume() {
cr.paused = false
}
func isClosedConnectionError(err error) bool {
operr, ok := err.(*net.OpError)
return ok && operr.Err.Error() == "use of closed network connection"
}
// checkStreamSupport checks whether the stream type is supported in the
// given version.
func checkStreamSupport(v *semver.Version, t streamType) bool {

58
vendor/github.com/coreos/etcd/rafthttp/transport.go generated vendored
View File

@ -15,6 +15,7 @@
package rafthttp
import (
"context"
"net/http"
"sync"
"time"
@ -26,9 +27,10 @@ import (
"github.com/coreos/etcd/raft"
"github.com/coreos/etcd/raft/raftpb"
"github.com/coreos/etcd/snap"
"github.com/coreos/pkg/capnslog"
"github.com/xiang90/probing"
"golang.org/x/net/context"
"golang.org/x/time/rate"
)
var plog = logutil.NewMergeLogger(capnslog.NewPackageLogger("github.com/coreos/etcd", "rafthttp"))
@ -83,6 +85,8 @@ type Transporter interface {
// If the connection is active since peer was added, it returns the adding time.
// If the connection is currently inactive, it returns zero time.
ActiveSince(id types.ID) time.Time
// ActivePeers returns the number of active peers.
ActivePeers() int
// Stop closes the connections and stops the transporter.
Stop()
}
@ -94,8 +98,12 @@ type Transporter interface {
// User needs to call Start before calling other functions, and call
// Stop when the Transport is no longer used.
type Transport struct {
DialTimeout time.Duration // maximum duration before timing out dial of the request
TLSInfo transport.TLSInfo // TLS information used when creating connection
DialTimeout time.Duration // maximum duration before timing out dial of the request
// DialRetryFrequency defines the frequency of streamReader dial retrial attempts;
// a distinct rate limiter is created per every peer (default value: 10 events/sec)
DialRetryFrequency rate.Limit
TLSInfo transport.TLSInfo // TLS information used when creating connection
ID types.ID // local member ID
URLs types.URLs // local peer URLs
@ -119,7 +127,8 @@ type Transport struct {
remotes map[types.ID]*remote // remotes map that helps newly joined member to catch up
peers map[types.ID]Peer // peers map
prober probing.Prober
pipelineProber probing.Prober
streamProber probing.Prober
}
func (t *Transport) Start() error {
@ -134,7 +143,15 @@ func (t *Transport) Start() error {
}
t.remotes = make(map[types.ID]*remote)
t.peers = make(map[types.ID]Peer)
t.prober = probing.NewProber(t.pipelineRt)
t.pipelineProber = probing.NewProber(t.pipelineRt)
t.streamProber = probing.NewProber(t.streamRt)
// If client didn't provide dial retry frequency, use the default
// (100ms backoff between attempts to create a new stream),
// so it doesn't bring too much overhead when retry.
if t.DialRetryFrequency == 0 {
t.DialRetryFrequency = rate.Every(100 * time.Millisecond)
}
return nil
}
@ -195,7 +212,8 @@ func (t *Transport) Stop() {
for _, p := range t.peers {
p.stop()
}
t.prober.RemoveAll()
t.pipelineProber.RemoveAll()
t.streamProber.RemoveAll()
if tr, ok := t.streamRt.(*http.Transport); ok {
tr.CloseIdleConnections()
}
@ -274,8 +292,8 @@ func (t *Transport) AddPeer(id types.ID, us []string) {
}
fs := t.LeaderStats.Follower(id.String())
t.peers[id] = startPeer(t, urls, id, fs)
addPeerToProber(t.prober, id.String(), us)
addPeerToProber(t.pipelineProber, id.String(), us, RoundTripperNameSnapshot, rtts)
addPeerToProber(t.streamProber, id.String(), us, RoundTripperNameRaftMessage, rtts)
plog.Infof("added peer %s", id)
}
@ -302,7 +320,8 @@ func (t *Transport) removePeer(id types.ID) {
}
delete(t.peers, id)
delete(t.LeaderStats.Followers, id.String())
t.prober.Remove(id.String())
t.pipelineProber.Remove(id.String())
t.streamProber.Remove(id.String())
plog.Infof("removed peer %s", id)
}
@ -319,8 +338,10 @@ func (t *Transport) UpdatePeer(id types.ID, us []string) {
}
t.peers[id].update(urls)
t.prober.Remove(id.String())
addPeerToProber(t.prober, id.String(), us)
t.pipelineProber.Remove(id.String())
addPeerToProber(t.pipelineProber, id.String(), us, RoundTripperNameSnapshot, rtts)
t.streamProber.Remove(id.String())
addPeerToProber(t.streamProber, id.String(), us, RoundTripperNameRaftMessage, rtts)
plog.Infof("updated peer %s", id)
}
@ -362,6 +383,20 @@ func (t *Transport) Resume() {
}
}
// ActivePeers returns a channel that closes when an initial
// peer connection has been established. Use this to wait until the
// first peer connection becomes active.
func (t *Transport) ActivePeers() (cnt int) {
t.mu.RLock()
defer t.mu.RUnlock()
for _, p := range t.peers {
if !p.activeSince().IsZero() {
cnt++
}
}
return cnt
}
type nopTransporter struct{}
func NewNopTransporter() Transporter {
@ -378,6 +413,7 @@ func (s *nopTransporter) RemovePeer(id types.ID) {}
func (s *nopTransporter) RemoveAllPeers() {}
func (s *nopTransporter) UpdatePeer(id types.ID, us []string) {}
func (s *nopTransporter) ActiveSince(id types.ID) time.Time { return time.Time{} }
func (s *nopTransporter) ActivePeers() int { return 0 }
func (s *nopTransporter) Stop() {}
func (s *nopTransporter) Pause() {}
func (s *nopTransporter) Resume() {}

41
vendor/github.com/coreos/etcd/rafthttp/util.go generated vendored
View File

@ -15,8 +15,6 @@
package rafthttp
import (
"crypto/tls"
"encoding/binary"
"fmt"
"io"
"net"
@ -27,7 +25,6 @@ import (
"github.com/coreos/etcd/pkg/transport"
"github.com/coreos/etcd/pkg/types"
"github.com/coreos/etcd/raft/raftpb"
"github.com/coreos/etcd/version"
"github.com/coreos/go-semver/semver"
)
@ -39,8 +36,8 @@ var (
// NewListener returns a listener for raft message transfer between peers.
// It uses timeout listener to identify broken streams promptly.
func NewListener(u url.URL, tlscfg *tls.Config) (net.Listener, error) {
return transport.NewTimeoutListener(u.Host, u.Scheme, tlscfg, ConnReadTimeout, ConnWriteTimeout)
func NewListener(u url.URL, tlsinfo *transport.TLSInfo) (net.Listener, error) {
return transport.NewTimeoutListener(u.Host, u.Scheme, tlsinfo, ConnReadTimeout, ConnWriteTimeout)
}
// NewRoundTripper returns a roundTripper used to send requests
@ -61,31 +58,6 @@ func newStreamRoundTripper(tlsInfo transport.TLSInfo, dialTimeout time.Duration)
return transport.NewTimeoutTransport(tlsInfo, dialTimeout, ConnReadTimeout, ConnWriteTimeout)
}
func writeEntryTo(w io.Writer, ent *raftpb.Entry) error {
size := ent.Size()
if err := binary.Write(w, binary.BigEndian, uint64(size)); err != nil {
return err
}
b, err := ent.Marshal()
if err != nil {
return err
}
_, err = w.Write(b)
return err
}
func readEntryFrom(r io.Reader, ent *raftpb.Entry) error {
var l uint64
if err := binary.Read(r, binary.BigEndian, &l); err != nil {
return err
}
buf := make([]byte, int(l))
if _, err := io.ReadFull(r, buf); err != nil {
return err
}
return ent.Unmarshal(buf)
}
// createPostRequest creates a HTTP POST request that sends raft message.
func createPostRequest(u url.URL, path string, body io.Reader, ct string, urls types.URLs, from, cid types.ID) *http.Request {
uu := u
@ -203,3 +175,12 @@ func setPeerURLsHeader(req *http.Request, urls types.URLs) {
}
req.Header.Set("X-PeerURLs", strings.Join(peerURLs, ","))
}
// addRemoteFromRequest adds a remote peer according to an http request header
func addRemoteFromRequest(tr Transporter, r *http.Request) {
if from, err := types.IDFromString(r.Header.Get("X-Server-From")); err == nil {
if urls := r.Header.Get("X-PeerURLs"); urls != "" {
tr.AddRemote(from, strings.Split(urls, ","))
}
}
}

29
vendor/github.com/coreos/etcd/snap/db.go generated vendored
View File

@ -15,18 +15,24 @@
package snap
import (
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"path"
"path/filepath"
"time"
"github.com/coreos/etcd/pkg/fileutil"
)
var ErrNoDBSnapshot = errors.New("snap: snapshot file doesn't exist")
// SaveDBFrom saves snapshot of the database from the given reader. It
// guarantees the save operation is atomic.
func (s *Snapshotter) SaveDBFrom(r io.Reader, id uint64) (int64, error) {
start := time.Now()
f, err := ioutil.TempFile(s.dir, "tmp")
if err != nil {
return 0, err
@ -34,14 +40,16 @@ func (s *Snapshotter) SaveDBFrom(r io.Reader, id uint64) (int64, error) {
var n int64
n, err = io.Copy(f, r)
if err == nil {
fsyncStart := time.Now()
err = fileutil.Fsync(f)
snapDBFsyncSec.Observe(time.Since(fsyncStart).Seconds())
}
f.Close()
if err != nil {
os.Remove(f.Name())
return n, err
}
fn := path.Join(s.dir, fmt.Sprintf("%016x.snap.db", id))
fn := s.dbFilePath(id)
if fileutil.Exist(fn) {
os.Remove(f.Name())
return n, nil
@ -54,21 +62,22 @@ func (s *Snapshotter) SaveDBFrom(r io.Reader, id uint64) (int64, error) {
plog.Infof("saved database snapshot to disk [total bytes: %d]", n)
snapDBSaveSec.Observe(time.Since(start).Seconds())
return n, nil
}
// DBFilePath returns the file path for the snapshot of the database with
// given id. If the snapshot does not exist, it returns error.
func (s *Snapshotter) DBFilePath(id uint64) (string, error) {
fns, err := fileutil.ReadDir(s.dir)
if err != nil {
if _, err := fileutil.ReadDir(s.dir); err != nil {
return "", err
}
wfn := fmt.Sprintf("%016x.snap.db", id)
for _, fn := range fns {
if fn == wfn {
return path.Join(s.dir, fn), nil
}
if fn := s.dbFilePath(id); fileutil.Exist(fn) {
return fn, nil
}
return "", fmt.Errorf("snap: snapshot file doesn't exist")
return "", ErrNoDBSnapshot
}
func (s *Snapshotter) dbFilePath(id uint64) string {
return filepath.Join(s.dir, fmt.Sprintf("%016x.snap.db", id))
}

24
vendor/github.com/coreos/etcd/snap/metrics.go generated vendored
View File

@ -33,9 +33,33 @@ var (
Help: "The marshalling cost distributions of save called by snapshot.",
Buckets: prometheus.ExponentialBuckets(0.001, 2, 14),
})
snapDBSaveSec = prometheus.NewHistogram(prometheus.HistogramOpts{
Namespace: "etcd",
Subsystem: "snap_db",
Name: "save_total_duration_seconds",
Help: "The total latency distributions of v3 snapshot save",
// lowest bucket start of upper bound 0.1 sec (100 ms) with factor 2
// highest bucket start of 0.1 sec * 2^9 == 51.2 sec
Buckets: prometheus.ExponentialBuckets(0.1, 2, 10),
})
snapDBFsyncSec = prometheus.NewHistogram(prometheus.HistogramOpts{
Namespace: "etcd",
Subsystem: "snap_db",
Name: "fsync_duration_seconds",
Help: "The latency distributions of fsyncing .snap.db file",
// lowest bucket start of upper bound 0.001 sec (1 ms) with factor 2
// highest bucket start of 0.001 sec * 2^13 == 8.192 sec
Buckets: prometheus.ExponentialBuckets(0.001, 2, 14),
})
)
func init() {
prometheus.MustRegister(saveDurations)
prometheus.MustRegister(marshallingDurations)
prometheus.MustRegister(snapDBSaveSec)
prometheus.MustRegister(snapDBFsyncSec)
}

25
vendor/github.com/coreos/etcd/snap/snappb/snap.pb.go generated vendored
View File

@ -1,6 +1,5 @@
// Code generated by protoc-gen-gogo.
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: snap.proto
// DO NOT EDIT!
/*
Package snappb is a generated protocol buffer package.
@ -20,6 +19,8 @@ import (
math "math"
_ "github.com/gogo/protobuf/gogoproto"
io "io"
)
@ -78,24 +79,6 @@ func (m *Snapshot) MarshalTo(dAtA []byte) (int, error) {
return i, nil
}
func encodeFixed64Snap(dAtA []byte, offset int, v uint64) int {
dAtA[offset] = uint8(v)
dAtA[offset+1] = uint8(v >> 8)
dAtA[offset+2] = uint8(v >> 16)
dAtA[offset+3] = uint8(v >> 24)
dAtA[offset+4] = uint8(v >> 32)
dAtA[offset+5] = uint8(v >> 40)
dAtA[offset+6] = uint8(v >> 48)
dAtA[offset+7] = uint8(v >> 56)
return offset + 8
}
func encodeFixed32Snap(dAtA []byte, offset int, v uint32) int {
dAtA[offset] = uint8(v)
dAtA[offset+1] = uint8(v >> 8)
dAtA[offset+2] = uint8(v >> 16)
dAtA[offset+3] = uint8(v >> 24)
return offset + 4
}
func encodeVarintSnap(dAtA []byte, offset int, v uint64) int {
for v >= 1<<7 {
dAtA[offset] = uint8(v&0x7f | 0x80)
@ -342,7 +325,7 @@ func init() { proto.RegisterFile("snap.proto", fileDescriptorSnap) }
var fileDescriptorSnap = []byte{
// 126 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x09, 0x6e, 0x88, 0x02, 0xff, 0xe2, 0xe2, 0x2a, 0xce, 0x4b, 0x2c,
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x2a, 0xce, 0x4b, 0x2c,
0xd0, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x03, 0xb1, 0x0b, 0x92, 0xa4, 0x44, 0xd2, 0xf3,
0xd3, 0xf3, 0xc1, 0x42, 0xfa, 0x20, 0x16, 0x44, 0x56, 0xc9, 0x8c, 0x8b, 0x03, 0x24, 0x5f, 0x9c,
0x91, 0x5f, 0x22, 0x24, 0xc6, 0xc5, 0x9c, 0x5c, 0x94, 0x2c, 0xc1, 0xa8, 0xc0, 0xa8, 0xc1, 0xeb,

10
vendor/github.com/coreos/etcd/snap/snapshotter.go generated vendored
View File

@ -21,7 +21,7 @@ import (
"hash/crc32"
"io/ioutil"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
@ -84,13 +84,13 @@ func (s *Snapshotter) save(snapshot *raftpb.Snapshot) error {
marshallingDurations.Observe(float64(time.Since(start)) / float64(time.Second))
}
err = pioutil.WriteAndSyncFile(path.Join(s.dir, fname), d, 0666)
err = pioutil.WriteAndSyncFile(filepath.Join(s.dir, fname), d, 0666)
if err == nil {
saveDurations.Observe(float64(time.Since(start)) / float64(time.Second))
} else {
err1 := os.Remove(path.Join(s.dir, fname))
err1 := os.Remove(filepath.Join(s.dir, fname))
if err1 != nil {
plog.Errorf("failed to remove broken snapshot file %s", path.Join(s.dir, fname))
plog.Errorf("failed to remove broken snapshot file %s", filepath.Join(s.dir, fname))
}
}
return err
@ -114,7 +114,7 @@ func (s *Snapshotter) Load() (*raftpb.Snapshot, error) {
}
func loadSnap(dir, name string) (*raftpb.Snapshot, error) {
fpath := path.Join(dir, name)
fpath := filepath.Join(dir, name)
snap, err := Read(fpath)
if err != nil {
renameBroken(fpath)

299
vendor/github.com/coreos/etcd/test generated vendored
View File

@ -1,299 +0,0 @@
#!/usr/bin/env bash
#
# Run all etcd tests
# ./test
# ./test -v
#
# Run tests for one package
#
# PKG=./wal ./test
# PKG=snap ./test
#
# Run code coverage
# COVERDIR=coverage PASSES=cov ./test
set -e
source ./build
# build tests with vendored dependencies
etcd_setup_gopath
if [ -z "$PASSES" ]; then
PASSES="fmt dep compile build unit"
fi
# Invoke ./cover for HTML output
COVER=${COVER:-"-cover"}
# Hack: gofmt ./ will recursively check the .git directory. So use *.go for gofmt.
IGNORE_PKGS="(cmd|vendor|etcdserverpb|rafttest|gopath.proto)"
INTEGRATION_PKGS="(integration|e2e|contrib|functional-tester)"
TEST_PKGS=`find . -name \*_test.go | while read a; do dirname $a; done | sort | uniq | egrep -v "$IGNORE_PKGS" | sed "s|\./||g"`
FORMATTABLE=`find . -name \*.go | while read a; do echo $(dirname $a)/"*.go"; done | sort | uniq | egrep -v "$IGNORE_PKGS" | sed "s|\./||g"`
TESTABLE_AND_FORMATTABLE=`echo "$TEST_PKGS" | egrep -v "$INTEGRATION_PKGS"`
# TODO: 'client' pkg fails with gosimple from generated files
# TODO: 'rafttest' is failing with unused
GOSIMPLE_UNUSED_PATHS=`find . -name \*.go | while read a; do dirname $a; done | sort | uniq | egrep -v "$IGNORE_PKGS" | grep -v 'client'`
if [ -z "$GOARCH" ]; then
GOARCH=$(go env GOARCH);
fi
# user has not provided PKG override
if [ -z "$PKG" ]; then
TEST=$TESTABLE_AND_FORMATTABLE
FMT=$FORMATTABLE
# user has provided PKG override
else
# strip out leading dotslashes and trailing slashes from PKG=./foo/
TEST=${PKG/#./}
TEST=${TEST/#\//}
TEST=${TEST/%\//}
# only run gofmt on packages provided by user
FMT="$TEST"
fi
# split TEST into an array and prepend REPO_PATH to each local package
split=(${TEST// / })
TEST=${split[@]/#/${REPO_PATH}/}
# determine whether target supports race detection
if [ "$GOARCH" == "amd64" ]; then
RACE="--race"
fi
function unit_pass {
echo "Running unit tests..."
# only -run=Test so examples can run in integration tests
go test -timeout 3m ${COVER} ${RACE} -cpu 1,2,4 -run=Test $@ ${TEST}
}
function integration_pass {
echo "Running integration tests..."
go test -timeout 15m -v -cpu 1,2,4 $@ ${REPO_PATH}/integration
go test -timeout 1m -v ${RACE} -cpu 1,2,4 $@ ${REPO_PATH}/client/integration
go test -timeout 10m -v ${RACE} -cpu 1,2,4 $@ ${REPO_PATH}/clientv3/integration
go test -timeout 1m -v -cpu 1,2,4 $@ ${REPO_PATH}/contrib/raftexample
go test -timeout 1m -v ${RACE} -cpu 1,2,4 -run=Example $@ ${TEST}
}
function cov_pass {
echo "Running code coverage..."
# install gocovmerge before running code coverage from github.com/wadey/gocovmerge
# gocovmerge merges coverage files
if ! which gocovmerge >/dev/null; then
echo "gocovmerge not installed"
exit 255
fi
if [ -z "$COVERDIR" ]; then
echo "COVERDIR undeclared"
exit 255
fi
mkdir -p "$COVERDIR"
# PKGS_DELIM contains all the core etcd pkgs delimited by ',' which will be profiled for code coverage.
# Integration tests will generate code coverage for those pkgs
PKGS_DELIM=$(echo $TEST | sed 's/ /,/g')
# TODO create coverage to e2e test
PKGS=`echo "$TEST_PKGS" | egrep -v "(e2e|functional-tester)"`
for t in ${PKGS}; do
tf=`echo $t | tr / _`
# uses -run=Test to skip examples because clientv3/ example tests will leak goroutines
go test -covermode=set -coverpkg $PKGS_DELIM -timeout 15m -run=Test -v -coverprofile "$COVERDIR/${tf}.coverprofile" ${REPO_PATH}/$t
done
gocovmerge "$COVERDIR"/*.coverprofile >"$COVERDIR"/cover.out
}
function e2e_pass {
echo "Running e2e tests..."
go test -timeout 10m -v -cpu 1,2,4 $@ ${REPO_PATH}/e2e
}
function integration_e2e_pass {
echo "Running integration and e2e tests..."
go test -timeout 10m -v -cpu 1,2,4 $@ ${REPO_PATH}/e2e &
e2epid="$!"
go test -timeout 15m -v -cpu 1,2,4 $@ ${REPO_PATH}/integration &
intpid="$!"
wait $e2epid
wait $intpid
go test -timeout 1m -v ${RACE} -cpu 1,2,4 $@ ${REPO_PATH}/client/integration
go test -timeout 10m -v ${RACE} -cpu 1,2,4 $@ ${REPO_PATH}/clientv3/integration
go test -timeout 1m -v -cpu 1,2,4 $@ ${REPO_PATH}/contrib/raftexample
go test -timeout 1m -v ${RACE} -cpu 1,2,4 -run=Example $@ ${TEST}
}
function grpcproxy_pass {
go test -timeout 15m -v ${RACE} -tags cluster_proxy -cpu 1,2,4 $@ ${REPO_PATH}/integration
}
function release_pass {
rm -f ./bin/etcd-last-release
# to grab latest patch release; bump this up for every minor release
UPGRADE_VER=$(git tag -l --sort=-version:refname "v3.0.*" | head -1)
if [ -n "$MANUAL_VER" ]; then
# in case, we need to test against different version
UPGRADE_VER=$MANUAL_VER
fi
local file="etcd-$UPGRADE_VER-linux-$GOARCH.tar.gz"
echo "Downloading $file"
set +e
curl --fail -L https://github.com/coreos/etcd/releases/download/$UPGRADE_VER/$file -o /tmp/$file
local result=$?
set -e
case $result in
0) ;;
22) return 0
;;
*) exit $result
;;
esac
tar xzvf /tmp/$file -C /tmp/ --strip-components=1
mkdir -p ./bin
mv /tmp/etcd ./bin/etcd-last-release
}
function fmt_pass {
toggle_failpoints disable
echo "Checking gofmt..."
fmtRes=$(gofmt -l -s -d $FMT)
if [ -n "${fmtRes}" ]; then
echo -e "gofmt checking failed:\n${fmtRes}"
exit 255
fi
echo "Checking govet..."
vetRes=$(go vet $TEST)
if [ -n "${vetRes}" ]; then
echo -e "govet checking failed:\n${vetRes}"
exit 255
fi
echo "Checking 'go tool vet -shadow'..."
for path in $FMT; do
if [ "${path##*.}" != "go" ]; then
path="${path}/*.go"
fi
vetRes=$(go tool vet -shadow ${path})
if [ -n "${vetRes}" ]; then
echo -e "govet -shadow checking ${path} failed:\n${vetRes}"
exit 255
fi
done
if which goword >/dev/null; then
echo "Checking goword..."
# get all go files to process
gofiles=`find $FMT -iname '*.go' 2>/dev/null`
# ignore tests and protobuf files
gofiles=`echo ${gofiles} | sort | uniq | sed "s/ /\n/g" | egrep -v "(\\_test.go|\\.pb\\.go)"`
# only check for broken exported godocs
gowordRes=`goword -use-spell=false ${gofiles} | grep godoc-export | sort`
if [ ! -z "$gowordRes" ]; then
echo -e "goword checking failed:\n${gowordRes}"
exit 255
fi
else
echo "Skipping goword..."
fi
if which gosimple >/dev/null; then
echo "Checking gosimple..."
for path in $GOSIMPLE_UNUSED_PATHS; do
simplResult=`gosimple ${path} 2>&1 || true`
if [ -n "${simplResult}" ]; then
echo -e "gosimple checking ${path} failed:\n${simplResult}"
exit 255
fi
done
else
echo "Skipping gosimple..."
fi
if which unused >/dev/null; then
echo "Checking unused..."
for path in $GOSIMPLE_UNUSED_PATHS; do
unusedResult=`unused ${path} 2>&1 || true`
if [ -n "${unusedResult}" ]; then
echo -e "unused checking ${path} failed:\n${unusedResult}"
exit 255
fi
done
else
echo "Skipping unused..."
fi
echo "Checking for license header..."
licRes=$(for file in $(find . -type f -iname '*.go' ! -path './cmd/*' ! -path './gopath.proto/*'); do
head -n3 "${file}" | grep -Eq "(Copyright|generated|GENERATED)" || echo -e " ${file}"
done;)
if [ -n "${licRes}" ]; then
echo -e "license header checking failed:\n${licRes}"
exit 255
fi
echo "Checking commit titles..."
git log --oneline `git merge-base HEAD master`...HEAD | while read l; do
commitMsg=`echo "$l" | cut -f2- -d' '`
if [[ "$commitMsg" == Merge* ]]; then
# ignore "Merge pull" commits
continue
fi
if [[ "$commitMsg" == Revert* ]]; then
# ignore revert commits
continue
fi
pkgPrefix=`echo "$commitMsg" | cut -f1 -d':'`
spaceCommas=`echo "$commitMsg" | sed 's/ /\n/g' | grep -c ',$' || echo 0`
commaSpaces=`echo "$commitMsg" | sed 's/,/\n/g' | grep -c '^ ' || echo 0`
if [[ `echo $commitMsg | grep -c ":..*"` == 0 || "$commitMsg" == "$pkgPrefix" || "$spaceCommas" != "$commaSpaces" ]]; then
echo "$l"...
echo "Expected commit title format '<package>{\", \"<package>}: <description>'"
echo "Got: $l"
exit 255
fi
done
}
function dep_pass {
echo "Checking package dependencies..."
# don't pull in etcdserver package
pushd clientv3 >/dev/null
badpkg="(etcdserver|mvcc)"
deps=`go list -f '{{ .Deps }}' | sed 's/ /\n/g' | egrep "${badpkg}" | egrep -v "${badpkg}/" || echo ""`
popd >/dev/null
if [ ! -z "$deps" ]; then
echo -e "clientv3 has masked dependencies:\n${deps}"
exit 255
fi
}
function compile_pass {
echo "Checking build..."
go build -v ./tools/...
}
# fail fast on static tests
function build_pass {
GO_BUILD_FLAGS="-a -v" etcd_build
}
for pass in $PASSES; do
${pass}_pass $@
done
echo "Success"

2
vendor/github.com/coreos/etcd/version/version.go generated vendored
View File

@ -26,7 +26,7 @@ import (
var (
// MinClusterVersion is the min cluster version this etcd binary is compatible with.
MinClusterVersion = "3.0.0"
Version = "3.1.0"
Version = "3.3.13"
APIVersion = "unknown"
// Git SHA Value will be set during build

9
vendor/github.com/coreos/etcd/wal/decoder.go generated vendored
View File

@ -29,6 +29,9 @@ import (
const minSectorSize = 512
// frameSizeBytes is frame size in bytes, including record size and padding size.
const frameSizeBytes = 8
type decoder struct {
mu sync.Mutex
brs []*bufio.Reader
@ -104,7 +107,7 @@ func (d *decoder) decodeRecord(rec *walpb.Record) error {
}
}
// record decoded as valid; point last valid offset to end of record
d.lastValidOff += recBytes + padBytes + 8
d.lastValidOff += frameSizeBytes + recBytes + padBytes
return nil
}
@ -116,7 +119,7 @@ func decodeFrameSize(lenField int64) (recBytes int64, padBytes int64) {
// padding is stored in lower 3 bits of length MSB
padBytes = int64((uint64(lenField) >> 56) & 0x7)
}
return
return recBytes, padBytes
}
// isTornEntry determines whether the last entry of the WAL was partially written
@ -126,7 +129,7 @@ func (d *decoder) isTornEntry(data []byte) bool {
return false
}
fileOff := d.lastValidOff + 8
fileOff := d.lastValidOff + frameSizeBytes
curOff := 0
chunks := [][]byte{}
// split data on sector boundaries

4
vendor/github.com/coreos/etcd/wal/encoder.go generated vendored
View File

@ -52,7 +52,7 @@ func newEncoder(w io.Writer, prevCrc uint32, pageOffset int) *encoder {
// newFileEncoder creates a new encoder with current file offset for the page writer.
func newFileEncoder(f *os.File, prevCrc uint32) (*encoder, error) {
offset, err := f.Seek(0, os.SEEK_CUR)
offset, err := f.Seek(0, io.SeekCurrent)
if err != nil {
return nil, err
}
@ -103,7 +103,7 @@ func encodeFrameSize(dataBytes int) (lenField uint64, padBytes int) {
if padBytes != 0 {
lenField |= uint64(0x80|padBytes) << 56
}
return
return lenField, padBytes
}
func (e *encoder) flush() error {

6
vendor/github.com/coreos/etcd/wal/file_pipeline.go generated vendored
View File

@ -17,7 +17,7 @@ package wal
import (
"fmt"
"os"
"path"
"path/filepath"
"github.com/coreos/etcd/pkg/fileutil"
)
@ -55,7 +55,7 @@ func (fp *filePipeline) Open() (f *fileutil.LockedFile, err error) {
case f = <-fp.filec:
case err = <-fp.errc:
}
return
return f, err
}
func (fp *filePipeline) Close() error {
@ -65,7 +65,7 @@ func (fp *filePipeline) Close() error {
func (fp *filePipeline) alloc() (f *fileutil.LockedFile, err error) {
// count % 2 so this file isn't the same as the one last published
fpath := path.Join(fp.dir, fmt.Sprintf("%d.tmp", fp.count%2))
fpath := filepath.Join(fp.dir, fmt.Sprintf("%d.tmp", fp.count%2))
if f, err = fileutil.LockFile(fpath, os.O_CREATE|os.O_WRONLY, fileutil.PrivateFileMode); err != nil {
return nil, err
}

6
vendor/github.com/coreos/etcd/wal/repair.go generated vendored
View File

@ -17,7 +17,7 @@ package wal
import (
"io"
"os"
"path"
"path/filepath"
"github.com/coreos/etcd/pkg/fileutil"
"github.com/coreos/etcd/wal/walpb"
@ -62,7 +62,7 @@ func Repair(dirpath string) bool {
}
defer bf.Close()
if _, err = f.Seek(0, os.SEEK_SET); err != nil {
if _, err = f.Seek(0, io.SeekStart); err != nil {
plog.Errorf("could not repair %v, failed to read file", f.Name())
return false
}
@ -94,6 +94,6 @@ func openLast(dirpath string) (*fileutil.LockedFile, error) {
if err != nil {
return nil, err
}
last := path.Join(dirpath, names[len(names)-1])
last := filepath.Join(dirpath, names[len(names)-1])
return fileutil.LockFile(last, os.O_RDWR, fileutil.PrivateFileMode)
}

249
vendor/github.com/coreos/etcd/wal/wal.go generated vendored
View File

@ -21,7 +21,7 @@ import (
"hash/crc32"
"io"
"os"
"path"
"path/filepath"
"sync"
"time"
@ -97,7 +97,7 @@ func Create(dirpath string, metadata []byte) (*WAL, error) {
}
// keep temporary wal directory so WAL initialization appears atomic
tmpdirpath := path.Clean(dirpath) + ".tmp"
tmpdirpath := filepath.Clean(dirpath) + ".tmp"
if fileutil.Exist(tmpdirpath) {
if err := os.RemoveAll(tmpdirpath); err != nil {
return nil, err
@ -107,12 +107,12 @@ func Create(dirpath string, metadata []byte) (*WAL, error) {
return nil, err
}
p := path.Join(tmpdirpath, walName(0, 0))
p := filepath.Join(tmpdirpath, walName(0, 0))
f, err := fileutil.LockFile(p, os.O_WRONLY|os.O_CREATE, fileutil.PrivateFileMode)
if err != nil {
return nil, err
}
if _, err = f.Seek(0, os.SEEK_END); err != nil {
if _, err = f.Seek(0, io.SeekEnd); err != nil {
return nil, err
}
if err = fileutil.Preallocate(f.File, SegmentSizeBytes, true); err != nil {
@ -143,7 +143,7 @@ func Create(dirpath string, metadata []byte) (*WAL, error) {
}
// directory was renamed; sync parent dir to persist rename
pdir, perr := fileutil.OpenDir(path.Dir(w.dir))
pdir, perr := fileutil.OpenDir(filepath.Dir(w.dir))
if perr != nil {
return nil, perr
}
@ -157,6 +157,48 @@ func Create(dirpath string, metadata []byte) (*WAL, error) {
return w, nil
}
func (w *WAL) renameWal(tmpdirpath string) (*WAL, error) {
if err := os.RemoveAll(w.dir); err != nil {
return nil, err
}
// On non-Windows platforms, hold the lock while renaming. Releasing
// the lock and trying to reacquire it quickly can be flaky because
// it's possible the process will fork to spawn a process while this is
// happening. The fds are set up as close-on-exec by the Go runtime,
// but there is a window between the fork and the exec where another
// process holds the lock.
if err := os.Rename(tmpdirpath, w.dir); err != nil {
if _, ok := err.(*os.LinkError); ok {
return w.renameWalUnlock(tmpdirpath)
}
return nil, err
}
w.fp = newFilePipeline(w.dir, SegmentSizeBytes)
df, err := fileutil.OpenDir(w.dir)
w.dirFile = df
return w, err
}
func (w *WAL) renameWalUnlock(tmpdirpath string) (*WAL, error) {
// rename of directory with locked files doesn't work on windows/cifs;
// close the WAL to release the locks so the directory can be renamed.
plog.Infof("releasing file lock to rename %q to %q", tmpdirpath, w.dir)
w.Close()
if err := os.Rename(tmpdirpath, w.dir); err != nil {
return nil, err
}
// reopen and relock
newWAL, oerr := Open(w.dir, walpb.Snapshot{})
if oerr != nil {
return nil, oerr
}
if _, _, _, err := newWAL.ReadAll(); err != nil {
newWAL.Close()
return nil, err
}
return newWAL, nil
}
// Open opens the WAL at the given snap.
// The snap SHOULD have been previously saved to the WAL, or the following
// ReadAll will fail.
@ -181,44 +223,16 @@ func OpenForRead(dirpath string, snap walpb.Snapshot) (*WAL, error) {
}
func openAtIndex(dirpath string, snap walpb.Snapshot, write bool) (*WAL, error) {
names, err := readWalNames(dirpath)
names, nameIndex, err := selectWALFiles(dirpath, snap)
if err != nil {
return nil, err
}
nameIndex, ok := searchIndex(names, snap.Index)
if !ok || !isValidSeq(names[nameIndex:]) {
return nil, ErrFileNotFound
rs, ls, closer, err := openWALFiles(dirpath, names, nameIndex, write)
if err != nil {
return nil, err
}
// open the wal files
rcs := make([]io.ReadCloser, 0)
rs := make([]io.Reader, 0)
ls := make([]*fileutil.LockedFile, 0)
for _, name := range names[nameIndex:] {
p := path.Join(dirpath, name)
if write {
l, err := fileutil.TryLockFile(p, os.O_RDWR, fileutil.PrivateFileMode)
if err != nil {
closeAll(rcs...)
return nil, err
}
ls = append(ls, l)
rcs = append(rcs, l)
} else {
rf, err := os.OpenFile(p, os.O_RDONLY, fileutil.PrivateFileMode)
if err != nil {
closeAll(rcs...)
return nil, err
}
ls = append(ls, nil)
rcs = append(rcs, rf)
}
rs = append(rs, rcs[len(rcs)-1])
}
closer := func() error { return closeAll(rcs...) }
// create a WAL ready for reading
w := &WAL{
dir: dirpath,
@ -232,7 +246,7 @@ func openAtIndex(dirpath string, snap walpb.Snapshot, write bool) (*WAL, error)
// write reuses the file descriptors from read; don't close so
// WAL can append without dropping the file lock
w.readClose = nil
if _, _, err := parseWalName(path.Base(w.tail().Name())); err != nil {
if _, _, err := parseWalName(filepath.Base(w.tail().Name())); err != nil {
closer()
return nil, err
}
@ -242,6 +256,52 @@ func openAtIndex(dirpath string, snap walpb.Snapshot, write bool) (*WAL, error)
return w, nil
}
func selectWALFiles(dirpath string, snap walpb.Snapshot) ([]string, int, error) {
names, err := readWalNames(dirpath)
if err != nil {
return nil, -1, err
}
nameIndex, ok := searchIndex(names, snap.Index)
if !ok || !isValidSeq(names[nameIndex:]) {
err = ErrFileNotFound
return nil, -1, err
}
return names, nameIndex, nil
}
func openWALFiles(dirpath string, names []string, nameIndex int, write bool) ([]io.Reader, []*fileutil.LockedFile, func() error, error) {
rcs := make([]io.ReadCloser, 0)
rs := make([]io.Reader, 0)
ls := make([]*fileutil.LockedFile, 0)
for _, name := range names[nameIndex:] {
p := filepath.Join(dirpath, name)
if write {
l, err := fileutil.TryLockFile(p, os.O_RDWR, fileutil.PrivateFileMode)
if err != nil {
closeAll(rcs...)
return nil, nil, nil, err
}
ls = append(ls, l)
rcs = append(rcs, l)
} else {
rf, err := os.OpenFile(p, os.O_RDONLY, fileutil.PrivateFileMode)
if err != nil {
closeAll(rcs...)
return nil, nil, nil, err
}
ls = append(ls, nil)
rcs = append(rcs, rf)
}
rs = append(rs, rcs[len(rcs)-1])
}
closer := func() error { return closeAll(rcs...) }
return rs, ls, closer, nil
}
// ReadAll reads out records of the current WAL.
// If opened in write mode, it must read out all records until EOF. Or an error
// will be returned.
@ -322,7 +382,7 @@ func (w *WAL) ReadAll() (metadata []byte, state raftpb.HardState, ents []raftpb.
// not all, will cause CRC errors on WAL open. Since the records
// were never fully synced to disk in the first place, it's safe
// to zero them out to avoid any CRC errors from new writes.
if _, err = w.tail().Seek(w.decoder.lastOffset(), os.SEEK_SET); err != nil {
if _, err = w.tail().Seek(w.decoder.lastOffset(), io.SeekStart); err != nil {
return nil, state, nil, err
}
if err = fileutil.ZeroToEnd(w.tail().File); err != nil {
@ -356,12 +416,91 @@ func (w *WAL) ReadAll() (metadata []byte, state raftpb.HardState, ents []raftpb.
return metadata, state, ents, err
}
// Verify reads through the given WAL and verifies that it is not corrupted.
// It creates a new decoder to read through the records of the given WAL.
// It does not conflict with any open WAL, but it is recommended not to
// call this function after opening the WAL for writing.
// If it cannot read out the expected snap, it will return ErrSnapshotNotFound.
// If the loaded snap doesn't match with the expected one, it will
// return error ErrSnapshotMismatch.
func Verify(walDir string, snap walpb.Snapshot) error {
var metadata []byte
var err error
var match bool
rec := &walpb.Record{}
names, nameIndex, err := selectWALFiles(walDir, snap)
if err != nil {
return err
}
// open wal files in read mode, so that there is no conflict
// when the same WAL is opened elsewhere in write mode
rs, _, closer, err := openWALFiles(walDir, names, nameIndex, false)
if err != nil {
return err
}
// create a new decoder from the readers on the WAL files
decoder := newDecoder(rs...)
for err = decoder.decode(rec); err == nil; err = decoder.decode(rec) {
switch rec.Type {
case metadataType:
if metadata != nil && !bytes.Equal(metadata, rec.Data) {
return ErrMetadataConflict
}
metadata = rec.Data
case crcType:
crc := decoder.crc.Sum32()
// Current crc of decoder must match the crc of the record.
// We need not match 0 crc, since the decoder is a new one at this point.
if crc != 0 && rec.Validate(crc) != nil {
return ErrCRCMismatch
}
decoder.updateCRC(rec.Crc)
case snapshotType:
var loadedSnap walpb.Snapshot
pbutil.MustUnmarshal(&loadedSnap, rec.Data)
if loadedSnap.Index == snap.Index {
if loadedSnap.Term != snap.Term {
return ErrSnapshotMismatch
}
match = true
}
// We ignore all entry and state type records as these
// are not necessary for validating the WAL contents
case entryType:
case stateType:
default:
return fmt.Errorf("unexpected block type %d", rec.Type)
}
}
if closer != nil {
closer()
}
// We do not have to read out all the WAL entries
// as the decoder is opened in read mode.
if err != io.EOF && err != io.ErrUnexpectedEOF {
return err
}
if !match {
return ErrSnapshotNotFound
}
return nil
}
// cut closes current file written and creates a new one ready to append.
// cut first creates a temp wal file and writes necessary headers into it.
// Then cut atomically rename temp wal file to a wal file.
func (w *WAL) cut() error {
// close old wal file; truncate to avoid wasting space if an early cut
off, serr := w.tail().Seek(0, os.SEEK_CUR)
off, serr := w.tail().Seek(0, io.SeekCurrent)
if serr != nil {
return serr
}
@ -372,7 +511,7 @@ func (w *WAL) cut() error {
return err
}
fpath := path.Join(w.dir, walName(w.seq()+1, w.enti+1))
fpath := filepath.Join(w.dir, walName(w.seq()+1, w.enti+1))
// create a temp wal file with name sequence + 1, or truncate the existing one
newTail, err := w.fp.Open()
@ -401,7 +540,7 @@ func (w *WAL) cut() error {
return err
}
off, err = w.tail().Seek(0, os.SEEK_CUR)
off, err = w.tail().Seek(0, io.SeekCurrent)
if err != nil {
return err
}
@ -413,12 +552,13 @@ func (w *WAL) cut() error {
return err
}
// reopen newTail with its new path so calls to Name() match the wal filename format
newTail.Close()
if newTail, err = fileutil.LockFile(fpath, os.O_WRONLY, fileutil.PrivateFileMode); err != nil {
return err
}
if _, err = newTail.Seek(off, os.SEEK_SET); err != nil {
if _, err = newTail.Seek(off, io.SeekStart); err != nil {
return err
}
@ -460,11 +600,15 @@ func (w *WAL) ReleaseLockTo(index uint64) error {
w.mu.Lock()
defer w.mu.Unlock()
if len(w.locks) == 0 {
return nil
}
var smaller int
found := false
for i, l := range w.locks {
_, lockIndex, err := parseWalName(path.Base(l.Name()))
_, lockIndex, err := parseWalName(filepath.Base(l.Name()))
if err != nil {
return err
}
@ -477,7 +621,7 @@ func (w *WAL) ReleaseLockTo(index uint64) error {
// if no lock index is greater than the release index, we can
// release lock up to the last one(excluding).
if !found && len(w.locks) != 0 {
if !found {
smaller = len(w.locks) - 1
}
@ -552,7 +696,7 @@ func (w *WAL) Save(st raftpb.HardState, ents []raftpb.Entry) error {
return nil
}
mustSync := mustSync(st, w.state, len(ents))
mustSync := raft.MustSync(st, w.state, len(ents))
// TODO(xiangli): no more reference operator
for i := range ents {
@ -564,7 +708,7 @@ func (w *WAL) Save(st raftpb.HardState, ents []raftpb.Entry) error {
return err
}
curOff, err := w.tail().Seek(0, os.SEEK_CUR)
curOff, err := w.tail().Seek(0, io.SeekCurrent)
if err != nil {
return err
}
@ -611,22 +755,13 @@ func (w *WAL) seq() uint64 {
if t == nil {
return 0
}
seq, _, err := parseWalName(path.Base(t.Name()))
seq, _, err := parseWalName(filepath.Base(t.Name()))
if err != nil {
plog.Fatalf("bad wal name %s (%v)", t.Name(), err)
}
return seq
}
func mustSync(st, prevst raftpb.HardState, entsnum int) bool {
// Persistent state on all servers:
// (Updated on stable storage before responding to RPCs)
// currentTerm
// votedFor
// log entries[]
return entsnum != 0 || st.Vote != prevst.Vote || st.Term != prevst.Term
}
func closeAll(rcs ...io.ReadCloser) error {
for _, f := range rcs {
if err := f.Close(); err != nil {

44
vendor/github.com/coreos/etcd/wal/wal_unix.go generated vendored
View File

@ -1,44 +0,0 @@
// Copyright 2016 The etcd Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build !windows
package wal
import (
"os"
"github.com/coreos/etcd/pkg/fileutil"
)
func (w *WAL) renameWal(tmpdirpath string) (*WAL, error) {
// On non-Windows platforms, hold the lock while renaming. Releasing
// the lock and trying to reacquire it quickly can be flaky because
// it's possible the process will fork to spawn a process while this is
// happening. The fds are set up as close-on-exec by the Go runtime,
// but there is a window between the fork and the exec where another
// process holds the lock.
if err := os.RemoveAll(w.dir); err != nil {
return nil, err
}
if err := os.Rename(tmpdirpath, w.dir); err != nil {
return nil, err
}
w.fp = newFilePipeline(w.dir, SegmentSizeBytes)
df, err := fileutil.OpenDir(w.dir)
w.dirFile = df
return w, err
}

Some files were not shown because too many files have changed in this diff Show More