poa-devops/docs/Blockchain-backup.md

Setup blockchain backup on a node

1. connect to the node as root.

2. clone this repository to root's home folder:

git clone https://github.com/poanetwork/poa-devops.git
cd poa-devops

3. make sure python (v2.6+ or v3.5+) is installed on the node.

4. install pip and/or upgrade it to the newest version:

apt-get install python-pip
pip install --upgrade pip

5. upgrade setuptools to the newest version:

pip install --upgrade setuptools

6. install boto and boto3 packages:

pip install boto boto3

7. install ansible:

pip install ansible

8. create group_vars/backup file:

cp group_vars/backup.example group_vars/backup

and set the following variables:

• poa_role - node's role (one of bootnode, validator, moc, explorer, netstat)
• backup_parity_data - true/false - backup whole parity_data folder "as is" or not (default: true)
• backup_parity_blocks - true/false - use or not parity export blocks to create a file with exported blocks and backup it (default: true)
• access_key - s3 access key
• secret_key - s3 secret key
• s3_bucket - s3 bucket name
• node_name - short descriptive name of the node (e.g. sokol-arche). It will be used as part of a backup file name, so it must be lowercase, must not contain spaces, commas, etc

9. create hosts file:

touch hosts

and set it to run backup on localhost:

[backup]
localhost

10. run playbook (still, do this on the node)

ansible-playbook -i hosts -c local site.yml

11. if all is well, setup a cronjob to run every hour:

crontab -e

append the following line:

30 * * * * /bin/bash /root/poa-devops/bkp-blockchain-cron.sh

12. configure logrotate to archive old log files. Create file /etc/cron.hourly/poa-devops-logrotate with the following content:

#!/bin/bash
/usr/sbin/logrotate /root/poa-devops/bkp-blockchain-logrotate.conf

and set permission to run it:

chmod 755 /etc/cron.hourly/poa-devops-logrotate

How to restore from the backup

Restore from parity_data

Must be done under root account

0. place backup file to the user's home directory, e.g. /home/bootnode (or /home/moc, etc)

1. stop netstats (if it's installed) and parity

# may fail if netstats is not installed, ignore it then:
systemctl stop poa-netstats
systemctl stop poa-parity

2. backup your current parity_data

# replace bootnode with other role's name if necessary (e.g. moc, validator, etc)
cd /home/bootnode
mv parity_data parity_data.bkp.$(date -u +%Y%m%d-%H%M%S)

3. unpack parity_data folder from backup

# exact filename will be different
tar -xzf 20180127-134045-bootnode-sokol-archive-parity_data.tar.gz

4. make sure that bootnode (or moc, etc) is owner of the unpacked directory

ls -lh

check owner of the parity_data, if it's another user, run

# replace bootnode with correct user name if necessary (e.g. moc, validator, etc)
chown -R bootnode:bootnode parity_data

5. restart parity and netstats (if it's installed)

systemctl start poa-parity
# unnecessary if netstats is not installed:
systemctl start poa-netstats

6. open NETSTATS_URL in your browser to see if node is up and accepts blocks. If not, explore parity logs in /home/bootnode/logs/parity.log or try to start parity manually to see if it fails at startup:

# assuming you're still in home folder
./parity --config node.toml

Restore from parity_blocks

Parity docs: https://github.com/paritytech/parity/wiki/FAQ:-Backup,-Restore,-and-Files#how-do-i-backup-my-blockchain

Must be done under root account_

0. place backup file to the user's home directory, e.g. /home/bootnode (or /home/moc, etc)

1. stop netstats (if it's installed) and parity

# may fail if netstats is not installed, ignore it then:
systemctl stop poa-netstats
systemctl stop poa-parity

2. backup your current parity_data

# replace bootnode with other role's name if necessary (e.g. moc, validator, etc)
cd /home/bootnode
cp -a parity_data parity_data.bkp.$(date -u +%Y%m%d-%H%M%S)

3. remove current chain

rm -rf parity_data/cache parity_data/chains

4. unpack parity_blocks.rlp file from backup

# exact filename will be different
gunzip 20180127-134045-bootnode-sokol-archive-parity_blocks.rlp.gz

5. import blocks from backup

./parity --config node.toml import 20180127-134045-bootnode-sokol-archive-parity_blocks.rlp

6. make sure that bootnode (or moc, etc) is owner of parity_data and all its subfolders, most importantly newly created parity_data/cache and parity_data/chains

ls -lh
ls -lh parity_data

if owner is another user, run

# replace bootnode with correct user name if necessary (e.g. moc, validator, etc)
chown -R bootnode:bootnode parity_data

7. restart parity and netstats (if it's installed)

systemctl start poa-parity
# unnecessary if netstats is not installed:
systemctl start poa-netstats

8. open NETSTATS_URL in your browser to see if node is up and accepts blocks. If not, explore parity logs in /home/bootnode/logs/parity.log or try to start parity manually to see if it fails at startup:

# assuming you're still in home folder
./parity --config node.toml