poa-network-monitor

Tests for network health checks and monitoring.

• network-test folder contains tests and helper script. Use the command line arguments to detect network name and url. If no command line arguments received, parameters from the toml file will be used.
  unit-test folder contains unit tests for checking missing-round test.
  contracts folder contains abi and address of needed contract
• common folder contains file with configuration information obtained from toml file and dao for working with sqlite database.
• webapp folder contains web server for retrieving test results
• client folder contains React app
• scripts folder contains script for new accounts creating and bash scripts for running tests and monitor. test-runner.sh file contains logic to prevent duplicate test executions using PID files. It takes name of the test as argument and executes bash script for running this test.
• test-result-monitor.js file checks test results via web server and send alert to slack channel. Also uses the command line arguments to detect network name and url
• config-sample.toml is example of file with settings. Needs to be renamed to config.toml and filled with valid settings (as account and password)

Setup

Run Parity nodes

1. Install Parity and obtain spec.json and bootnodes.txt files using these instructions: POA Installation.
2. Clone Github repository:

git clone https://github.com/poanetwork/poa-network-monitor.git

Install dependencies

cd poa-network-monitor
npm install

3.Run parity nodes
For running parity node enable JSONRPC when connecting to POA Network on Parity --jsonrpc-apis all
For running two nodes for the each network it's needed to specify different ports for them.

Example of running Sokol node on ubuntu:

nohup parity --chain /path/to/sokol/spec.json --reserved-peers /path/to/sokol/bootnodes.txt --jsonrpc-apis all --port 30300 --jsonrpc-port 8540 --ws-port 8450 --ui-port 8180 --no-ipc > parity-sokol.log 2>&1 &

url will be http://localhost:8540

For the Core node:

nohup parity --chain /path/to/core/spec.json --reserved-peers /path/to/core/bootnodes.txt --jsonrpc-apis all --port 30301 --jsonrpc-port 8541 --ws-port 8451 --ui-port 8181 --no-ipc > parity-core.log 2>&1 &

url will be http://localhost:8541

Edit the configuration file

Rename config-sample.toml to the config.toml (or copy and rename). Specify urls for requests to Parity, slackWebHookUrl and channel. Webhook can be created as here. Other settings can be changed too, accounts creation is described below.

Create test accounts

For account creating newAccount.js script can be used. It will create encrypted account using specified password and print it's address. For the Sokol:

cd /home/user/poa-network-monitor
node ./scripts/newAccount.js sokol http://localhost:8540 password

Core:

cd /home/user/poa-network-monitor
node ./scripts/newAccount.js core http://localhost:8541 password

For sending txs test (missing-rounds.js)

1. Create 2 accounts (in each network), add POA to the one of them. For the Sokol network test POA can be added here
2. Add created addresses and passwords to the Sending txs test section of the config.toml file. addressFrom.. must be address with POA

For sending txs via public RPC test (txs-public-rpc-test.js) and Infura (txs-infura-test.js)

This tests can't unlock account as it uses remote node, so it creates raw tx and signs with private key. For getting public and private keys it uses keystore file.
1.Create 2 accounts in each network, add POA to the one of them.
2.Add created addresses and passwords to the Sending txs via public RPC test section of the config.toml file.
3.Add path to the keystore of the account with poa (keyStorePath parameter). Keystore file is usually located in the ~/.local/share/io.parity.ethereum/keys/ folder.
Repeat these steps for the txs-infura-test.js as it needs separate accounts. Save accounts parameters in the Sending txs via Infura test section of the config.

Setup scripts for running monitor and tests.

Tests

Bash scripts for running tests and monitor are located in the scripts folder. They can be used for adding to cron.
Example script for running separate test:

#!/bin/sh
cd /home/user/poa-network-monitor; node ./network-test/mining-block-test.js core $coreRpcUrl >> ./logs/txs-core-log 2>&1;
node ./network-test/mining-block-test.js sokol $sokolRpcUrl >> ./logs/txs-sokol-log 2>&1;

Reorgs

Run reorgs test for the each network:

cd /home/user/poa-network-monitor;
nohup node ./network-test/reorgs-check.js core ws://localhost:8451  >> ./logs/reorgs_core.log 2>&1 &

cd /home/user/poa-network-monitor;
nohup node ./network-test/reorgs-check.js sokol ws://localhost:8450  >> ./logs/reorgs_sokol.log 2>&1 &

Test for reorgs runs continuously so it's not needed to add it on cron.

Monitor

When running monitor the time in seconds can be specified for checking last result.
Script for separate run:

#!/bin/sh <br>
cd /home/user/poa-network-monitor; node ./test-result-monitor.js sokol 1800 >> ./logs/monitor-sokol-log 2>&1;
node ./test-result-monitor.js core 1800 >> ./logs/monitor-core-log 2>&1

Scripts for monitor running are located in the scripts folder.

Add scripts to the crontab

Run sudo crontab -e -u user
Crontab example with timeout:

*/10 * * * * cd /home/user/poa-network-monitor; timeout -s 2 8m ./scripts/test-runner.sh missing-rounds-sokol
*/12 * * * * cd /home/user/poa-network-monitor; timeout -s 2 8m ./scripts/test-runner.sh missing-rounds-core
*/16 * * * * cd /home/user/poa-network-monitor; timeout -s 2 8m ./scripts/test-runner.sh mining-reward-sokol
*/18 * * * * cd /home/user/poa-network-monitor; timeout -s 2 8m ./scripts/test-runner.sh mining-reward-core
0,30 * * * * cd /home/user/poa-network-monitor; timeout -s 2 25m ./scripts/test-runner.sh txs-sokol
5,35 * * * * cd /home/user/poa-network-monitor; timeout -s 2 25m ./scripts/test-runner.sh txs-core
*/15 * * * * cd /home/user/poa-network-monitor; timeout -s 2 12m ./scripts/test-runner.sh txs-public-rpc-sokol
*/17 * * * * cd /home/user/poa-network-monitor; timeout -s 2 12m ./scripts/test-runner.sh txs-public-rpc-core
1 * * * * cd /home/user/poa-network-monitor; timeout -s 2 8m ./scripts/test-runner.sh reward-transfer-sokol
2 * * * * cd /home/user/poa-network-monitor; timeout -s 2 8m ./scripts/test-runner.sh reward-transfer-core
*/3 * * * * cd /home/user/poa-network-monitor; timeout -s 2 2m ./scripts/test-runner.sh reward-by-block-core
*/4 * * * * cd /home/user/poa-network-monitor; timeout -s 2 2m ./scripts/test-runner.sh reward-by-block-sokol
*/18 * * * * cd /home/user/poa-network-monitor; timeout -s 2 12m ./scripts/test-runner.sh txs-infura-core
7,37 * * * * cd /home/user/poa-network-monitor; timeout -s 2 15m ./scripts/test-runner.sh monitor-sokol
8,38 * * * * cd /home/user/poa-network-monitor; timeout -s 2 15m ./scripts/test-runner.sh monitor-core

Run web server

cd /home/user/poa-network-monitor;
nohup node ./webapp/index.js >> ./logs/web_server.log 2>&1 &

Run UI

Install dependencies

cd /home/user/poa-network-monitor/client
npm install

Run

nohup npm start &