The basic scheduler has shown weakness when handling high load, which
manifests with the occasional deadlock situation. This introduces an
advanced scheduler, which contains additional features in an attempt to
ensure that regardless of the concurrent load no job is ever left in an
inconsistent state, explicitly with outstanding items to read on its
channels.
The Dirk accountmanager was using a local scatter/gather concurrency
method to obtain wallets, however this uses the parallelism of the Vouch
server rather than the Dirk server. This chnages the Dirk
accountmanager to use a configuration value to select the concurrency
level.
This also standardizes the use of process concurency to allow for
hierarchical definition of the value.
`vouch_ready` is a prometheus metric that is `0` if Vouch is not yet
ready to validate, and `1` if it is.
`vouch_release` has a labeled value `version` that contains the release
version of Vouch.
When a chain reorganisation occurs it is possible that this impacts
Vouch's attestations and proposals for the current and next epoch. This
patch listens to the Ethereum 2 events stream for reorganisation
notifications. If it finds one it cancels existing and creates new
duties, as required.
This option allows users to control the maximum amount of time that
Vouch will wait for a block to arrive before starting its attestation
process. Note that this is a spec value, so changes can have a highly
detrimental impact on Vouch's behaviour.
The "best" aggregate attestation strategy obtains aggregate attestations
from all listed nodes, scores them according to their attestation
coverage, and signs and broadcasts the one with the highest coverage.
The "first" aggregate attestation strategy signs and broadcasts the
first aggregate attestation returned from all listed nodes.
This reintroduces prometheus metrics for the account manager module.
The metrics track the validating state of each account, and are found
under the `vouch_accountmanager_accounts_total` metric. The metrics are
differentiated using the `state` label.