zecfoundation
/
cloud-foundation-fabric
mirror of https://github.com/ZcashFoundation/cloud-foundation-fabric.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #303 from rosmo/decentralized-firewall-validator 

Added decentralized firewall rule validator
This commit is contained in:
Taneli Leppä 2021-09-08 13:51:51 +02:00 committed by GitHub
parent 5fee253ab6 ff254facd4
commit b86d696e17
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 553 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
networking/decentralized-firewall/validator/Dockerfile Normal file
View File

@ -0,0 +1,29 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM python:3.9-slim
RUN mkdir /validator
COPY requirements.txt /validator/requirements.txt
RUN pip install -r /validator/requirements.txt
COPY validator.py /validator/validator.py
RUN mkdir /schemas
COPY firewallSchema.yaml /schemas/firewallSchema.yaml
COPY firewallSchemaAutoApprove.yaml /schemas/firewallAutoApprove.yaml
COPY firewallSchemaSettings.yaml /schemas/firewallSchemaSettings.yaml
RUN mkdir /rules
CMD ["/rules/**/*.yaml"]
ENTRYPOINT ["python3", "/validator/validator.py"]

80
networking/decentralized-firewall/validator/README.md Normal file
View File

@ -0,0 +1,80 @@
# Decentralized firewall validator
The decentralized firewall validator is a Python scripts that utilizes [Yamale](https://github.com/23andMe/Yamale) schema
validation library to validate the configured firewall rules.
## Configuring schemas
There are three configuration files:
- [firewallSchema.yaml](firewallSchema.yaml), where the basic validation schema is configured
- [firewallSchemaAutoApprove.yaml](firewallSchemaAutoApprove.yaml), where the a different schema for auto-approval
can be configured (in case more validation is required than what is available in the schema settings)
- [firewallSchemaSettings.yaml](firewallSchemaSettings.yaml), configures list of allowed and approved
source and destination ranges, ports, network tags and service accounts.
## Building the container
You can build the container like this:
```sh
docker build -t eu.gcr.io/YOUR-PROJECT/firewall-validator:latest .
docker push eu.gcr.io/YOUR-PROJECT/firewall-validator:latest
```
## Running the validator
Example:
```sh
docker run -v $(pwd)/firewall:/rules/ -t eu.gcr.io/YOUR-PROJECT/firewall-validator:latest
```
Output is JSON with keys `ok` and `errors` (if any were found).
## Using as a GitHub action
An `action.yml` is provided for this validator to be used as a GitHub action.
Example of being used in a pipeline:
```yaml
- uses: actions/checkout@v2
- name: Get changed files
if: ${{ github.event_name == 'pull_request' }}
id: changed-files
uses: tj-actions/changed-files@v1.1.2
- uses: ./.github/actions/validate-firewall
if: ${{ github.event_name == 'pull_request' }}
id: validation
with:
files: ${{ steps.changed-files.outputs.all_modified_files }}
- uses: actions/github-script@v3
if: ${{ github.event_name == 'pull_request' && steps.validation.outputs.ok != 'true' }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
var comments = [];
var errors = JSON.parse(process.env.ERRORS);
for (const filename in errors) {
var fn = filename.replace('/github/workspace/', '');
comments.push({
path: fn,
body: "```\n" + errors[filename].join("\n") + "\n```\n",
position: 1,
});
}
github.pulls.createReview({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number,
event: "REQUEST_CHANGES",
body: "Firewall rule validation failed.",
comments: comments,
});
core.setFailed("Firewall validation failed");
env:
ERRORS: '${{ steps.validation.outputs.errors }}'
```

44
networking/decentralized-firewall/validator/action.yml Normal file
View File

@ -0,0 +1,44 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
name: 'Validate firewall rules'
description: 'Validate firewall rule YAML files'
inputs:
files:
description: 'Files to scan (supports wildcards)'
required: false
default: '/github/workspace/firewall/**/*.yaml'
mode:
description: 'Mode (validate or approve)'
required: false
default: 'validate'
schema:
description: 'Schema'
required: false
default: '/schemas/firewallSchema.yaml'
outputs:
ok:
description: 'Validation successful'
errors:
description: 'Validation results'
runs:
using: 'docker'
image: 'Dockerfile'
args:
- ${{ inputs.files }}
- "--mode"
- ${{ inputs.mode }}
- "--schema"
- ${{ inputs.schema }}
- "--github"

32
networking/decentralized-firewall/validator/firewallSchema.yaml Normal file
View File

@ -0,0 +1,32 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('rule'), key=str(min=3, max=30))
---
rule:
disabled: bool(required=False)
deny: list(include('trafficSpec'), required=False)
allow: list(include('trafficSpec'), required=False)
direction: enum('ingress', 'INGRESS', 'egress', 'EGRESS')
priority: int(min=1, max=65535, required=False)
destination_ranges: list(netmask(type='destination'), max=256, required=False)
source_ranges: list(netmask(type='source'), max=256, required=False)
source_tags: list(networktag(), max=30, required=False)
target_tags: list(networktag(), max=70, required=False)
source_service_accounts: list(serviceaccount(), max=10, required=False)
target_service_account: list(serviceaccount(), max=10, required=False)
---
trafficSpec:
ports: list(networkports())
protocol: enum('all', 'tcp', 'udp', 'icmp', 'esp', 'ah', 'ipip', 'sctp')

42
networking/decentralized-firewall/validator/firewallSchemaAutoApprove.yaml Normal file
View File

@ -0,0 +1,42 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
map(include('ingress'), include('egress'), key=str(min=3, max=30))
---
ingress:
disabled: bool(required=False)
deny: list(include('trafficSpec'), required=False)
allow: list(include('trafficSpec'), required=False)
direction: enum('ingress', 'INGRESS')
priority: int(min=1, max=65535, required=False)
source_ranges: list(netmask(type='source'), max=256, required=False)
source_tags: list(networktag(), max=30, required=False)
target_tags: list(networktag(), max=70, required=False)
source_service_accounts: list(serviceaccount(), max=10, required=False)
target_service_account: list(serviceaccount(), max=10, required=False)
---
egress:
disabled: bool(required=False)
deny: list(include('trafficSpec'), required=False)
allow: list(include('trafficSpec'), required=False)
direction: enum('egress', 'EGRESS')
priority: int(min=1, max=65535, required=False)
destination_ranges: list(netmask(type='destination'), max=256, required=False)
source_tags: list(networktag(), max=30, required=False)
target_tags: list(networktag(), max=70, required=False)
source_service_accounts: list(serviceaccount(), max=10, required=False)
target_service_account: list(serviceaccount(), max=10, required=False)
---
trafficSpec:
ports: list()
protocol: enum('all', 'tcp', 'udp', 'icmp', 'esp', 'ah', 'ipip', 'sctp')

49
networking/decentralized-firewall/validator/firewallSchemaSettings.yaml Normal file
View File

@ -0,0 +1,49 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
allowedPorts:
- ports: 22 # SSH
approved: false
- ports: 80 # HTTP
approved: true
- ports: 443 # HTTPS
approved: true
- ports: 3306 # MySQL
approved: false
- ports: 8000-8999
approved: true
allowedSourceRanges:
- cidr: 10.0.0.0/8 # Example on-premise range
approved: true
- cidr: 35.191.0.0/16 # Load balancing & health checks
approved: true
- cidr: 130.211.0.0/22 # Load balancing & health checks
approved: false
- cidr: 35.235.240.0/20 # IAP source range
approved: true
allowedDestinationRanges:
- cidr: 10.0.0.0/8
approved: true
- cidr: 0.0.0.0/0
approved: false
allowedNetworkTags:
- tag: '*'
approved: true
allowedServiceAccounts:
- serviceAccount: '*'
approved: true

16
networking/decentralized-firewall/validator/requirements.txt Normal file
View File

@ -0,0 +1,16 @@
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
yamale~=3.0.0
PyYAML~=5.4.0
click~=7.1.0

261
networking/decentralized-firewall/validator/validator.py Normal file
View File

@ -0,0 +1,261 @@
#!/usr/bin/env python3
# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import glob
import ipaddress
import json
import sys
import click
import yaml
import yamale
from fnmatch import fnmatch
from types import SimpleNamespace
from yamale.validators import DefaultValidators, Validator
class Netmask(Validator):
""" Custom netmask validator """
tag = 'netmask'
settings = {}
mode = None
_type = None
def __init__(self, *args, **kwargs):
self._type = kwargs.pop('type', 'source-or-dest')
super().__init__(*args, **kwargs)
def fail(self, value):
dir_str = 'source or destination'
mode_str = 'allowed'
if self._type == 'source':
dir_str = 'source'
elif self._type == 'destination':
dir_str = 'destination'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s %s network.' % (value, mode_str, dir_str)
def _is_valid(self, value):
is_ok = False
network = ipaddress.ip_network(value)
if self._type == 'source' or self._type == 'source-or-dest':
for ip_range in self.settings['allowedSourceRanges']:
allowed_network = ipaddress.ip_network(ip_range['cidr'])
if network.subnet_of(allowed_network):
if self.mode != 'approve' or ip_range['approved']:
is_ok = True
break
if self._type == 'destination' or self._type == 'source-or-dest':
for ip_range in self.settings['allowedDestinationRanges']:
allowed_network = ipaddress.ip_network(ip_range['cidr'])
if network.subnet_of(allowed_network):
if self.mode != 'approve' or ip_range['approved']:
is_ok = True
break
return is_ok
class NetworkTag(Validator):
""" Custom network tag validator """
tag = 'networktag'
settings = {}
mode = None
_type = None
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def fail(self, value):
mode_str = 'allowed'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s network tag.' % (value, mode_str)
def _is_valid(self, value):
is_ok = False
for tag in self.settings['allowedNetworkTags']:
if fnmatch(value, tag['tag']):
if self.mode != 'approve' or tag['approved']:
is_ok = True
break
return is_ok
class ServiceAccount(Validator):
""" Custom service account validator """
tag = 'serviceaccount'
settings = {}
mode = None
_type = None
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def fail(self, value):
mode_str = 'allowed'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s service account.' % (value, mode_str)
def _is_valid(self, value):
is_ok = False
for sa in self.settings['allowedServiceAccounts']:
if fnmatch(value, sa['serviceAccount']):
if self.mode != 'approve' or sa['approved']:
is_ok = True
break
return is_ok
class NetworkPorts(Validator):
""" Custom ports validator """
tag = 'networkports'
settings = {}
mode = None
_type = None
allowed_port_map = []
approved_port_map = []
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
for port in self.settings['allowedPorts']:
ports = self._process_port_definition(port['ports'])
self.allowed_port_map.extend(ports)
if port['approved']:
self.approved_port_map.extend(ports)
def _process_port_definition(self, port_definition):
ports = []
if not isinstance(port_definition, int) and '-' in port_definition:
start, end = port_definition.split('-', 2)
for port in range(int(start), int(end) + 1):
ports.append(int(port))
else:
ports.append(int(port_definition))
return ports
def fail(self, value):
mode_str = 'allowed'
if self.mode == 'approve':
mode_str = 'automatically approved'
return '\'%s\' is not an %s IP port.' % (value, mode_str)
def _is_valid(self, value):
ports = self._process_port_definition(value)
is_ok = True
for port in ports:
if self.mode == 'approve' and port not in self.approved_port_map:
is_ok = False
break
elif port not in self.allowed_port_map:
is_ok = False
break
return is_ok
class FirewallValidator:
schema = None
settings = None
validators = None
def __init__(self, settings, mode):
self.settings = settings
self.validators = DefaultValidators.copy()
Netmask.settings = self.settings
Netmask.mode = mode
self.validators[Netmask.tag] = Netmask
NetworkTag.settings = self.settings
NetworkTag.mode = mode
self.validators[NetworkTag.tag] = NetworkTag
ServiceAccount.settings = self.settings
ServiceAccount.mode = mode
self.validators[ServiceAccount.tag] = ServiceAccount
NetworkPorts.settings = self.settings
NetworkPorts.mode = mode
self.validators[NetworkPorts.tag] = NetworkPorts
def set_schema_from_file(self, schema):
self.schema = yamale.make_schema(path=schema, validators=self.validators)
def set_schema_from_string(self, schema):
self.schema = yamale.make_schema(content=schema, validators=self.validators)
def validate_file(self, file):
print('Validating %s...' % (file), file=sys.stderr)
data = yamale.make_data(file)
yamale.validate(self.schema, data)
@click.command()
@click.argument('files')
@click.option('--schema',
default='/schemas/firewallSchema.yaml',
help='YAML schema file')
@click.option('--settings',
default='/schemas/firewallSchemaSettings.yaml',
help='schema configuration file')
@click.option('--mode',
default='validate',
help='select mode (validate or approve)')
@click.option('--github',
is_flag=True,
default=False,
help='output GitHub action compatible variables')
def main(**kwargs):
args = SimpleNamespace(**kwargs)
files = [args.files]
if '*' in args.files:
files = glob.glob(args.files, recursive=True)
print('Arguments: %s' % (str(sys.argv)), file=sys.stderr)
f = open(args.settings)
settings = yaml.load(f, Loader=yaml.SafeLoader)
firewall_validator = FirewallValidator(settings, args.mode)
firewall_validator.set_schema_from_file(args.schema)
output = {'ok': True, 'errors': {}}
for file in files:
try:
firewall_validator.validate_file(file)
except yamale.yamale_error.YamaleError as e:
if file not in output['errors']:
output['errors'][file] = []
output['ok'] = False
for result in e.results:
for err in result.errors:
output['errors'][file].append(err)
if args.github:
print('::set-output name=ok::%s' % ('true' if output['ok'] else 'false'))
print('::set-output name=errors::%s' % (json.dumps(output['errors'])))
print(json.dumps(output), file=sys.stderr)
else:
print(json.dumps(output))
if not output['ok'] and not args.github:
sys.exit(1)
if __name__ == '__main__':
main()