168 lines
7.5 KiB
Markdown
168 lines
7.5 KiB
Markdown
# Usage
|
|
|
|
## Prerequisites
|
|
|
|
The bootstrap script included in this project expects the AWS CLI, jq, and Terraform to be installed and on the PATH.
|
|
|
|
On macOS, with Homebrew installed, just run: `brew install awscli jq terraform`
|
|
|
|
For other platforms, or if you don't have Homebrew installed, please see the following links:
|
|
|
|
- [jq](https://stedolan.github.io/jq/download/)
|
|
- [awscli](https://docs.aws.amazon.com/cli/latest/userguide/installing.html)
|
|
- [terraform](https://www.terraform.io/intro/getting-started/install.html)
|
|
|
|
You will also need the following information for the installer:
|
|
|
|
- A unique prefix to use for provisioned resources (5 alphanumeric chars or less)
|
|
- A password to use for the RDS database (at least 8 characters long)
|
|
- The name of a IAM key pair to use for EC2 instances, if you provide a name which
|
|
already exists it will be used, otherwise it will be generated for you.
|
|
|
|
## AWS
|
|
|
|
You will need to set up a new AWS account (or subaccount), and then either login
|
|
to that account using the AWS CLI (via `aws configure`) or create a user account
|
|
that you will use for provisioning, and login to that account. The account used
|
|
requires full access to all AWS services, as a wide variety of services are used,
|
|
a mostly complete list is as follows:
|
|
|
|
- VPCs and associated networking resources (subnets, routing tables, etc.)
|
|
- Security Groups
|
|
- EC2
|
|
- S3
|
|
- SSM
|
|
- DynamoDB
|
|
- Route53
|
|
- RDS
|
|
- ElastiCache
|
|
- CodeDeploy
|
|
|
|
Given the large number of services involved, and the unpredictability of which
|
|
specific API calls will be needed during provisioning, it is recommended that
|
|
you provide a user account with full access. You do not need to keep this user
|
|
around (or enabled) except during the initial provisioning, and any subsequent
|
|
runs to update the infrastructure. How you choose to handle this user is up to you.
|
|
|
|
## Usage
|
|
|
|
Once the prerequisites are out of the way, you are ready to spin up your new infrastructure!
|
|
|
|
From the root of the project:
|
|
|
|
```
|
|
$ bin/infra help
|
|
```
|
|
|
|
This will show you the tasks and options available to you with this script.
|
|
|
|
The infra script will request any information it needs to proceed, and then call Terraform to bootstrap the necessary infrastructure
|
|
for its own state management. This state management infra is needed to ensure that Terraforms state is stored in a centralized location,
|
|
so that multiple people can use Terraform on the same infra without stepping on each others toes. Terraform prevents this from happening by
|
|
holding locks (via DynamoDB) against the state data (stored in S3). Generating the S3 bucket and DynamoDB table has to be done using local state
|
|
the first time, but once provisioned, the local state is migrated to S3, and all further invocations of `terraform` will use the state stored in S3.
|
|
|
|
The infra created, at a high level, is as follows:
|
|
|
|
- An SSH keypair (or you can choose to use one which was already created), this is used with any EC2 hosts
|
|
- A VPC containing all of the resources provisioned
|
|
- A public subnet for the app servers, and a private subnet for the database (and Redis for now)
|
|
- An internet gateway to provide internet access for the VPC
|
|
- An ELB which exposes the app server HTTP endpoints to the world
|
|
- A security group to lock down ingress to the app servers to 80/443 + SSH
|
|
- A security group to allow the ELB to talk to the app servers
|
|
- A security group to allow the app servers access to the database
|
|
- An internal DNS zone
|
|
- A DNS record for the database
|
|
- An autoscaling group and launch configuration for each chain
|
|
- A CodeDeploy application and deployment group targeting the corresponding autoscaling groups
|
|
|
|
Each configured chain will receive its own ASG (autoscaling group) and deployment group, when application updates
|
|
are pushed to CodeDeploy, all autoscaling groups will deploy the new version using a blue/green strategy. Currently,
|
|
there is only one EC2 host to run, and the ASG is configured to allow scaling up, but no triggers are set up to actually perform the
|
|
scaling yet. This is something that may come in the future.
|
|
|
|
**IMPORTANT**: This repository's `.gitignore` prevents the storage of several files generated during provisioning, but it is important
|
|
that you keep them around in your own fork, so that subsequent runs of the `infra` script are using the same configuration and state.
|
|
These files are `backend.tfvars`, `main.tfvars`, and the Terraform state directories. If you generated
|
|
a private key for EC2 (the default), then you will also have a `*.privkey** file in your project root, you need to store this securely out of
|
|
band once created, but does not need to be in the repository.
|
|
|
|
## Migration Prompt
|
|
|
|
The installer will prompt during its initial run to ask if you want to migrate
|
|
the Terraform state to S3, this is a necessary step, and is only prompted due to
|
|
a bug in the Terraform CLI, in a future release, this shouldn't occur, but in
|
|
the meantime, you will need to answer yes to this prompt.
|
|
|
|
## Configuring Installer
|
|
|
|
The `infra` script generates config files for storing the values provided for
|
|
future runs. You can provide overrides to this configuration in
|
|
`terraform.tfvars` or any file with the `.tfvars` extension.
|
|
|
|
## Defining Chains/Adding Chains
|
|
|
|
The default of this repo is to build infra for the `sokol` chain, but you may not want that, or want a different set, so you need to
|
|
create/edit `user.tfvars` and add the following configuration:
|
|
|
|
```terraform
|
|
chains = {
|
|
"mychain" = "url/to/endpoint"
|
|
}
|
|
chain_trace_endpoints = {
|
|
"mychain" = "url/to/debug/endpoint/or/the/main/chain/endpoint"
|
|
}
|
|
```
|
|
|
|
This will ensure that those chains are used when provisioning the infrastructure.
|
|
|
|
## Configuration
|
|
|
|
Config is stored in the Systems Manager Parameter Store, each chain has its own set of config values. If you modify one of these values,
|
|
you will need to go and terminate the instances for that chain so that they are reprovisioned with the new configuration.
|
|
|
|
You will need to make sure to import the changes into the Terraform state though, or you run the risk of getting out of sync.
|
|
|
|
## Destroying Provisioned Infrastructure
|
|
|
|
You can use `bin/infra destroy` to remove any generated infrastructure. It is
|
|
important to note though that if you run this script on partially generated
|
|
infrastructure, or if an error occurs during the destroy process, that you may
|
|
need to manually check for, and remove, any resources that were not able to be
|
|
deleted for you. You can use the `bin/infra resources` command to list all ARNs
|
|
that are tagged with the unique prefix you supplied to the installer, but not
|
|
all AWS resources support tags, and so will not be listed. Here's a list of such
|
|
resources I am aware of:
|
|
|
|
- Route53 hosted zone and records
|
|
- ElastiCache/RDS subnet groups
|
|
- CodeDeploy applications
|
|
|
|
If the `destroy` command succeeds, then everything has been removed, and you do
|
|
not have to worry about leftover resources hanging around.
|
|
|
|
## Common Errors and Questions
|
|
|
|
### Error Applying Plan (diffs didn't match)
|
|
|
|
If you see something like the following:
|
|
|
|
```
|
|
Error: Error applying plan:
|
|
|
|
1 error(s) occurred:
|
|
|
|
* module.stack.aws_autoscaling_group.explorer: aws_autoscaling_group.explorer: diffs didn't match during apply. This is a bug with Terraform and should be reported as a GitHub Issue.
|
|
|
|
Please include the following information in your report:
|
|
|
|
Terraform Version: 0.11.7
|
|
Resource ID: aws_autoscaling_group.explorer
|
|
Mismatch reason: attribute mismatch: availability_zones.1252502072
|
|
```
|
|
|
|
This is due to a bug in Terraform, however the fix is to just rerun `bin/infra
|
|
provision` again, and Terraform will pick up where it left off. This does not
|
|
always happen, but this is the current workaround if you see it.
|