New extra stage for FAST gitlab setup (#2232)
* new extra stage for gitlab setup * removed wrong link * small fixes README.md --------- Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
This commit is contained in:
simonebruzzechesse
GitHub
parent
a04d63e991
commit
10ae9bc824
|
|
@ -0,0 +1,388 @@
|
|||
# FAST Gitlab repository management
|
||||
|
||||
This small extra stage allows creating and populating Gitlab groups and projects
|
||||
used to host FAST stages code, including rewriting of module sources and secrets
|
||||
used for private modules repository access.
|
||||
|
||||
It is designed for use in a Gitlab self-managed, and is only meant as a one-shot
|
||||
solution with perishable state especially when used for initial population, as
|
||||
you don't want Terraform to keep overwriting your changes with initial versions
|
||||
of files.
|
||||
|
||||
## Gitlab provider credentials
|
||||
|
||||
A [Gitlab token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html)
|
||||
is needed to authenticate against their API. The token needs admin permissions,
|
||||
like shown in this screenshot:
|
||||
|
||||
<p align="center">
|
||||
<img src="gitlab_token.png" alt="Gitlab token scopes.">
|
||||
</p>
|
||||
|
||||
Once a token is available set it in the `gitlab_config.access_token` terraform
|
||||
variable before running Terraform. You might also want to update the default
|
||||
Gitlab hostname and SSH port on the `gitlab_config` section.
|
||||
|
||||
## Variable configuration
|
||||
|
||||
### Modules project and sources
|
||||
|
||||
The `modules_config` variable controls creation and management of both the
|
||||
Gitlab project hosting fast modules and key and secret used to access it, and
|
||||
indirectly control population of initial files: if the `modules_config` variable
|
||||
is not specified no module repository is know to the code, so module source
|
||||
paths cannot be replaced, and initial population of files cannot happen. If the
|
||||
variable is specified, an optional `source_ref` attribute can be set to the
|
||||
reference used to pin modules versions.
|
||||
|
||||
This is an example that configures the modules project in Gitlab in a "shared"
|
||||
group with source ref:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
project_name = "modules"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
group = "shared"
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
If the modules are located in a non modules only repository, use the
|
||||
module_prefix attribute to set the location of your modules within the
|
||||
repository:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
project_name = "modules"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
group = "shared"
|
||||
module_prefix = "modules/"
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
In the above example, no key options are set so it's assumed modules will be
|
||||
fetched from a public repository. If modules repository authentication is needed
|
||||
the `key_config` attribute also needs to be set.
|
||||
|
||||
If no keypair path is specified an internally generated key will be stored as a
|
||||
deploy key in the modules project, and as secrets in the stage repositories:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
project_name = "modules"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
group = "shared"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
To use an existing keypair pass the path to the private key, the public key name
|
||||
is assumed to have the same name ending with the `.pub` suffix. This is useful
|
||||
in cases where the access key has already been set in the modules repository,
|
||||
and new repositories need to be created and their corresponding secret set:
|
||||
|
||||
```hcl
|
||||
modules_config = {
|
||||
project_name = "modules"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
group = "shared"
|
||||
key_config = {
|
||||
create_secrets = true
|
||||
keypair_path = "~/modules-repository-key"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
### Projects
|
||||
|
||||
The `projects` variable is where you configure which projects to create in which
|
||||
groups and whether initial population of files is desired.
|
||||
|
||||
This is an example that creates repositories for stages 00 and 01 and 02:
|
||||
|
||||
```tfvars
|
||||
projects = {
|
||||
fast_00_bootstrap = {
|
||||
create_options = {
|
||||
description = "FAST bootstrap."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/0-bootstrap"
|
||||
workflow_file = "bootstrap-workflow.yaml"
|
||||
}
|
||||
fast_01_resman = {
|
||||
create_options = {
|
||||
description = "FAST resource management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/1-resman"
|
||||
workflow_file = "resman-workflow.yaml"
|
||||
}
|
||||
fast_02_networking = {
|
||||
create_options = {
|
||||
description = "FAST networking management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "net-admins"
|
||||
populate_from = "../../stages/2-networking-a-peering"
|
||||
workflow_file = "networking-workflow.yaml"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
The `create_options` repository attribute controls creation: if the attribute is
|
||||
not present, the repository is assumed to be already existing.
|
||||
|
||||
The group attribute
|
||||
|
||||
Initial population depends on a modules repository being configured in
|
||||
the `modules_config` variable described in the preceding section and on
|
||||
the`populate_from` attributes in each repository where population is required,
|
||||
which point to the folder holding the files to be committed.
|
||||
|
||||
Each repository may contain some sample tfvars and data files that can be used
|
||||
as a starting point for your own files. By default, the samples are not
|
||||
populate. However, you can enable this by setting the `populate_samples`
|
||||
attribute to `true`. Here's an updated example:
|
||||
|
||||
```tfvars
|
||||
projects = {
|
||||
fast_00_bootstrap = {
|
||||
create_options = {
|
||||
description = "FAST bootstrap."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/0-bootstrap"
|
||||
populate_sample = true
|
||||
workflow_file = "bootstrap-workflow.yaml"
|
||||
}
|
||||
fast_01_resman = {
|
||||
create_options = {
|
||||
description = "FAST resource management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/1-resman"
|
||||
populate_sample = true
|
||||
workflow_file = "resman-workflow.yaml"
|
||||
}
|
||||
fast_02_networking = {
|
||||
create_options = {
|
||||
description = "FAST networking management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "net-admins"
|
||||
populate_from = "../../stages/2-networking-a-peering"
|
||||
populate_sample = true
|
||||
workflow_file = "networking-workflow.yaml"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
Please note that setting `populate_samples` to `true` will populate the sample
|
||||
files to the repository, potentially overwriting any existing files with the
|
||||
same name. To minimize the risk of overwriting existing files, we populate the
|
||||
original `data` directory to a `data.sample` directory. In any case, be careful
|
||||
when enabling this option and review commit history to check any changes made to
|
||||
the sample files.
|
||||
|
||||
Each project is created inside a group, this mapping is done via `group`
|
||||
attribute being set on the project. The `group` attribute is either a reference
|
||||
to the key of a group to create in the `groups` variable or the id of an
|
||||
existing group in Gilab. Please find below a full example of projects + groups
|
||||
initialization.
|
||||
|
||||
```tfvars
|
||||
projects = {
|
||||
fast_00_bootstrap = {
|
||||
create_options = {
|
||||
description = "FAST bootstrap."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/0-bootstrap"
|
||||
populate_sample = true
|
||||
workflow_file = "bootstrap-workflow.yaml"
|
||||
}
|
||||
fast_01_resman = {
|
||||
create_options = {
|
||||
description = "FAST resource management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/1-resman"
|
||||
populate_sample = true
|
||||
workflow_file = "resman-workflow.yaml"
|
||||
}
|
||||
fast_02_networking = {
|
||||
create_options = {
|
||||
description = "FAST networking management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "net-admins"
|
||||
populate_from = "../../stages/2-networking-a-peering"
|
||||
populate_sample = true
|
||||
workflow_file = "networking-workflow.yaml"
|
||||
}
|
||||
}
|
||||
|
||||
groups = {
|
||||
org-admins = {
|
||||
name = "gcp-org-admins"
|
||||
path = "gcp-org-admins"
|
||||
description = "GCP Organization administrators"
|
||||
}
|
||||
net-admins = {
|
||||
name = "gcp-net-admins"
|
||||
path = "gcp-net-admins"
|
||||
description = "GCP Network administrators"
|
||||
}
|
||||
shared = {
|
||||
name = "shared"
|
||||
path = "shared"
|
||||
description = "Shared repositories"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
### Groups configuration
|
||||
|
||||
The `groups` variable can be used to create groups in Gitlab, please find below
|
||||
a sample usage of this variable.
|
||||
|
||||
```tfvars
|
||||
groups = {
|
||||
org-admins = {
|
||||
name = "gcp-org-admins"
|
||||
path = "gcp-org-admins"
|
||||
description = "GCP Organization administrators"
|
||||
}
|
||||
net-admins = {
|
||||
name = "gcp-net-admins"
|
||||
path = "gcp-net-admins"
|
||||
description = "GCP Network administrators"
|
||||
}
|
||||
shared = {
|
||||
name = "shared"
|
||||
path = "shared"
|
||||
description = "Shared repositories"
|
||||
}
|
||||
}
|
||||
# tftest skip
|
||||
```
|
||||
|
||||
### Commit configuration
|
||||
|
||||
An optional variable `commit_config` can be used to configure the author, email,
|
||||
and message used in commits for the initial population of files. Its defaults
|
||||
are probably fine for most use cases.
|
||||
|
||||
## How to run this stage:
|
||||
|
||||
Connect to Gitlab as root user and create a personal access token with permissions as per the [gitlab provider credentials](#gitlab-provider-credentials).
|
||||
Set the newly created personal access as `gitlab_config.access_token` variable
|
||||
and then issue the following commands:
|
||||
|
||||
```bash
|
||||
gcloud alpha storage cp gs://${prefix}-prod-iac-core-outputs-0/workflows/*-workflow.yaml ./workflows/
|
||||
```
|
||||
|
||||
This will download Gitlab CICD workflow files generated during 0-bootstrap stage
|
||||
on the local .workflows directory for later being uploaded on the new Gitlab
|
||||
projects.
|
||||
|
||||
Set `http_proxy` and `https_proxy` env vars to http://localhost:3128 and then
|
||||
run:
|
||||
|
||||
```bash
|
||||
terraform init
|
||||
terraform apply
|
||||
```
|
||||
|
||||
Navigate Gitlab URL and check whether new groups and projects has been
|
||||
bootstrapped successfully. Update the default pipeline template replacing the
|
||||
ssh-keyscan default configuration with the following commands:
|
||||
|
||||
```bash
|
||||
ssh-keyscan -p 2222 -H 'gitlab.gcp.example.com' >> ~/.ssh/known_hosts
|
||||
ssh-keyscan -p 2222 gitlab.gcp.example.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts
|
||||
```
|
||||
|
||||
Please specify the right hostname and port for the gitlab executor to fetch the
|
||||
right host key otherwise you might get an _Host key verification failed._ during
|
||||
terraform init.
|
||||
|
||||
Try to create a merge request to trigger a CI pipeline on one of FAST stages and
|
||||
check if the plan pipeline executes successfully.
|
||||
|
||||
|
||||
<!-- TFDOC OPTS files:1 -->
|
||||
<!-- BEGIN TFDOC -->
|
||||
## Files
|
||||
|
||||
| name | description | resources |
|
||||
|---|---|---|
|
||||
| [cicd-versions.tf](./cicd-versions.tf) | Provider version. | |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | <code>gitlab_deploy_key</code> · <code>gitlab_group</code> · <code>gitlab_project</code> · <code>gitlab_project_variable</code> · <code>gitlab_repository_file</code> · <code>tls_private_key</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | |
|
||||
| [providers.tf](./providers.tf) | Provider configuration. | |
|
||||
| [variables.tf](./variables.tf) | Module variables. | |
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [gitlab_config](variables.tf#L28) | Gitlab config. | <code title="object({ access_token = string hostname = optional(string, "gitlab.gcp.example.com") ssh_port = optional(number, 2222) })">object({…})</code> | ✓ | |
|
||||
| [groups](variables.tf#L37) | Gitlab groups. | <code title="map(object({ name = string path = string description = string }))">map(object({…}))</code> | ✓ | |
|
||||
| [commit_config](variables.tf#L17) | Configure commit metadata. | <code title="object({ author = optional(string, "FAST loader") email = optional(string, "fast-loader@fast.gcp.tf") message = optional(string, "FAST initial loading") })">object({…})</code> | | <code>{}</code> |
|
||||
| [modules_config](variables.tf#L46) | Gitlab modules config. | <code title="object({ bootstrap = optional(bool, true) module_prefix = optional(string, "") group = optional(string) project_name = string source_ref = optional(string) key_config = optional(object({ create_key = optional(bool, false) create_secrets = optional(bool, false) keypair_path = optional(string) }), {}) })">object({…})</code> | | <code>null</code> |
|
||||
| [projects](variables.tf#L71) | Gitlab projects to create. | <code title="map(object({ create_options = optional(object({ allow = optional(object({ auto_merge = optional(bool) merge_commit = optional(bool) rebase_merge = optional(bool) squash_merge = optional(bool) })) auto_init = optional(bool) description = optional(string) features = optional(object({ issues = optional(bool) projects = optional(bool) wiki = optional(bool) })) templates = optional(object({ gitignore = optional(string, "Terraform") license = optional(string) repository = optional(object({ name = string owner = string })) }), {}) visibility = optional(string, "private") })) group = string populate_from = optional(string) populate_samples = optional(bool, false) workflow_file = optional(string, null) }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [clone](outputs.tf#L17) | Clone projects commands. | |
|
||||
<!-- END TFDOC -->
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
/**
|
||||
* Copyright 2024 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Provider version.
|
||||
|
||||
terraform {
|
||||
required_providers {
|
||||
gitlab = {
|
||||
source = "gitlabhq/gitlab"
|
||||
version = "~> 16.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 182 KiB |
|
|
@ -0,0 +1,171 @@
|
|||
/**
|
||||
* Copyright 2024 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
_repository_files = flatten([
|
||||
for k, v in var.projects : [
|
||||
for f in concat(
|
||||
[for f in fileset(path.module, "${v.populate_from}/*.svg") : f],
|
||||
[for f in fileset(path.module, "${v.populate_from}/*.md") : f],
|
||||
(v.populate_samples ? [
|
||||
for f in fileset(path.module, "${v.populate_from}/*.sample") : f
|
||||
] : []),
|
||||
(v.populate_samples ? [
|
||||
for f in fileset(path.module, "${v.populate_from}/data/**/*.*") : f
|
||||
] : []),
|
||||
[for f in fileset(path.module, "${v.populate_from}/*.tf") : f],
|
||||
[for f in fileset(path.module, "${v.populate_from}/templates/*.tpl") : f],
|
||||
[for f in fileset(path.module, "${v.populate_from}/terraform.tfvars") : f]
|
||||
) : {
|
||||
project = k
|
||||
file = f
|
||||
name = replace(f, "${v.populate_from}/", "")
|
||||
}
|
||||
] if v.populate_from != null
|
||||
])
|
||||
modules_files = {
|
||||
for f in concat(
|
||||
[for f in fileset(path.module, "../../../modules/*/*.svg") : f],
|
||||
[for f in fileset(path.module, "../../../modules/*/*.md") : f],
|
||||
[for f in fileset(path.module, "../../../modules/*/*.tf") : f],
|
||||
[for f in fileset(path.module, "../../../modules/*/*.yaml") : f]
|
||||
) : f => replace(f, "../../../modules/", "")
|
||||
}
|
||||
modules_ref = (
|
||||
try(var.modules_config.source_ref, null) == null
|
||||
? ""
|
||||
: "?ref=${var.modules_config.source_ref}"
|
||||
)
|
||||
modules_project = try(var.modules_config.project_name, null)
|
||||
modules_group = var.modules_config.bootstrap ? gitlab_group.default[var.modules_config.group].path : var.modules_config.group
|
||||
module_prefix = try(var.modules_config.module_prefix, null)
|
||||
projects = {
|
||||
for k, v in var.projects :
|
||||
k => v.create_options == null ? k : gitlab_project.default[k].id
|
||||
}
|
||||
repository_files = merge(
|
||||
{
|
||||
for k in local._repository_files :
|
||||
"${k.project}/${k.name}" => k
|
||||
if !endswith(k.name, ".tf") || (
|
||||
!startswith(k.name, "0") && k.name != "globals.tf"
|
||||
)
|
||||
},
|
||||
{
|
||||
for k, v in var.projects :
|
||||
"${k}/workflows/${v.workflow_file}" => {
|
||||
project = k
|
||||
file = "./workflows/${v.workflow_file}"
|
||||
name = ".gitlab-ci.yml"
|
||||
}
|
||||
if v.workflow_file != null
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
resource "gitlab_group" "default" {
|
||||
for_each = var.groups
|
||||
name = each.value.name
|
||||
path = each.value.path
|
||||
description = each.value.description
|
||||
}
|
||||
|
||||
resource "gitlab_project" "modules" {
|
||||
count = try(var.modules_config.bootstrap, false) ? 1 : 0
|
||||
name = var.modules_config.project_name
|
||||
description = "FAST Shared modules"
|
||||
namespace_id = gitlab_group.default[var.modules_config.group].id
|
||||
}
|
||||
|
||||
resource "gitlab_repository_file" "modules" {
|
||||
for_each = try(var.modules_config.bootstrap, false) ? local.modules_files : {}
|
||||
project = gitlab_project.modules.0.id
|
||||
branch = "main"
|
||||
file_path = "${var.modules_config.module_prefix}${each.value}"
|
||||
content = endswith(each.key, ".png") || endswith(each.key, ".gif") || endswith(each.key, ".svg") ? filebase64(each.key) : base64encode(file(each.key))
|
||||
commit_message = "${var.commit_config.message} (${each.value})"
|
||||
author_name = var.commit_config.author
|
||||
author_email = var.commit_config.email
|
||||
}
|
||||
|
||||
resource "gitlab_project" "default" {
|
||||
for_each = {
|
||||
for k, v in var.projects : k => v if v.create_options != null
|
||||
}
|
||||
name = each.key
|
||||
description = (
|
||||
each.value.create_options.description != null
|
||||
? each.value.create_options.description
|
||||
: "FAST stage ${each.key}."
|
||||
)
|
||||
visibility_level = each.value.create_options.visibility
|
||||
merge_requests_enabled = try(each.value.create_options.features.merge_requests, null)
|
||||
namespace_id = try(gitlab_group.default[each.value.group].id, each.value.group)
|
||||
issues_enabled = try(each.value.create_options.features.issues, null)
|
||||
wiki_enabled = try(each.value.create_options.features.wiki, null)
|
||||
}
|
||||
|
||||
resource "tls_private_key" "default" {
|
||||
algorithm = "ED25519"
|
||||
}
|
||||
|
||||
resource "gitlab_deploy_key" "example" {
|
||||
count = (
|
||||
try(var.modules_config.key_config.create_key, null) == true ? 1 : 0
|
||||
)
|
||||
project = var.modules_config.bootstrap ? gitlab_project.modules.0.id : "${var.modules_config.namespace}/${var.modules_config.project_name}"
|
||||
title = "Modules repository access"
|
||||
key = (
|
||||
try(var.modules_config.key_config.keypair_path, null) == null
|
||||
? tls_private_key.default.public_key_openssh
|
||||
: file(pathexpand("${var.modules_config.key_config.keypair_path}.pub"))
|
||||
)
|
||||
}
|
||||
|
||||
resource "gitlab_project_variable" "default" {
|
||||
for_each = (
|
||||
try(var.modules_config.key_config.create_secrets, null) == true
|
||||
? local.projects
|
||||
: {}
|
||||
)
|
||||
project = local.projects[each.key]
|
||||
key = "CICD_MODULES_KEY"
|
||||
value = (
|
||||
try(var.modules_config.key_config.keypair_path, null) == null
|
||||
? tls_private_key.default.private_key_openssh
|
||||
: file(pathexpand(var.modules_config.key_config.keypair_path))
|
||||
)
|
||||
protected = false
|
||||
}
|
||||
|
||||
resource "gitlab_repository_file" "default" {
|
||||
for_each = local.modules_project == null ? {} : local.repository_files
|
||||
project = local.projects[each.value.project]
|
||||
branch = "main"
|
||||
file_path = each.value.name
|
||||
content = (
|
||||
endswith(each.value.name, ".tf") && local.modules_project != null
|
||||
? base64encode(replace(
|
||||
file(each.value.file),
|
||||
"/source(\\s*)=\\s*\"../../../modules/([^/\"]+)\"/",
|
||||
"source$1= \"git::ssh://git@${var.gitlab_config.hostname}:${var.gitlab_config.ssh_port}/${local.modules_group}/${local.modules_project}.git//${local.module_prefix}$2${local.modules_ref}\"" # "
|
||||
))
|
||||
: endswith(each.value.name, ".png") || endswith(each.value.name, ".gif") || endswith(each.value.name, ".svg") ? filebase64(each.value.file) : base64encode(file(each.value.file))
|
||||
)
|
||||
commit_message = "${var.commit_config.message} (${each.value.name})"
|
||||
author_name = var.commit_config.author
|
||||
author_email = var.commit_config.email
|
||||
}
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
/**
|
||||
* Copyright 2024 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
output "clone" {
|
||||
description = "Clone projects commands."
|
||||
value = {
|
||||
for k, v in var.projects :
|
||||
k => "ssh://git@${var.gitlab_config.hostname}:${var.gitlab_config.ssh_port}/${v.group}/${k}.git"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
/**
|
||||
* Copyright 2024 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
# tfdoc:file:description Provider configuration.
|
||||
|
||||
provider "gitlab" {
|
||||
base_url = "https://${var.gitlab_config.hostname}"
|
||||
token = var.gitlab_config.access_token
|
||||
insecure = true
|
||||
}
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
gitlab_config = {
|
||||
access_token = "{token}"
|
||||
}
|
||||
modules_config = {
|
||||
project_name = "modules"
|
||||
key_config = {
|
||||
create_key = true
|
||||
create_secrets = true
|
||||
}
|
||||
group = "shared"
|
||||
}
|
||||
projects = {
|
||||
fast_00_bootstrap = {
|
||||
create_options = {
|
||||
description = "FAST bootstrap."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/0-bootstrap"
|
||||
populate_samples = true
|
||||
workflow_file = "bootstrap-workflow.yaml"
|
||||
}
|
||||
fast_01_resman = {
|
||||
create_options = {
|
||||
description = "FAST resource management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "org-admins"
|
||||
populate_from = "../../stages/1-resman"
|
||||
populate_samples = true
|
||||
workflow_file = "resman-workflow.yaml"
|
||||
}
|
||||
fast_02_networking = {
|
||||
create_options = {
|
||||
description = "FAST networking management."
|
||||
features = {
|
||||
issues = true
|
||||
}
|
||||
}
|
||||
group = "net-admins"
|
||||
populate_from = "../../stages/2-networking-a-peering"
|
||||
populate_samples = true
|
||||
workflow_file = "networking-workflow.yaml"
|
||||
}
|
||||
}
|
||||
groups = {
|
||||
org-admins = {
|
||||
name = "gcp-org-admins"
|
||||
path = "gcp-org-admins"
|
||||
description = "GCP Organization administrators"
|
||||
}
|
||||
net-admins = {
|
||||
name = "gcp-net-admins"
|
||||
path = "gcp-net-admins"
|
||||
description = "GCP Network administrators"
|
||||
}
|
||||
shared = {
|
||||
name = "shared"
|
||||
path = "shared"
|
||||
description = "Shared repositories"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
/**
|
||||
* Copyright 2024 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
variable "commit_config" {
|
||||
description = "Configure commit metadata."
|
||||
type = object({
|
||||
author = optional(string, "FAST loader")
|
||||
email = optional(string, "fast-loader@fast.gcp.tf")
|
||||
message = optional(string, "FAST initial loading")
|
||||
})
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "gitlab_config" {
|
||||
description = "Gitlab config."
|
||||
type = object({
|
||||
access_token = string
|
||||
hostname = optional(string, "gitlab.gcp.example.com")
|
||||
ssh_port = optional(number, 2222)
|
||||
})
|
||||
}
|
||||
|
||||
variable "groups" {
|
||||
description = "Gitlab groups."
|
||||
type = map(object({
|
||||
name = string
|
||||
path = string
|
||||
description = string
|
||||
}))
|
||||
}
|
||||
|
||||
variable "modules_config" {
|
||||
description = "Gitlab modules config."
|
||||
type = object({
|
||||
bootstrap = optional(bool, true)
|
||||
module_prefix = optional(string, "")
|
||||
group = optional(string)
|
||||
project_name = string
|
||||
source_ref = optional(string)
|
||||
key_config = optional(object({
|
||||
create_key = optional(bool, false)
|
||||
create_secrets = optional(bool, false)
|
||||
keypair_path = optional(string)
|
||||
}), {})
|
||||
})
|
||||
default = null
|
||||
validation {
|
||||
condition = (
|
||||
var.modules_config == null
|
||||
||
|
||||
try(var.modules_config.project_name, null) != null
|
||||
)
|
||||
error_message = "Modules configuration requires a modules repository name."
|
||||
}
|
||||
}
|
||||
|
||||
variable "projects" {
|
||||
description = "Gitlab projects to create."
|
||||
type = map(object({
|
||||
create_options = optional(object({
|
||||
allow = optional(object({
|
||||
auto_merge = optional(bool)
|
||||
merge_commit = optional(bool)
|
||||
rebase_merge = optional(bool)
|
||||
squash_merge = optional(bool)
|
||||
}))
|
||||
auto_init = optional(bool)
|
||||
description = optional(string)
|
||||
features = optional(object({
|
||||
issues = optional(bool)
|
||||
projects = optional(bool)
|
||||
wiki = optional(bool)
|
||||
}))
|
||||
templates = optional(object({
|
||||
gitignore = optional(string, "Terraform")
|
||||
license = optional(string)
|
||||
repository = optional(object({
|
||||
name = string
|
||||
owner = string
|
||||
}))
|
||||
}), {})
|
||||
visibility = optional(string, "private")
|
||||
}))
|
||||
group = string
|
||||
populate_from = optional(string)
|
||||
populate_samples = optional(bool, false)
|
||||
workflow_file = optional(string, null)
|
||||
}))
|
||||
default = {}
|
||||
nullable = true
|
||||
validation {
|
||||
condition = alltrue([
|
||||
for k, v in var.projects :
|
||||
try(regex("^[a-zA-Z0-9-_.]+$", k), null) != null
|
||||
])
|
||||
error_message = "Project names must match '^[a-zA-Z0-9_.]+$'."
|
||||
}
|
||||
}
|
||||
|
|
@ -3,3 +3,4 @@
|
|||
This folder contains additional helper stages for FAST, which can be used to simplify specific operational tasks:
|
||||
|
||||
- [GitHub repository management](./0-cicd-github/)
|
||||
- [Gitlab repository management](./0-cicd-gitlab/)
|
||||
|
|
|
|||
|
|
@ -193,7 +193,7 @@ This stage also implements initial support for two interrelated features
|
|||
|
||||
Workload Identity Federation support allows configuring external providers independently from CI/CD, and offers predefined attributes for a few well known ones (more can be easily added by editing the `identity-providers.tf` file). Once providers have been configured their names are passed to the following stages via interface outputs, and can be leveraged to set up access or impersonation in IAM bindings.
|
||||
|
||||
CI/CD support is fully implemented for GitHub, Gitlab, and Cloud Source Repositories / Cloud Build. For GitHub, we also offer a [separate supporting setup](../../extras/0-cicd-github/) to quickly create / configure repositories.
|
||||
CI/CD support is fully implemented for GitHub, Gitlab, and Cloud Source Repositories / Cloud Build. For GitHub, we also offer a [separate supporting setup](../../extras/0-cicd-github/) to quickly create / configure repositories. The same applies for Gitlab with the [following extra stage](../../extras/0-cicd-gitlab/).
|
||||
|
||||
<!-- TODO: add a general overview of our design -->
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue