cloud-foundation-fabric/blueprints/third-party-solutions/gitlab

Gitlab Blueprint

This blueprint is responsible for provisioning a production ready Gitlab instance on the landing zone infrastructure. The reference architecture of this deployment target 1K users, updates to the current code is required in of HA and/or higher capacity requirements.

The following diagram illustrates the high-level design of created resources, which can be adapted to specific requirements via variables:

Table of contents

• Gitlab Blueprint
  • Table of contents
  • Managed Services for Seamless Operations
    • Object Storage <-> Google Cloud Storage
  • Identity
    • SAML Integration
      • Google Workspace Setup
    • Others Identity Integration
  • Email
    • Sendgrid integration
  • SSL Certificate Configuration
  • Networking and scalability
  • HA
    • Deployment
      • Step 0: Cloning the repository
      • Step 2: Prepare the variables
      • Step 3: Deploy resources
      • Step 4: Use the created resources
  • Reference and useful links
  • Files
  • Variables
  • Outputs

Managed Services for Seamless Operations

This Gitlab installation prioritizes the use of Google Cloud managed services to streamline infrastructure management and optimization. Here's a breakdown of the managed services incorporated:

1. Google Cloud Storage: is a highly scalable and secure object storage service for storing and accessing data in Google Cloud.
   
   
2. Cloud SQL PostgreSQL: Cloud SQL for Postgres is a fully managed database service on Google Cloud Platform. It eliminates database administration tasks, allowing you to focus on your application, while offering high performance, automatic scaling, and secure management of your PostgreSQL databases.
   
   
3. Memorystore: GCP Memorystore offers a fully managed Redis service for in-memory data caching and high-performance data access.

Benefits:

• Reduced Operational Overhead: Google handles infrastructure setup, maintenance, and updates, freeing up your time and resources.
• Enhanced Security: Managed services often benefit from Google's comprehensive security measures and expertise.
• Scalability: Easily adjust resource allocation to meet evolving demands.
• Cost Optimization: Pay for the resources you use, benefiting from Google's infrastructure optimization. Integration: Managed services seamlessly integrate with other GCP services, promoting a cohesive cloud environment. This module embraces managed services to deliver a resilient, scalable, and cost-effective application architecture on Google Cloud.

Object Storage <-> Google Cloud Storage

GitLab supports using an object storage service for holding numerous types of data. It’s recommended over NFS and in general it’s better in larger setups as object storage is typically much more performant, reliable, and scalable.

A single storage connection to Cloud Storage is configured for all object types, which leverages default Google Compute Engine credential (the so called " consolidated form"). A Cloud Storage bucket is bootstrapped for each object type, the table below summarized such a configuration:

Object Type	Description	Cloud Storage Bucket
artifacts	CI artifacts	${prefix}-gitlab-artifacts
external_diffs	Merge request diffs	${prefix}-mr-diffs
uploads	User uploads	${prefix}-gitlab-uploads
lfs	Git Large File Storage objects	${prefix}-gitlab-lfs
packages	Project packages (e.g. PyPI, Maven ..)	${prefix}-gitlab-packages
dependency_proxy	Dependency Proxy	${prefix}-gitlab-dependency-proxy
terraform_state	Terraform state files	${prefix}-gitlab-terraform-state
pages	Pages	${prefix}-gitlab-pages

For more information on Gitlab object storage and Google Cloud Storage integration please refer to the official Gitlab documentation available at the following link.

• PostgreSQL service

Updated postgres configuration to match documentation, created required database in postgres instance.

• Redis

Identity

GitLab integrates with a number of OmniAuth providers as well as external authentication and authorization providers such as Google Secure LDAP and many other providers. At this time this stage can deal with SAML integration for both user authentication and provisioning, in order to setup SAML integration please provide the saml block on gitlab_config variable.

SAML Integration

This section details how configure GitLab to act as a SAML service provider ( SP). This allows GitLab to consume assertions from a SAML identity provider ( IdP), such as Cloud Identity, to authenticate users. Please find instructions below for integration with:

• Google Workspace

Google Workspace Setup

Setup of Google Workspace is documented in the official Gitlab documentation available at the following link which are also reported below for simplicity.

Create a custom SAML webapp following instructions available at the following link, providing these information in the service provider configuration:

Configuration	Typical Value	Cloud Storage Bucket
Name of SAML App	Gitlab	Name of the app
ACS URL	https://<GITLAB_DOMAIN>/users/auth/saml/callback	Assertion Consumer Service URL.
GITLAB_DOMAIN	gitlab.example.com	Your GitLab instance domain.
Entity ID	https://gitlab.example.com	A value unique to your SAML application. Set it to the issuer in your GitLab configuration.
Name ID	EMAIL	Required value. Also known as name_identifier_format.

Then setup the following SAML attribute mappings:

Google Directory attributes	App attributes
Basic information > Email	email
Basic Information > First name	first_name
Basic Information > Last name	last_name

After configuring the Google Workspace SAML application, record the following information:

Value	Description
SSO URL	Setup in gitlab_config.saml.sso_target_url variable
Certificate (download)	Setup in gitlab_config.saml.idp_cert_fingerprint (obtain value with the following command openssl x509 -in <your_certificate.crt> -noout -fingerprint -sha1)

Others Identity Integration

• OpenID Connect OmniAuth
• Google Secure LDAP

Email

Gmail / Workspace

• documentation

Sendgrid integration

Use the Google Cloud Marketplace to sign up for the SendGrid email service. Make a note of your SendGrid SMTP account credentials, which include username, password, and hostname. Your SMTP username and password are the same as what you used to sign up for the service. The SendGrid hostname is smtp.sendgrid.net. Create an API key: Sign in to SendGrid and go to Settings > API Keys.

1. Create an API key.
2. Select the permissions for the key. At a minimum, the key must have Mail send permissions to send email.
3. Click Save to create the key.
4. SendGrid generates a new key. This is the only copy of the key, so make sure that you copy the key and save it for later.

Configure the sendgrid API key in the gitlab_config variable, under mail, sendgrid arguments as per the following example:

gitlab_config = {
  hostname = "gitlab.example.com"
  mail     = {
    sendgrid = {
      api_key = "test"
    }
  }
}

SSL Certificate Configuration

This module provides flexibility in configuring SSL certificates for the server. You have two options:

1. Provide Your Own Certificates: If you have existing SSL certificates, you can place them in the certs folder within the module's directory. The module will automatically detect and use them. File Names: Ensure the files are named ${gitlab_hostname}.crt (for the certificate) and gitlab_hostname.key (for the private key). Although it is not required in this stage it is mandatory to also place inside the certs folder the server CA certificate which is later use to secure HTTPS access from the Gitlab runner. Name of the CA certificate should be: ${gitlab_hostname}.ca.crt
2. Use Automatically Generated Self-Signed Certificates: If you don't provide certificates, the module will generate a self-signed certificate for immediate use. Updating Later: You can replace the self-signed certificate with your own certificates at any time by placing them in the certs folder and re-running Terraform.

Important Notes:

Certificate Validation: Self-signed certificates are not validated by browsers and will trigger warnings. Use them only for development or testing environments.

For more information on how to configure HTTPS on Gitlab please refer to the original Gitlab documentation available at the following link.

Networking and scalability

• Load balancer

HA

• High Availability

Deployment

Step 0: Cloning the repository

If you want to deploy from your Cloud Shell, click on the image below, sign in if required and when the prompt appears, click on “confirm”.

Otherwise, in your console of choice:

git clone https://github.com/GoogleCloudPlatform/cloud-foundation-fabric

Before you deploy the architecture, you will need at least the following information (for more precise configuration see the Variables section):

• The project ID

The VPC host project, VPC and subnets should already exist and the following networking requirements are satisfied:

• configured PSA for Cloud SQL on the VPC
• subnets configured with PGA and Cloud NAT for internet access
• Inbound firewall rule for IAP on port 22
• Inbound firewall rule for TCP ports 80, 443, 2222 from proxy subnet CIDR (gitlab)

Step 2: Prepare the variables

Once you have the required information, head back to your cloned repository. Make sure you’re in the directory of this tutorial (where this README is in).

Configure the Terraform variables in your terraform.tfvars file. See terraform.tfvars.sample as starting point - just copy it to terraform.tfvars and edit the latter. See the variables documentation below.

Step 3: Deploy resources

Initialize your Terraform environment and deploy the resources:

terraform init
terraform apply

Step 4: Use the created resources

Connect to squid-proxy for accessing gitlab instance using the gcloud command available in the ssh_to_bastion terraform output.

terraform output ssh_to_bastion

A gcloud command like the following should be available

gcloud compute ssh squid-vm --project ${project} --zone europe-west8-b -- -L 3128:127.0.0.1:3128 -N -q -f

Set as system proxy ip 127.0.0.1 and port 3128 and connect to Gitlab hostname https://gitlab.gcp.example.com. Use default admin password available in /run/gitlab/config/initial_root_password or reset admin password via the following command on the Docker container:

gitlab-rake “gitlab:password:reset”

Reference and useful links

• Reference architecture up to 1k users
• /etc/gitlab/gitlab.rb template
• /etc/gitlab/gitlab.rb default options

Files

name	description	modules	resources
gitlab.tf	None	compute-vm · iam-service-account · net-lb-int
main.tf	Module-level locals and resources.	project
outputs.tf	Module outputs.
services.tf	None	cloudsql-instance · gcs	google_redis_instance
ssl.tf	None		tls_cert_request · tls_locally_signed_cert · tls_private_key · tls_self_signed_cert
variables.tf	Module variables.

Variables

name	description	type	required	default	producer
gitlab_instance_config	Gitlab Compute Engine instance config.	object({…})	✓
network_config	Shared VPC network configurations to use for Gitlab Runner VM.	object({…})	✓
prefix	Prefix used for resource names.	string	✓
project_id	Project id, references existing project if project_create is null.	string	✓
region	GCP Region.	string	✓
admin_principals	Users, groups and/or service accounts that are assigned roles, in IAM format (group:foo@example.com).	list(string)		[]
cloudsql_config	Cloud SQL Postgres config.	object({…})		{}
gcs_config	GCS for Object Storage config.	object({…})		{}
gitlab_config	Gitlab configuration.	object({…})		{}
project_create	Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format.	object({…})		null
redis_config	Redis Config.	object({…})		{}

Outputs

name	description	sensitive	consumers
gitlab_ilb_ip	Gitlab Internal Load Balancer IP Address.
instance	Gitlab compute engine instance.
postgresql_users	Gitlab postgres user password.	✓
project	GCP project.
ssh_to_gitlab	gcloud command to ssh gitlab instance.
ssl_certs	Gitlab SSL Certificates.	✓

Test

module "test" {
  source = "./fabric/blueprints/third-party-solutions/gitlab"
  gitlab_config = {
    hostname = "gitlab.gcp.example.com"
    mail = {
      sendgrid = {
        api_key = "sample_api_key"
      }
    }
    saml = {
      idp_cert_fingerprint = "67:90:96.....REPLACE_ME"
      sso_target_url       = "https://accounts.google.com/o/saml2/idp?idpid=REPLACE_ME"
    }
  }
  gitlab_instance_config = {
    replica_zone = "europe-west8-c"
    zone         = "europe-west8-b"
    data_disk = {
      replica_zone = "europe-west8-c"
    }
  }
  network_config = {
    host_project      = "host-project"
    network_self_link = "https://www.googleapis.com/compute/v1/projects/prod-net-landing-0/global/networks/prod-landing-0"
    subnet_self_link  = "https://www.googleapis.com/compute/v1/projects/prod-net-landing-0/regions/europe-west1/subnetworks/landing-default-ew1"
  }
  prefix = "prefix"
  project_create = {
    billing_account_id = "1234-ABCD-1234"
    parent             = "folders/1234563"
  }
  project_id = "my-project"
  region     = "europe-west8"
}
# tftest modules=14 resources=50