Our team embarked on the transition to Terraform in the spring. We established our essential code requirements and naming conventions, aligning them with other departments. This process is closely intertwined with our shift to AWS Cloud. Our decision was to create a robust foundation for our infrastructure using the Infrastructure as Code (IaC) approach.
This could have been the introduction to a standard Terraform use article. However, our focus today is on the configuration and settings of our development tools.
While most people utilize Terraform for managing cloud infrastructure on platforms like Amazon, Google, and Azure, this tool offers a broader range of functionality. It allows us to manage third-party services through a substantial selection of providers, both official and community-based. We employ Terraform as a comprehensive tool for managing the infrastructure we work with, including GitLab, SonarQube, and potentially Artifactory in the future. These tools serve as the foundation for team workspace organization.
How We Facilitate New Project Onboarding
Consider a scenario where a new project within the company needs the creation of GitLab subgroups, repositories, and related actions. As we also employ SonarQube for code analysis, we replicate similar actions for a new project on SonarQube, ensuring seamless code quality testing integration for teams. Terraform generates a group for the project’s repositories in GitLab, adhering to our naming conventions. Additionally, we create security groups for communication with Active Directory, which simplifies access to GitLab projects for our developers.
To streamline the integration of these two tools, we’ve automated the process. In the traditional manual approach, users need to add links, integration tokens, or specific tasks to the pipeline. Our approach simplifies this by first creating entities on GitLab and SonarQube and then adding integration tokens to the desired GitLab project.
Once this is completed, the team can commence work with automation at the forefront. We’ve also implemented automatic token rotation, enhancing security by automatically regenerating tokens in case of a compromise. This eliminates key management issues and bolsters security.
Artifactory can be seamlessly integrated into this pipeline, providing a platform for private repositories of various technologies, including containers and Terraform modules. Typically, the outcome of an activity is an artifact, making separate repositories for these artifacts a best practice. When a new project is initiated, Artifactory automatically creates a space for teams to upload their work results.
The Engineer’s Workflow
The Terraform code configuration with SonarQube is illustrated in the provided example. This demonstrates the key advantages of Terraform.
“`
#################################################################################
# SonarQube project configuration
#################################################################################
resource “sonarqube_project” “this” {
name = var.name
project = var.name
visibility = var.project_visability_level
tags = [lower(var.project_tag)]
}
resource “sonarqube_permissions” “this” {
group_name = “temabit/${lower(var.project_tag)}”
project_key = sonarqube_project.this.project
permissions = [“codeviewer”, “user”]
}
#################################################################################
# SonarQube Project Analysis token
#################################################################################
resource “terraform_data” “retention_trigger” {
input = var.project_token_retention
triggers_replace = true
}
resource “sonarqube_user_token” “this” {
name = “${lower(var.project_tag)}-${var.name}-project-token”
type = var.token_access_level
project_key = sonarqube_project.this.project
expiration_date = local.user_token_expires_at
login_name = var.sonarqube_integration_user
lifecycle {
ignore_changes = [
expiration_date
]
replace_triggered_by = [
terraform_data.retention_trigger
]
}
}
#################################################################################
# Gitlab Project SonarQube token
#################################################################################
data “gitlab_project” “this” {
path_with_namespace = var.project_subgroup == null ? “temabit/${var.project_tag}/${var.name}” : “temabit/${var.project_tag}/${var.project_subgroup}/${var.name}”
}
resource “gitlab_project_variable” “sonar_token” {
project = data.gitlab_project.this.id
key = “SONAR_TOKEN”
value = sonarqube_user_token.this.token
masked = true
}
resource “gitlab_project_variable” “sonar_url” {
project = data.gitlab_project.this.id
key = “SONAR_HOST_URL”
value = var.sonarqube_url
masked = false
}
“`
Key Terraform Advantages
One of the significant benefits of our process is the streamlined pipeline. The process is highly efficient. We initiate a ticket, assign it a name, and our engineer configures Terraform, ensuring everything functions correctly. Once the pipeline is approved, we obtain all the necessary settings for working on a new project within minutes. The ticket processing time stands at 30 minutes, although our future plans include full automation of this process.
Why do we follow this approach? It significantly reduces the time engineers spend on request processing and minimizes routine tasks. It allows engineers to concentrate their focus without distractions. Everything is monitored, ensuring visibility into all changes.
Adapting to Terraform as a Team
We essentially began anew with GitLab as our primary tool, embarking on comprehensive automation alongside our experienced team. While our team possessed prior Terraform experience, our approach to using Terraform evolved, broadening our horizons and enhancing our technical expertise. For instance, we explore the potential benefits of the AWS CDK, but that’s a topic for another article.
Our efforts have yielded positive results, reducing the onboarding process from three hours to just 15 minutes. Of course, human factors may come into play, such as an engineer’s absence from the workplace, which may need additional time. However, we are actively exploring solutions to minimize such factors, striving to find the optimal set of tools. The main goal remains the substantial reduction of time spent on technical and routine tasks.
We will continue sharing updates — stay tuned.
Oleksandr Sapozhnikov, Lead of SRE Team, TemaBit Fozzy Group
We’ve said it before, our team is out of this world. And we have some…