Git ops & Continuous Infrastructure with terra*

Copyright © 2021 HashiCorp
GitOps & Continuous
Infrastructure with Terra*
Haggai Philip Zagury, DevOps Group & Tech lead
September 2021

Focus for today's talk - why GitOps
GitOps
Every Change is Driven by a change in source control

Focus for today's talk - why GitOps
GitOps
Every Change is Driven by a change in source
control
Alexis Richardson coined the term Gitops in
2017. Gitops is the new phase of DevOps
that many organizations are adopting, in
which all the infrastructure will be stored in
Git as code and will be used for continuous
deployment
#1 - The entire system described declaratively.
#2 - The canonical desired system state versioned in Git.
#3 - Approved changes that can be automatically applied to the
system.
------
#4 - Software agents to ensure correctness and alert on
divergence.

My Journey.tf
Using terraform in various use cases, adv, caveats
Working with terraform for the past 5 years (at least)
Love & Hate relationship with State management
- From tiny projects (single main.tf file)

My Journey.tf
- To 10’s of *.tf files
- Have you seen 001_s3.tf, 002_iam.tf …
- Things can break very fast ....

My Journey.tf
- Upgrading to terraform 0.13
- What comes first ? Account / S3 / Dynamodb lock table ?
- Terraform without terraform cloud ?!
(for small teams / startups)

My Journey.tf
- Seamless integration with Consul / Vault (more on that later)
The ones I worked with ;)

My Journey.tf
- Seamless integration with Consul / Vault (more on that later)
- Cloud Native Technologies

My Journey.tf
For a while my life was waiting for these ...

My Journy.tf
Life getting D.R.Yer

Focus for today's talk
GitOps
Every Change is Driven by a change in source
control (git is standard scm hence GitOps)
Continuous Infrastructure &
Operations
Utilizing DevOps best-practices and tools
enabling and entire SDLC based on Git
Operations.

01 GitOps the early days ...
Terraform Cloud

01 GitOps the early days ...
Terraform Cloud || Run from your laptop

CODE EDITOR
dev-mode
#!/bin/bash
__git_check_if_changed() {
git diff --name-only --quiet HEAD^..HEAD -- $1
echo $?
}
export CHANGED_FRONTEND=$(__git_check_if_changed ./frontend)
export CHANGED_BACKEND=$(__git_check_if_changed ./backend)
export CHANGED_INFRA=$(__git_check_if_changed ./infra)

CODE EDITOR
Google codebuild
Base on -> https://cloud.google.com/architecture/managing-infrastructure-as-code
@eavichay

CODE EDITOR
ci-mode
#!/bin/bash
__git_check_if_changed() {
if [ -z $IS_GITHUB ]; then
git diff --name-only --quiet HEAD -- $1
else
git diff --name-only --quiet HEAD^..HEAD -- $1
fi
# git diff --quiet ${github.ref}HEAD -- $1
echo $?
}

CODE EDITOR
ci-mode
If ./terraform files in git changeset
then terraform workspace select $env
tf apply -auto-approve
If ./backend files in git changeset
Docker build + push all backend components
Then terraform workspace select $env
If ./frontend files in git changeset
Upload files to s3 static + invalidation cloudfront
end
```
`

CODE EDITOR
Pseudo
pipeline
If we could freestyle it ...
If ./terraform files in git changeset
then terraform workspace select $env
If ./backend files in git changeset
Docker build + push all backend components
Then terraform workspace select $env
If ./frontend files in git changeset
Upload files to s3 static + invalidation cloudfront
end
`

My Journey.tf
Everything in 1 repo [ fe + be code, terraform iac, helm deployment]

Running Terraform with Github Actions
Ahoy! -> Insurance for Recreational boats
The Continuous Infrastructure:
Terraform Workspace == Protected / Official Branch
● 1 Infrastructure Repository
● 3 environments - 3 branches (main, staging, dev)
● Trigger based on certain directory in changeset
-> quirky but works !
●

CODE EDITOR
ci-mode | brach == workspace
steps:
- name: gcr.io/${PROJECT_ID}/terraform:0.14.7
entrypoint: bash
args:
- '-c'
- |
cd /workspace/infrastructure/terraform/cloud-core;
terraform init;
if [ "${BRANCH_NAME}" == "master" ]; then
terraform workspace select production;
cp /workspace/infrastructure/terraform/production.tfvars /workspace/apply.tfvars;
gcloud container clusters get-credentials production-cluster --zone=us-east1-b
elif [ "${BRANCH_NAME}" == "staging" ]; then
cp /workspace/infrastructure/terraform/staging.tfvars /workspace/apply.tfvars;
gcloud container clusters get-credentials staging-cluster --zone=us-east1-b
terraform workspace select staging;

CODE EDITOR
ci-mode
steps:
- name: gcr.io/${PROJECT_ID}/terraform:0.14.7
entrypoint: bash
args:
...
exit
fi
terraform plan -var-file=/workspace/apply.tfvars -no-color -out=/workspace/tfplan-core;
terraform apply -input=false /workspace/tfplan-core;

My Journey.tf
Github Actions
https://learn.hashicorp.com/tutorials/terraform/github-actions

04 Gitlab Runner + terragrunt || terraform

GitOps -> gitlab-ci.yml pipelines for terra*
Gitlab ci / pipelines

CODE EDITOR
gitlab-ci.yml
image:
name: hashicorp/terraform:1.0.4
entrypoint:
- '/usr/bin/env'
- 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

CODE EDITOR
Vault (secrets Manager) ping
vault ping:
stage: .pre
script:
- echo "Check status of $VAULT_ADDR"
- |
until vault status
do
echo "Vault returned error or sealed"
sleep 5
done
rules:
- if: '$VAULT_ADDR'
when: always

CODE EDITOR
terraform apply -auto-approve
apply:
stage: apply
extends: .secrets
script:
- *install-curl-jq
- *gitlab-tf-backend
- terraform apply -auto-approve
- DYNAMIC_ENVIRONMENT_URL=$(terraform output -no-color env-dynamic-url)
- echo "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" >> deploy.env
dependencies:
- plan production
artifacts:
expire_in: 1 week
name: $CI_COMMIT_REF_SLUG
reports:
dotenv: deploy.env

Git ops & Continuous Infrastructure with terra*

Increase Developers & Operations Productivity
Continuous deployment automation with an integrated feedback control loop speeds up
Mean Time to Deployment -> ship often isn’t just a catchy phrase !
Enhanced Developer Experience
Push code and not containers.
Developers can use familiar tools like Git to manage updates and features to Kubernetes more
rapidly without having to know the internal of Kubernetes. Newly on-boarded developers can get
quickly up to speed and be productive within days instead of months.

Improved Stability
When you use Git workflows to manage your cluster, you automatically gain a convenient audit
log of all cluster changes outside of Kubernetes. An audit trail of who did what, and when to your
cluster can be used to meet SOC 2 compliance and ensure stability.
Higher Reliability
With Git’s capability to revert/rollback and fork, you gain stable and reproducible rollbacks.
Because your entire system is described in Git, you also have a single source of truth from which
to recover after a meltdown, reducing your meantime to recovery (MTTR) from hours to minutes.

Consistency and Standardization
GitOps provides one model for making infrastructure, apps and Kubernetes add-on
changes, you have consistent end-to-end workflows across your entire organization.
Not only are your continuous integration and continuous deployment pipelines all driven by pull
request, but your operations tasks are also fully reproducible through Git.
Stronger Security Guarantees
Git’s strong correctness and security guarantees, backed by the strong cryptography used to
track and manage changes, as well as the ability to sign changes to prove authorship and origin
is key to a secure definition of the desired state of the cluster.

Git ops & Continuous Infrastructure with terra*

Related slideshows

More Related Content

Git ops & Continuous Infrastructure with terra*