GitOps with Kubernetes: Automating Deployments with Git as Single Source of Truth
Implement GitOps workflows for Kubernetes using ArgoCD and Flux, enabling declarative infrastructure and automated continuous deployment.
Cirrosys Team
Author
July 01, 2025
12 min read
gitopskubernetesargocdfluxcontinuous deployment
GitOps transforms how teams deploy and manage Kubernetes applications by using Git as the single source of truth for declarative infrastructure and applications. This approach brings the benefits of version control, code review, and automation to infrastructure management. This guide explores implementing production-ready GitOps workflows.
Understanding GitOps
Core Principles
GitOps is built on four key principles:
# GitOps Principles
gitops_principles:
declarative:
description: 'Entire system described declaratively'
implementation: 'YAML manifests in Git'
benefits: ['Version controlled', 'Reviewable', 'Reproducible']
versioned_and_immutable:
description: 'Desired state stored in Git'
implementation: 'Git commits as immutable snapshots'
benefits: ['Audit trail', 'Easy rollback', 'History tracking']
pulled_automatically:
description: 'Changes pulled and applied automatically'
implementation: 'Operators monitor Git and apply changes'
benefits: ['No manual kubectl', 'Consistent state', 'Self-healing']
continuously_reconciled:
description: 'Actual state continuously reconciled with desired'
implementation: 'Control loops ensure convergence'
benefits: ['Drift detection', 'Auto-correction', 'High availability']
GitOps vs Traditional CI/CD
# Comparing Deployment Approaches
class DeploymentComparison:
def traditional_cicd(self):
"""Traditional push-based CI/CD"""
return {
'trigger': 'CI pipeline pushes changes',
'credentials': 'CI system needs cluster access',
'state_management': 'Scripts and imperative commands',
'rollback': 'Re-run previous pipeline',
'audit': 'CI logs',
'drift_detection': 'Manual or additional tooling',
'security': 'CI system is attack vector'
}
def gitops_approach(self):
"""GitOps pull-based deployment"""
return {
'trigger': 'Git commit triggers reconciliation',
'credentials': 'Operator in cluster pulls changes',
'state_management': 'Declarative Git repository',
'rollback': 'Git revert',
'audit': 'Git history',
'drift_detection': 'Built-in continuous reconciliation',
'security': 'Reduced attack surface'
}
Implementing GitOps with ArgoCD
1. ArgoCD Architecture
Set up ArgoCD for GitOps:
# ArgoCD Installation and Configuration
apiVersion: v1
kind: Namespace
metadata:
name: argocd
---
# Install ArgoCD
# kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
# ArgoCD Server Configuration
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-server-config
namespace: argocd
data:
url: https://argocd.example.com
# OIDC Configuration
oidc.config: |
name: Auth0
issuer: https://auth.example.com/
clientId: argocd
clientSecret: $oidc.auth0.clientSecret
requestedScopes: ["openid", "profile", "email", "groups"]
requestedIDTokenClaims: {"groups": {"essential": true}}
# RBAC Configuration
policy.csv: |
# Admin access for platform team
g, platform-team, role:admin
# Read-only for developers
g, developers, role:readonly
# App-specific permissions
p, role:app-admin, applications, *, */*, allow
p, role:app-admin, repositories, *, *, allow
g, app-admins, role:app-admin
# Repository credentials
repositories: |
- url: https://github.com/company/k8s-configs
passwordSecret:
name: repo-creds
key: password
usernameSecret:
name: repo-creds
key: username
2. Application Definitions
Define applications in ArgoCD:
# ArgoCD Application Definition
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: production-app
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: production
source:
repoURL: https://github.com/company/k8s-configs
targetRevision: main
path: apps/production
# Helm specific
helm:
valueFiles:
- values-production.yaml
parameters:
- name: image.tag
value: '1.2.3'
# Kustomize specific
kustomize:
images:
- app=myapp:1.2.3
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true # Delete resources not in Git
selfHeal: true # Correct drift automatically
allowEmpty: false # Don't sync empty directories
syncOptions:
- CreateNamespace=true
- PruneLast=true # Prune after sync
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
# Health assessment
ignoreDifferences:
- group: apps
kind: Deployment
jsonPointers:
- /spec/replicas # Ignore HPA-managed replicas
# Rev history limit
revisionHistoryLimit: 10
3. Progressive Delivery with ArgoCD
Implement advanced deployment strategies:
# Argo Rollouts for Progressive Delivery
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: production-app
spec:
replicas: 10
strategy:
canary:
# Traffic management
trafficRouting:
istio:
virtualService:
name: production-app-vsvc
routes:
- primary
# Canary steps
steps:
# Step 1: 10% traffic
- setWeight: 10
- pause: { duration: 5m }
# Step 2: Analysis
- analysis:
templates:
- templateName: success-rate
clusterScope: true
args:
- name: service-name
value: production-app
# Step 3: 30% traffic
- setWeight: 30
- pause: { duration: 5m }
# Step 4: 50% traffic with manual approval
- setWeight: 50
- pause: {} # Manual approval required
# Step 5: Full rollout
- setWeight: 100
# Automatic rollback triggers
analysis:
templates:
- templateName: success-rate
startingStep: 2
args:
- name: service-name
value: production-app
# Anti-affinity during canary
antiAffinity:
requiredDuringSchedulingIgnoredDuringExecution: {}
selector:
matchLabels:
app: production-app
template:
metadata:
labels:
app: production-app
spec:
containers:
- name: app
image: myapp:1.2.3
ports:
- containerPort: 8080
---
# Analysis Template
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
spec:
args:
- name: service-name
metrics:
- name: success-rate
interval: 1m
failureLimit: 3
provider:
prometheus:
address: http://prometheus.monitoring:9090
query: |
sum(rate(
http_requests_total{
service="{{args.service-name}}",
status=~"2.."
}[1m]
)) /
sum(rate(
http_requests_total{
service="{{args.service-name}}"
}[1m]
)) * 100
successCondition: result[0] >= 99.0 # 99% success rate
Implementing GitOps with Flux
1. Flux v2 Setup
Bootstrap Flux in your cluster:
#!/bin/bash
# Bootstrap Flux v2
# Install Flux CLI
curl -s https://fluxcd.io/install.sh | sudo bash
# Check prerequisites
flux check --pre
# Bootstrap Flux with GitHub
flux bootstrap github \
--owner=company \
--repository=k8s-fleet \
--branch=main \
--path=./clusters/production \
--personal \
--token-auth
2. Flux Repository Structure
Organize your GitOps repository:
# Repository Structure
k8s-fleet/
├── clusters/
│ ├── production/
│ │ ├── flux-system/ # Flux components
│ │ ├── infrastructure/ # Platform services
│ │ │ ├── sources/ # Helm repositories
│ │ │ ├── nginx/ # Ingress controller
│ │ │ ├── cert-manager/ # Certificate management
│ │ │ └── monitoring/ # Prometheus stack
│ │ └── apps/ # Application workloads
│ │ ├── backend/
│ │ └── frontend/
│ └── staging/
│ └── ... (similar structure)
├── infrastructure/
│ ├── base/ # Base configurations
│ └── overlays/ # Environment overrides
└── apps/
├── base/
└── overlays/
3. Flux GitOps Workflow
Define sources and deployments:
# Git Repository Source
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: k8s-fleet
namespace: flux-system
spec:
interval: 1m
ref:
branch: main
url: https://github.com/company/k8s-fleet
secretRef:
name: github-auth
---
# Helm Repository Source
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
name: bitnami
namespace: flux-system
spec:
interval: 30m
url: https://charts.bitnami.com/bitnami
---
# Kustomization for Infrastructure
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: infrastructure
namespace: flux-system
spec:
interval: 10m
path: ./infrastructure/production
prune: true
sourceRef:
kind: GitRepository
name: k8s-fleet
validation: client
healthChecks:
- apiVersion: apps/v1
kind: Deployment
name: nginx-ingress-controller
namespace: ingress-nginx
patches:
- patch: |
- op: replace
path: /spec/values/controller/replicaCount
value: 3
target:
kind: HelmRelease
name: nginx-ingress
---
# HelmRelease for Application
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
name: production-app
namespace: apps
spec:
interval: 5m
chart:
spec:
chart: ./charts/app
version: '1.2.3'
sourceRef:
kind: GitRepository
name: k8s-fleet
interval: 1m
values:
image:
repository: myapp
tag: '1.2.3'
autoscaling:
enabled: true
minReplicas: 3
maxReplicas: 10
targetCPUUtilizationPercentage: 70
ingress:
enabled: true
className: nginx
hosts:
- host: app.example.com
paths:
- path: /
pathType: Prefix
# Automated upgrades
upgrade:
remediation:
retries: 3
remediateLastFailure: true
# Rollback on failure
rollback:
cleanupOnFail: true
# Post-deployment tests
test:
enable: true
# Dependencies
dependsOn:
- name: infrastructure
namespace: flux-system
4. Multi-Tenancy with Flux
Implement multi-tenant GitOps:
# Tenant Onboarding
apiVersion: v1
kind: Namespace
metadata:
name: tenant-a
labels:
toolkit.fluxcd.io/tenant: tenant-a
---
# Service Account for Tenant
apiVersion: v1
kind: ServiceAccount
metadata:
name: tenant-a
namespace: tenant-a
---
# RBAC for Tenant
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: tenant-a-reconciler
namespace: tenant-a
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tenant-a
namespace: tenant-a
---
# Tenant GitRepository
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
name: tenant-a
namespace: tenant-a
spec:
interval: 1m
ref:
branch: main
url: https://github.com/tenant-a/k8s-config
secretRef:
name: git-auth
---
# Tenant Kustomization
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: tenant-a
namespace: tenant-a
spec:
interval: 5m
path: './deploy'
prune: true
serviceAccountName: tenant-a
sourceRef:
kind: GitRepository
name: tenant-a
validation: server
# Restrict tenant to their namespace
patches:
- target:
group: '*'
version: '*'
kind: '*'
patch: |
- op: add
path: /metadata/namespace
value: tenant-a
Advanced GitOps Patterns
1. Environment Promotion
Automate promotion between environments:
# GitOps Environment Promotion
import git
import yaml
from github import Github
class GitOpsPromoter:
def __init__(self, repo_url, github_token):
self.repo = git.Repo.clone_from(repo_url, '/tmp/gitops-repo')
self.github = Github(github_token)
self.gh_repo = self.github.get_repo('company/k8s-configs')
def promote_application(self, app_name, from_env, to_env):
"""Promote application version between environments"""
# Get current version in source environment
source_file = f"apps/{from_env}/{app_name}/values.yaml"
with open(f"/tmp/gitops-repo/{source_file}", 'r') as f:
source_values = yaml.safe_load(f)
current_version = source_values['image']['tag']
# Update target environment
target_file = f"apps/{to_env}/{app_name}/values.yaml"
target_path = f"/tmp/gitops-repo/{target_file}"
with open(target_path, 'r') as f:
target_values = yaml.safe_load(f)
# Create promotion record
promotion = {
'app': app_name,
'version': current_version,
'from': from_env,
'to': to_env,
'timestamp': datetime.utcnow().isoformat(),
'promoted_by': 'gitops-automation'
}
# Update version
target_values['image']['tag'] = current_version
# Write updated values
with open(target_path, 'w') as f:
yaml.dump(target_values, f, default_flow_style=False)
# Commit and push
self.repo.index.add([target_file])
self.repo.index.commit(
f"Promote {app_name} {current_version} from {from_env} to {to_env}"
)
# Create PR for production promotions
if to_env == 'production':
self._create_promotion_pr(promotion)
else:
# Direct push for non-production
self.repo.remote('origin').push()
return promotion
def _create_promotion_pr(self, promotion):
"""Create PR for production promotion"""
# Create branch
branch_name = f"promote-{promotion['app']}-{promotion['version']}"
self.repo.create_head(branch_name)
self.repo.head.reference = self.repo.heads[branch_name]
# Push branch
self.repo.remote('origin').push(branch_name)
# Create PR
pr = self.gh_repo.create_pull(
title=f"Promote {promotion['app']} to {promotion['version']}",
body=self._generate_pr_body(promotion),
head=branch_name,
base='main'
)
# Add labels and reviewers
pr.add_to_labels('promotion', 'production')
pr.create_review_request(reviewers=['platform-team'])
return pr
2. Secret Management
Integrate secret management with GitOps:
# Sealed Secrets for GitOps
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: app-secrets
namespace: production
spec:
encryptedData:
api-key: AgBvA8JqZzN2R5I1VH6K...encrypted...data...
db-password: AgCYz8JqZzN2R5I1VH6K...encrypted...data...
template:
metadata:
name: app-secrets
namespace: production
type: Opaque
---
# External Secrets Operator
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
name: vault-backend
namespace: production
spec:
provider:
vault:
server: 'https://vault.example.com:8200'
path: 'secret'
version: 'v2'
auth:
kubernetes:
mountPath: 'kubernetes'
role: 'production-app'
serviceAccountRef:
name: 'production-app'
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: app-secrets
namespace: production
spec:
refreshInterval: 15s
secretStoreRef:
name: vault-backend
kind: SecretStore
target:
name: app-secrets
creationPolicy: Owner
data:
- secretKey: api-key
remoteRef:
key: production/app
property: api_key
- secretKey: db-password
remoteRef:
key: production/database
property: password
3. Policy as Code
Enforce policies in GitOps workflows:
# OPA Gatekeeper Policies
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
name: k8srequiredlabels
spec:
crd:
spec:
names:
kind: K8sRequiredLabels
validation:
openAPIV3Schema:
type: object
properties:
labels:
type: array
items:
type: string
targets:
- target: admission.k8s.gatekeeper.sh
rego: |
package k8srequiredlabels
violation[{"msg": msg}] {
required := input.parameters.labels
provided := input.review.object.metadata.labels
missing := required[_]
not provided[missing]
msg := sprintf("Label '%v' is required", [missing])
}
---
# Apply Policy
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sRequiredLabels
metadata:
name: must-have-environment
spec:
match:
kinds:
- apiGroups: ['apps']
kinds: ['Deployment', 'StatefulSet']
namespaces: ['production', 'staging']
parameters:
labels: ['environment', 'team', 'app']
---
# Kyverno Policy
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-pod-security-standards
spec:
validationFailureAction: enforce
background: true
rules:
- name: check-security-context
match:
any:
- resources:
kinds:
- Pod
namespaces:
- production
validate:
message: 'Security context is required for production pods'
pattern:
spec:
securityContext:
runAsNonRoot: true
runAsUser: '>0'
fsGroup: '>0'
containers:
- name: '*'
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
Monitoring GitOps
1. GitOps Metrics
Track GitOps performance:
# GitOps Metrics Collector
from prometheus_client import Counter, Histogram, Gauge
class GitOpsMetrics:
def __init__(self):
# Sync metrics
self.sync_total = Counter(
'gitops_sync_total',
'Total number of sync operations',
['app', 'status']
)
self.sync_duration = Histogram(
'gitops_sync_duration_seconds',
'Time to complete sync',
['app']
)
self.drift_detected = Counter(
'gitops_drift_detected_total',
'Number of times drift was detected',
['app', 'resource']
)
# Repository metrics
self.repo_fetch_duration = Histogram(
'gitops_repo_fetch_duration_seconds',
'Time to fetch from git',
['repo']
)
self.commit_lag = Gauge(
'gitops_commit_lag_seconds',
'Time since last processed commit',
['app']
)
# Health metrics
self.apps_health = Gauge(
'gitops_app_health',
'Application health status',
['app', 'health']
)
def record_sync(self, app_name, duration, status):
"""Record sync operation metrics"""
self.sync_total.labels(
app=app_name,
status=status
).inc()
self.sync_duration.labels(
app=app_name
).observe(duration)
if status == 'drifted':
self.drift_detected.labels(
app=app_name,
resource='unknown'
).inc()
2. GitOps Dashboard
Comprehensive GitOps monitoring:
# Grafana Dashboard for GitOps
apiVersion: v1
kind: ConfigMap
metadata:
name: gitops-dashboard
namespace: monitoring
data:
dashboard.json: |
{
"dashboard": {
"title": "GitOps Operations",
"panels": [
{
"title": "Sync Status",
"targets": [{
"expr": "sum by (app, status) (rate(gitops_sync_total[5m]))"
}]
},
{
"title": "Drift Detection",
"targets": [{
"expr": "sum by (app) (increase(gitops_drift_detected_total[1h]))"
}]
},
{
"title": "Sync Duration",
"targets": [{
"expr": "histogram_quantile(0.95, rate(gitops_sync_duration_seconds_bucket[5m]))"
}]
},
{
"title": "Application Health",
"targets": [{
"expr": "gitops_app_health"
}]
}
]
}
}
Best Practices
1. Repository Structure
# Recommended GitOps Repository Structure
gitops_structure:
separation_of_concerns:
- 'Separate app code from k8s configs'
- 'Environment-specific directories'
- 'Shared base configurations'
security:
- 'Never store secrets in Git'
- 'Use sealed secrets or external secret operators'
- 'Implement RBAC for Git repositories'
automation:
- 'Automated testing of manifests'
- 'Policy validation in CI'
- 'Automated rollback on failures'
practices:
- 'Small, atomic commits'
- 'Meaningful commit messages'
- 'PR reviews for production changes'
- 'Protect main branch'
2. Disaster Recovery
#!/bin/bash
# GitOps Disaster Recovery
# Backup GitOps state
backup_gitops() {
# Export ArgoCD applications
kubectl get applications -n argocd -o yaml > argocd-apps-backup.yaml
# Export Flux resources
flux export all > flux-backup.yaml
# Backup Git repository
git clone --mirror https://github.com/company/k8s-configs k8s-configs-backup
# Store in secure location
aws s3 sync . s3://backup-bucket/gitops-backup/$(date +%Y%m%d)/
}
# Restore GitOps state
restore_gitops() {
# Restore Git repository
git clone s3://backup-bucket/gitops-backup/latest/k8s-configs-backup
# Re-bootstrap Flux
flux bootstrap github \
--owner=company \
--repository=k8s-configs-restored \
--branch=main \
--path=./clusters/production
# Or restore ArgoCD apps
kubectl apply -f argocd-apps-backup.yaml
}
Conclusion
GitOps revolutionizes Kubernetes deployments by bringing the benefits of Git—version control, collaboration, and automation—to infrastructure management. Key benefits include:
- Declarative everything: Infrastructure and applications defined in Git
- Automated synchronization: Continuous deployment without manual intervention
- Enhanced security: Reduced access to clusters, audit trails
- Easy rollbacks: Git revert equals infrastructure rollback
- Improved collaboration: Code review for infrastructure changes
Whether using ArgoCD or Flux, the principles remain the same: Git as the single source of truth, automated reconciliation, and continuous monitoring. Start small with a single application, establish patterns and practices, then scale across your entire infrastructure.
The future of Kubernetes deployment is declarative, automated, and Git-driven. Embrace GitOps today to transform how your team delivers applications.
Share this article