Q: Your product and QA teams are bottlenecked by shared staging environments. Merging untested frontend and backend changes to the shared `dev` cluster causes frequent regressions. The engineering lead wants an automated ephemeral environment created for every open pull request (e.g., `https://pr-412.preview.example.com`). The environment must spin up within 3 minutes, deploy full backend microservices with isolated database schemas, and automatically tear down immediately when the PR is merged or closed to eliminate cloud cost waste.
Build fully automated on-demand ephemeral preview environments for every pull request using Argo CD ApplicationSets, PR generators, dynamic DNS wildcards, and automated resource cleanup on PR closure.
Want to master this scenario in a live sandbox? KodeKloud's Enterprise GitOps with ArgoCD & Kubernetes Rollouts covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: Build fully automated on-demand ephemeral preview environments for every pull request using Argo CD ApplicationSets, PR generators, dynamic DNS wildcards, and automated resource cleanup on PR closure.
Configure Argo CD ApplicationSet with Pull Request Generator
Deploy an Argo CD ApplicationSet using the GitHub Pull Request Generator. The generator continuously polls or receives webhooks from the GitHub API, filtering for active PRs with the label preview-env.
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: pr-preview-environments
namespace: argocd
spec:
generators:
- pullRequest:
github:
owner: myorg
repo: e-commerce-app
labels:
- preview-env
tokenRef:
secretName: github-pr-token
key: token
requeueAfterSeconds: 60
template:
metadata:
name: 'preview-pr-{{number}}'
spec:
project: default
source:
repoURL: 'https://github.com/myorg/e-commerce-app.git'
targetRevision: '{{head_sha}}'
path: helm/app
helm:
valuesObject:
ingress:
host: 'pr-{{number}}.preview.example.com'
database:
schema: 'pr_{{number}}'
destination:
server: 'https://kubernetes.default.svc'
namespace: 'preview-pr-{{number}}'
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
Implement Dynamic Wildcard Ingress and Cert-Manager DNS-01
Set up an AWS Route53 or Cloudflare wildcard DNS record *.preview.example.com pointing to the cluster Ingress Controller load balancer. Use cert-manager with Let's Encrypt DNS-01 challenge to issue a single wildcard SSL certificate *.preview.example.com, eliminating per-PR Let's Encrypt rate limits.
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: wildcard-preview-cert
namespace: ingress-nginx
spec:
secretName: wildcard-preview-tls
issuerRef:
name: letsencrypt-route53-dns
kind: ClusterIssuer
dnsNames:
- '*.preview.example.com'
Automate Ephemeral Database Provisioning and Seeding
Add a Helm pre-install hook Job that connects to a dedicated PostgreSQL preview cluster, executes CREATE DATABASE pr_412 TEMPLATE staging_seed;, and runs initial database migrations. Add a post-delete hook that drops the ephemeral database when the Helm release is uninstalled.
Configure Automated Teardown and GitHub PR Commenting
When a developer closes or merges the PR, the Argo CD PR Generator detects the closed PR state and automatically deletes the corresponding Application resource. With cascade deletion enabled (prune: true), Argo CD tears down the namespace, pods, ingress, and PVs immediately.
- W
- e
- e
- l
- i
- m
- i
- n
- a
- t
- e
- d
- s
- t
- a
- g
- i
- n
- g
- b
- o
- t
- t
- l
- e
- n
- e
- c
- k
- s
- b
- y
- i
- m
- p
- l
- e
- m
- e
- n
- t
- i
- n
- g
- G
- i
- t
- O
- p
- s
- -
- d
- r
- i
- v
- e
- n
- p
- r
- e
- v
- i
- e
- w
- e
- n
- v
- i
- r
- o
- n
- m
- e
- n
- t
- s
- .
- U
- s
- i
- n
- g
- A
- r
- g
- o
- C
- D
- A
- p
- p
- l
- i
- c
- a
- t
- i
- o
- n
- S
- e
- t
- s
- w
- i
- t
- h
- t
- h
- e
- P
- R
- g
- e
- n
- e
- r
- a
- t
- o
- r
- ,
- a
- n
- y
- P
- R
- l
- a
- b
- e
- l
- e
- d
- `
- p
- r
- e
- v
- i
- e
- w
- -
- e
- n
- v
- `
- a
- u
- t
- o
- m
- a
- t
- i
- c
- a
- l
- l
- y
- s
- p
- i
- n
- s
- u
- p
- a
- n
- i
- s
- o
- l
- a
- t
- e
- d
- n
- a
- m
- e
- s
- p
- a
- c
- e
- a
- n
- d
- d
- a
- t
- a
- b
- a
- s
- e
- s
- c
- h
- e
- m
- a
- a
- c
- c
- e
- s
- s
- i
- b
- l
- e
- v
- i
- a
- `
- p
- r
- -
- <
- n
- u
- m
- b
- e
- r
- >
- .
- p
- r
- e
- v
- i
- e
- w
- .
- e
- x
- a
- m
- p
- l
- e
- .
- c
- o
- m
- `
- .
- U
- p
- o
- n
- m
- e
- r
- g
- i
- n
- g
- o
- r
- c
- l
- o
- s
- i
- n
- g
- t
- h
- e
- P
- R
- ,
- A
- r
- g
- o
- C
- D
- c
- a
- s
- c
- a
- d
- e
- s
- t
- h
- e
- d
- e
- l
- e
- t
- i
- o
- n
- ,
- e
- n
- s
- u
- r
- i
- n
- g
- z
- e
- r
- o
- i
- d
- l
- e
- c
- l
- o
- u
- d
- s
- p
- e
- n
- d
- .
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
-
Configuring Service selector labels that mismatch underlying Pod manifest metadata when addressing pr-412.preview.example.com
A typo in
spec.selectorcauses the Service endpoints list (kubectl get endpoints) to remain empty, resulting in503 Service Unavailableor timeouts. -
Omitting explicit ingressClassName specifications when running multiple ingress controllers
Without an explicit
ingressClassName, ingress resources are either ignored or picked up unpredictably by competing controllers. -
Depleting worker node subnet IP address allocations under VPC CNI or Azure CNI
Direct VPC pod networking allocates real subnet IPs; failing to monitor IP pools causes new pods to stay stuck indefinitely in
ContainerCreating. - Attempting network triage without inspecting kubectl describe service and EndpointSlice resources Verifying EndpointSlice population immediately determines whether a connectivity failure is caused by pod labeling, readiness probes, or routing.