Q: How would you set up an automated rollback strategy in Kubernetes for failed deployments?

Replace standard Kubernetes `Deployment` manifests with `Rollout` custom resources. Define structured canary steps (e.g. 10% traffic for 5 minutes, then 30%, then 100%) to restrict the failure blast radius.

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payment-engine
spec:
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: { duration: 3m }
      - analysis:
          templates:
          - templateName: success-rate-metric
      - setWeight: 50
      - pause: { duration: 5m }
      - setWeight: 100

Configure automated metric queries that execute continuously during the rollout. If HTTP 5xx error rate exceeds 1% or p99 latency spikes above 500ms, the analysis job marks the deployment as Failed, halting traffic progression instantly.

apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate-metric
spec:
  metrics:
  - name: success-rate
    interval: 30s
    failureLimit: 2
    successCondition: result[0] >= 0.99
    provider:
      prometheus:
        address: http://prometheus.monitoring:9090
        query: |
          sum(rate(http_requests_total{app="payment-engine",status!~"5.*"}[1m]))
          /
          sum(rate(http_requests_total{app="payment-engine"}[1m]))

When the AnalysisTemplate fails, Argo Rollouts immediately routes 100% of user traffic back to the stable ReplicaSet via Ingress/Service mesh within seconds. Simultaneously, an event webhook notifies GitHub/GitLab to create an automated revert commit on the GitOps repository.

If pods crash immediately (`CrashLoopBackOff` or failed liveness probes), Kubernetes native `progressDeadlineSeconds` aborts rollout progression and notifies alerting channels.