Q: A sudden Layer 7 HTTP flood (300,000 req/sec) targets your login API on GCP, degrading backend Cloud Run and GKE services. How do you deploy and tune Cloud Armor to block the attack in real-time without rejecting legitimate customers?
Production implementation of Google Cloud Armor security policies, rate-limiting rules, Adaptive Protection ML anomaly detection, and OWASP Top 10 mitigation on Global External HTTP(S) Load Balancers.
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: During a promotional launch, our checkout API experienced a sophisticated Layer 7 HTTP flood from distributed IP addresses that bypassed traditional rate limiters. We leveraged Cloud Armor to neutralize the attack at the Google edge network.
Analyze Edge Telemetry in Cloud Monitoring & Log Explorer
Inspect Cloud Load Balancing access logs to identify request fingerprinting patterns:
- Log Query: Ran
httpRequest.requestUrl=~'/api/v1/auth/login' AND httpRequest.status=504in Cloud Logging. - Fingerprinting: Isolated anomalous User-Agent signatures, JA3 TLS fingerprints, and source ASN concentrations.
- Target Backend: Verified backend service saturation on the GKE Ingress controller.
Enable Cloud Armor Adaptive Protection
Leverage Google's machine learning anomaly detection to auto-generate mitigation rules:
- Enable ML: Executed
gcloud compute security-policies update prod-armor-policy --enable-layer7-ddos-defense. - Alert Review: Evaluated the auto-generated Cloud Armor security policy alert containing the calculated attack signature.
Deploy Threshold Rate-Limiting Rules
Apply strict client-based rate limits on sensitive endpoints using Cloud Armor rules:
- Action: Configured
throttleaction when a client exceeds 50 requests per minute on/api/v1/auth/login. - Ban Action: Added
ban-thresholdto temporarily drop repeat offenders for 10 minutes (returning HTTP 429). - Key Extraction: Keyed by Client IP (
SRC_IP) and enforced Google reCAPTCHA Enterprise challenge for suspicious tokens.
Enforce Pre-Configured WAF Rulesets (OWASP CRS 3.3)
Protect the backend against SQLi, XSS, and Remote Code Execution:
- SQLi Mitigation: Added
evaluatePreconfiguredWaf('sqli-v33-stable', {'sensitivity': 1}). - Geo-Restriction: Blocked high-risk non-operational regions while allowing verified CDN/partner IP blocks.
- Analyzed edge access logs to identify attack URL patterns, TLS signatures, and ASN concentrations.
- Enabled Cloud Armor Adaptive Protection to calculate machine-learning traffic signatures automatically.
- Deployed rate-limiting and client throttling rules returning HTTP 429 for IPs exceeding 50 req/min on sensitive login routes.
- Enforced preconfigured OWASP Core Rulesets for SQL injection and cross-site scripting mitigation.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Defending Global Microservices Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
- Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Defending Global Microservices Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
- Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Defending Global Microservices Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
- Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Defending Global Microservices Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.