Q: A production application works fine for internal users but fails for external ones with a 403 Forbidden error. How will you isolate and resolve the issue?
Step-by-step diagnostic strategy to isolate why an enterprise banking application succeeds for corporate network users but returns HTTP 403 Forbidden to public external clients.
Want to master this scenario in a live sandbox? KodeKloud's Istio Service Mesh & Advanced Kubernetes Networking Course covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Inspect HTTP Response Headers to Identify the Rejecting Component
Run `curl -v` against the public endpoint. Examine the 'Server' and custom error headers in the 403 response. If the response header contains `Server: Cloudflare`, `Server: awselb/2.0`, or `x-azure-ref`, the request was rejected at the edge proxy, not by the backend pod.
curl -i -k https://api.banking.company.com/v1/accounts
# Response analysis:
# HTTP/2 403
# server: Microsoft-Azure-Application-Gateway/v2
# x-ms-forbidden-reason: WAF-GeoBlockAudit Web Application Firewall (WAF) & Geo-Blocking Rules
Check WAF logs (Azure WAF, AWS WAF, Cloudflare). Banking platforms enforce strict IP whitelisting, Rate-Limiting rules, and Geo-Blocking. External clients may originate from non-whitelisted geographical regions or trigger OWASP Core Rule Set (CRS) false positives due to specific cookie formats or User-Agent strings.
# Query Azure Application Gateway WAF logs via Log Analytics
AzureDiagnostics
| where Category == "ApplicationGatewayFirewallLog"
| where action_s == "Blocked"
| project TimeGenerated, clientIp_s, ruleId_s, message_sCheck Ingress Controller CIDR Whitelisting & Reverse Proxy Headers
Verify whether the Kubernetes Ingress manifest or NGINX configuration contains `nginx.ingress.kubernetes.io/whitelist-source-range`. If external clients traverse an intermediate load balancer that does NOT preserve the client IP via X-Forwarded-For, the Ingress sees the load balancer's private IP (or vice-versa).
kubectl get ingress <ingress-name> -o yaml | grep -i whitelistVerify API Gateway Authentication & OAuth Scope Policies
Internal users may authenticate automatically via intranet Kerberos/NTLM tokens or trusted mTLS certificates, whereas external clients require Bearer JWT tokens from an external Identity Provider (Azure AD / Okta). If the API gateway fails to parse the external token or missing scope, it returns 403 Forbidden.
- Inspect curl response headers to identify whether WAF, Ingress, or the application backend returned the 403.
- Query WAF logs to check for triggered OWASP rules, Geo-blocking, or rate-limiting filters.
- Verify Ingress IP whitelist annotations and X-Forwarded-For client IP preservation.
- Audit API Gateway OAuth2 token scopes to ensure external identity claims have appropriate permissions.