Q: AWS app works internally but not internet — how would you troubleshoot?
Outside-in OSI network layer troubleshooting model for resolving private VPC services that remain inaccessible over the public internet.
AWS App Works Internally But Not Internet (60s SRE Runbook)
Watch Naveed break down how to troubleshoot VPC ingress failures—tracing Route 53 public DNS, Internet Gateway routes, public subnet ALB placement, Security Groups, and NACLs in 60 seconds.
- Verify Route 53 public hosted zone DNS resolution vs private zone records
- Check subnet route table has an active 0.0.0.0/0 route to Internet Gateway
- Ensure public-facing ALB is deployed across public subnets with public IPs
- Audit Security Group ingress on 80/443 and verify stateless NACL return rules
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: When an AWS app works internally but not internet, the compute instance and container runtime are verified healthy. The root cause is isolated strictly along the outside-in ingress network boundary between the public internet and the AWS VPC.
DNS & IP Resolution (Outside-in Step 1)
Verify public DNS mapping from an external workstation to ensure client traffic reaches AWS edge endpoints:
# 1. Inspect public DNS resolution against authoritative resolvers
dig +noall +answer app.example.com @8.8.8.8
# Expected stdout: Public ALB IP or CNAME
# app.example.com. 300 IN CNAME dualstack.app-alb-102938475.us-east-1.elb.amazonaws.com.
# dualstack.app-alb-102938475.us-east-1.elb.amazonaws.com. 60 IN A 52.204.142.89
- Execute
dig +short app.example.comandnslookup app.example.comagainst public recursive resolvers (e.g.8.8.8.8or1.1.1.1). - Verify Route 53 Public vs Private Hosted Zones: A common failure occurs when an internal record resolves to a private VPC IP (
10.0.x.x) because only a private hosted zone was configured, leaving the public hosted zone missing or pointing to a stale endpoint. - Confirm the public A/AAAA Alias record correctly maps to the Application Load Balancer canonical DNS name (
dualstack.app-alb-xxxx.us-east-1.elb.amazonaws.com).
Entry Point Topology & Subnet Routing (Step 2)
Audit Internet Gateway (IGW) attachments and subnet routing table associations:
# 2. Verify subnet route table target points to an Internet Gateway
aws ec2 describe-route-tables \n --filters "Name=association.subnet-id,Values=subnet-0123456789abcdef0" \n --query "RouteTables[].Routes[?DestinationCidrBlock=='0.0.0.0/0'].[DestinationCidrBlock,GatewayId]" \n --output text
# Expected clean output (no ASCII formatting):
# 0.0.0.0/0 igw-0123456789abcdef0
- Public Subnet Route Table Verification: The subnets hosting the Application Load Balancer must have an explicit route table entry:
0.0.0.0/0 → igw-xxxx(Internet Gateway). If an ALB is mistakenly deployed into private subnets, inbound packets cannot cross the VPC perimeter. - Load Balancer Scheme Audit: Confirm the ALB scheme is configured as
internet-facingrather thaninternal. An internal ALB only receives private ENI IPs without public routability. - Avoid NAT Gateway Confusion: NAT Gateways only enable outbound egress from private subnets to the internet; they do not accept inbound ingress from the public internet.
Firewalls: Security Groups & Stateless NACLs (Step 3)
Inspect AWS stateful Security Groups and stateless Network Access Control Lists (NACLs) across ingress and egress paths:
- ALB Security Group: Must explicitly allow inbound TCP port 80/443 from
0.0.0.0/0(Public Internet). - Target Security Group Chaining: Backend EC2 or ECS tasks in private subnets must allow ingress from the ALB's Security Group ID (e.g.
sg-alb123), preventing direct unauthorized access while permitting forwarded traffic. - Stateless NACL Ephemeral Return Ports: Network ACLs evaluate inbound and outbound rules independently without connection tracking. The subnet NACL must allow inbound 80/443 AND allow outbound ephemeral traffic on ports
1024–65535to return SYN-ACK packets to client browsers.
AWS WAF & VPC Reachability Analyzer (Step 4)
Inspect perimeter firewall rules and mathematically prove network path connectivity using AWS VPC Reachability Analyzer:
# 4. Mathematically prove network path using AWS VPC Reachability Analyzer
aws ec2 create-network-insights-path \n --source igw-0123456789abcdef0 \n --destination eni-0123456789abcdef0 \n --protocol TCP \n --destination-port 443
# Expected result: PathFound: false (or true once rules are resolved)
# Explanations: Security group sg-0123 does not allow inbound TCP on port 443
- AWS WAF Inspection: Review CloudWatch metrics for
BlockedRequests. Common false positives stem from aggressive IP rate limiting thresholds, geographic restrictions, or overly sensitive AWS Managed Core Rule Set (CRS) SQLi rules. - VPC Reachability Analyzer: Run an automated diagnostic path between the Internet Gateway (
igw-xxxx) and the load balancer or target instance ENI. The tool computes hop-by-hop configuration analysis and pinpoints the exact misconfigured route or security rule without injecting live probe packets. - Cross-reference operational runbooks with our companion guides: inspect the AWS ALB 5XX Troubleshooting Runbook, review the AWS EC2 SSH Access Diagnostics, and explore the AWS DevOps Architecture Guide for enterprise VPC landing zones.
- First, isolate the failure layer: since the app works internally on private VPC IPs, compute and application processes are proven healthy.
- Trace outside-in starting at DNS: run dig app.example.com to verify public DNS resolves to the ALB CNAME rather than a private 10.x IP.
- Check subnet route tables: ensure the ALB is deployed in public subnets containing an active 0.0.0.0/0 route to an Internet Gateway (IGW), not a NAT Gateway.
- Audit Security Groups: verify the ALB SG permits inbound ports 80/443 from 0.0.0.0/0, and backend instance SGs permit ingress chained from the ALB SG.
- Verify stateless Network ACLs: confirm both inbound 80/443 allow rules and outbound ephemeral return rules (ports 1024–65535) are active.
- Execute AWS VPC Reachability Analyzer: trigger an automated network insights path between the IGW and ENI to mathematically identify the exact blocking security rule.
Mastered this procedure? Subscribe to @naveedkumbhar01 on YouTube for daily 60-second SRE incident runbooks, Kubernetes triage shortcuts, and high-impact interview elevator pitches.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Purging compromised IAM user credentials immediately prior to querying CloudTrail forensic event logs when configuring Works Internally Internet Deactivating a compromised key prevents abuse while preserving identity records for CloudTrail forensic queries to assess blast radius.
- Conflating local IAM Permissions Boundaries with organizational Service Control Policies (SCPs) when configuring Works Internally Internet SCPs define maximum permission guards at the AWS Organization level across accounts, whereas permission boundaries limit delegate privileges within a single account.
- Deploying single-AZ cloud infrastructure for production workloads without multi-AZ failover resilience when configuring Works Internally Internet Availability Zone hardware failures or localized network partitions cause complete service downtime when resources are not distributed across >=2 AZs.
- Jumping straight to SSH or instance restarts without reviewing CloudWatch metrics and AWS Systems Manager logs when configuring Works Internally Internet Senior SREs triage control plane and hypervisor metrics via CloudWatch and SSM Session Manager before executing invasive host-level actions.