⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All AWS & Cloud Architecture Interview Questions Scenario 60 of 179 in AWS & Cloud Architecture
Senior DevOps / SRE [L2] AWS Cost & Architecture Production Scenario [L2]

Q: CloudFront is serving stale content after you updated S3.

When CloudFront is serving stale content after you updated S3, the cause is almost always caching, not S3: S3 gives strong read-after-write consistency, so the origin is correct while an edge location, or the user's browser,.

#AWS #Cost & Architecture #L2 #Cloud #Infrastructure #S3
🎙️ Candidate Opening & Architectural Context
"When <strong>CloudFront is serving stale content after you updated S3</strong>, the cause is almost always caching, not S3: S3 gives strong read-after-write consistency, so the origin is correct while an edge location, or the user's browser, still holds the old object until its TTL expires. The fast fix is an invalidation; the lasting fix is versioned asset names and sane cache headers. Interviewers use this scenario to see whether you diagnose methodically (is it the browser, the edge, or the origin?) rather than blindly invalidating <code>/*</code> and hoping."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: When <strong>CloudFront is serving stale content after you updated S3</strong>, the cause is almost always caching, not S3: S3 gives strong read-after-write consistency, so the origin is correct while an edge location, or the user's browser, still holds the old object until its TTL expires. The fast fix is an invalidation; the lasting fix is versioned asset names and sane cache headers. Interviewers use this scenario to see whether you diagnose methodically (is it the browser, the edge, or the origin?) rather than blindly invalidating <code>/*</code> and hoping.

1️⃣

Symptom to cause: three caches in the path

Work out which layer is serving the old bytes before changing anything.

  • Browser cache: governed by Cache-Control returned to the client. A long max-age means the browser never even asks CloudFront.
  • CloudFront edge (and regional edge) cache: governed by the cache policy's minimum, default and maximum TTL combined with the origin's Cache-Control/Expires. The default TTL is typically 24 hours when the origin sends no headers.
  • Origin: confirm S3 really holds the new object. An upload to a different key, bucket or prefix than the origin path is a surprisingly common cause.
  • Query strings, headers, cookies: if they are not part of the cache key, differing requests collapse into one cached object.
  • Practical order: test with curl first (no browser), then compare edge versus origin, then inspect cache policy. This sequence takes minutes and avoids unnecessary invalidations.
  • Rule out replication delays: if you use a multi-origin or cross-region replicated bucket, confirm the object reached the bucket CloudFront actually reads.
2️⃣

Hands-on: CloudFront serving stale content, inspect the headers

Use response headers to prove which layer answered.

# What does the edge say?
curl -sI https://cdn.example.com/app.js | egrep -i 'x-cache|age|etag|cache-control|last-modified'
# x-cache: Hit from cloudfront
# age: 61342

# What does the origin say?
aws s3api head-object --bucket my-assets-bucket --key app.js \
  --query '{ETag:ETag,LastModified:LastModified,CacheControl:CacheControl}'

# What caching rules apply to the distribution?
aws cloudfront get-distribution-config --id E123EXAMPLE \
  --query 'DistributionConfig.DefaultCacheBehavior.CachePolicyId'
  • X-Cache: Hit from cloudfront means the edge served a cached copy; Miss from cloudfront means it fetched from the origin.
  • Age: seconds the object has lived in the edge cache. A large value on a file you just updated confirms an edge cache problem.
  • ETag / Last-Modified: compare with aws s3api head-object to prove edge and origin differ.
  • Bypass the browser: use curl so browser caching is excluded from the test.
  • Cache-Control from S3: S3 returns whatever metadata was set at upload. Re-uploading with the same key but without the header resets it, so check the object's metadata after each pipeline change.
  • Compare with a cache-busting request: adding ?v=2 only proves a difference if query strings are in the cache key; use it as a hint, not as proof.
Advertisement
3️⃣

Fix now: create an invalidation and wait for it

An invalidation removes objects from edge caches before TTL expiry.

INV=$(aws cloudfront create-invalidation --distribution-id E123EXAMPLE \
  --paths "/index.html" "/app.js" --query Invalidation.Id --output text)

aws cloudfront wait invalidation-completed --distribution-id E123EXAMPLE --id "$INV"
curl -sI https://cdn.example.com/app.js | grep -i x-cache   # expect: Miss then Hit
  • Targeted paths: invalidate only what changed, for example /index.html and /app.js. Wildcards like /* work but are wasteful; check current pricing and free-path allowances.
  • Wait for completion: invalidations are not instant. Use aws cloudfront wait invalidation-completed before telling stakeholders it is fixed.
  • Paths are case sensitive and must match the cached URL, including a leading slash. Query strings are part of the cached key if forwarded.
  • Browser copies are not purged: users with a long max-age still need a hard refresh or a changed URL.
  • Verify afterwards: a first request after invalidation should report Miss from cloudfront and return the new ETag; the next should be a Hit with the new content.
Pro Tip: An invalidation is a repair tool, not a deployment strategy. If every release needs one, your caching design is wrong.
4️⃣

Root causes and edge cases

Match the symptom to a configuration fault.

  • Long TTL on mutable names: app.js served with max-age=31536000 will be stale in browsers for a year.
  • Wrong origin or path: the behavior's origin path or a different origin group points at an older bucket or prefix.
  • Cache policy too aggressive: a high minimum TTL overrides short origin headers. Review the policy rather than the object.
  • Multiple behaviors: a more specific path pattern may match your file with different caching than the default behavior.
  • Origin Shield or regional cache: additional layers delay propagation, which is why a verified invalidation matters.
5️⃣

Production practice: versioned assets and cache headers

Make staleness impossible by design.

  • Content-hashed filenames: publish app.3f9c1a.js with Cache-Control: public, max-age=31536000, immutable; a new build means a new URL, so nothing needs invalidating.
  • Short TTL for the entry document: keep index.html at a low max-age or no-cache (revalidate) and invalidate only that file in the deploy pipeline.
  • Set headers at upload: aws s3 sync dist/ s3://my-assets-bucket --cache-control ... per file type, or use a response headers policy.
  • Pipeline step: end the deploy job with the invalidation and wait command, and fail the job if it errors.
  • Stale-while-revalidate and stale-if-error: CloudFront can honor these directives; useful for resilience, but they also deliberately serve old content, so set them knowingly.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Read the X-Cache and Age headers first to locate the stale layer, then ship hashed asset filenames with long TTLs and invalidate only index.html."
⚡ 60-Second Elevator Pitch Talking Points
  • S3 is strongly consistent, so stale content means an edge or browser cache is still holding the old object until its TTL ends.
  • I confirm with curl -I for X-Cache and Age, compare against aws s3api head-object, then run a targeted invalidation and wait for it to complete.
  • I automate content-hashed filenames, explicit Cache-Control headers and a post-deploy invalidation of only the entry file.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Invalidating /* on every deploy It is slow, can incur charges beyond the free allowance, and hides the real problem of mutable asset names.
  2. Testing in the browser only The browser cache may be the stale layer; curl isolates the edge from the client.
  3. Ignoring the cache policy minimum TTL A high minimum TTL overrides short origin Cache-Control headers, so fixing the object headers has no effect.
  4. Assuming S3 is eventually consistent S3 offers strong read-after-write consistency, so chasing the origin wastes time.
Advertisement
Want more AWS & Cloud Architecture scenarios?
Explore our complete collection of scenario-based AWS & Cloud Architecture interview runbooks.
Browse All AWS & Cloud Architecture Questions →