Q: CloudFront is serving stale content after you updated S3.
When CloudFront is serving stale content after you updated S3, the cause is almost always caching, not S3: S3 gives strong read-after-write consistency, so the origin is correct while an edge location, or the user's browser,.
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: When <strong>CloudFront is serving stale content after you updated S3</strong>, the cause is almost always caching, not S3: S3 gives strong read-after-write consistency, so the origin is correct while an edge location, or the user's browser, still holds the old object until its TTL expires. The fast fix is an invalidation; the lasting fix is versioned asset names and sane cache headers. Interviewers use this scenario to see whether you diagnose methodically (is it the browser, the edge, or the origin?) rather than blindly invalidating <code>/*</code> and hoping.
Symptom to cause: three caches in the path
Work out which layer is serving the old bytes before changing anything.
- Browser cache: governed by
Cache-Controlreturned to the client. A longmax-agemeans the browser never even asks CloudFront. - CloudFront edge (and regional edge) cache: governed by the cache policy's minimum, default and maximum TTL combined with the origin's
Cache-Control/Expires. The default TTL is typically 24 hours when the origin sends no headers. - Origin: confirm S3 really holds the new object. An upload to a different key, bucket or prefix than the origin path is a surprisingly common cause.
- Query strings, headers, cookies: if they are not part of the cache key, differing requests collapse into one cached object.
- Practical order: test with curl first (no browser), then compare edge versus origin, then inspect cache policy. This sequence takes minutes and avoids unnecessary invalidations.
- Rule out replication delays: if you use a multi-origin or cross-region replicated bucket, confirm the object reached the bucket CloudFront actually reads.
Hands-on: CloudFront serving stale content, inspect the headers
Use response headers to prove which layer answered.
# What does the edge say?
curl -sI https://cdn.example.com/app.js | egrep -i 'x-cache|age|etag|cache-control|last-modified'
# x-cache: Hit from cloudfront
# age: 61342
# What does the origin say?
aws s3api head-object --bucket my-assets-bucket --key app.js \
--query '{ETag:ETag,LastModified:LastModified,CacheControl:CacheControl}'
# What caching rules apply to the distribution?
aws cloudfront get-distribution-config --id E123EXAMPLE \
--query 'DistributionConfig.DefaultCacheBehavior.CachePolicyId'
- X-Cache:
Hit from cloudfrontmeans the edge served a cached copy;Miss from cloudfrontmeans it fetched from the origin. - Age: seconds the object has lived in the edge cache. A large value on a file you just updated confirms an edge cache problem.
- ETag / Last-Modified: compare with
aws s3api head-objectto prove edge and origin differ. - Bypass the browser: use
curlso browser caching is excluded from the test. - Cache-Control from S3: S3 returns whatever metadata was set at upload. Re-uploading with the same key but without the header resets it, so check the object's metadata after each pipeline change.
- Compare with a cache-busting request: adding
?v=2only proves a difference if query strings are in the cache key; use it as a hint, not as proof.
Fix now: create an invalidation and wait for it
An invalidation removes objects from edge caches before TTL expiry.
INV=$(aws cloudfront create-invalidation --distribution-id E123EXAMPLE \
--paths "/index.html" "/app.js" --query Invalidation.Id --output text)
aws cloudfront wait invalidation-completed --distribution-id E123EXAMPLE --id "$INV"
curl -sI https://cdn.example.com/app.js | grep -i x-cache # expect: Miss then Hit
- Targeted paths: invalidate only what changed, for example
/index.htmland/app.js. Wildcards like/*work but are wasteful; check current pricing and free-path allowances. - Wait for completion: invalidations are not instant. Use
aws cloudfront wait invalidation-completedbefore telling stakeholders it is fixed. - Paths are case sensitive and must match the cached URL, including a leading slash. Query strings are part of the cached key if forwarded.
- Browser copies are not purged: users with a long
max-agestill need a hard refresh or a changed URL. - Verify afterwards: a first request after invalidation should report
Miss from cloudfrontand return the new ETag; the next should be a Hit with the new content.
Root causes and edge cases
Match the symptom to a configuration fault.
- Long TTL on mutable names:
app.jsserved withmax-age=31536000will be stale in browsers for a year. - Wrong origin or path: the behavior's origin path or a different origin group points at an older bucket or prefix.
- Cache policy too aggressive: a high minimum TTL overrides short origin headers. Review the policy rather than the object.
- Multiple behaviors: a more specific path pattern may match your file with different caching than the default behavior.
- Origin Shield or regional cache: additional layers delay propagation, which is why a verified invalidation matters.
Production practice: versioned assets and cache headers
Make staleness impossible by design.
- Content-hashed filenames: publish
app.3f9c1a.jswithCache-Control: public, max-age=31536000, immutable; a new build means a new URL, so nothing needs invalidating. - Short TTL for the entry document: keep
index.htmlat a lowmax-ageorno-cache(revalidate) and invalidate only that file in the deploy pipeline. - Set headers at upload:
aws s3 sync dist/ s3://my-assets-bucket --cache-control ...per file type, or use a response headers policy. - Pipeline step: end the deploy job with the invalidation and wait command, and fail the job if it errors.
- Stale-while-revalidate and stale-if-error: CloudFront can honor these directives; useful for resilience, but they also deliberately serve old content, so set them knowingly.
- S3 is strongly consistent, so stale content means an edge or browser cache is still holding the old object until its TTL ends.
- I confirm with curl -I for X-Cache and Age, compare against aws s3api head-object, then run a targeted invalidation and wait for it to complete.
- I automate content-hashed filenames, explicit Cache-Control headers and a post-deploy invalidation of only the entry file.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Invalidating /* on every deploy It is slow, can incur charges beyond the free allowance, and hides the real problem of mutable asset names.
- Testing in the browser only The browser cache may be the stale layer; curl isolates the edge from the client.
- Ignoring the cache policy minimum TTL A high minimum TTL overrides short origin Cache-Control headers, so fixing the object headers has no effect.
- Assuming S3 is eventually consistent S3 offers strong read-after-write consistency, so chasing the origin wastes time.