Q: Your application writes millions of small files to S3. Performance is slow on listing and retrieval. How do you optimize?

• S3 automatically partitions by prefix — requests are distributed across S3 partitions. More unique prefixes = better parallelism. Add a hash/timestamp prefix to distribute keys: abc123/2024/01/filename instead of logs/filename.
• Avoid sequential keys — old S3 had hot partition issues with sequential keys (dates). Modern S3 handles this better but prefixing is still good practice.
• S3 Select or Athena — for querying/filtering data, use S3 Select to retrieve only needed data instead of downloading entire files.
• Aggregate small files — if files are <1MB, aggregate into larger files. S3 works best with larger objects. Use multipart for >100MB.

## 🟢 Networking & VPC

• Request parallelism — use multipart download with parallel part fetching for large files.
• Batch operations — for bulk operations on millions of objects, use S3 Batch Operations instead of single API calls.