⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All AWS & Cloud Architecture Interview Questions Scenario 125 of 179 in AWS & Cloud Architecture
Staff Cloud Architect AWS AWS Large-Scale Cloud Data Migration Staff Architecture
🎯 Target Role / Context: Staff Cloud Architect & Data Platform Scenario

Q: You have around 500 TB of data in one AWS account and need to migrate it to another account. How would you design the secure, automated migration solution?

Enterprise architectural blueprint for migrating 500 Terabytes of live object data between two distinct AWS accounts securely, with automated checksum verification, KMS key grant delegation, and minimal cost.

#AWS #S3 #Data Migration #DataSync #Architecture #FinOps #Comparisons
🎙️ Candidate Opening & Architectural Context
"Migrating 500 TB of data cross-account cannot be done with a naive 'aws s3 sync' CLI script. It requires high-throughput managed services: S3 Cross-Account Replication paired with S3 Batch Replication, or AWS DataSync, with cross-account IAM delegation and KMS CMK key grants."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: Migrating 500 TB of data cross-account cannot be done with a naive 'aws s3 sync' CLI script. It requires high-throughput managed services: S3 Cross-Account Replication paired with S3 Batch Replication, or AWS DataSync, with cross-account IAM delegation and KMS CMK key grants.

1

Select Architecture: S3 Batch Replication vs. AWS DataSync

For pure S3-to-S3 object migration while preserving metadata and versioning, S3 Batch Replication is ideal. For scheduled differential syncs with bandwidth rate-limiting, AWS DataSync provides automated verification.

2

Configure Cross-Account IAM Role & Destination Bucket Policy

Create an IAM role in Account A with permission to read source objects and replicate to Account B. Configure Account B's bucket policy to allow Account A's replication role to write objects with BucketOwnerEnforced ownership.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"AWS": "arn:aws:iam::111111111111:role/s3-replication-role"},
    "Action": ["s3:ReplicateObject", "s3:ReplicateDelete", "s3:ObjectOwnerOverrideToBucketOwner"],
    "Resource": "arn:aws:s3:::destination-bucket-account-b/*"
  }]
}
Advertisement
3

Delegate KMS CMK Encryption Permissions

If source objects use KMS CMK encryption, the KMS key policy in Account A must grant kms:Decrypt to the replication role, and Account B's KMS key must grant kms:Encrypt.

4

Execute S3 Batch Replication for Historical Data

Enable S3 Cross-Region/Cross-Account Replication for all new incoming objects. Generate an S3 Inventory report on the source bucket and initiate an S3 Batch Replication job to backfill the existing 500 TB.

💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Use S3 Replication for ongoing writes combined with S3 Batch Replication on an S3 Inventory report for the 500 TB backfill. Enforce BucketOwnerEnforced to guarantee destination ownership."
⚡ 60-Second Elevator Pitch Talking Points
  • Enable S3 Cross-Account Replication to replicate all new incoming objects in real-time.
  • Generate an S3 Inventory list of the source bucket and launch an S3 Batch Replication job for the 500 TB.
  • Configure cross-account KMS CMK key grants for seamless decryption and re-encryption.
  • Set Object Ownership to BucketOwnerEnforced in Account B so destination account owns all migrated objects.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Relying on legacy bucket ACLs instead of modern S3 Block Public Access and IAM bucket policies when configuring Migrating Data Across AWS recommends disabling ACLs (Bucket Owner Enforced) in favor of centralized declarative bucket policies and account-level Block Public Access controls.
  2. Confusing SSE-S3 encryption with KMS customer-managed key (SSE-KMS) authorization policies when configuring Migrating Data Across Accessing SSE-KMS encrypted objects requires both s3:GetObject and kms:Decrypt permissions; neglecting KMS key policies causes silent 403 Forbidden errors.
  3. Using sequential timestamps or predictable prefixes for high-throughput S3 read and write keys when configuring Migrating Data Across Sequential key prefixes limit S3 horizontal partitioning; randomized hash prefixes distribute workloads across partitions to achieve 3,500 PUT and 5,500 GET requests/sec.
  4. Failing to audit S3 object delete markers when troubleshooting missing versioned objects when configuring Migrating Data Across Issuing a DELETE on a versioned bucket only adds a delete marker; engineers must retrieve previous versions or purge markers to restore data.
Advertisement
Want more AWS & Cloud Architecture scenarios?
Explore our complete collection of scenario-based AWS & Cloud Architecture interview runbooks.
Browse All AWS & Cloud Architecture Questions →