Q: Your microservice running in Google Kubernetes Engine (GKE) needs to write objects to an AWS S3 bucket and publish events to AWS SQS. Storing static AWS IAM Access Keys and Secret Keys in GCP Secret Manager violates security policy and poses credential leak risks. How do you implement zero-secret cross-cloud IAM role assumption using OIDC federation?
Engineering a zero-static-credential cross-cloud identity federation pipeline enabling Google Kubernetes Engine (GKE) workloads to assume AWS IAM roles using Google OIDC identity tokens.
Want to master this scenario in a live sandbox? Stephane Maarek's AWS Certified DevOps Engineer Professional Masterclass on Udemy covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: Hardcoded AWS credentials in foreign clouds are one of the most common causes of enterprise data breaches. We eliminated all long-lived AWS IAM access keys in GCP by configuring direct OIDC federation between Google Cloud IAM and AWS IAM.
Configure GKE Workload Identity to Emit Google OIDC Identity Tokens
Equip the GKE pod to acquire cryptographically signed Google JSON Web Tokens (JWT):
- GKE Workload Identity: Bound Kubernetes Service Account (KSA) to Google Service Account (GSA)
gke-s3-writer@proj.iam.gserviceaccount.com. - Fetch Identity Token: Application or helper fetches OIDC token from local metadata server:
curl -H 'Metadata-Flavor: Google' 'http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=aws-crosscloud'.
Create AWS IAM OIDC Identity Provider for Google Accounts
Establish trust in AWS for tokens signed by Google Cloud:
- Create OIDC Provider in AWS: Executed
aws iam create-open-id-connect-provider --url https://accounts.google.com --client-id-list aws-crosscloud --thumbprint-list 2c9c7820.... - Cryptographic Trust: AWS fetches and validates Google's public JWKS keys from
https://www.googleapis.com/oauth2/v3/certs.
Create AWS IAM Role with Scoped Condition Keys
Define the AWS role and restrict assumption strictly to the specific GCP service account:
- Trust Policy JSON: Configured
Principal: { Federated: 'arn:aws:iam::ACCOUNT:oidc-provider/accounts.google.com' }withAction: 'sts:AssumeRoleWithWebIdentity'. - Condition Keys: Added
Condition: { StringEquals: { 'accounts.google.com:aud': 'aws-crosscloud', 'accounts.google.com:sub': '$GSA_UNIQUE_NUMERIC_ID' } }.
Execute sts:AssumeRoleWithWebIdentity & Access AWS S3
Exchange the short-lived Google token for temporary AWS STS credentials:
- STS Exchange: Executed
aws sts assume-role-with-web-identity --role-arn arn:aws:iam::ACCOUNT:role/gke-s3-role --role-session-name gke-session --web-identity-token $GOOGLE_TOKEN. - Temporary Credentials: AWS STS returns temporary AWS access key, secret key, and session token valid for 1 hour.
- Verification: Pod writes file to S3:
aws s3 cp report.parquet s3://prod-lake/with zero static credentials stored anywhere.
- Configure GKE Workload Identity to fetch Google-signed OIDC identity tokens.
- Create an OIDC Identity Provider in AWS IAM trusting https://accounts.google.com.
- Define an AWS IAM role with trust conditions validating the exact Google Service Account ID.
- Exchange the Google token via sts:AssumeRoleWithWebIdentity to obtain temporary 1-hour credentials.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Relying on legacy bucket ACLs instead of modern S3 Block Public Access and IAM bucket policies when configuring Zero Secret Cross AWS recommends disabling ACLs (Bucket Owner Enforced) in favor of centralized declarative bucket policies and account-level Block Public Access controls.
-
Confusing SSE-S3 encryption with KMS customer-managed key (SSE-KMS) authorization policies when configuring Zero Secret Cross
Accessing SSE-KMS encrypted objects requires both
s3:GetObjectandkms:Decryptpermissions; neglecting KMS key policies causes silent 403 Forbidden errors. - Using sequential timestamps or predictable prefixes for high-throughput S3 read and write keys when configuring Zero Secret Cross Sequential key prefixes limit S3 horizontal partitioning; randomized hash prefixes distribute workloads across partitions to achieve 3,500 PUT and 5,500 GET requests/sec.
- Failing to audit S3 object delete markers when troubleshooting missing versioned objects when configuring Zero Secret Cross Issuing a DELETE on a versioned bucket only adds a delete marker; engineers must retrieve previous versions or purge markers to restore data.