⚡ ~/naveed Interview Prep
⚡ Portfolio Home 👤 About Naveed Bio & Credentials 📫 Contact Naveed Get In Touch ✍️ Engineering Blog Deep Dives 🎯 Interview Hub 1,000+ Scenarios ☸️ Kubernetes Mastery Hub 24 Modules 🎮 DevOps Arcade & Quizzes Subnet Blitz ⚡ 🗺️ DevOps Roadmaps PDFs & Guides 🤖 Morpheus Analysis AI Quant ↗ 🛠️ Developer Tools Utilities 🧪 Labs & Experiments 📄 Interactive CV & Certs 🔗 All Links & Socials ⚡ Join The Dispatch (Weekly SRE Newsletter) →
← Back to All Platform Engineering & IDP Interview Questions Scenario 37 of 50 in Platform Engineering & IDP
Staff Platform Engineer Platform Engineering Control Planes & Crossplane Control Plane Security
🎯 Target Role / Context: Staff Platform Engineer Loop · Security & Control Plane Governance

Q: Your company's Crossplane AWS provider uses static IAM user access keys stored in a Kubernetes Secret. A security audit mandates immediate revocation of all static keys and non-disruptive rotation every 24 hours. How do you migrate Crossplane to IRSA without disrupting ongoing infrastructure reconciliation?

Zero-downtime credential rotation for Crossplane cloud provider controllers using AWS IRSA and short-lived STS tokens.

#Crossplane #AWS IAM #IRSA #Security #Secrets #Platform Engineering
🎙️ Candidate Opening & Architectural Context
"Static long-lived access keys in Crossplane create catastrophic credential leakage risks. Enterprise platform engineering requires migrating Crossplane providers to AWS IAM Roles for Service Accounts (IRSA) or EKS Pod Identity, which issue short-lived, automatically rotated STS tokens."
Advertisement
⚡ Recommended Practice Lab

Want to master this scenario in a live sandbox? KodeKloud's CKA & CKAD Hands-On Certification Track covers this exact problem with hands-on terminal drills.

🛠️ Production Runbook & Step-by-Step Resolution

Direct Answer: Static long-lived access keys in Crossplane create catastrophic credential leakage risks. Enterprise platform engineering requires migrating Crossplane providers to AWS IAM Roles for Service Accounts (IRSA) or EKS Pod Identity, which issue short-lived, automatically rotated STS tokens.

1

Create IAM Role with OIDC Trust Policy for Crossplane Provider

Create an AWS IAM role trusting the EKS cluster's OIDC identity provider, scoped specifically to the Crossplane provider's ServiceAccount.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E080826BA80423"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E080826BA80423:sub": "system:serviceaccount:crossplane-system:provider-aws-upjet"
        }
      }
    }
  ]
}
2

Configure ProviderConfig to Use Injected IRSA Identity

Update the Crossplane ProviderConfig from source: Secret to source: InjectedIdentity (IRSA).

apiVersion: aws.upjet.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: InjectedIdentity
Advertisement
3

Annotate Provider ServiceAccount and Trigger Rolling Update

Annotate the provider ServiceAccount with eks.amazonaws.com/role-arn. The AWS SDK inside the provider pod automatically reads the mounted web identity token, rotating credentials every hour with zero manual intervention.

kubectl annotate serviceaccount -n crossplane-system provider-aws-upjet \
  eks.amazonaws.com/role-arn=arn:aws:iam::123456789012:role/crossplane-provider-role \
  --overwrite
4

Revoke Legacy Static IAM Access Keys in AWS

After verifying that kubectl get managed resources remain in Ready=True, safely delete the static IAM access keys in AWS Console/CLI.

Pro Tip: Zero-Trust Win: IRSA eliminates long-lived secrets from cluster storage; credentials rotate automatically every 60 minutes via AWS STS.
💡 The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Eliminate static Crossplane credentials by migrating ProviderConfig to InjectedIdentity powered by AWS IAM Roles for Service Accounts (IRSA)."
⚡ 60-Second Elevator Pitch Talking Points
  • Create an IAM role federated with the cluster's OIDC provider scoped to the provider's ServiceAccount.
  • Configure Crossplane ProviderConfig to source credentials from InjectedIdentity.
  • Rely on automated STS token rotation to eliminate static cloud access keys completely.
⛔ Critical Candidate Mistakes to Avoid in This Scenario

Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:

  1. Engineering bespoke in-house developer tools instead of adopting standardized platform Golden Paths when configuring Crossplane Provider Secret Bespoke internal tools create long-term maintenance debt and fragmented workflows, whereas standardized Golden Paths accelerate delivery.
  2. Overlooking infrastructure cost visibility and FinOps allocation tagging across multi-tenant environments when configuring Crossplane Provider Secret Without granular resource tagging, engineering organizations cannot attribute cloud infrastructure costs to specific teams or business units.
  3. Failing to define explicit Service Level Objectives (SLOs) and Error Budgets to balance feature velocity with reliability when configuring Crossplane Provider Secret Without agreed SLOs, engineering and product teams lack an objective data-driven framework to prioritize technical debt over new features.
  4. Neglecting automated continuous drift detection between declarative configuration and deployed cloud state when configuring Crossplane Provider Secret Unmanaged drift accumulates over time, causing future automated deployments to fail or overwrite emergency manual adjustments.
Advertisement
Want more Platform Engineering & IDP scenarios?
Explore our complete collection of scenario-based Platform Engineering & IDP interview runbooks.
Browse All Platform Engineering & IDP Questions →