Q: Your team practices trunk-based development. A long-running feature takes 3 weeks to build. How do you keep it out of production?
Use feature flags (feature toggles):
Want to master this scenario in a live sandbox? KodeKloud's Enterprise GitOps with ArgoCD & Kubernetes Rollouts covers this exact problem with hands-on terminal drills.
🛠️ Production Runbook & Step-by-Step Resolution
Direct Answer: "In enterprise CI/CD, you cannot rely on manual interventions; every rollback and promotion must be declarative. When addressing this question, I walk the interviewer through our production incident runbook: isolating the blast radius, checking diagnostic logs and metrics, and applying a safe fix."
Initial Diagnostics & Root Cause Analysis
Use feature flags (feature toggles):
- Wrap the new feature code in a flag:
if (featureFlags.isEnabled("new-checkout-flow")) { ... }. - The code is deployed to production but the feature is OFF by default.
- Enable it gradually: for internal users first → beta users → all users.
- Rollback is instant — just turn the flag off without redeployment.
- Continuous deployment without exposing incomplete features.
Remediation & Permanent Safeguards
Tools: LaunchDarkly, Unleash, AWS AppConfig, or a simple Redis/DynamoDB-backed flag store. Benefits: Downside: flag debt — flags must be cleaned up after feature fully rolls out. Accumulating old flags makes code messy.
- Separate deployment from release.
- Easy A/B testing.
- Instant kill switch in production.
- Wrap the new feature code in a flag: if (featureFlags.isEnabled("new-checkout-flow")) { ... }.
- The code is deployed to production but the feature is OFF by default.
- Enable it gradually: for internal users first → beta users → all users.
Senior hiring managers and bar raisers evaluate how you prioritize during outages. Avoid these common anti-patterns:
- Promoting application releases to production without an automated, canary-validated rollback mechanism when configuring team practices trunk Pipelines that lack automated rollback mechanisms force engineering teams to scramble manual hotfixes during severe production regressions.
- Running concurrent deployment pipeline jobs without stage mutex locking on shared target environments when configuring team practices trunk Concurrent pipeline runs on the same environment race to apply state changes, causing out-of-order commits and unpredictable deployment state.
- Omitting dependency lockfiles (package-lock.json, go.sum, Cargo.lock) from continuous integration builds when configuring team practices trunk Omitting lockfiles allows upstream package updates to break builds unpredictably without any changes to application codebase.
- Failing to define dynamic cache invalidation keys for CI package managers and layer caches when configuring team practices trunk Stale build caches mask underlying dependency incompatibilities or pull outdated libraries, while un-cached runs slow feedback loops.