Q: A developer on your team commits a 500 MB binary model file directly into the Git repository. The clone time for your monorepo is now 45 minutes. How do you remove it without losing history and prevent it from ever happening again?
Immediate cleanup:
#CI/CD #๐ Supply Chain Security & Advanced CI/CD #L2 #DevOps #Automation #Pipelines
๐๏ธ Candidate Opening & Architectural Context
""During a high-stakes release, we hit a similar deployment challenge and resolved it with automated safeguards. The interviewer is testing: Git LFS, `git filter-repo`, pre-receive hooks.. I structure my answer around systematic triage first, root cause analysis second, and permanent remediation third.""
Advertisement
๐ ๏ธ Production Runbook & Step-by-Step Resolution
1๏ธโฃ
Initial Diagnostics & Root Cause Analysis
Immediate cleanup:
- Git LFS: Move all binary assets to Git LFS (
git lfs track "*.bin" "*.pkl"). LFS stores a small pointer in Git, and the actual binary lives in LFS storage (S3). Clone remains fast. - Pre-receive hook (server side): Configure the GitHub/GitLab server-side hook to reject any push where a single file exceeds 50 MB without also updating
.gitattributesto track it with LFS.
2๏ธโฃ
Remediation & Permanent Safeguards
This rewrites history to completely remove the file from every commit. All team members must re-clone or run git fetch --all && git reset --hard origin/main. Long-term prevention โ two layers:
git filter-repo --path models/big_model.bin --invert-paths
git push --force --all
๐ก The Senior SRE Gold Nugget (Key Architectural Takeaway)
"Pro-Tip: Git LFS: Move all binary assets to Git LFS (git lfs track "*.bin" "*.pkl"). LFS stores a small pointer in Git, and the actual bina."
โก 60-Second Elevator Pitch Talking Points
- Git LFS: Move all binary assets to Git LFS (git lfs track "*.bin" "*.pkl"). LFS stores a small po...
- Pre-receive hook (server side): Configure the GitHub/GitLab server-side hook to reject any push w...
Advertisement