Q: Your test suite has 1000 tests. Some pass, some fail, some are flaky (fail randomly). How do you manage test reliability?

Categorize tests:

• Stable tests — always pass or always fail deterministically. Block the pipeline.
• Flaky tests — pass sometimes, fail sometimes. Don't block the pipeline. Quarantine them.
• Slow tests — run in separate parallel job or nightly.
• Tag flaky tests: @pytest.mark.flaky or // @flaky in Jest.
• Run them in a separate non-blocking pipeline step.
• Report results but don't fail the pipeline.

Quarantine approach: Fix flaky tests: Flaky test tracking:

• Track flakiness rate. Priority fix if a test is > 20% flaky.
• Common causes: timing issues (use proper waits, not sleep), shared state between tests (isolate), network calls (mock them), race conditions.
• Use retry mechanism as a short-term fix: pytest-rerunfailures retries failed tests 2-3 times before marking as failure.
• Tools like BuildPulse, Gradle Enterprise, or custom dashboards track flakiness rate over time.