Module: utils/flakyDetector

Compute and persist flaky scores for tests (DIF-004).

A test is "flaky" when it sometimes passes and sometimes fails across runs. The flaky score (0–100) represents the pass/fail balance ratio: flakyScore = (min(passes, fails) / total) * 100

A score of 0 means the test always produces the same result (all pass or all fail). A score of 50 means it passes and fails equally often (maximally flaky). Note: the score measures the proportion of minority outcomes, not the sequential alternation pattern — PPPPPFFFFF and PFPFPFPFPF both score 50.

Called after each test run by testRunner.js via runFeedbackLoop. The score is persisted to tests.flakyScore for dashboard display, filtering, and badge rendering.

Source:
Example
import { computeAndPersistFlakyScores } from "../utils/flakyDetector.js";
const result = computeAndPersistFlakyScores("PRJ-1");
// { updated: 5, flaky: 2 }

Members

(inner, constant) MIN_RESULTS :number

Minimum number of run results before computing a flaky score.

Type:
  • number
Source:

Methods

(static) computeAndPersistFlakyScores(projectId, maxRunsopt) → {Object}

Compute flaky scores for all tests in a project and persist them.

Scans the last N completed test runs (default 20) for the project, aggregates pass/fail counts per test, computes the flaky score, and bulk-updates the flakyScore column.

Parameters:
Name Type Attributes Default Description
projectId string
maxRuns number <optional>
 20

— Maximum number of recent runs to consider.
Source:
Returns:
Type
Object

(static) getTopFlakyTests(projectIds, limitopt) → {Array.<{testId: string, name: string, flakyScore: number, projectId: string}>}

Get the top N flakiest tests for a set of project IDs. Used by the dashboard to display the "Flaky Tests" panel.

Parameters:
Name Type Attributes Default Description
projectIds Array.<string>
limit number <optional>
 10
Source:
Returns:
Type
Array.<{testId: string, name: string, flakyScore: number, projectId: string}>