History & Flakiness Trending

Two things are built from history data and shown in the detailed template:

• Pass-rate trend — a mini sparkline and delta showing how your overall pass rate changed across the last N runs.
• Flakiness table — top-N tests ranked by flake rate, each with a per-test dot sparkline (amber dot = flaky in that run).

Both sections are controlled by showTrend (default true). They appear once at least two history entries exist.

Zero configuration needed for local development. History is written automatically to ~/.reportforge/history.json.

// playwright.config.tsreporter: [['@reportforge/playwright-pdf', {  template: 'detailed',  // historyFile defaults to ~/.reportforge/history.json}]]

The sparkline and flakiness table appear automatically once two or more runs have been recorded. No extra configuration required.

Ephemeral CI runners discard the workspace after each run. Set historyFile to a project-relative path and cache it between runs so history accumulates across CI jobs.

// playwright.config.tsreporter: [['@reportforge/playwright-pdf', {  template: 'detailed',  historyFile: '.reportforge/history.json',}]]

# .github/workflows/test.yml- uses: actions/cache@v4  with:    path: .reportforge/history.json    key: reportforge-history-${{ github.ref }}    # Omit restore-keys — cross-branch fallback causes misleading sparklines on PRs- run: npx playwright test  env:    RF_LICENSE_KEY: ${{ secrets.RF_LICENSE_KEY }}    PUPPETEER_EXECUTABLE_PATH: /usr/bin/google-chrome-stable

Using github.ref as the cache key keeps history separate per branch. Drop restore-keys entirely on PR branches — cross-branch history makes the sparkline misleading.

In a monorepo with multiple Playwright configs, give each config its own historyFile so runs from different packages do not overwrite each other.

// packages/frontend/playwright.config.tsreporter: [['@reportforge/playwright-pdf', {  historyFile: '.reportforge/frontend-history.json',  outputFile: 'reports/{date}-frontend.pdf',}]]// packages/api/playwright.config.tsreporter: [['@reportforge/playwright-pdf', {  historyFile: '.reportforge/api-history.json',  outputFile: 'reports/{date}-api.pdf',}]]

The detailed template shows a Top flaky tests table below the pass-rate trend card. Each row shows the test name, flake rate, how many runs it was flaky in, and a dot sparkline across stored history (amber dot = flaky in that run, grey = clean).

reporter: [['@reportforge/playwright-pdf', {  template: 'detailed',  flakinessTopN: 5,   // default — show top 5 flakiest tests                      // set 0 to hide the table entirely}]]

The flakiness table is gated on showTrend: true (the default) and only appears when history contains at least one qualifying run. Runs recorded before this feature was introduced are excluded from the flake-rate denominator — the table fills in correctly as newer runs accumulate. No history migration needed.

A test is counted as flaky when Playwright marks it flaky — i.e. it failed on at least one retry attempt but ultimately passed.