Eval suite

Tab

Moves through Run, the run history, the case filter, each case row, Show more, and inside an open case the triage buttons.

Enter, Space

On Run, starts the run; while running the same button stops it. On a case row, opens or closes its drilldown. On Mark as expected or Open issue, triages the case and moves focus to Undo or the confirmation that replaces the buttons; Undo brings the buttons back with focus on the first.

Arrow leftorArrow right

On the case filter, moves between All, Failed, Regressions and Improved. On the run history, steps through the runs and reads out each version, score and date.

HomeorEnd

On the case filter or the run history, jumps to the first or last item.

Escape

On the run history, returns the readout to the latest run.

• Each category row is one image with a full text summary (passed, failed, errors, not yet scored, percent and the change against the baseline), so the 120 cells never become 120 announcements.
• Outcomes never rely on color: pass, fail and error each have their own glyph, a regression is the only solid cell, and every list row spells out both outcomes in hidden text.
• Case rows with a drilldown are buttons with aria-expanded and aria-controls; the drilldown is a labelled region. Rows without one are plain text.
• The case filter is a radio group with roving focus. The run history is a slider whose value text names the run, its score and date.
• Run start, finish (score, delta, regressions, improvements), stop, failure and each triage are announced in a polite status region, including when a run or an issue is simulated.
• The score and every category percent count with tabular numerals; hidden text carries the final value so screen readers never read a number mid count.
• Pointer focus draws no ring; keyboard focus shows Arc's shared focus ring.

• Cells fill as results arrive: the background tints over 240ms and the glyph scales in from 40 percent with an ease out. Cases in flight breathe once a second.
• The score, category percents and cost count to their values on a critically damped spring, so a burst of results reads as one smooth count; a new run resets them at once.
• Bars fill with a clip-path tween and tint green or red only when the run finishes; the deltas fade up 3px at the same moment.
• The current run lands on the sparkline as a short drawn segment and a dot that scales in after it.
• Rows that appear mid run, the drilldown and Show more open on the smooth spring with no overshoot. The filter highlight glides on a critically damped spring and the list height follows the new filter.
• Run, Stop and Run again crossfade in place while the pill width follows the measured label.
• Reduced motion fills cells, numbers, bars and panels immediately, stops the breathing, and keeps every state and announcement.

• The card is 760px wide at most. From 680px a category is one line: label, 24 cells and its bar; below that the cells move under the label, and below 480px they wrap into two rows of 12.
• Below 480px the header's model chips become a small aligned table (v12, v11 and Judge on the left, the model on the right) and the subtitle takes the full width under the title and Run.
• The summary goes from three columns to two at 620px and to one at 420px, with cost and time side by side.
• The two outputs share one frame split down the middle and stack below 560px; the drilldown labels move above their text. Below 480px each rubric row puts its name, score and change on one line with the bar under them.
• Case categories hide below 480px, triage tags keep only their icon, and titles truncate with the full title in hidden text.
• The filter drops its counts below 480px and tightens to fit 320px without scrolling.

• Scores, counts and per category numbers are derived from one results map in memoized passes; cells only change attributes, and their fills are CSS transitions.
• Counting numbers write their text through a motion value, so the spring never re-renders React per frame.
• The word diff runs once per opened case and is memoized on the two outputs.
• The simulation uses timers on eight workers and clears them on stop and unmount.

• Use it when a team compares a candidate version of a prompt, model or agent against a baseline on a fixed suite and needs to know which cases regressed, not only the average.
• Score is the pass rate over all cases in percent; the delta compares it with the baseline pass rate computed from each case's baseline outcome. A regression is a case that passed on the baseline and does not pass now; improved is the reverse.
• Stream results from onRun as an async iterable for the live grid. Results may arrive in any order; the regression and improved lists keep arrival order while the run streams.
• Rubric scores are 1 to 5, one per suite.rubric criterion, for both baselineScores and result scores. The rubric bars reuse the category bar: new score as the fill, baseline as the ghost tick.
• When the two replies share little wording, the drilldown shows both in full without highlights instead of striking through the whole baseline.
• Model marks: pass brand "claude" or "openai" for the built-in marks, or any inline SVG as mark. The sample uses Claude Sonnet 5.5 against GPT-6 Luna with Claude Opus 5.5 as judge; costs and timings are sample values.

The full library index for assistants is at /llms.txt.