Alternative-Route accessibility benchmark: kimi-k2.6

Summary

kimi-k2.6 was scored on a 200-item open-ended benchmark, judged independently by claude-sonnet-4.5 (primary) and gpt-4o-mini (secondary) against the three-criterion rubric (2-of-3). The two judges agree on 86.0% of items (Cohen's κ = 0.652, “substantial”). No human labels have been recorded for this run yet, so the report defaults to the primary judge alone — score the sample on the Labeling page for a validated recommendation. The reported headline is 68.5% (137/200), with 65.5%–76.5% as the full judge-dependent range.

Accuracy by judge configuration

The same 200 items scored three ways: Primary (claude-sonnet-4.5 alone), Secondary (gpt-4o-mini alone), and AND (both judges must independently mark an item correct — the most conservative reading).

Physical mobility shows the largest primary/secondary split of any category — 58.14% vs. 74.42%, a 16.3-point gap. Physical mobility is the weakest category under the reported configuration, at 58.14% (25/43).

Judge reliability

Every item is scored twice — once by claude-sonnet-4.5 (primary) and once by gpt-4o-mini (secondary), each applying the same rubric independently. Raw agreement is 86.0% and Cohen's κ = 0.652— “substantial” (Landis & Koch: <0 poor, 0–0.2 slight, 0.2–0.4 fair, 0.4–0.6 moderate, 0.6–0.8 substantial, 0.8–1.0 almost perfect).

Human-label validation

Failure taxonomy

Every item scored 0 by the primaryjudge, bucketed into a failure sub-category by keyword match against its ground-truth cues, intent and the model's answer. 63 failures total.