Alternative-Route accessibility benchmark: minimax-m3

Summary

minimax-m3 was scored on a 200-item open-ended benchmark, judged independently by claude-sonnet-4.5 (primary) and gpt-4o-mini (secondary) against the three-criterion rubric (2-of-3). The two judges agree on 86.0% of items (Cohen's κ = 0.715, “substantial”). No human labels have been recorded for this run yet, so the report defaults to the primary judge alone — score the sample on the Labeling page for a validated recommendation. The reported headline is 56.0% (112/200), with 50.0%–58.0% as the full judge-dependent range.

Accuracy by judge configuration

The same 200 items scored three ways: Primary (claude-sonnet-4.5 alone), Secondary (gpt-4o-mini alone), and AND (both judges must independently mark an item correct — the most conservative reading).

Sensory impairments shows the largest primary/secondary split of any category — 67.65% vs. 61.76%, a 5.9-point gap. Physical mobility is the weakest category under the reported configuration, at 41.86% (18/43).

Judge reliability

Every item is scored twice — once by claude-sonnet-4.5 (primary) and once by gpt-4o-mini (secondary), each applying the same rubric independently. Raw agreement is 86.0% and Cohen's κ = 0.715— “substantial” (Landis & Koch: <0 poor, 0–0.2 slight, 0.2–0.4 fair, 0.4–0.6 moderate, 0.6–0.8 substantial, 0.8–1.0 almost perfect).

Human-label validation

Failure taxonomy

Every item scored 0 by the primaryjudge, bucketed into a failure sub-category by keyword match against its ground-truth cues, intent and the model's answer. 88 failures total.