Model comparison — 26-event sample-resolved set

Same pipeline (Brave retrieval + market-anchor prompt + 0.10 longshot floor), five different LLMs. Each cell shows p = probability assigned to outcome[0] and brier = Brier loss for the resolved outcome. Hover any cell for the model's rationale on that event. Rows sorted by inter-model spread — most contested events first. Lower Brier = better. Random binary baseline = 0.25, uniform-prior baseline = 0.22.

< 0.05 (excellent) < 0.10 < 0.25 < 0.50 ≥ 0.50 (catastrophic)

Opus 4.7 (production)

0.2329

n=26 resolved

Sonnet 4.6 (prev prod)

0.0879

n=14 resolved

Opus 4.6

0.2264

n=26 resolved

GPT-5.2

0.2584

n=26 resolved

Gemini 3.1 Pro (post-harden)

0.4259

n=26 resolved

Reliability diagram

Production model calibration on resolved rows: Opus 4.7 (production). Small-n view: 26 outcome[0] probabilities, ECE 0.144.

26 events × 5 models = 130 predictions in the grid below. Tooltips show rationale.

Category	Event (outcomes; resolved winner)	Opus 4.7 (production)	Sonnet 4.6 (prev prod)	Opus 4.6	GPT-5.2	Gemini 3.1 Pro (post-harden)
Sports	Who won the 2025-26 Liga Portugal (Portuguese top-flight football)? n=18 winner: FC Porto	p=0.05 brier 0.792	p=0.05 brier —	p=0.05 brier 1.789	p=0.05 brier 1.789	p=0.06 brier 0.944
Sports	Who won the 2025-26 NHL Calder Memorial Trophy (Rookie of the Year)? n=30 winner: Matthew Schaefer	p=0.05 brier 0.083	p=0.05 brier —	p=0.05 brier 0.083	p=0.05 brier 0.073	p=0.05 brier 0.975
Sports	Who won the 2025-26 La Liga (Spanish top-flight football)? n=20 winner: Barcelona	p=0.05 brier 0.054	p=0.05 brier —	p=0.05 brier 0.050	p=0.05 brier 0.049	p=0.05 brier 0.950
Entertainment	Which contestants were eliminated from Survivor Season 50 Episode 7? n=14 winner: Dee Valladares	p=0.05 brier 0.035	p=0.05 brier —	p=0.05 brier 0.036	p=0.05 brier 0.037	p=0.07 brier 0.929
Politics	How many US Senators voted for Trump's Federal Reserve Chair pick (2027 confirmation vote) n=12 winner: 54	p=0.05 brier 0.369	p=0.05 brier —	p=0.05 brier 0.028	p=0.05 brier 0.028	p=0.08 brier 0.917
Sports	Who won the 2025-26 Ligue 1 (French top-flight football)? n=18 winner: PSG	p=0.85 brier 0.073	p=0.78 brier —	p=0.75 brier 0.105	p=0.78 brier 0.097	p=0.06 brier 0.944
Politics	How many Supreme Court justices voted in favor of Louisiana in Louisiana v. Callais (2026) n=10 winner: 3	p=0.10 brier 0.900	p=0.05 brier —	p=0.05 brier 0.225	p=0.10 brier 0.900	p=0.10 brier 0.900
Politics	Which coalition won the 2026 Colombia Senate election? n=2 winner: Government Alliance	p=0.82 brier 0.032	p=0.75 brier 0.062	p=0.88 brier 0.014	p=0.72 brier 0.078	p=0.35 brier 0.423
Sports	Who won the 2025-26 Serie A (Italian top-flight football)? n=20 winner: Inter	p=0.22 brier 0.611	p=0.05 brier —	p=0.18 brier 0.722	p=0.14 brier 0.761	p=0.05 brier 0.950
Entertainment	Who won The Masked Singer Season 14? n=18 winner: Galaxy Girl	p=0.05 brier 1.260	p=0.05 brier —	p=0.05 brier 0.952	p=0.06 brier 0.944	p=0.06 brier 0.944
Sports	Who won the Sussex vs Leicestershire 2026 County Championship cricket match? n=2 winner: Sussex	p=0.90 brier 0.010	p=0.75 brier 0.062	p=0.57 brier 0.181	p=0.60 brier 0.160	p=0.50 brier 0.250
Elections	Who won the 2026 Democratic primary for West Virginia's 1st Congressional District? n=2 winner: Vince George	p=0.10 brier 0.010	p=0.25 brier 0.062	p=0.10 brier 0.010	p=0.10 brier 0.010	p=0.50 brier 0.250
Sports	Who won the Bax vs Arcon ATP Challenger tennis match on May 5, 2026? n=2 winner: Florent Bax	p=0.55 brier 0.202	p=0.50 brier 0.250	p=0.75 brier 0.062	p=0.52 brier 0.230	p=0.50 brier 0.250
Sports	Who won the Watson vs Okamura WTA Challenger tennis match on May 5, 2026? n=2 winner: Heather Watson	p=0.50 brier 0.250	p=0.52 brier 0.230	p=0.55 brier 0.202	p=0.57 brier 0.185	p=0.73 brier 0.073
Sports	Who won the Najzer vs Ebster tennis match in the 2026 W15 Klagenfurt Round of 32? n=2 winner: Anna Lena Ebster	p=0.10 brier 0.010	p=0.25 brier 0.062	p=0.10 brier 0.010	p=0.10 brier 0.010	p=0.10 brier 0.010
Sports	Who won the Rocha vs Johns ATP Challenger tennis match on May 10, 2026? n=2 winner: Garrett Johns	p=0.10 brier 0.010	p=0.25 brier 0.062	p=0.10 brier 0.010	p=0.10 brier 0.010	p=0.10 brier 0.010
Sports	Who won the Perez vs Lalami Laaroussi ATP Challenger tennis match on May 5, 2026? n=2 winner: Younes Lalami Laaroussi	p=0.10 brier 0.010	p=0.25 brier 0.062	p=0.25 brier 0.062	p=0.10 brier 0.010	p=0.10 brier 0.010
Sports	Who won the Worcestershire vs Durham 2026 County Championship cricket match? n=2 winner: Durham	p=0.10 brier 0.010	p=0.25 brier 0.062	p=0.10 brier 0.010	p=0.10 brier 0.010	p=0.10 brier 0.010
Sports	Who won the Bangladesh vs Pakistan men's Test cricket match starting May 8, 2026? n=2 winner: Bangladesh	p=0.90 brier 0.010	p=0.75 brier 0.062	p=0.90 brier 0.010	p=0.90 brier 0.010	p=0.90 brier 0.010
Sports	Who won the Glamorgan vs Somerset 2026 County Championship cricket match? n=2 winner: Glamorgan	p=0.90 brier 0.010	p=0.75 brier 0.062	p=0.90 brier 0.010	p=0.90 brier 0.010	p=0.90 brier 0.010
Sports	Who won the Los Angeles Lakers vs Oklahoma City NBA Round 2 playoff series in 2026? n=2 winner: Oklahoma City	p=0.90 brier 0.010	p=0.75 brier 0.062	p=0.90 brier 0.010	p=0.90 brier 0.010	p=0.90 brier 0.010
Elections	Who became Prime Minister of Hungary after the 2026 election? n=2 winner: Péter Magyar	p=0.90 brier 0.010	p=0.75 brier 0.062	p=0.90 brier 0.010	p=0.90 brier 0.010	p=0.90 brier 0.010
Elections	Who won the 2026 Democratic primary for Ohio's 15th Congressional District? n=2 winner: Don Leonard	p=0.90 brier 0.010	p=0.75 brier 0.062	p=0.90 brier 0.010	p=0.90 brier 0.010	p=0.90 brier 0.010
Entertainment	Who was Netflix's next live roast subject after Tom Brady? n=11 winner: Kevin Hart	p=0.50 brier 0.275	p=0.50 brier —	p=0.50 brier 0.275	p=0.50 brier 0.275	p=0.50 brier 0.275
Sports	Who won the Breda vs Heerenveen Eredivisie football match on May 10, 2026? n=3 winner: Breda	p=0.80 brier 0.060	p=0.67 brier —	p=0.80 brier 0.060	p=0.80 brier 0.060	p=0.80 brier 0.060
Entertainment	Who won Tournament of Champions Season 7 (Food Network)? n=20 winner: Bryan Voltaggio	p=0.05 brier 0.950	p=0.05 brier —	p=0.05 brier 0.950	p=0.05 brier 0.950	p=0.05 brier 0.950

Source files in data/predictions/ablation_*.json + data/predictions/multi_outcome_retrieval{.phase1_sonnet,}.json. Ground truth in data/resolved.json. Full methodology + per-decision postmortem in docs/DECISIONS.md. ← back to live dashboard