THE ACCOUNTABILITY NUMBERS
These checks show their source window and computation date. Player-strength estimates and descriptive box-score game contributions have separate results. Methodology ↗
opportunity-adjusted-impact-v3 · Published 2026-11-10
FULL IMPACT: THE HISTORICAL TEST
Separate offensive and defensive lineup fits, with passing opportunities, shooting threat, opportunity-adjusted rebounds and defended shots. Values estimate strength per 48 minutes above league average; they are not points a player caused.
Each season selects settings from December and January, then tests February through season end using inputs frozen at January 31. The 2024–25 fit has no later-season inputs; the 2025–26 fit can borrow from 2024–25. These are retrospective corrected sources and previously studied seasons, not a new prospective test.
| Model | RMSE | MAE |
|---|---|---|
| New full impact | 14.95 | 11.84 |
| Previous full model, retuned | 14.89 | 11.71 |
| Baseline selected before testing | 15.46 | 12.18 |
| Without creation | 14.99 | 11.80 |
| Without shooting-threat proxy | 15.02 | 11.90 |
| Without defensive events / shots | 15.02 | 11.94 |
| Without rebound adjustment | 14.94 | 11.83 |
The new model beats the preselected box/lineup baseline. Its error is close to the previous full model, within the preset 1% tolerance—not an accuracy improvement. Removing a feature group tests its predictive contribution; it does not establish causal skill credit.
Creation
Potential, secondary and free-throw assists; screens and drives. Useful-action coefficients cannot turn negative.
Spacing proxy
Three-point volume and stabilized accuracy. Direct gravity and off-ball defensive attention remain unmeasured.
Defense
Separate defensive lineup strength, events and non-overlapping defended-shot zones. Small shot samples shrink toward average.
Rebound credit compares recovered balls with tracked chances from the same sample. Duplicate box/play-by-play shooting fields and overlapping shot-zone totals are removed. Positions are never forced to share a rating or ordering.
Validation gates, sample sizes & uncertainty
| Gate | Result |
|---|---|
| sample | Passed |
| comparative accuracy | Passed |
| legacy noninferiority | Passed |
| per season noninferiority | Passed |
| role and exposure | Passed |
| missing source stress | Passed |
| future game interval | Passed |
| historical source coverage | Passed |
| duplicate inputs removed | Passed |
| feature coverage | Passed |
| useful groups active | Passed |
| feature ablations | Passed |
Paired 95% interval for mean squared error versus the preselected baseline: -22.94 to -8.59 points². Negative favors this model. Predictions use actual later lineups and minutes; these are not pregame odds.
The tuning-calibrated 90% game interval covers 90.3% of test games. Player intervals use 100 game-cluster fits conditional on recorded features; that game coverage is not a calibration claim for individual players. Role/minutes checks group team-game errors and do not prove position-neutral individual attribution.
462 players · through 2026-11-09 · 5 games held from lineup fitting. Missing feature samples remain unknown and pool toward the training average.
Current game exceptions
0022600129 · ValueError: Lineups: plus_minus_mismatch:1626157,plus_minus_mismatch:203083
0022600138 · ValueError: Lineups: inconsistent_substitution:4:100,inconsistent_substitution:4:92,inconsistent_substitution:4:97,minutes_mismatch:1631131,minutes_mismatch:1631172,minutes_mismatch:1642263,minutes_mismatch:1643417
0022600186 · ValueError: Lineups: incomplete_five_player_minutes,minutes_mismatch:1641796,plus_minus_mismatch:1641796,unresolved_period_starters:5
0022600193 · ValueError: Lineups: incomplete_five_player_minutes,minutes_mismatch:1631288,plus_minus_mismatch:1631288,unresolved_period_starters:5
0022600206 · ValueError: Lineups: plus_minus_mismatch:1641869
Estimates, components & coverage ↗ · Full validation receipts ↗
SEPARATE BOX-SCORE CONTRIBUTION CHECKS
2026-27 season · 154 games · 3,479 player-games · games through 2026-11-09T00:00:00 · computed 2026-11-10T16:06:00Z
How much of every real final margin the model's per-player credit explains when summed by team.
Share of games where the team with more total Impact actually won.
Spearman-Brown adjusted correlation between each player's even- and odd-game box contribution (0 players, 30+ games each). Full-season RAPM is excluded from this check. Shared feature baselines remain a limitation.
RAPM agreement is available only for a fit that passed publication checks. Unversioned legacy coefficients cannot establish this comparison. The current fit is unverified, so this value is unavailable.
CALIBRATION BY DECILE
Games bucketed by the model's predicted margin (sum of home Impact minus sum of away Impact). A calibrated model's actual margins should rise monotonically with its predicted ones. Slope: 0.443 · mean absolute error: 5.3 pts.
HOW TO READ THESE — AND HOW NOT TO
- The margin fit is explanatory, not predictive. Per-game Impact is computed from the same game's events, so it must broadly agree with the scoreboard. The honest test here is whether credit assignment sums back to real margins with a stable slope and calibrated deciles — a check on attribution, not a forecast. Pregame prediction is a different (and harder) problem.
- Reliability is the number to watch. A metric can "explain" games while being noise at the player level. Split-half reliability measures whether a player's Impact in half his games predicts his Impact in the other half.
- RAPM agreement has limits. We exclude the published blend from this comparison. Prior-informed RAPM can still share box-score information, so this correlation is not proof of independent predictive accuracy.
- Game context remains a limitation. The published box-only formula does not adjust for garbage time, opponent strength or defensive assignments. Those effects require separate validation.
- Known blind spots: off-ball defense, screening value and gravity are under-measured by the box signal. The current formula centers credit against a game-wide per-minute baseline; it does not apply the retired blend's team demeaning.
PREGAME FORECASTS — PROSPECTIVE RESULTS
9 games · Brier 0.232 · Log loss 0.658. A 50/50 baseline has Brier 0.250.
Home-court baseline Brier: 0.282.
Season-total interval coverage is unavailable until completed seasons and archived intervals exist.