As you may remember, one of the things I did when creating pre-season projection using Compu-Picks was show win distribution
probabilities. Below is a summary of how those probability distributions did.
Overall, the system did extremely well evaluating probability distributions, neither being biased towards the middle or towards the tail.
The below table shows a breakdown of how many teams finished in each range, compared to the theoretical expectation (where an even
number of teams would finish in each probability bucket). There's plenty of noise, but there's no particular bias towards the tails (19 teams
finished under the 10th percentile or over the 90th percentile, compared to the theoretical expectation of 24) or the middle
(22 teams finished between the 40th and 60th percentile compared to the theoretical expectation of 24).
| Min | Max | Number | Expected |
| 0.0% | 5.0% | 2 | 6 |
| 5.0% | 10.0% | 7 | 6 |
| 10.0% | 15.0% | 10 | 6 |
| 15.0% | 20.0% | 8 | 6 |
| 20.0% | 25.0% | 6 | 6 |
| 25.0% | 30.0% | 6 | 6 |
| 30.0% | 35.0% | 4 | 6 |
| 35.0% | 40.0% | 8 | 6 |
| 40.0% | 45.0% | 5 | 6 |
| 45.0% | 50.0% | 9 | 6 |
| 50.0% | 55.0% | 5 | 6 |
| 55.0% | 60.0% | 3 | 6 |
| 60.0% | 65.0% | 4 | 6 |
| 65.0% | 70.0% | 3 | 6 |
| 70.0% | 75.0% | 7 | 6 |
| 75.0% | 80.0% | 9 | 6 |
| 80.0% | 85.0% | 3 | 6 |
| 85.0% | 90.0% | 11 | 6 |
| 90.0% | 95.0% | 4 | 6 |
| 95.0% | 100.0% | 6 | 6 |
To clarify exactly what this table means and how it was calculated, a bit of explanation is in order. I'll use LSU, who went a perfect
12-0 in the regular season, as an example. The system projected them at 7.93 wins,
so clearly they did much better than projected,
but what does that mean on a probibility basis? In this case, they were projected to have a 5% chance of going 12-0,
so they did better than 95% of the simulations, and equalled 5% of them. For the sake of this calculation, that would mean that
they finished at the 97.5% percentile (95% + 5%/2). The actual number was 97.6% since the 5% display was rounded, but I think you get
the general idea.
A full list of how each team did compared to their probability distributions is below:
| Team | E(wins) | Actual | Diff | Percentile |
| Kansas State | 4.71 | 10 | 5.29 | 99.0% |
| Arkansas State | 4.66 | 10 | 5.34 | 97.9% |
| Louisiana State | 7.93 | 12 | 4.07 | 97.6% |
| Baylor | 4.50 | 9 | 4.50 | 96.8% |
| Houston | 8.47 | 12 | 3.53 | 96.1% |
| Iowa State | 2.47 | 6 | 3.53 | 95.1% |
| Wyoming | 4.67 | 8 | 3.33 | 94.3% |
| Michigan | 6.24 | 10 | 3.76 | 93.5% |
| Georgia | 6.25 | 10 | 3.75 | 93.5% |
| Louisiana Tech | 4.94 | 8 | 3.06 | 90.1% |
| Louisiana-Lafayette | 4.84 | 8 | 3.16 | 89.0% |
| Marshall | 3.23 | 6 | 2.77 | 88.9% |
| Utah State | 4.11 | 7 | 2.89 | 88.6% |
| Colorado | 1.27 | 3 | 1.73 | 88.5% |
| Oklahoma State | 8.48 | 11 | 2.52 | 88.4% |
| Rutgers | 4.91 | 8 | 3.09 | 87.6% |
| Eastern Michigan | 3.23 | 6 | 2.77 | 87.5% |
| Clemson | 6.03 | 9 | 2.97 | 87.2% |
| Southern California | 7.37 | 10 | 2.63 | 86.1% |
| Vanderbilt | 3.82 | 6 | 2.18 | 85.5% |
| North Texas | 2.73 | 5 | 2.27 | 85.1% |
| Toledo | 5.54 | 8 | 2.46 | 83.7% |
| Michigan State | 8.14 | 10 | 1.86 | 83.3% |
| Western Kentucky | 4.95 | 7 | 2.05 | 82.0% |
| Temple | 5.81 | 8 | 2.19 | 79.8% |
| California | 5.18 | 7 | 1.82 | 79.1% |
| New Mexico State | 2.58 | 4 | 1.42 | 78.3% |
| Virginia | 6.11 | 8 | 1.89 | 77.7% |
| Stanford | 9.26 | 11 | 1.74 | 77.1% |
| Florida International | 6.00 | 8 | 2.00 | 76.7% |
| Wake Forest | 4.46 | 6 | 1.54 | 75.8% |
| Wisconsin | 8.36 | 10 | 1.64 | 75.8% |
| San Diego State | 6.42 | 8 | 1.58 | 75.5% |
| Tulsa | 6.50 | 8 | 1.50 | 74.2% |
| Northern Illinois | 7.26 | 9 | 1.74 | 74.1% |
| Washington | 5.62 | 7 | 1.38 | 74.1% |
| Virginia Tech | 9.45 | 11 | 1.55 | 73.6% |
| Cincinnati | 7.57 | 9 | 1.43 | 71.6% |
| Arkansas | 8.66 | 10 | 1.34 | 70.7% |
| Ball State | 4.67 | 6 | 1.33 | 70.5% |
| Southern Mississippi | 8.65 | 10 | 1.35 | 68.8% |
| Alabama | 9.95 | 11 | 1.05 | 66.3% |
| Louisville | 6.06 | 7 | 0.94 | 66.1% |
| Georgia Tech | 6.86 | 8 | 1.14 | 64.7% |
| Kent | 4.26 | 5 | 0.74 | 63.2% |
| Oregon | 10.16 | 11 | 0.84 | 62.5% |
| Iowa | 6.25 | 7 | 0.75 | 60.9% |
| San Jose State | 4.56 | 5 | 0.44 | 59.2% |
| Texas-El Paso | 4.71 | 5 | 0.29 | 57.4% |
| Penn State | 8.49 | 9 | 0.51 | 56.8% |
| East Carolina | 4.82 | 5 | 0.18 | 54.6% |
| Ohio | 8.40 | 9 | 0.60 | 54.1% |
| Northwestern | 5.83 | 6 | 0.17 | 53.1% |
| Miami (Ohio) | 4.07 | 4 | -0.07 | 51.3% |
| Purdue | 6.00 | 6 | 0.00 | 50.2% |
| Florida | 6.16 | 6 | -0.16 | 49.8% |
| Buffalo | 3.32 | 3 | -0.32 | 49.7% |
| Texas | 7.09 | 7 | -0.09 | 47.3% |
| Miami (Florida) | 6.22 | 6 | -0.22 | 47.1% |
| Boise State | 10.76 | 11 | 0.24 | 46.6% |
| Nevada-Las Vegas | 2.42 | 2 | -0.42 | 45.7% |
| Nebraska | 9.01 | 9 | -0.01 | 45.2% |
| Western Michigan | 7.15 | 7 | -0.15 | 45.2% |
| Illinois | 6.33 | 6 | -0.33 | 45.2% |
| Bowling Green State | 5.43 | 5 | -0.43 | 44.4% |
| South Carolina | 9.92 | 10 | 0.08 | 44.1% |
| Texas Christian | 10.04 | 10 | -0.04 | 43.4% |
| Syracuse | 5.59 | 5 | -0.59 | 42.5% |
| Brigham Young | 9.22 | 9 | -0.22 | 41.1% |
| Memphis | 2.86 | 2 | -0.86 | 39.5% |
| Connecticut | 5.68 | 5 | -0.68 | 39.4% |
| Auburn | 7.62 | 7 | -0.62 | 38.1% |
| Alabama-Birmingham | 3.97 | 3 | -0.97 | 37.7% |
| Southern Methodist | 7.61 | 7 | -0.61 | 37.6% |
| Pittsburgh | 6.78 | 6 | -0.78 | 37.5% |
| Rice | 4.93 | 4 | -0.93 | 37.1% |
| Mississippi State | 6.78 | 6 | -0.78 | 36.8% |
| Washington State | 5.02 | 4 | -1.02 | 34.6% |
| West Virginia | 9.56 | 9 | -0.56 | 34.5% |
| North Carolina State | 7.88 | 7 | -0.88 | 34.0% |
| Utah | 8.13 | 7 | -1.13 | 30.9% |
| Oklahoma | 9.72 | 9 | -0.72 | 29.8% |
| UCLA | 7.30 | 6 | -1.30 | 29.2% |
| Missouri | 8.19 | 7 | -1.19 | 28.0% |
| Nevada | 7.96 | 7 | -0.96 | 27.5% |
| North Carolina | 8.43 | 7 | -1.43 | 25.9% |
| Tulane | 3.79 | 2 | -1.79 | 25.5% |
| Arizona | 5.91 | 4 | -1.91 | 22.2% |
| Notre Dame | 9.38 | 8 | -1.38 | 22.0% |
| Colorado State | 5.05 | 3 | -2.05 | 22.0% |
| Akron | 2.99 | 1 | -1.99 | 20.9% |
| Boston College | 6.05 | 4 | -2.05 | 20.6% |
| Kentucky | 6.91 | 5 | -1.91 | 20.5% |
| Minnesota | 4.90 | 3 | -1.90 | 19.2% |
| Oregon State | 5.31 | 3 | -2.31 | 19.0% |
| Arizona State | 8.01 | 6 | -2.01 | 18.3% |
| Florida Atlantic | 3.16 | 1 | -2.16 | 17.8% |
| Louisiana-Monroe | 6.28 | 4 | -2.28 | 16.4% |
| Duke | 5.37 | 3 | -2.37 | 15.7% |
| Army | 5.64 | 3 | -2.64 | 15.6% |
| Texas Tech | 7.32 | 5 | -2.32 | 15.2% |
| Air Force | 8.71 | 7 | -1.71 | 14.6% |
| Fresno State | 6.75 | 4 | -2.75 | 14.4% |
| Navy | 7.60 | 5 | -2.60 | 12.7% |
| Idaho | 4.61 | 2 | -2.61 | 12.6% |
| Central Florida | 7.68 | 5 | -2.68 | 11.7% |
| Florida State | 10.19 | 8 | -2.19 | 11.4% |
| Texas A&M | 8.73 | 6 | -2.73 | 11.1% |
| Kansas | 4.62 | 2 | -2.62 | 10.9% |
| South Florida | 8.02 | 5 | -3.02 | 10.8% |
| Hawaii | 8.95 | 6 | -2.95 | 10.5% |
| Tennessee | 7.85 | 5 | -2.85 | 9.0% |
| Maryland | 5.15 | 2 | -3.15 | 8.7% |
| New Mexico | 3.53 | 1 | -2.53 | 8.3% |
| Middle Tennessee State | 5.46 | 2 | -3.46 | 8.2% |
| Central Michigan | 6.64 | 3 | -3.64 | 7.9% |
| Indiana | 4.00 | 1 | -3.00 | 7.5% |
| Mississippi | 5.79 | 2 | -3.79 | 5.1% |
| Troy | 7.56 | 3 | -4.56 | 3.8% |
| Ohio State | 10.20 | 6 | -4.20 | 2.7% |
You may have noticed that the win distribution probabilities don't exactly correlate to the difference between actual and expected
win totals. As an example of why, consider two teams (team A and team B), each projected at exactly six wins. Team A has every
game rated an absolute toss-up, while team B has six games rated at a 90% chance of winning and another six games rated at only a
10% chance of winning. In each case, the win expectation is six, but the probability distributions are much different. For team A,
nothing between 10-2 and 2-10 would really be outside the realm of reasonable likelihood, while for team B, 8-4 or 4-8 would be extremely
surprising. So in that case a 9-3 record for team A would actually be considered a less extreme result than an 8-4 record for team B.
Similar types of situations occur in the below table, though not to the extreme example as just described.
I've spent a bit of time looking through the teams who finished especially well or poorly and haven't found any particularly meaningful
tendencies on either end. If anyone has any good ideas I can dig into them a bit, but for now at least I haven't seen anything obvious
that I'm missing, other than "replacing a top ten coach with a guy who has zero coaching experience" variable, and I'm guessing that situation isn't
going to come up often enough to be useful going forward.
One other thing that I'm going to be spending some time trying to do is to tighten the win probability distribution bands. Right now the model is accurately
reflecting what it sees as "noise", but at least some of that is data that it either hasn't seen or doesn't understand. My hope is that next season
I can get a similarly accurate depiction of the limits of its knowledge, but with bands that are at least a bit less wide than I had to use in 2011.
There are a few important notes and caveats I need to make about this model:
1) Compu-Picks does not endorse implicitly or explicitly any form of illegal gambling.
Compu-Picks is intended to be used for entertainment purposes only.
2) No guarantee or warranty is offered or implied by Compu-Picks for any information provided and/or predictions made.
2011 Compu-Picks Blog
Questions, comments or suggestions? Email me at cfn_ms@hotmail.com
