by Kevin Harlow
February 23, 2003
It's 40 games into the season and your favorite team is tearing the league up with a 28-12 record. Are they really a .700 team? Probably not, a team's record is a combination of skill and luck. Although you may be tempted to develop creative ways to convince yourself that they are in fact a .700 team, that level of ability is a historical rarity. Unfortunately it is much more likely that the team has been a bit lucky, albeit still a good team. So how good is your team? Equivalently, without major changes to your team, what can you expect your team's winning percentage to be over the course of the rest of the season?
The general concept for solving the problem is that the estimation of a team's ability requires the resolution of two competing factors. First, MLB teams are approximately normally distributed around the league average .500 winning percentage. In other words, most teams are close to .500 and there are fewer and fewer teams as you get further away (either above or below) from .500. Second, there is a certain element of luck in winning games, which is also approximately normally distributed, but around 0. The best estimate is the most probable combination of ability and luck that would result in the observed winning percentage.
Your team with the 28-12 record may be a .480 team that has gotten incredibly lucky. Although the probability of a team having a .480 winning percentage is pretty high, the chances of having such a large amount of luck required to get a .700 winning percentage over 40 games (with .480 talent) is extremely low. The combined probability of ability and luck is low. Similarly, they may be a .750 team who has been unlucky! In this case the chance of experiencing that amount of luck is reasonably high, but the chances are close to zero that they are truly a .700 (or .750) team. Most likely they are a good team, perhaps a .600 team, that has gotten a little bit lucky.
We need an equation to calculate the team's true talent level. The first step is to define the total variance in winning percentages to be a combination of the variances of luck and ability.
VAR(Total,G) = VAR(Luck,G) + VAR(Talent)
VAR(Luck,G) = VAR(Binom,G) + VAR(Misc,G)
VAR(Total,G) = [VAR(Binom,G)+VAR(Misc,G)] + VAR(Talent)
Some definitions might be helpful!
P = talent level, or Winning% for ROS
G = Number of games played
Y = Winning% in G games
VAR(Luck,G) = Variance of winning% due to luck
VAR(Talent) = Variance of winning% of MLB teams due to differences in ability
VAR(Total,G) = Total combined variance of winning percentage
VAR(Binom,G) = Variance of winning percentage due to binomial distribution
VAR(Misc,G) = Variance of winning percentage due to miscellaneous sources (not binomial or talent)
The team's true talent level, P, is estimated from the following equation:
P = [.5*VAR(Luck,G) + Y*VAR(Talent)] / [VAR(total)]
Notice that the higher the variance due to luck the more you regress the team's current winning percentage to the mean of a .500 team. For example, you would regress a team's winning percentage over 5 games much more than you would over 50 games because there is much more luck involved in just 5 games.
We'll calculate the above "Luck" terms as follows:
VAR(Binom,G) = P*(1-P)/G
VAR(Misc,G) = a/G
where "a" is a constant. This leaves us with four unknowns - P, VAR(Total,G), "a", and VAR(Talent). The calculation of these values requires real game-by-game results such as those that you can get at Retrosheet. The data set that I am using includes all the games from 1980-2001, excluding the strike-shortened seasons of 1981, 1994, and 1995.
In order to estimate 'a' and VAR(Talent), I calculated, using the above data, the average for the variance of the winning percentages over a certain numbers of games and eyeballed a best fit with guesses of 'a' and VAR(Talent). For instance, in the table below, the VAR(Total) for 40 games is the average of the variance of the winning percentages over games 1-40, games 41-80, games 81-120, and games 121-160. Similarly, the VAR(Total) for 20 games is the average of the variance of the winning percentages over games 1-20, games 21-40, games 41-60, ..., games 141-160.
#G VAR(Total) EstVAR(Total) 160 0.00439 0.00439 80 0.00602 0.00598 40 0.00911 0.00915 20 0.01539 0.01550 10 0.02820 0.02820
EstVAR(Total) was calculated using the following equation while varying 'a' and VAR(Talent) until good estimates were produced across the full range of number of games.
EstVAR(Total,G) = (.5*.5)/G + a/G + VAR(Talent)
My "eyeballed" best fit for the parameters were 'a'=0.004 and VAR(Total)=0.0028. Substituting these numbers into the equation
P = [.5*VAR(Luck,G) + Y*VAR(Talent)] / [VAR(Luck,G) + VAR(Talent)]
gives
P = [.5*(P*(1-P)/G + 0.004/G) + Y*0.0028] / [P*(1-P)/G + 0.004/G + 0.0028]
The team's true ability, or winning percentage for the rest of the season is the solution to the cubic equation above. (Tip: If you are using Excel, go to Tools-Options-Calculations and mark iterative calculations and then enter the above equation for P as a circular reference.)
To test the accuracy over a wide range of games and winning percentages, I regressed the average rest-of-season winning percentage against P for each set of games and wins, where the number of games is divisible by 5 and is between 19 and 121. In other words, for each number of games played 20, 25, 30, ..., 120, I averaged the winning percentage over the rest of the season for (20 G, 10 W), (20 G, 11 W), (20 G, 9 W), etc. The following linear equation was calculated using Excel:
ROS_W% = 1.01*P; R^2=0.788
The solution to the cubic P equation above does a reasonable job at predicting ROS winning percentages. Also, the variance in ability between MLB teams has been about 0.0028 in the non-strike 1980-2001 seasons. Thanks to Retrosheet for being such a great historical resource, without whom none of the above analysis (except the errors!) would have been possible.
Go to Kevin Harlow's home page.