Baseball Musings
Baseball Musings
November 29, 2003
Significance

I must be tired lately, because it's taken me a few days to get a handle on what's bothering me about productive outs. Here's the line that's bothering me:


Base on balls are a fundamental piece of the Athletics' offensive philosophy, but statistically, they have shown to have slightly less significance than Productive Outs in the post-season.

That's a very misleading line. Significance is in the eye of the beholder. When we talk about significance, we're talking about the probability of something being very low; how low is up to the person studying the data, but most people look for a probability under .05 for something to be significant.

So let's do a thought experiment. According to the article, of 130 series in which one team made more productive outs than the other, the team with more productive outs won 62.3% series, or 81 total. We can define this as a Bernoulli random variable; it has the value 1 if the team which wins the series has more productive outs, 0 if the team which wins the series has fewer productive outs. Now, imagine a bag filled with balls labeled 1 or 0 in the proportion 81 1's and 49 0's. Taking a ball from the bag is a Bernoulli trial. If we do this many times (replacing the removed ball each time), the probability of getting a certain number of 1's in a certain number of trials is given by the binomial distribution.

With the binomial, we can ask questions like, "What is the probability of getting exactly 78 balls with a 1 if I make 130 trials," or, "What is the probability of getting at least 85 1's in 130 trials." But more importantly, we can ask, if I repeatedly sample 130 balls from the bag, what range will the result be in 95% of the time? To be clear, here's the experiment:

Perform 130 Bernoulli trials with our bag of balls. Record the number of balls labeled with a 1. Repeat this experiment thousands of times. Make a histogram of the results.

The histogram will look like a normal distribution (and in fact, for a large number of trials, it can be approximated with the normal) with the highest bar on 81, the mean. The height of each bar of the histogram represents the probability of getting that number of 1's. At 81, the height is .072. If we sum the height of the bars around the mean until we get .95, we've found the range where we expect 95% of the results to be. For this distribution, with 130 trials, that range is 69-91.

Now, according to the article, more walks win a series 60% of the time, or 78 out of 130. Seventy-eight is well within our 69-91 range of 95%. So walks are not less significant than productive outs. The fact is, this difference could easily be sampling error. If we did another trial, we might get 75 wins on productive outs and 85 wins on walks, and we still couldn't tell if they came from the same distribution or not.

The sample size is too small. They have not shown there is any significant difference in any of the stats they mention.


Posted by David Pinto at 08:43 AM | Statistics | TrackBack (0)