In November I made note of this post, Jeter Uncertainty Principle, which linked to a Baseball Prospectus article that wondered if the batted ball fielding systems were overestimating the range of players.

the spread of observed performance in metrics like UZR and DRS is much, much smaller than that of metrics like nFRAA (or Tom Tango’s With Or Without You system, which is similarly down on Jeter’s fielding ability).

This got me thinking, but unfortunately my computer crashed and I lost my MSSql database with my fielding data. I recently downloaded the retrosheet data through 2009, and decided to do an experiment.

The Probabilistic Model of Range uses six parameters to determine the probability of a ball in play being turned into an out. Three of those, the direction, velocity, and batted ball type, are subjective measures. Three of them, the handedness of the batter, the handedness of the pitcher, and the park are objective. If I used just those three objective measures to construct the model, would I see a bigger spread in the data?

I last computed PMR in 2008, and here is the team listing from that year. I built the retrosheet model a bit differently. I only use visiting fielders so a great or terrible home fielder doesn’t over influence the model, but I used a different set of data to build the model. Instead of using the same year, I used the previous three seasons and the following season. So for 2008, I used the 2005, 2006, 2007 and 2009 seasons. Here are the results for the teams:

The number of balls in play do not match up precisely between BIS and Retrosheet, and I’m exploring why (in the case of Baltimore, it was a foul ball error). The ordering is different, which isn’t surprising given the different model and the way the model was built. What I would like you to notice, however, is that the spread of the index is indeed wider than the original PMR model shows. So a probabilistic model that uses only objective parameters also shows a larger spread.

I’m going to try to get more comfortable with the data, making sure the differences are just foul pops. Then, I’ll build some positional models to see what happens there.