This just makes me believe we are still not close with defensive metrics. The Yankees simply are not a good fielding team.

Would love to see how these match up with Tango's "Scouting Report"

This just makes me believe we are finally close with defensive metrics. The Yankees simply are a good fielding team.

(After years of ineptitude, of course.)

This just makes me deeply unsure whether we are finally close with defensive metrics. After all these years of reading about Derek Jeter's shortcomings on this blog, I'm just not convinced the Yankees are a good fielding team.

I still find it difficult to accept that all of the variables that go into fielding can be reduced to a formula. With all respect, I suspect the results are a little like the SAT scores, which really only measure how well a kid does on SAT exams, a measurement that correlates only roughly to performance in college. In something of the same way, these stats really only measure how well a team does by various measures that are inconsequential in themselves, and it's a leap of faith - how much of a leap is the question - whether that has any correlation to the actual manufacture of outs.

The teams at the bottom had pretty bad years, which suggests some level of correlation. But two and a half (the Padres) playoff teams are in the second half of the list, which makes me a bit dubious.

I think it's funny that faced with David's analysis of reams of data, the main response is that since it doesn't match up with our impressions, it's the analysis that must be wrong! Come on. Look at the methodology and say what's wrong with the method, or else have some integrity and admit that your *impressions* were mistaken.

Look, Jeter's range up the middle is not good. But Cano and Rodriguez were great, and the Yankees' OF (esp. Cabrera) were excellent. You can probably find some details of the Probabilistic Model of Range to quibble over, but even haters should admit that the Yankees were *among* the top-fielding teams last year.

At the team level, this model concludes that the difference between the best fielding team and the worst fielding team was about 0.85 plays a game (measured against their predicted plays). What it really suggests is that there is really very little overall difference between teams, and that it should difficult to tell them apart based on observation.

David - how do park factors come into play with this? Could the high grass at Yankee Stadium be helping the stats for the Yankees? They do play 81 games there. I've always suspected that the grass helps Wang's home/road splits. Could it be helping the Yanks, as a team, here too?

Yes, park is a factor.

So this rates the Yankees as the 13th best real-life fielding team in baseball, when they were on this model expected to be the 27th "best."

So they were most successful team in "lucking" into outs.

Big deal! All you Yankee suck-ups need to reread the table - or else realize you're beating your chests about outperforming analytical expectations simply in order to become average.

Woo woo - Yankees are average, rock on!!!!

Hey wait a second, is it any coincidence that the Big Three are the top three here. Any chance the Yankers, Bosux, and Cubs are simply benefitting from bad umpiring here - seems like those three teams get more calls going their way than any other teams in baseball anyway. A call a game and boom your actual DER outperforms the predicted.

When is anyone gonna publish umpire evaluations of this nature?

Ok maybe not a call a game but bad calls even at a lower rate could still explain the discrepancy que no.

I don't think it's hard to believe the results from the model - IF you are willing to accept the underlying assumptions surrounding "predicted DER".

The Yankees gave a lot of innings to guys who were pretty hittable: Igawa, Clippard, Henn, Farnsworth, DeSalvo, Rasner. Clemens is showing signs of being near the end of the road, which is reflected in his K rate (by far the lowest of his career); it's not hard to believe that batters are making contact more consistently and with more authority against him. Ditto Pettitte, ditto Mussina.

Freddie, you didn't understand the chart. The predicted DER of .674 for the Yanks implies that that's what an average team of fielders, given the Yankees context (batted ball distribution, park, etc) would have achieved. Basically, it's real hard to get an out with the Yankees pitchers and playing half your games at Yankee stadium.

Anyway, that's what PMR is saying. Whether that's true is another question.

"I think it's funny that faced with David's analysis of reams of data, the main response is that since it doesn't match up with our impressions, it's the analysis that must be wrong!"

Maybe the impressions are right. PMR is not the only pbp defense analysis out there. When the advanced metrics don't agree then we either have to pick one and trust it implicitly, or go by our impressions.

For the Yanks, here's how certain players did on the John Dewan plus/minus leaders and trailers:

Jeter -34
Melky -22
Abreu -12

Cano +17

If I assume the maximum possible rating for the others: A-Rod didn't make the leader or trailer list, so if the #10 3B is +7, then A-Rod could be at most a +6. So:

A-Rod +6
Damon +7
1B +1

That's -37 for the starters, so plus minus does not see the Yankees as the best defensive team, or anywhere near it. I'm pretty sure UZR did not either.

If your inititial impression was that the Yankees are poor defenders, then you should stick with it unless you have some reason to believe PMR is more accurate than the other systems.

Well, for what it's worth I think PMR does a much better job at taking the actual distribution of balls in play allowed by a team, and modeling defensive expectations against that distribution instead of against a league-average distribution, than either plus/minus or UZR do.

Rally, I have no problem with bringing in alternative measures to contest David Pinto's. I was laughing at the idea that one's impressions are a good rebuttal -- that's ludicrous, it's Joe Morganesque.
So, let's ask what the Dewan measure is measuring, and what the Pinto PMR formula is measuring, and then decide which of those better gets at the true defensive ability of a team. (Keeping in mind JJ's point way up near the top of the comments, to the effect that there could just be several different things we could mean by 'defensive ability'.)

If a team's fielders see a lot of very difficult balls to field, should that go into the model? I would say it should. PMR is much more comprehensive than other models in incorporate the difficulty of batted balls in play. So I'll tend to trust PMR when it conflicts with other measures.

I'd like to see the balls in air/balls on ground distributions. In the absence of hang-time measurements, the ball-in-air type decision is quite subjective and of significant importance.

I am pretty sure that the Yankee infield was far from the best in the league, but as for the outfield, it could have been quite good. There were two centerfielders out there.

This is the "visiting player model". It may be that Yankee opponents fielded fewer balls in play than anticipated, in part because of the park but also because of the quality of the Yankee offence. It will be interesting to see what the results are using the other models.

.85 plays per game is definitely a big deal -- that's about .7 runs per game. If league-average ERA is 4.50, that bumps the best teams up to a 4.15 ERA and the worst teams down to 4.85. Sure, that's not as big a difference as hitting or pitching, but I think we already knew that. Defense is important.

Rally,

Melky -22? I'll have to run CF first when I do players. I find that tough to believe. Melky ranked very high last season in PMR among LF. I suppose he could be lost in CF, but he didn't look that bad.

It is a little hard to believe, but here it is:

http://www.billjamesonline.net/fieldingbible/2007-plus-minus-leaders.asp

Mike Green,

I see a correlation of 0.5 between difficulty of a team's fielding chances and that team's batters out percentage (1-babip). You should expect some correlation because of the ballparks, whether a .5 is too high I don't know.

Interesting model, Mr. Pinto. I'd love to get a look at the data, but I suppose you've got to pay BIS some good cash for that. :-) Mostly, I'd like to see how standardized the classifications for "line drive"/"fly ball"/"etc. and "slow"/"medium"/"fast".

As for some reasons for the differences from these results and those of the plus/minus:

Since plus/minus measures plays made vs. "expected" plays made, a LF with very good range may take a few fly balls that the CF might otherwise get to. Presumably, Damon's presence in LF could account for some of Melky's low score. (OTOH, this would also mean that Damon should have a similarly higher score, which isn't something we saw.)

"Since plus/minus measures plays made vs. "expected" plays"

Its PMR that compares plays made to expected plays. +/- assigns a positive vlaue for making a play and a negative value for not making it, with the magnitude of that value depending on how difficult a chance was.

With PMR, you may see some instances of a player inflating their rating through ball-hogging. In prior year Orlando Hudson did this by grabbing every infield pop fly. But that doesn't matter for team ratings, for the Yankees it really doesn't matter if Melky or Damon makes the catch, as long as it was made.

Does this mean that a team like the Angels had a staff that was one of the easiest to hit or that they play in a relatively easy hitters park or a little of both? Their park factor this year was 105, but I'm not sure how much of that is balls in play and how much is other factors.

Angel hitters hit .337 at home on balls in play, and .294 on the road. The pitchers allowed .260 at home and .273 on the road.

The park didn't change or anything, it hasn't been a hitters park in the past, but for some reason Angel hitters went wild at home this year.

Jon, that's something I've tried to tease out, but I haven't come up with a good way to do it. I think it is a combination of both.

I think Mike Green and Rally have identified a serious problem with the visitor-dominated model: it effectively lowers the defensive bar for teams whose hitters have a high BABIP. I sorted the teams by predicted DER, and the 8 teams with the lowest predicted DER -- i.e. the teams whose pitchers ostensibly allowed harder to field balls -- had an above-average BABIP in every single case. The 8 teams with the highest predicted DER look just the opposite: 6 of the 8 have below-average BABIP, 1 was average (NYM), and just 1 was above average (CO).

The low predicted DER teams had an average BABIP of .313, while the high predicted DER teams were just .295 (MLB average was .303). All figures from B-Ref.

This can't be a coincidence. What is likely happening is that the model, looking at the experience of visiting fielders in Anaheim trying to stop the rockets hit by Angels hitters (.315 BABIP), concludes "boy, it's hard to turn balls into outs in this park." But down the street in S.D. (.291), or up the road in SF (.281), it looks mighty easy to field those same balls.

Well, Guy, I suggest you take this up with the people who insisted that including the home team polluted the model too much.

I can appreciate you may feel a bit whip-sawed by contradictory criticisms, David. But I think both approaches pose real problems. The answer may lie in other directions. For example:

1) use multi-year data to create the model;

2) control for quality of home team hitting -- if visiting fielders turn a given ball in Anaheim into an out at a .72 rate, and the Angels hitters are .020 above-average on BABIP, then the expected DER is .74 rather than .72 on that ball.

3) Use park-specific parameters for the OF, but not the IF (assuming you use them for IFs). I could be wrong, but with turf gone it seems infields are similar enough that any park factor you use probably adds more bias than it removes. This would give you much more robust samples for all IF BIP, and reduce the impact of having an especially good or bad group of home hitters in a ballpark.

Mike Green, clearly Rally wasn't going on the general "impression" that Jeter sucks really bad at shortstop. The general "impression" has him winning gold gloves and considering him good. It is the defensive metrics that have pointed out that he's a joke. Dewan's +- system puts him as the 2nd worst defender in the league after Manny, and those two are blowing away the competition in being atrocious. Maybe you consider it common knowledge that Jeter's a horse's ass in the field because you're knowledgeable about this stuff and discuss it with other such informed people. But really you or me or whoever likely haven't just guesstimated that Jeter's worthless out there. We know it from reading the analysis.

If the Yankees are actually interested in being, like, really good, why have Jeter at ss and weak-hitting plus-defense firstbaseman Doug M??? A sane thing to do would be to play Jeter's broke-ass at 1st base and get a slick fielding ss- the premium defensive position.

Also, regarding infields. It made me curious when I heard the Colorado had one of if not THE best fielding groups of infielders, and that Colorado, to help diminish the effect of the hitter-favored park, now has one of, if not THE longest infield grass in baseball.

Does adjusting for park eliminate this long grass effect?

Overall, there are 33 more actual outs than predicted outs. Is that just accumulation of rounding errors? The presentation of the difference between actual and predicted DER to 5 decimal places implies that the maximum discrepancy from rounding error should be about 1 when adding up 133,000 balls in play. I believe I suggested in last year's discussion that the visiting team model would lead to a net positive outcome, since (for whatever reason), one symptom of the home field advantage is that home teams' batted balls are hits more often; primarily using the visitors' fielding sets a baseline slightly below the overall (home+visitor) average.

I think Mike, Rally, and Guy's critique amounts to saying that if a team hits particularly well at home (on balls staying in play), then this lowers the visitor's performance and the lowers the expectation for the home team's defense at home, making it easier for them to exceed expectations. In road games, the team's offensive performance will have far less to do with the baseline used to measure its defense. So home and road PMR splits for teams may shed some light on how much of an issue this is. [I calculated a correlation of -0.36 between teams' BA on BIP specifically at home and their overall predicted DER; the top 5 teams in BABIP at home averaged a predicted DER of .679; the bottom 5 teams in BABIP at home averaged a predicted DER of .692. Obviously 1 year of data doesn't prove anything, but it does support their hypothesis.]

With only one year of data used to construct this year's model, there's still a lot of room for luck in setting the baselines at each park. I do expect that this is significant, in comparison to the relatively small margins you currently find between the teams.

Multi year data would help with that, as Guy suggested. Before 2006 you used multi-year data in constructing the baselines for PMR, and I believe you changed to a one year basis last year (because a change in the data feed you received from BIS made prior years' data incompatible). Do you have a technical reason for excluding 2006 data this time around, or have you changed your mind about the propriety of a multi-year baseline?

Brittainator, the park is figured into every set of parameters, so if long grass helps defense, that should fall out in the model.

Joe Arthur, there is year to year line drive variation, and that ends up making everyone in a season positive or negative. If 2007 was a better defensive year than 2006, everyone looks good in 2007.

David,

can't you use single year values for hit types and multi-year for parks?

Say a line drive at location X is fielded 17% in 2007 and 15% in 2006, a multiyear park factor of .95, just multiply the year values by the park factor.

Coming into this late, but isnt the park factored into the model simply by using the park average out for each play each single year, and not using a league wide average out on each play each single year. So how is David using mulityear park factors with single year values for hit type?