One of the seminal pieces for bringing advanced NBA stats into the daily conversation is Michael Lewis’s expose on Shane Battier. Although the piece touches on a variety of concepts relating to NBA analytics (advanced scouting reports, the shortcomings of the boxscore, counterpart statistics) the primary focus is the idea that there are a variety of contributions that are not measured in the box score but help a team win. Among those are plus minus all-stars like Battier, Nick Collison and recently Epke Udoh.
On first blush, there’s an appeal to this idea. Battier and Collison are clearly valuable players for reasons not entirely evident in their individual stats. They both play tenacious defense in limiting their opponent’s production. Unfortunately, I think Lewis vastly overstated the case for plus/minus and his article has forced very smart people to include it in analyses where other tools are better suited.
The intuitive case for counting stats
This is a fairly easy case to make. The best way to figure out if a player is helping his team win is to count up all the good and bad things that happen on a basketball court, assign values to those things, and go from there. Of course there are also fit considerations, but generally getting the ball, putting the ball in the basket, and helping a teammate put the ball in the basket will help the team.
The intuitive case for plus/minus is also somewhat persuasive. After all, how better to measure how much a player “has a magical ability to help his team win” than count the number of points the team scores and allows while on the court? Then you could adjust that for the ratings of the teammates and the ratings of the opponents on the floor. The problem with this is that most of the data acquired has absolutely no relation to player quality. When a center scores in the post, the “magical ability” philosophy says a better defensive point guard would have done something to stop it. Occam’s Razor says that’s ridiculous. Let’s move on.
What are the lessons of Shane Battier?
The Lewis article attributes Memphis and Houston’s strong performance with Battier on the court to his defense. That is a sensible conclusion. However, it then acts like Battier’s defense is some weird intangible that can only be picked up by plus/minus. That’s where the whole case for plus/minus goes off the rails. The king of the boxscore himself concedes that the traditional boxscore does not really pick up on eFG defense. Additionally, it’s not like Battier’s defense was some sort of mystery before plus/minus. Kill lists and the eye test both tell us about his value at that end.
Secondarily, traditional box score metrics have underrated Battier because he takes almost 50% of his shots from three. There is a real offensive benefit to floor spacing that goes beyond the traditional usage and efficiency tradeoff. Has plus/minus really told us about Battier’s value? To me it tells us about the value of perimeter defense and three point shooting, something that can probably be better measured by counterpart stats (something relied on extensively in the Lewis article) and three point shooting, which can be better measured by counting three pointers. Really, plus/minus is more valuable in identifying classes of players that are incorrectly evaluated, and calibrating more precise tools to do the real evaluating.
Minimizing defense further
With the floor spacing rationale addressed and eliminated, let’s turn our attention to the alleged inability of the boxscore to evaluate defense. The biggest hole in the box score is figuring out how an individual player affects his team’s effective field goal percentage defense. This is clearly the most important part of defense, and generally the best defenses are better here than in the areas of not fouling and turning the opposition over. It’s very, very difficult to attribute a miss or a make to an individual player.
That said, I think this rationale for using plus/minus to play a major role in player evaluation dies through death by a few different cuts. Start with the Oliver concept that defensive eFG is 20% of the game (40% of defense). We know intuitively, anecdotally, and with some data that is interesting but should be taken with a shaker of salt, that coaches really impact defense. Defense is more of a team/scheme/effort game than offense.
The quality of the defenders does matter, but a good coach with players who care makes a huge difference. Someone like Thibodeau, Skiles or Brown can take what was once a poor eFG team and turn it into an above average to great team. That knocks out a solid amount of the value of individual evaluation here.
Next we know that certain statistics can tell us about defensive attributes which affect opponent eFG. Defensive rebounding correlates well with defensive eFG, and which lineups will rebound well can be predicted by how good the individual rebounders on the floor are. Blocks tell us a little bit about help defense, but I’ve been unable to find rigorous analysis. So defensive rebounding and blocks can help generally figure out who the better defenders are with a few exceptions (Troy Murphy).
Finally, opponent production stats and situational stats can fill the final gap, along with the eye test. Those all say that Collison and Battier are excellent players despite failing to show up in the primary box-score numbers.
I think the box score contains largely accurate credit apportionment for 80% of the action. For the 20% that is eFG defense, scheme and effort represent a good chunk of that. While the rest of the information is sketchy and sometimes difficult to come by, my guess is that 50-60% of a player’s defensive value can be approximated with existing individual stats or extensive game film. That means the legendary “black box” is much smaller than people imagine. Using total plus/minus, even when heavily adjusted, to plug this gap seems unlikely to get an analyst any closer to a true approximation of a player’s value.
The failure of plus-minus to identify anything novel
It’s now been three years since Lewis’s article blasted plus/minus to the forefront. Besides identifying Collison and Battier (two guys everybody knew were very good defenders), it really seems like plus/minus has failed to find other serious contributors. Part of this of course is that there is very little good analytical basketball writing in the public, but the fact that this supposedly powerful tool is basically limited to a footnote in everybody’s all-defense ratings is fairly conspicuous.
The test case here I suppose is Epke Udoh who notably has an awesome plus-minus despite looking like absolute garbage in every other category on the basketball court. If Udoh really keeps this up, then we may have something, though my impression is that his eye-popping number is more the result of him playing with other atrocious bigs and guys who don’t care about defense.
Why is this a discussion?
This isn’t to downplay the complexity of the game of basketball. Of course you can’t just throw the guys with the best PERs or Win Percentages together and win a championship. But I’d still contend that counting stats and the eye test are a better way to quantify fit and defense than plus/minus, and probably do a better job summarizing a player’s contribution to winning than anybody is really letting on.
It seems that guys like Hollinger and Pelton are downplaying the explanatory power of their boxscore statistics to appear reasonable and remain relevant. Since nobody cares about me though, I’m going to go ahead and say the black box that’s not explained by boxscore and advanced statistics is probably no more than about 5% of the game. Whether we are doing the explaining correctly is another matter for another post.