As some of you may know the Seahawks super bowl opponents, the New England Patriots, have been accused of cheating by deflating footballs during last week’s AFC championship game against the Indianapolis Colts. Under inflation allegedly gives the quarterback a better grip on the ball. The primary evidence against the Patriots is a report that, when inspected during or after the game, eleven of the twelve balls provided by the Patriots were significantly underinflated.
This report has led to a lot of speculation and accusation based on the “evidence” of eleven out of twelve.
But what does this evidence actually show?
Because, as the great statistician Edward Tufte wrote, “At the heart of quantitative reasoning is a single question: Compared to what?”
Whether eleven of twelve (91%) is a lot depends entirely on what you are comparing it to. A lot more than eleven of every twelve soldiers returned from Vietnam but does not mean that our casualties were light. People who drive while drunk survive a lot more than eleven out of twelve times but that doesn’t make drunk driving safe.
One of the ways people lie with statistics is by presenting them as if they were self-referential. “I mean, I could see if it were three or four balls but eleven out of twelve? C’mon somthin’s gotta be goin on there”
Maybe or maybe not.
We know that inflated things (car tires, footballs, air mattresses) tend to deflate over time. We can see the punishment a football takes during a game. Perhaps eleven out of twelve balls are generally deflated by the end of a game. Of course we can’t go back and collect data from games past to find the average deflation of a game ball but in this case that wasn’t necessary.
It turns out that each team provides game balls. The key piece of evidence in this case is the inflation of the Colts game balls. If eleven of the twelve Colts balls were deflated after the game then the state of the Patriots balls is just the state of game balls after a game. If only one or two Colts balls are deflated then we have pretty strong evidence that something was done to the Patriots balls.
If the NFL was able to measure the Patriots game balls surely they could have measured the Colts balls as well, compared the two sets of balls and had very good evidence from which to draw conclusions. So far it doesn’t look as though they did this and, as a result, we have only the appearance of evidence; heat but no light.
The take away is: Every time you see a statistic you should ask yourself “Compared to what?”