Compared to What?

As some of you may know the Seahawks super bowl opponents, the New England Patriots, have been accused of cheating by deflating footballs during last week’s AFC championship game against the Indianapolis Colts. Under inflation allegedly gives the quarterback a better grip on the ball. The primary evidence against the Patriots is a report that, when inspected during or after the game, eleven of the twelve balls provided by the Patriots were significantly underinflated.

This report has led to a lot of speculation and accusation based on the “evidence” of eleven out of twelve.

But what does this evidence actually show?



Because, as the great statistician Edward Tufte wrote, “At the heart of quantitative reasoning is a single question: Compared to what?”
Whether eleven of twelve (91%) is a lot depends entirely on what you are comparing it to. A lot more than eleven of every twelve soldiers returned from Vietnam but does not mean that our casualties were light. People who drive while drunk survive a lot more than eleven out of twelve times but that doesn’t make drunk driving safe.

One of the ways people lie with statistics is by presenting them as if they were self-referential. “I mean, I could see if it were three or four balls but eleven out of twelve? C’mon somthin’s gotta be goin on there”

Maybe or maybe not.

We know that inflated things (car tires, footballs, air mattresses) tend to deflate over time. We can see the punishment a football takes during a game. Perhaps eleven out of twelve balls are generally deflated by the end of a game. Of course we can’t go back and collect data from games past to find the average deflation of a game ball but in this case that wasn’t necessary.

It turns out that each team provides game balls. The key piece of evidence in this case is the inflation of the Colts game balls. If eleven of the twelve Colts balls were deflated after the game then the state of the Patriots balls is just the state of game balls after a game. If only one or two Colts balls are deflated then we have pretty strong evidence that something was done to the Patriots balls.

If the NFL was able to measure the Patriots game balls surely they could have measured the Colts balls as well, compared the two sets of balls and had very good evidence from which to draw conclusions. So far it doesn’t look as though they did this and, as a result, we have only the appearance of evidence; heat but no light.

The take away is: Every time you see a statistic you should ask yourself “Compared to what?”

6 responses to “Compared to What?

  1. Anonymous

    I wonder if one of our science peeps could come close to determining how much a ball would deflate being in cold weather for the entirety of the game? I’m sure there is information about the temperature and the length of the game. Of course there are more variables than that, but if that is the baseline, we might know if ball deflation is a normal phenomenon.

  2. Anonymous

    Will the Patriots be accused of inflating the balls in that hot AZ weather?

  3. Qualitative data submitted from the field from multiple expert observers first discovered and reported this deflated football data through proper data collection and investigation protocols that are evenly available to both teams are designed to assure greater objectivity and impartiality.

    One team exercised this open and available (even encouraged) opportunity to provide this sort of quantitative data reporting opportunity and presented a testable hypothesis based on their focus group of expert analysis; the other team did not. The hypothesis (“Patriot’s footballs had less air pressure than the league rules allow”) was objectively tested and proven to be true 92% of the time in the testing sample of 12 via reliable and valid measurements.

    Compare these statistical facts: Teams who exercised equal and encouraged opportunities to report qualitative data about potential rule violations based on expert field analysis which was substantiated with objective verification through valid and reliable measurements:


  4. Correction: 2nd Paragraph in last post should have read “qualitative”, not “quantitative”.

    The statistical comparison is not to the Colt’s footballs, but rather to the Patriot’s same footballs before the game. The air pressure was tested before the game and again at halftime (PRE-POST MEASURE) that firmly established the discrepancy from approved (legal) to unauthorized (illegal) footballs.

    The Colts don’t need to prove their innocence as no charges have been levied against them, but the Patriots certainly have some explaining to do.

    Until then, only one team has been factually and determined with confidence beyond dispute to be playing with illegal footballs. How they got caught with illegal footballs? Tell it to the judge.

    “Gerry Austin, longtime referee, says halftime Pats-Colts footballs brought in, checked at half. Colts footballs still legal. Pats were not.”


    GO HAWKS!!!!

    • The issue here is not the Colts innocence or guilt but the need for a control group. It is possible that game conditions will generally reduce ball inflation to levels below those allowed for by the rules. This is exactly the approach the Patriots took when protesting their innocence. They claimed to have subjected balls to game like conditions and to have discovered “natural” deflation. This was also the approach taken by much of the media when looking for explanations in physics and engineering. The point of examining the Colts balls is that if their balls were also deflated then there would be a good case for external environmental conditions as the cause of the deflation. If the Colts balls were not deflated, despite being subject to exactly the same environmental conditions, then the case for Patriot malfeasance is supported.

  5. Louis

    Subjectively I almost got mad at this post. Objectively though you are completely right. Good spin on empirical deduction.