(Guest post by Greg Forster)
It may not be Super Chart! or Son of Super Chart! but this graph of fumble rates sure doesn’t make the Patriots look good. Jack Fowler breaks it down in The Corner:
Over the last five seasons, the average NFL team fumble-to-play ratio is 1 in every 50. The Patriots record is 1 in every 73. Why such a disparity? The obvious argument: Under-inflated balls are much easier for running backs to protect, and therefore less likely to be fumbled.
Yeeeeeeeeeeeeeeeeeeesh.
Jay P. Greene's Blog 
There’s so much wrong with this analysis. The Pats pass more, hence fewer fumbles because there is more open space for ball carriers to run instead of getting gang tackled in a cluster of bodies. The Pats score a lot, thus they score on more plays than most teams, hence fewer fumbles (you don’t fumble when you’re dancing in the end zone, that’s called spiking). The Pats, on average, win. You don’t fumble when you’re cruising to victory because you don’t have to play as risky. You call conservative plays, you throw the ball away, you take a knee.
Finally, and most importantly, fumbles aren’t driven by multiple data points distributed randomly across multiple players. In fact, most fumbles are committed by quarterbacks. They touch the ball on every play. Wanna take a guess as to how many QBs are in the top 20 in the 2014 list of most fumbles committed? The answer is 20. You don’t get to a WR until #21, and you don’t get to an RB until #24. Whoever Jack Fowler is (who spent the time to do this analysis), should have at least known that QBs commit most fumbles before concluding that the numbers are driven by “Under-inflated balls” which are “easier for running backs to protect, and therefore less likely to be fumbled.”
When you compare fumble rates, you aren’t really comparing one team to another, you are mostly comparing QBs, in this case Tom Brady to other QBs. It’s not surprising at all that one of the top QBs in the league, who regularly leads in completions, yards per attempt, and TDs per possession would, conversely, not regularly fumble the ball. He’s already thrown it.
http://www.sportingcharts.com/nfl/stats/most-fumbles-and-fumbles-lost/2014/
In short, we know that Brady’s outstanding performance is not due to cheating because his performance is so outstanding.
We don’t prove the null hypothesis is true when we fail to reject it. In other words, we don’t know from this test that he did not cheat, we only fail to prove that he did.
The failure of the analysis is that it assumes the treatment (hypothetically deflated balls) is influencing running backs, and the lack of fumbles by the team’s (mediocre) running backs are portrayed as independent observations that verify the effect of the treatment on the outcome (low fumbles). In truth, the lack of fumbles is an outcome to which the quarterback contributes the most, in which case the quarterback and the outcome are dependent and confounded. In which case you can not distinguish Brady’s talent from cheating.
In science, if you follow the rules of science, it is illegitimate for people to dismiss your work on grounds that your results align with your preferences, you got your funding from an unpopular source, etc. But once you have been caught falsifying your data – even on one occasion – all your work becomes suspect, and a discussion of motives, funding sources, etc. becomes not only permissible but mandatory. You have lost the presumption of innocence; a presumption of guilt is now appropriate. Your earlier work should be regarded as unreliable unless it can be proven reliable (such as by replication).
Same applies here. If we had never caught Brady cheating, your arguments here would be valid. But Brady has lost the presumption of innocence; a presumption of guilt is now appropriate.
You must really love the Colts!
Or really hate cheaters.
What proof is there that Tom Brady has cheated in the past, sufficient that he should now be presumed guilty before innocent?
If we had proof he had cheated in the past, we wouldn’t be debating about presumptions – we would know.
The standard “innocent until proven guilty” applies in criminal law but not in other contexts. My argument here is that athletes should be held to a higher standard. The very nature of the athletic enterprise seems to me to require it – sports loses all interest if we can’t trust the athletes to follow the rules.
But hey, if you think Tom Brady doesn’t deserve to be held to a higher standard than criminals, well, okay then.
I guess you can’t believe every blog post you come across on the internet.
http://fivethirtyeight.com/datalab/your-guide-to-deflate-gateballghazi-related-statistical-analyses/
Thanks for posting this link! Most of that 538 article is just making the same empty arguments I refute above, but he links to more serious authors who have shown the actual data analysis to be questionable. Obviously in my post I was assuming the hypothesis that the data in the graph were responsibly handled. That’s the kind of risk you run when blogging – you’re diving into the discussion before everything has been extensively reviewed. I appreciate your setting the record straight.
Greg, just because I have wasted a wildly inappropriate amount of time analyzing this issue, I would like to add this link, which I thought was compelling.
http://drewfustin.com/2015/01/27/patriots-fumble-comments/
In any case, I’d like to make a friendly bet on NEXT year’s Patriots, to be paid next time you happen to be up at Pioneer Institute in Boston.
…With what we presumably agree will be very carefully vetted footballs new season, I’ll bet Pats are Top 5 in least fumbles (since I think player selection and coaching is driver here); you get what should be a very safe Pats are 6th thru 32nd.
It’s fascinating to me that some people want to shift the debate to the question of whether Tom Brady is a highly skilled quarterback, which of course no one is disputing, and as far away as possible from the only question that matters: whether Tom Brady is a liar and a cheater.
Obviously Tom Brady would perform excellently (e.g. be in the top 5 on an important metric such as fumbles) whether he cheated or not. But that is not the question at hand.