I was having a conversation with someone the other day about statistics, and their relationship to the draft, when it dawned on me that I might have sounded a bit too dismissive of the film study portion of talent evaluation. That was probably a mistake on my part. so I thought I would clarify things a bit more.
draft geek crowd, there is a bit of a divide between the guys who
examine the numbers, and the guys who feel that it all comes down to
studying the film. Things sometimes get a bit heated (as much as they
can between geeks), and insulting comments are thrown around regarding
the opposing groups viewpoints. In general, both sides come out looking
rather badly. Everybody wants to be perceived as some sort of guru, and
to show that the other side is composed of morons. It's sort of a very lame
version of the Middle East peace talks, but with more spreadsheets and a
slightly lower body count.
Since I would probably get
lumped in with the stat geeks, more often than not, I thought I would
poke some holes in their balloon first. The men of numbers tend to be
very dismissive of any subjective analysis. Sometimes this is a good
thing, but occasionally they get carried away with things. If a
player's traits can't be measured, they must be ignored. Only the
quantifiable matters. Except for when the numbers come out wrong, and
then they have to come up with new statistics to explain the
shortcomings of the old ones.
If there is one issue
that I think screws up the statistics crowd, it is probably an
over-reliance on regression analysis, and the scatter-plots that come
with it. It's not that this approach has no value. It's just that
people get carried away with it. They try to find correlations between
certain attributes, and an arbitrary measure of success, and forget to
question whether this attribute has any sort of sensible relationship to
the game of football. Why does a defensive ends' 3-cone time have a
0.05 correlation to his CarAV
(I'm just making this up as an example)? There can be a reason for
this, or it can mean nothing. If you can't envision how it translates
to what a player is doing on the field, it becomes a bit sketchy. This
becomes an issue before we even get to the question of whether CarAV is
an adequate measure of success. Cameron Wake has a CarAV of 37, and is
one of the league's premier pass rushers. Tyler Brayton was a
mediocrity, with a CarAV of 39. Something doesn't add up there. It
ends up being Sex Panther statistics.
not that this sort of analysis is bad; it's just that it has some
limitations that people sometimes overlook. In baseball, questions of
sample size are less problematic (more games played, more at bats,
etc.). Plus, the situations that players are put in are much more
similar, at least when they are batting. They all face the same
pitchers (for the most part), and they have comparable and numerous
opportunities. Does Denarius Moore, playing receiver with the Raiders,
have a similar opportunity to Greg Jennings, playing with the Packers?
This is before we even come to the question of whether the best players
are getting on the field in the first place. See Cameron Wake's four
years in the CFL, when no NFL team wanted him, as an example. At some
point, expecting the results to be in any way linear, becomes
ridiculous. Maybe it is a product of our childhood education involving
coloring books, that makes our obsession with "staying inside the lines"
so powerful. Regardless, I suspect we could find a high correlation
between a stat geek's sense of superiority and his proximity to someone
who doesn't know how to use Excel spreadsheets.
though, how complicated are spreadsheets? I could teach a five year
old how to do a regression analysis in five minutes (true, a sixty year
old might take longer). Despite my affection for spreadsheets, I have
to admit that only people with OCD are likely to spend much time using
them. Spreadsheets do a good job of tidying things up, and making us
look more clever than we are, and for that I am grateful. They don't
make us geniuses.
So, now we come to the other side of
the table, the "film study" boys. They generally dismiss statistics, in
favor of a pure observational approach. By eschewing statistics, they
also insulate themselves from any analysis that might show what
percentage of the time their eyes were malfunctioning. It's sort of like my view
on going to the doctor. If I never go in for a check up, I am
guaranteed to die of natural, or at the very least undiagnosed, causes.
That certainly sounds preferable to finding out about a tumor in my
colon. "Here lies Sid, his ass killed him." That will never be on my
headstone, so long as I avoid doctors. People will just say,"We don't
know what killed him. It was just his time."
my occasional prodding of the film study crowd, I think there are some
areas where they are correct. No matter what my computer is telling me
about a particular prospect, it seems sensible to watch them play at some
point. My only gripe with the film guys, is their short term memory.
This year's 'waist bender with a bad bubble', can be next year's All
Pro. It's good to keep track of how often we turn out to be wrong, so
that we can learn from our mistakes. Everyone wants to point to a
player like Vernon Gholston, and say they knew he was going to be a
bust, or that they could have spotted Tom Brady. If you don't account
for all the times you were wrong, however, you aren't telling the whole
The film study crowd will often say that talent
evaluation is more art than science. Based on my experience, guru
types who make such statements are generally to be avoided. Yes, there
are things that can be difficult to quantify, and some people might
indeed have a good eye for talent. However, people who make these
statements are kind of setting themselves up on a theoretically
unassailable position. I can't debate with someone on their subjective
views, nor can I prove them wrong. They might be right, but making mystical claims of insight, is problematic for me..
frequently, watching game film does give you an interesting and valuable
perspective. On the one hand, I remember watching Melvin Ingram
(DE/OLB, taken with the 18th pick in 2012, by the Chargers), and being
really excited about him. Unfortunately, the computer wasn't quite as
sold on him as a prospect, so I was forced to demote him in favor of
safer prospects. On the other hand, I also remember watching Hakeem
Nicks play in college (WR, taken with the 28th pick in 2009, by the
Giants), and also being fan. In this case, the computer liked Hakeem
quite a bit. What really sealed the deal though was watching this
If you only watched the beginning of this clip, do yourself favor
and watch the whole thing. The slow motion replay is what you want to
see. Now maybe I made too big of a deal out of this play, back in 2009,
but it stuck with me. I remember thinking,"My god, how did he maintain
control of that ball?" It seemed to show a lot of concentration.
Someone else might suggest that is was just a fluke, or that he got
lucky. To me, at the time, it was just amazing. I enjoy seeing those
"Oh shit, did you see that?" moments, and put a lot of stock in them.
If a player can do something like that, even just once, there is reason to hope that
they can do it again....and hopefully again, and again.
When both perspectives, the statistical and the observed, are combined, I think
you will get a better view of things, than if you are just an adherent
side of the argument. These two opposed groups are the peanut butter
and chocolate of analysis. They work better together. Or, since peanut
butter and chocolate are actually quite pleasing on their own, maybe
these two groups are more like vinegar and oil. Individually they are
repulsive, but capable of making a decent salad dressing when combined.
what I will write in this blog, I will generally try to refrain from
expressing my own subjective views. There are already enough people out
there who will give you subjective opinions, and a select few of them
are worth listening to (not he bozos on ESPN). For the most part, I
will just give you the numbers, and you can go watch the players
yourself, to form your own conclusions. There's no reason to think that
your impressions of a player would be any worse than mine.