Monday, December 1, 2014

Perhaps A Lobotomy Would Help?

I've been a bit distracted the past two months, folding paper cranes and writing haikus about sheepdogs, but those are the sorts of things which pay the bills.  Regardless, I'm back for the moment for some more deranged ranting about the potential benefits of lobotomizing your favorite NFL team's general manager.

Okay, in the previous post we set the computer up to behave like a bit of an imbecile.  We asked it to pick one player per draft class (from 2004-2013), who was between 245 and 285 pounds, based on their known physical traits, and their number of tackles for a loss in college. We were strictly looking at how productive these players were as pass rushers, and chose to use this range of weights because that is where you typically find the majority of the league's 3-4 outside linebackers and 4-3 defensive ends.  The computer also couldn't pick anyone who was selected before the 3rd round, or anyone who went undrafted.  In the area below, you will find a basic tally of the computer's results for its 10 selections.

  POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Total 880 635 343 200 0.227 72.15 38.97

POTGP is simply the number of Potential Games Played for a given player.  For example, if a player was selected in the 2011 draft then, by the end of the 2013 season, they could have potentially played in 32 games.  GP is simply the number of games they actually played.  GS is the number of games where they were listed as a starter.  Sack/POTGP is their number of sacks per potential game played.  % GP is the percentage of 'potential games' in which they players appeared.  % GS is the percentage of 'potential games' in which the players were listed as a starter.

I should also mention that the actual average and median draft position of the computer's picks came at the 119th and 115th picks respectively.  Also, the computer's overall Sack/POTGP result of 0.227, would be the equivalent of its typical selection generating 3.632 sacks per 16 game season.

To make some comparisons a bit easier, I will list here the average and median results of the computer's selection, when it came to athletic ability, and the average number of tackles for a loss in a player's final two years in college.  The Kangaroo Score and Agility Score are given in the form of how many standard deviations that a player is away from the average results for someone in their position group.

Computer    Avg. TFL    Kangaroo     Agility
AVG 17 1.067 0.482
MED 16.625 1.279 0.664

Now, we finally get to compare the computer's results to those of a handful of actual NFL teams.  To do this, we'll list every player that these teams selected over the same period of time (2004-2013), who fell into the 245-285 pound weight class.  We also have to remember that we are only examining how these players performed between the time they were drafted, and the end of the 2013 NFL season.  Players who went undrafted, regardless of how they ended up performing, will be left off of these lists.  I won't include all 32 teams here, though I will say that the results we are about to show appear to present a pretty typical picture of most NFL teams.  For the sake of brevity (Ha!), I will just show a handful of teams that I thought were particularly interesting.  If you're curious about some other team's results, feel free to ask, or you can simply do the calculations yourself, since it really isn't that complicated.

We'll start with my favorite team to torment, the Baltimore Ravens.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
John Simon 16 7 0 0 0 43.75 0
Courtney Upshaw 32 32 22 3 0.093 100 68.75
Pernell McPhee 48 44 6 9.5 0.197 91.66 12.5
Sergio Kindle 64 3 0 0 0 4.68 0
Paul Kruger 80 67 23 22 0.275 83.75 28.75
Antwan Barnes 112 83 5 25.5 0.227 74.1 4.46
Ryan LaCasse 128 12 0 0 0 9.37 0
Dan Cody 144 2 0 0 0 1.38 0
Roderick Green 160 54 0 12 0.075 33.75 0

Total 784 304 56 72 0.091 38.77 7.14

The Ravens have selected 9 players over this period of time, which is just one player short of what the computer drafted.  The average and median draft position of the Ravens' picks would be the 109th and 129th pick.  So, the Ravens selected almost the same number of players as the computer did, at roughly a similar point in the draft.  Even if we ignore the insane difference in the Ravens' raw sack total compared to the computer, the computer still comes out well ahead in Sack/POTGP.  The Ravens' result of 0.091 would be the equivalent of a player producing 1.456 sacks per 16 game season, well below the computer's result of 3.632.  A total of 4 of the Ravens' picks (44.44%) came from the first two rounds of the draft, where the computer was barred from making a selection.

I suspect Ravens' fans will argue that including players like Dan Cody and Sergio Kindle in this list is a bit unfair, since injuries kept them from getting on the field.  All I can say to that is "Hey, that's life!".  Every team on this list faced an equal risk of this occurring, as did the computer.  I can also say that while both of these players were fairly productive in college, their measured athletic ability makes the likelihood of them becoming exceptional performers somewhat doubtful.  In Cody's case, his 0.187 Kangaroo Score and -0.289 Agility Score paint the picture of a fairly mediocre athlete.  With Kindle, we find he has a 0.203 Kangaroo Score and a -0.533 Agility Score, which again are extremely questionable results.  You can choose to ignore these factors if you wish to, much like the Ravens did, though the whole point of this exercise is to illustrate how that might be a mistake. 

While I have no real problem with the college level statistical production of this group, it seems obvious that actual athletic ability is something the Ravens don't place a lot of value in. Overall, the median athletic ability for their selections was a paltry 0.187 Kangaroo Score, and a -0.289 Agility Score.  This means that the team regularly bets on mediocre athletes, and they appear to get mediocre results.

I also seem to run across a lot of Ravens' fans who continue to express high hopes for Courtney Upshaw, despite his extremely limited contributions as a pass rusher.  We've discussed Upshaw in the past, so we'll cut to the chase.  The main defense that people present for Upshaw, is his supposed quality as a run stopper.  I have no interest in debating this, though I tend to view this argument as an attempt to see the silver lining in a bad situation.  So, here's a simple test, to see if you truly believe that being "good against the run" is as valuable as being a good pass rusher.  How quickly would you trade the one dimensional run stopper, Courtney Upshaw, for a younger but one dimensional pass rusher, like Dwight Freeney?

Now, let's take a look at the Steelers.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Jarvis Jones 16 14 8 1 0.062 87.5 50
Chris Carter 48 29 4 0 0 60.41 8.33
Jason Worilds 64 57 21 19 0.296 89.06 32.81
Bruce Davis 96 15 0 0 0 15.62 0
LaMarr Woodley 112 94 81 57 0.508 83.91 72.32
Shaun Nua 144 0 0 0 0 0 0
Nathaniel Adibi 160 0 0 0 0 0 0

Total 640 209 114 77 0.120 32.65 17.81

Compared to many of the other teams I looked at, the Steelers actually selected relatively few players in the weight class we are examining, just 7 in total.  While their combined 77 sacks is well short of the computer's 200, the Steelers actually do better than many teams on a per player basis, with a 0.120 Sack/POTGP.  That would give the typical Steelers' draft pick 1.92 sacks in a sixteen game season, which is better than many other team on this list (though well short the computer's average draft pick which produces 3.632 sacks per season).  The average and median draft positions of the Steelers picks would be the 105th and 88th pick respectively, which is about half a round higher than where the computer made its selections.  A total of 3 of these picks (42.85%) came from the first two rounds of the draft, where the computer was barred from making a pick.

I have no interest in criticizing the Steelers.  After all, they have produced better results than many of the NFL teams we'll be looking at, especially when you consider how few selections they made. Still, similar to the previously mentioned Ravens, they also have shown a relative lack of interest in quantifiable athletic ability.  The results for their median draft pick were a -0.144 Kangaroo Score and a -0.028 Agility Score, which is clearly very average.

It seems worth noting, however, that almost all of their sack production during this period of time came from two players, LaMarr Woodley and Jason Worilds, who were both taken in the 2nd round (James Harrison is excluded because he wasn't selected in the draft).  In case you don't remember, the computer was barred from selecting players who were drafted this highly, so the Steelers had a bit of an advantage in this area.  Despite that, I think we can safely say that the computer would have spotted these potential talents rather easily.  We've discussed Jason Worilds before, so we'll skip that topic, except to note his 0.604 Kangaroo Score and 0.727 Agility Score, as well as an average of 14.75 TFL in college.  As for LaMarr Woodley, his 1.195 Kangaroo Score along with a -0.075 Agility Score would have given him a combined 0.560 Total Score (according to the dumbed down methods the computer was using for this game).  When you factor in the 15.25 tackles for a loss that Woodley averaged in his last two years in college, this would have resulted in the computer giving him a 1st round grade, which is slightly higher than where Woodley was actually selected.  So, once again, exceptional athleticism, and a history of proven production seem to produce the best results.

Oh, and if you are still expecting Jarvis Jones to emerge as the next great Steeler's pass rusher, the computer would like to reiterate its strong doubts about that.

Now let's move on the the Patriots.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Jamie Collins  16 16 8 0 0 100 50
Michael Buchanan 16 15 0 2 0.125 93.75 0
Chandler Jones 32 30 29 17.5 0.546 93.75 90.62
Dont'a Hightower 32 30 27 5 0.156 93.75 84.37
Jake Bequette 32 8 0 0 0 25 0
Markell Carter 48 0 0 0 0 0 0
Jerm. Cunningham 64 38 14 3.5 0.054 59.37 21.87
Brandon Spikes 64 51 39 1 0.015 79.68 60.93
Shawn Crable 96 6 0 0.5 0.005 6.25 0
Justin Rogers 112 32 0 0 0 28.57 0
Jeremy Mincey 128 66 40 20 0.156 51.56 31.25
Ryan Claridge 144 0 0 0 0 0 0

Total 784 292 157 49.5 0.063 37.24 20.02

More than almost any team I have looked at so far, the Patriots have selected a lot of players in this weight class, with a total of 12.  Despite the abundance of picks that the Patriots have made, their total number of sacks is a horribly embarrassing 49.5.  Even if we look at a better measuring stick like Sack/POTGP, their result of 0.063, is still laughably bad.  That means their typical player would be producing just 1.008 sacks in a 16 game season.  The average and median draft position of these draft picks would come at the 111th pick and 84th pick respectively, which again, is slightly higher than where the computer made its selections.  A total of 5 of these picks (41.66%) came from the first 2 rounds of the draft, where the computer was prohibited from making a selection.

Now, unlike a number of the teams in this post, the Patriots' results get a bit skewed.  Their apparent preference for larger inside linebackers like Brandon Spikes and Dont'a Hightower, means that some of these players weren't likely to get as many pass rushing opportunities.  Still, even if we were to excuse that, it's hard to say that this would radically improve their overall results.  The median results for their selections were a very slightly above average 0.335 Kangaroo Score and a 0.301 Agility Score.  While these are better results than what we saw from the Ravens and Steelers, it unfortunately coincides with a dip in their typical players number of TFLs in their final two years in college, to a median result of just 10.62 (compared to a more respectable 15.25 for the Ravens, and 15.5 for the Steelers and well below the 16.62 for Team Kangaroo).  Obviously, we prefer prospects with great athletic ability and proven performance..

Their player with the highest Sacks/POTGP result, is Chandler Jones.  With a result of 0.546, that works out to about 8.73 sacks per 16 game season.  When you consider Jones' 0.859 Kangaroo Score, along with a 0.247 Agility Score, you start to see someone with some intriguing athletic ability.  When examining his average number of tackles for a loss, during his final two college seasons, we get a result of 11.25 (though we had to adjust this some since he only played 7 games in his final year at Syracuse).  While the computer wouldn't have given Jones a 1st round grade, he clearly had some intriguing potential.

So, how has a team like the Patriots managed to survive with so many of their draft picks producing such poor results?  Well, they've largely gotten by with a regular supply of mercenary free agent pass rushers.  Let's take a look at some of the players they have brought in to fill the void, during this time period.

Player    Kangaroo       Agility Avg. TFL
Rob Ninkovich 0.267 1.013 13.25
Andre Carter 1.230 0.577 19.5
Mark Anderson 1.344 0.846 12.5
Derrick Burgess 1.802            N/A          N/A
Adalius Thomas 1.573 -0.306 18

Well, how about that!  They've largely been signing freakishly gifted athletes who were highly productive in college.  While some of these players may not have provided exceptional results to their new team, some decline in performance should probably be expected when you are signing players who are going into their 2nd and 3rd NFL contract.   Still, it's funny to consider that these players are the ones that drew the Patriots eye in free agency, since they largely fit our mold for successful NFL pass rushers.  The only real question is, why don't the Patriots just draft players like this in the first place?  If there is one bit of good news, I do think things could potentially turn out quite well for Jamie Collins.

Let's see what a fairly terrible Falcons' defense has done.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Malliciah Goodman 16 14 1 0 0 87.5 6.25
Stansly Maponga 16 12 0 0 0 75 0
Jon Massaquoi 32 24 4 4 0.125 75 12.5
Cliff Matthews 48 25 0 0 0 52.08 0
Lawrence Sidbury 80 48 0 5 0.062 60 0
Curtis Lofton 96 96 95 7 0.072 100 98.95
Kroy Biermann 96 82 22 16.5 0.171 85.41 22.91
Chauncey Davis 144 102 25 11 0.076 80.83 17.36

Total 528 403 147 43.5 0.082 76.32 27.84

Over this period of time, the Falcons have selected 8 players in this weight class.  The average and median draft position of these picks came at the 139th and 140th pick, which is noticeably lower than where the computer (or any other team on this list) made its selections.  By a fairly large margin, the Falcons selections have produced the lowest number of total sacks, though when we look at Sack/POTGP, they at least manage to rise above the Patriots.  Their result of 0.082 would mean their typical pick produces the equivalent of 1.312 sacks in a 16 game season.

Most of the teams we are examining here were chosen due to their on-the-field success, or reputation for having a good defense.  I chose to include the Falcons for the complete opposite reason.  This isn't to say that there aren't good things about the team, but drafting quality pass rushers hasn't been their strong point during this period of time. 

The combination of making slightly fewer picks than the other teams, as well as making them later in the draft, suggests the Falcons really haven't viewed finding a pass rusher as much of a priority.  I can also say that the selections they did make almost invariably lacked the combination of athletic ability and college production that would have made them intriguing targets, in the eye's of the computer.  The median results for these selections were a 0.041 Kangaroo Score a -0.472 Agility Score, which obviously isn't anything to get excited about.  When you also consider their typical players poor median result of just 10 TFLs in their final two college seasons, success seemed quite unlikely.  Hey, that's their choice, and none of my business.  All I can say is that any complaints people might have about a poor pass rush were probably entirely foreseeable.

As one final note, I realize that Falcons' fans might object to Curtis Lofton being included in this list, since he clearly wasn't intended to be a pass rusher.  In the end though, he fell into the weight class that we had selected, so we couldn't exclude him.  If it makes any difference, just be glad we didn't include the Falcons' incredibly disappointing selection of Jamaal Anderson, with the 8th overall pick in 2007.  Anderson weighed 288 pounds at the combine, which excluded him from this list, and actually slightly improved the overall picture for the Falcons.  In the end, I think that sort of balances out including Lofton.

Finally, let's take a look at the Seahawks' defense.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Ty Powell 16 5 0 0 0 31.25 0
Bruce Irvin 32 28 12 10 0.312 87.5 37.5
Greg Scruggs 32 11 0 2 0.062 34.37 0
K.J. Wright 48 44 40 4.5 0.093 91.66 83.33
Aaron Curry 80 48 39 5.5 0.068 60 48.75
Nick Reed 80 26 0 1 0.012 32.5 0
Lawrence Jackson 96 69 24 19.5 0.203 71.87 25
Baraka Atkins 112 21 0 2 0.017 18.75 0
Darryl Tapp 128 114 35 25 0.195 89.06 27.34
Jeb Huckeba 144 0 0 0 0 0 0

Total 768 366 150 69.5 0.090 47.65 19.53

During this period of time the Seahawks have selected a total of 10 players in the weight class we are examining, the same as the computer.  The average and median draft position of these picks came at the 119th and 109th picks respectively, which is roughly the same area as where the computer and most of these other teams made their selections.  Their Sack/POTGP result of 0.090 would suggest that their typical draft pick produces about 1.44 sacks per 16 game season.  A total of 4 of these selections (40%) were chosen in the first two rounds of the draft, where the computer was barred from making a pick.

Despite the solid reputation of the Seahawks defense, their ability to successfully draft pass rushers is rather average to slightly below average, compared to these other teams.  Lawrence Jackson, actually ends up being credited with 28.05% of the sacks produced by Seahawks draft picks, though for 2/3 of these sacks Jackson was on another team, as he was only a Seahawk for his first 2 seasons.  That leaves only the somewhat mediocre Darryl Tapp and Bruce Irvin as the most productive pass rushers to be selected by the team during this period of time.

Again, the reasons for this apparent failure seem a bit obvious.  The median results for these selections would be a -0.007 Kangaroo Score and a 0.172 Agility Score, along with 13.5 TFLs in their final two college seasons.  Those are rather uninspiring results, and not surprisingly they produced uninspiring outcomes.

To fill this pass rushing void, the Seahawks have had to look outside the draft, similar to the Patriots.  In free agency they acquired Cliff Avril (0.287 Kangaroo Score and a 0.215 Agility Score), who was probably just a slightly above average athlete, though he did average 15 tackles for a loss in his final two years in college, which is quite good.  They also managed to pick up Michael Bennett as an undrafted free agent in 2009, and while his average of 9 tackles for a loss in college was fairly pedestrian, his 0.837 Kangaroo Score suggested some reasonable athletic potential (we don't have the data to calculate his Agility Score, unfortunately).  They also traded the previously mentioned Darryl Tapp, for the enigmatic Chris Clemons, who's eventual successes I admittedly have no real explanation for. 

Let's wrap this up...

I realize that only listing the results for 5 different teams may seem like I have been cherry picking the data a bit.  On the other hand, I tend to be a bit long-winded, and I doubt anyone would make it through a post where I did this for every single team.  All I can really say is, these teams seemed to do a good job of illustrating my overall point, and really appeared to capture the general problems most teams have in selecting pass rushers.  These teams, for the most part, are the norm.

That isn't to say that there aren't teams who have done significantly better.  There are.  The Giants, the Rams, the Titans, and a few others have done quite a bit better at selecting these sorts of players, though still quite a bit short of the computer's theoretical results.  Unfortunately, examining the methods to their success aren't that interesting, as their "hits" typically seem to be the very sort of players that the computer would approve of, the highly athletic freak who was productive in college.

Perhaps the most interesting team I looked at was the Kansas City Chiefs, which was the only team to slightly surpass the computer's results.  The combination of selections like Jared Allen, Tamba Hali, and Justin Houston makes up a rather intimidating group of pass rushers.  Still, with the possible exception of Tamba Hali, these players also fit the computer's mold for successful players, so I really don't see much for us to learn here, beyond what we already suspected.  The Chiefs have simply done a good job.

I'm sure some other people will criticize the amount of attention that I give to a player's ability to generate sacks.  To some extent, I suppose that is fair.  Almost any statistical category can be a bit overrated.  Despite that, I think it is interesting that even if we look beyond sack production, the computer's imaginary picks are still crushing pretty much everyone when it come to %GP and % GS.  So, when we simply consider a player's ability to get on the field and play, the computer is doing a much better job there as well.

While some might suspect that the computer had the advantage of hindsight, which is always 20/20, you have to remember that the computer made its selections based only on very basic pieces of data that would have been freely available at the time, so it really had no advantage in this sense.  You also have to remember the huge advantage that these NFL teams had, simply by being permitted to select players in the first two rounds of the draft, while the computer was prohibited from doing the same.  How much more lopsided do you suspect the computer's results would have been if this restriction had been lifted?

In the end, you can never really eliminate the risk that a player will be a failure.  Making any sort of guarantee about how following the path I proposed will assure success would be incredibly stupid.  The only real point that I am trying to convey is that, perhaps, if general managers resigned themselves to the likelihood that their instincts for identifying talented players were largely nonsensical beliefs manufactured by their egos, and instead based their decisions on measurable data, they probably couldn't do any worse, and quite possibly would actually improve (I suspect significantly).  When you really consider their histories of repeated failures, what is the real risk?


  1. Sidney/Reilly,
    I write for Rotoviz and am friends with Shawn Siegele. I just binge read your entire blog archives and its really...its just really, extraordinarily so even. I would greatly appreciate being able to get into contact with you, my email is so just write me if you don't want to publish an email or whatever.
    Thanks, Max

    1. Thanks for taking an interest in our deranged ramblings. I sent you an email, so feel to get in touch any time.