Thursday, January 1, 2015

The People's Champions

Our main focus around here has been speculation about the criteria by which team's make their picks in the NFL Draft.  The goal is simply to find ways of identifying prospects that have a better chance of meeting a team's hopes and expectations, rather than ending up a disappointment.  It's a tricky business, and inevitably we will be wrong sometimes (at which point, I will place the blame squarely on Reilly's shoulders). 

Oddly, if there is one aspect of doing this blog (the bloooog) that makes me nervous, it is the possibility of shouting "Hey, we were right about this guy!".  Yes, we want our idiotic hunches to turn out well, but chest thumping bravado gives me the heebie jeebies.  The data is there.  We attempt to sort it, and extract some meaning from it.  We think it is the sensible thing to do.  Despite that, I really cringe when I see some of the gurus of the internet, and the self-congratulatory wanking they engage in.  Some enthusiasm is understandable, but preferably in moderation.  I generally think we should behave more like monks, or maybe the Amish.  So, with that in mind, we've decided to try to make it a policy to avoid reflecting on the times when things go according to plan.  We want to avoid doing unnecessary and tacky victory laps.

At the same time, there are going to be some prospects whom we strongly disapproved of, who go on to do quite well for themselves.  Whether they are genuine outliers, or there was insufficient data to project them correctly, or maybe we were just plain wrong; it all works out the same.  Sometimes we're just going to be way off the mark.  It may not change our feelings about whether such prospects were sensible gambles for a team to make, but we do think these slip-ups should be given some consideration.  Maybe there is something to be learned from such embarrassing accidents.

Based on these feelings, we decided to start doing an annual list of our potentially most egregious errors.  Rather than include all of our mistakes, we'll mainly focus on the players that we had the most damning criticisms of, who still managed to exceed our expectations.  As time goes on, these lists will undoubtedly grow.  Since these players are possible victories for the traditional scouting crowd, we'll just refer to them as The People's Champions.  Our cilice has been tightened, let the flagellation begin...

Kelvin Benjamin

I believe our exact words were something like, any team that drafts him "should probably check their GM for symptoms of dementia".  That might have been a bit harsh.  Of course, Benjamin went on to accumulate 1,008 yards, and 9 touchdowns in his rookie season.  It wasn't our finest moment.  The worst part is, I still don't have a great theory as to how this happened.

One issue that makes Kelvin Benjamin particularly difficult to reexamine, relates to his claims that he intentionally bombed the combine in order to drop in the draft, so that Carolina would be able to select him.  If this is true, then we clearly didn't have much of a chance to evaluate him correctly, since we base a fair bit of our hunches on a player's measurable attributes.  On the one hand, it is hard to believe that Benjamin is serious about any of this, since you would have to be a moron to intentionally make yourself look bad at the combine.  On the other hand, this is an individual who scored a 7 on the Wonderlic test, so he probably needs his coaches to help him tie his shoes. 

Will Benjamin flame out like Michael Clayton, who was a 2004 rookie phenom that similarly defied explanation?  I have no idea.  While I'd love to have order restored to my universe, I will try to refrain from constructing any voodoo dolls shaped like a Panthers' player to do so.  I wish him luck, and sincerely hope that someone in Carolina has child-proofed his house.

In the end, if we have to be wrong at some point, I'd always prefer that it  be about someone as comically peculiar as Kelvin Benjamin.

C.J. Mosley

It's particularly painful for me to acknowledge that we might have been wrong about C.J. Mosley, since the Ravens are not only my home team, but one of my favorite organizations to torment.  Still, despite my apprehensions regarding Mosley, I have to say that I do think he probably is performing quite a bit better than our numbers suggested he would.

While we never really outright claimed that Mosley would be a bust, we certainly leaned towards the possibility that he would be a mediocrity.  Whether it was his measured athletic ability, or his statistical production in college, there wasn't much that grabbed our attention, or made him appear to be in the same class as some of the 1st round MLBs that preceded him.  Still, off the top of my head, I can say that there were a handful of similar players such as Navorro Bowman, Curtis Lofton, and maybe Jon Beason that were also a bit unlikely to become successful NFL middle linebackers.  So, these things do happen from time to time..

Has Mosley performed better than I believed he would?  Yes, almost certianly.  Now, do I really believe that Mosley deserved to be selected for the Pro Bowl this year?  Ehh...hmm...maybe not.  The degree to which his draft status may have enhanced the likelihood of receiving this honor is a strong possibility.  At the same time, we should also probably take a look at the players who have surrounded Mosley in his rookie season.

While the Ravens' 2014 secondary play was clearly horrendous, that shouldn't really have too much impact on an inside linebacker.  The front seven of their defense, however, was actually rather good.  The outside pass rushing tandem of Suggs and Dumervil, accumulated 29 combined sacks, more than holding up their part of the equation.  The team's three primary down linemen, of Ngata, Williams and Canty, are a massive 992 pound wall (possibly the largest trio in the league, though I still have to check on that), and generally conform (to varying degrees) to the athletic and statistical profiles we hope to see in these types of players.  They probably kept Mosley relatively untouched.  Finally, there is the team's other inside linebacker, the veteran Daryl Smith, who despite his age, is still performing at a high enough level to take a lot of the pressure off of Mosley.  All things considered, Mosley wound up in a pretty good environment.

Will Mosley be able to maintain this head of steam, as the surrounding talent retires or moves to other teams?  I don't know, but I think it is an interesting question.  Haloti Ngata is easily the most physically gifted linemen on the team, but his current contract makes him a ripe target to cut.  With the exception of Brandon Williams, everyone else in the front seven is at least 30 years old, so the clock is ticking.  How this will impact Mosley is hard to say.

The most interesting question, at least to me, is whether Mosley's apparent success may actually end up hurting the team in a more roundabout way.  If Mosley continues to do well, despite being a prospect that on paper appeared to present some risks, doesn't this just encourage the team to continue pursuing their questionable approach to the draft?  Let's consider the team's top draft picks in the past few years.

In 2013, the team's top pick was Matt Elam, who so far has been a fairly well recognized failure at the safety position.  Due to an inability to calculate his Agility Score, we weren't able to give him much of an assessment last year.  With defensive backs, the Agility Score is obviously a bit more of a factor than it might be at other positions.  While his 40 time was fairly good (4.43 seconds), his statistical production and Kangaroo Score (0.284) were little more than just average.  Still, while we couldn't fully appraise Elam, we certainly wouldn't have been able to endorse selecting someone about whom there wasn't enough data, or where the available data was relatively weak, so the computer would have viewed this pick as an unnecessary and unwise risk.

In 2012, the team's top pick was Courtney Upshaw, who is someone we have criticized numerous times already.  Due to the fact that he has been listed as a starter in 36 out of a possible 48 games during his young career, many people still choose to defend this pick.  Of course, Reilly and I tend to think that the whopping 3 sacks he has produced in this time are a bit more meaningful.  The data was there to suggest he would likely struggle as a pass rusher, which we still feel is the primary role for a 3-4 OLB, particularly when one is selected with a relatively high pick.  So, again, caution was thrown to the wind, and the team went with their gut instinct, rather than with the statistical data.

In 2011, the team selected cornerback Jimmy Smith, and...we really don't have a huge issue with this pick.  While there were other prospects that the computer might have preferred, Smith was still fairly intriguing.  While his Agility Score (0.316) was only slightly above average, he was a much larger sort of corner, and his Kangaroo Score (0.963), suggested he did have some potential.  While people seem to feel he has done well so far, injuries appear to be his main obstacle.  This seems to be shaping up to have been a reasonably effective draft pick.

In 2010, the team's top pick was Sergio Kindle.  While we could skip past this selection, since he never really played due to injury, the data still suggests that the team was ignoring the glaring warning signs here, as they frequently appear to do.  It's a shame that Kindle couldn't get on the field and confirm whether or not the data was accurate, but we still have to accept that this was a bust.

In 2009, the Ravens' first pick was the infamous Michael Oher.  What would the computer make of an offensive tackle prospect with a 0.134 Agility Score, and a 0.194 Kangaroo Score?  Well, you could certainly do worse, but those are pretty much the definition of average and forgettable results.  They certainly aren't the kinds of results that would make us feel comfortable with selecting someone in the first round.   Nonetheless, that is precisely what the team did, and Oher has gone on to average 1 sack surrendered in about 53% of his games started, which is clearly a rather poor result.  Oddly, none of this appears to have greatly impacted his starting status, because high draft picks probably get more second chances than they sometimes deserve.  We tried to delve into why this subject before,, and I think that brings us back around to why we should be equally suspicious of why Courtney Upshaw continues to get starts.

In the area that these players were selected, you can generally figure on making a successful pick about 60-70% of the time.  It seems reasonable to me that you could say that the only arguable success in this group may have been Jimmy Smith (though it may still be a bit early to declare that).  Most likely, it isn't a complete coincidence that Smith was a rather decent fit for the athletic/statistical profile of an NFL cornerback. On the other hand, what can we say about these other picks that the Ravens made?  Pretty much without exception, they all seem to have been enormous gambles, even if the conventional wisdom at the time suggested they were reasonable picks.

Now, it may sound as if I am still being a bit reticent about giving C.J. Mosley the level of praise he perhaps deserves, but that isn't my goal.  While I wish Mosley continued good fortune, I still have some concerns about the thought process that went into his selection, regardless of how things actually turned out.  To some extent, the process used to select Mosley appears to be the same one used to make all of these other regrettable picks.  It feels as if the Ravens are playing a strange form of Russian roulette.  While the computer would have expected the Ravens to shoot themselves in the foot 5 times out of 6, it only actually happened on 4 of these occasions.  That the team's most recent pull of the trigger turned out to be significantly less painful than it was in past years, might lead them to believe that their process is worth pursuing in the future.  People tend to sweep the past under the rug, in light of more positive recent outcomes.  This would make me very nervous.

Okay.  Maybe I'm feeling just a tiny bit bitter about C.J. Mosley.

Monday, December 1, 2014

Perhaps A Lobotomy Would Help?

I've been a bit distracted the past two months, folding paper cranes and writing haikus about sheepdogs, but those are the sorts of things which pay the bills.  Regardless, I'm back for the moment for some more deranged ranting about the potential benefits of lobotomizing your favorite NFL team's general manager.

Okay, in the previous post we set the computer up to behave like a bit of an imbecile.  We asked it to pick one player per draft class (from 2004-2013), who was between 245 and 285 pounds, based on their known physical traits, and their number of tackles for a loss in college. We were strictly looking at how productive these players were as pass rushers, and chose to use this range of weights because that is where you typically find the majority of the league's 3-4 outside linebackers and 4-3 defensive ends.  The computer also couldn't pick anyone who was selected before the 3rd round, or anyone who went undrafted.  In the area below, you will find a basic tally of the computer's results for its 10 selections.

  POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Total 880 635 343 200 0.227 72.15 38.97

POTGP is simply the number of Potential Games Played for a given player.  For example, if a player was selected in the 2011 draft then, by the end of the 2013 season, they could have potentially played in 32 games.  GP is simply the number of games they actually played.  GS is the number of games where they were listed as a starter.  Sack/POTGP is their number of sacks per potential game played.  % GP is the percentage of 'potential games' in which they players appeared.  % GS is the percentage of 'potential games' in which the players were listed as a starter.

I should also mention that the actual average and median draft position of the computer's picks came at the 119th and 115th picks respectively.  Also, the computer's overall Sack/POTGP result of 0.227, would be the equivalent of its typical selection generating 3.632 sacks per 16 game season.

To make some comparisons a bit easier, I will list here the average and median results of the computer's selection, when it came to athletic ability, and the average number of tackles for a loss in a player's final two years in college.  The Kangaroo Score and Agility Score are given in the form of how many standard deviations that a player is away from the average results for someone in their position group.

Computer    Avg. TFL    Kangaroo     Agility
AVG 17 1.067 0.482
MED 16.625 1.279 0.664

Now, we finally get to compare the computer's results to those of a handful of actual NFL teams.  To do this, we'll list every player that these teams selected over the same period of time (2004-2013), who fell into the 245-285 pound weight class.  We also have to remember that we are only examining how these players performed between the time they were drafted, and the end of the 2013 NFL season.  Players who went undrafted, regardless of how they ended up performing, will be left off of these lists.  I won't include all 32 teams here, though I will say that the results we are about to show appear to present a pretty typical picture of most NFL teams.  For the sake of brevity (Ha!), I will just show a handful of teams that I thought were particularly interesting.  If you're curious about some other team's results, feel free to ask, or you can simply do the calculations yourself, since it really isn't that complicated.

We'll start with my favorite team to torment, the Baltimore Ravens.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
John Simon 16 7 0 0 0 43.75 0
Courtney Upshaw 32 32 22 3 0.093 100 68.75
Pernell McPhee 48 44 6 9.5 0.197 91.66 12.5
Sergio Kindle 64 3 0 0 0 4.68 0
Paul Kruger 80 67 23 22 0.275 83.75 28.75
Antwan Barnes 112 83 5 25.5 0.227 74.1 4.46
Ryan LaCasse 128 12 0 0 0 9.37 0
Dan Cody 144 2 0 0 0 1.38 0
Roderick Green 160 54 0 12 0.075 33.75 0

Total 784 304 56 72 0.091 38.77 7.14

The Ravens have selected 9 players over this period of time, which is just one player short of what the computer drafted.  The average and median draft position of the Ravens' picks would be the 109th and 129th pick.  So, the Ravens selected almost the same number of players as the computer did, at roughly a similar point in the draft.  Even if we ignore the insane difference in the Ravens' raw sack total compared to the computer, the computer still comes out well ahead in Sack/POTGP.  The Ravens' result of 0.091 would be the equivalent of a player producing 1.456 sacks per 16 game season, well below the computer's result of 3.632.  A total of 4 of the Ravens' picks (44.44%) came from the first two rounds of the draft, where the computer was barred from making a selection.

I suspect Ravens' fans will argue that including players like Dan Cody and Sergio Kindle in this list is a bit unfair, since injuries kept them from getting on the field.  All I can say to that is "Hey, that's life!".  Every team on this list faced an equal risk of this occurring, as did the computer.  I can also say that while both of these players were fairly productive in college, their measured athletic ability makes the likelihood of them becoming exceptional performers somewhat doubtful.  In Cody's case, his 0.187 Kangaroo Score and -0.289 Agility Score paint the picture of a fairly mediocre athlete.  With Kindle, we find he has a 0.203 Kangaroo Score and a -0.533 Agility Score, which again are extremely questionable results.  You can choose to ignore these factors if you wish to, much like the Ravens did, though the whole point of this exercise is to illustrate how that might be a mistake. 

While I have no real problem with the college level statistical production of this group, it seems obvious that actual athletic ability is something the Ravens don't place a lot of value in. Overall, the median athletic ability for their selections was a paltry 0.187 Kangaroo Score, and a -0.289 Agility Score.  This means that the team regularly bets on mediocre athletes, and they appear to get mediocre results.

I also seem to run across a lot of Ravens' fans who continue to express high hopes for Courtney Upshaw, despite his extremely limited contributions as a pass rusher.  We've discussed Upshaw in the past, so we'll cut to the chase.  The main defense that people present for Upshaw, is his supposed quality as a run stopper.  I have no interest in debating this, though I tend to view this argument as an attempt to see the silver lining in a bad situation.  So, here's a simple test, to see if you truly believe that being "good against the run" is as valuable as being a good pass rusher.  How quickly would you trade the one dimensional run stopper, Courtney Upshaw, for a younger but one dimensional pass rusher, like Dwight Freeney?

Now, let's take a look at the Steelers.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Jarvis Jones 16 14 8 1 0.062 87.5 50
Chris Carter 48 29 4 0 0 60.41 8.33
Jason Worilds 64 57 21 19 0.296 89.06 32.81
Bruce Davis 96 15 0 0 0 15.62 0
LaMarr Woodley 112 94 81 57 0.508 83.91 72.32
Shaun Nua 144 0 0 0 0 0 0
Nathaniel Adibi 160 0 0 0 0 0 0

Total 640 209 114 77 0.120 32.65 17.81

Compared to many of the other teams I looked at, the Steelers actually selected relatively few players in the weight class we are examining, just 7 in total.  While their combined 77 sacks is well short of the computer's 200, the Steelers actually do better than many teams on a per player basis, with a 0.120 Sack/POTGP.  That would give the typical Steelers' draft pick 1.92 sacks in a sixteen game season, which is better than many other team on this list (though well short the computer's average draft pick which produces 3.632 sacks per season).  The average and median draft positions of the Steelers picks would be the 105th and 88th pick respectively, which is about half a round higher than where the computer made its selections.  A total of 3 of these picks (42.85%) came from the first two rounds of the draft, where the computer was barred from making a pick.

I have no interest in criticizing the Steelers.  After all, they have produced better results than many of the NFL teams we'll be looking at, especially when you consider how few selections they made. Still, similar to the previously mentioned Ravens, they also have shown a relative lack of interest in quantifiable athletic ability.  The results for their median draft pick were a -0.144 Kangaroo Score and a -0.028 Agility Score, which is clearly very average.

It seems worth noting, however, that almost all of their sack production during this period of time came from two players, LaMarr Woodley and Jason Worilds, who were both taken in the 2nd round (James Harrison is excluded because he wasn't selected in the draft).  In case you don't remember, the computer was barred from selecting players who were drafted this highly, so the Steelers had a bit of an advantage in this area.  Despite that, I think we can safely say that the computer would have spotted these potential talents rather easily.  We've discussed Jason Worilds before, so we'll skip that topic, except to note his 0.604 Kangaroo Score and 0.727 Agility Score, as well as an average of 14.75 TFL in college.  As for LaMarr Woodley, his 1.195 Kangaroo Score along with a -0.075 Agility Score would have given him a combined 0.560 Total Score (according to the dumbed down methods the computer was using for this game).  When you factor in the 15.25 tackles for a loss that Woodley averaged in his last two years in college, this would have resulted in the computer giving him a 1st round grade, which is slightly higher than where Woodley was actually selected.  So, once again, exceptional athleticism, and a history of proven production seem to produce the best results.

Oh, and if you are still expecting Jarvis Jones to emerge as the next great Steeler's pass rusher, the computer would like to reiterate its strong doubts about that.

Now let's move on the the Patriots.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Jamie Collins  16 16 8 0 0 100 50
Michael Buchanan 16 15 0 2 0.125 93.75 0
Chandler Jones 32 30 29 17.5 0.546 93.75 90.62
Dont'a Hightower 32 30 27 5 0.156 93.75 84.37
Jake Bequette 32 8 0 0 0 25 0
Markell Carter 48 0 0 0 0 0 0
Jerm. Cunningham 64 38 14 3.5 0.054 59.37 21.87
Brandon Spikes 64 51 39 1 0.015 79.68 60.93
Shawn Crable 96 6 0 0.5 0.005 6.25 0
Justin Rogers 112 32 0 0 0 28.57 0
Jeremy Mincey 128 66 40 20 0.156 51.56 31.25
Ryan Claridge 144 0 0 0 0 0 0

Total 784 292 157 49.5 0.063 37.24 20.02

More than almost any team I have looked at so far, the Patriots have selected a lot of players in this weight class, with a total of 12.  Despite the abundance of picks that the Patriots have made, their total number of sacks is a horribly embarrassing 49.5.  Even if we look at a better measuring stick like Sack/POTGP, their result of 0.063, is still laughably bad.  That means their typical player would be producing just 1.008 sacks in a 16 game season.  The average and median draft position of these draft picks would come at the 111th pick and 84th pick respectively, which again, is slightly higher than where the computer made its selections.  A total of 5 of these picks (41.66%) came from the first 2 rounds of the draft, where the computer was prohibited from making a selection.

Now, unlike a number of the teams in this post, the Patriots' results get a bit skewed.  Their apparent preference for larger inside linebackers like Brandon Spikes and Dont'a Hightower, means that some of these players weren't likely to get as many pass rushing opportunities.  Still, even if we were to excuse that, it's hard to say that this would radically improve their overall results.  The median results for their selections were a very slightly above average 0.335 Kangaroo Score and a 0.301 Agility Score.  While these are better results than what we saw from the Ravens and Steelers, it unfortunately coincides with a dip in their typical players number of TFLs in their final two years in college, to a median result of just 10.62 (compared to a more respectable 15.25 for the Ravens, and 15.5 for the Steelers and well below the 16.62 for Team Kangaroo).  Obviously, we prefer prospects with great athletic ability and proven performance..

Their player with the highest Sacks/POTGP result, is Chandler Jones.  With a result of 0.546, that works out to about 8.73 sacks per 16 game season.  When you consider Jones' 0.859 Kangaroo Score, along with a 0.247 Agility Score, you start to see someone with some intriguing athletic ability.  When examining his average number of tackles for a loss, during his final two college seasons, we get a result of 11.25 (though we had to adjust this some since he only played 7 games in his final year at Syracuse).  While the computer wouldn't have given Jones a 1st round grade, he clearly had some intriguing potential.

So, how has a team like the Patriots managed to survive with so many of their draft picks producing such poor results?  Well, they've largely gotten by with a regular supply of mercenary free agent pass rushers.  Let's take a look at some of the players they have brought in to fill the void, during this time period.

Player    Kangaroo       Agility Avg. TFL
Rob Ninkovich 0.267 1.013 13.25
Andre Carter 1.230 0.577 19.5
Mark Anderson 1.344 0.846 12.5
Derrick Burgess 1.802            N/A          N/A
Adalius Thomas 1.573 -0.306 18

Well, how about that!  They've largely been signing freakishly gifted athletes who were highly productive in college.  While some of these players may not have provided exceptional results to their new team, some decline in performance should probably be expected when you are signing players who are going into their 2nd and 3rd NFL contract.   Still, it's funny to consider that these players are the ones that drew the Patriots eye in free agency, since they largely fit our mold for successful NFL pass rushers.  The only real question is, why don't the Patriots just draft players like this in the first place?  If there is one bit of good news, I do think things could potentially turn out quite well for Jamie Collins.

Let's see what a fairly terrible Falcons' defense has done.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Malliciah Goodman 16 14 1 0 0 87.5 6.25
Stansly Maponga 16 12 0 0 0 75 0
Jon Massaquoi 32 24 4 4 0.125 75 12.5
Cliff Matthews 48 25 0 0 0 52.08 0
Lawrence Sidbury 80 48 0 5 0.062 60 0
Curtis Lofton 96 96 95 7 0.072 100 98.95
Kroy Biermann 96 82 22 16.5 0.171 85.41 22.91
Chauncey Davis 144 102 25 11 0.076 80.83 17.36

Total 528 403 147 43.5 0.082 76.32 27.84

Over this period of time, the Falcons have selected 8 players in this weight class.  The average and median draft position of these picks came at the 139th and 140th pick, which is noticeably lower than where the computer (or any other team on this list) made its selections.  By a fairly large margin, the Falcons selections have produced the lowest number of total sacks, though when we look at Sack/POTGP, they at least manage to rise above the Patriots.  Their result of 0.082 would mean their typical pick produces the equivalent of 1.312 sacks in a 16 game season.

Most of the teams we are examining here were chosen due to their on-the-field success, or reputation for having a good defense.  I chose to include the Falcons for the complete opposite reason.  This isn't to say that there aren't good things about the team, but drafting quality pass rushers hasn't been their strong point during this period of time. 

The combination of making slightly fewer picks than the other teams, as well as making them later in the draft, suggests the Falcons really haven't viewed finding a pass rusher as much of a priority.  I can also say that the selections they did make almost invariably lacked the combination of athletic ability and college production that would have made them intriguing targets, in the eye's of the computer.  The median results for these selections were a 0.041 Kangaroo Score a -0.472 Agility Score, which obviously isn't anything to get excited about.  When you also consider their typical players poor median result of just 10 TFLs in their final two college seasons, success seemed quite unlikely.  Hey, that's their choice, and none of my business.  All I can say is that any complaints people might have about a poor pass rush were probably entirely foreseeable.

As one final note, I realize that Falcons' fans might object to Curtis Lofton being included in this list, since he clearly wasn't intended to be a pass rusher.  In the end though, he fell into the weight class that we had selected, so we couldn't exclude him.  If it makes any difference, just be glad we didn't include the Falcons' incredibly disappointing selection of Jamaal Anderson, with the 8th overall pick in 2007.  Anderson weighed 288 pounds at the combine, which excluded him from this list, and actually slightly improved the overall picture for the Falcons.  In the end, I think that sort of balances out including Lofton.

Finally, let's take a look at the Seahawks' defense.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Ty Powell 16 5 0 0 0 31.25 0
Bruce Irvin 32 28 12 10 0.312 87.5 37.5
Greg Scruggs 32 11 0 2 0.062 34.37 0
K.J. Wright 48 44 40 4.5 0.093 91.66 83.33
Aaron Curry 80 48 39 5.5 0.068 60 48.75
Nick Reed 80 26 0 1 0.012 32.5 0
Lawrence Jackson 96 69 24 19.5 0.203 71.87 25
Baraka Atkins 112 21 0 2 0.017 18.75 0
Darryl Tapp 128 114 35 25 0.195 89.06 27.34
Jeb Huckeba 144 0 0 0 0 0 0

Total 768 366 150 69.5 0.090 47.65 19.53

During this period of time the Seahawks have selected a total of 10 players in the weight class we are examining, the same as the computer.  The average and median draft position of these picks came at the 119th and 109th picks respectively, which is roughly the same area as where the computer and most of these other teams made their selections.  Their Sack/POTGP result of 0.090 would suggest that their typical draft pick produces about 1.44 sacks per 16 game season.  A total of 4 of these selections (40%) were chosen in the first two rounds of the draft, where the computer was barred from making a pick.

Despite the solid reputation of the Seahawks defense, their ability to successfully draft pass rushers is rather average to slightly below average, compared to these other teams.  Lawrence Jackson, actually ends up being credited with 28.05% of the sacks produced by Seahawks draft picks, though for 2/3 of these sacks Jackson was on another team, as he was only a Seahawk for his first 2 seasons.  That leaves only the somewhat mediocre Darryl Tapp and Bruce Irvin as the most productive pass rushers to be selected by the team during this period of time.

Again, the reasons for this apparent failure seem a bit obvious.  The median results for these selections would be a -0.007 Kangaroo Score and a 0.172 Agility Score, along with 13.5 TFLs in their final two college seasons.  Those are rather uninspiring results, and not surprisingly they produced uninspiring outcomes.

To fill this pass rushing void, the Seahawks have had to look outside the draft, similar to the Patriots.  In free agency they acquired Cliff Avril (0.287 Kangaroo Score and a 0.215 Agility Score), who was probably just a slightly above average athlete, though he did average 15 tackles for a loss in his final two years in college, which is quite good.  They also managed to pick up Michael Bennett as an undrafted free agent in 2009, and while his average of 9 tackles for a loss in college was fairly pedestrian, his 0.837 Kangaroo Score suggested some reasonable athletic potential (we don't have the data to calculate his Agility Score, unfortunately).  They also traded the previously mentioned Darryl Tapp, for the enigmatic Chris Clemons, who's eventual successes I admittedly have no real explanation for. 

Let's wrap this up...

I realize that only listing the results for 5 different teams may seem like I have been cherry picking the data a bit.  On the other hand, I tend to be a bit long-winded, and I doubt anyone would make it through a post where I did this for every single team.  All I can really say is, these teams seemed to do a good job of illustrating my overall point, and really appeared to capture the general problems most teams have in selecting pass rushers.  These teams, for the most part, are the norm.

That isn't to say that there aren't teams who have done significantly better.  There are.  The Giants, the Rams, the Titans, and a few others have done quite a bit better at selecting these sorts of players, though still quite a bit short of the computer's theoretical results.  Unfortunately, examining the methods to their success aren't that interesting, as their "hits" typically seem to be the very sort of players that the computer would approve of, the highly athletic freak who was productive in college.

Perhaps the most interesting team I looked at was the Kansas City Chiefs, which was the only team to slightly surpass the computer's results.  The combination of selections like Jared Allen, Tamba Hali, and Justin Houston makes up for a rather intimidating group of pass rushers.  Still, with the possible exception of Tamba Hali, these players also fit the computer's mold for successful players, so I really don't see much for us to learn here, beyond what we already suspected.  The Chiefs have simply done a good job.

I'm sure some other people will criticize the amount of attention that I give to a player's ability to generate sacks.  To some extent, I suppose that is fair.  Almost any statistical category can be a bit overrated.  Despite that, I think it is interesting that even if we look beyond sack production, the computer's imaginary picks are still crushing pretty much everyone when it come to %GP and % GS.  So, when we simply consider a player's ability to get on the field and play, the computer is doing a much better job there as well.

While some might suspect that the computer had the advantage of hindsight, which is always 20/20, you have to remember that the computer made its selections based only on very basic pieces of data that would have been freely available at the time, so it really had no advantage in this sense.  You also have to remember the huge advantage that these NFL teams had, simply by being permitted to select players in the first two rounds of the draft, while the computer was prohibited from doing the same.  How much more lopsided do you suspect the computer's results would have been if this restriction had been lifted?

In the end, you can never really eliminate the risk that a player will be a failure.  Making any sort of guarantee about how following the path I proposed will assure success would be incredibly stupid.  The only real point that I am trying to convey is that, perhaps, if general managers resigned themselves to the likelihood that their instincts for identifying talented players were largely nonsensical beliefs manufactured by their egos, and instead based their decisions of measurable data, they probably couldn't do any worse, and quite possibly would actually improve (I suspect significantly).  When you really consider their histories of repeated failures, what is the real risk?

Friday, October 10, 2014

Pass Rushers...Now Lobotomized!

For reasons that are probably related to an undiagnosed mental disorder, our post on the Lobotomy Line was among the ones that I enjoyed the most.  Speculating on "what could have been", if a computer drafted offensive linemen based purely on measurable physical traits, gave me a bit of a chuckle.  For the most part, the computer had no problem matching, or simply outperforming, the draft picks that the Ravens made.  So, I thought we would play a similar game with pass rushers, to see how the computer could have theoretically done, in comparison to a handful of NFL teams.  The goal, just as it was when we looked at offensive linemen, was to go about this in as stupid and straightforward a manner as possible. 

When the NFL calendar rolls around to draft season in the spring, I frequently see people bemoaning the lack of quality pass rushers that are available.  Often, I hear people suggesting that if a team wants to improve their pass rush, they are pretty much required to take a player in the first round.  It's simply about supply and demand, so you have to pounce on the highly touted prospects, since there are so few quality players available.

To some extent they might be right.  The general quality of players taken in the first few rounds is a bit better, on average.  Unfortunately, I think a lot of this perception is also guided by the fact that teams just do a horrible job of identifying talent.  I suspect that there is a higher availability of talent than some may realize, though it sometimes gets overlooked.

When we discussed Reilly's views on drafting pass rushers (in the not very creatively titled Explosive Pass Rushers), we suggested a method that we believe should produce a successful pick between 65 - 82% of the time.  Of course, this suggested success rate hinged upon always drafting the best available pass rushers, in the eyes of the computer.  This doesn't necessarily mean taking somebody in the 1st round, as the computer's view on who the best player in a particular draft class can sometimes be surprising.  Still, it does frequently involve making a substantial investment of draft capital.  It's entirely possible that a team might not feel inclined to do that on a regular/annual basis.  When you start targeting lower tier pass rushers, your success rate will obviously go down a bit, though I think there are still some surprising opportunities out there.  Those less obvious opportunities are what we are going to explore today.

The rules for this game, however, will be slightly more complicated than when we were picking offensive linemen.  Just like we did before, the computer will pick one player per year, from the 2004-2013 draft classes.  We will discuss the computer's method of prioritizing these players in a second, but for now will just say that it is only looking at players whose weight ranges from 245-285 pounds (based on their combine weigh in), which is roughly the range in which you find most 4-3 defensive ends and 3-4 outside linebackers.  The computer will not be able to pick anyone who was taken before the 3rd round.  Even when we reach the 3rd round, the computer will be blocked from taking players if they wouldn't have been available to them according the additional rules which will follow.  The computer will also be barred from selecting players who went undrafted.  In the end, the computer will simply be looking for the best combination of athletic ability and statistical production in college.

First of all, let's discuss the athletic ability portion of this game.  Now, in the approach we took in the previously mentioned Explosive Pass Rushers post, we gave a bit more weight to some combine drills than others.  We're going to foolishly throw that idea out the window, for the moment.  Instead, we're just going to give equal weight to a player's Kangaroo Score (based in equal parts on their vertical jump and broad jump), and their Agility Score (based in equal parts on their short shuttle and 3-cone drill).  By simply combining these two scores together, we will arrive at a Total Score, which gives equal weight to all four of these drills.  Personally, I think this is a stupid thing to do, but stupid is what we are aiming for.  In the end, we are looking for players that are at least 0.500 standard deviations above average, relative to their peers, and when they are in short supply we will lower our standards to 0.300.  These thresholds were simply chosen because they are the ones we have used in the past, though they really weren't intended to be used in this way.  Again, stupidity and a lack of effort are our goals.  The 40-yard dash will almost completely be ignored in all of this, except that we will have a cutoff of 4.90 seconds for all of the prospects, which really isn't setting the bar very high at all.

Now, we get to the statistical production portion of the computer's judgment process.  All the computer will consider here is the average number of tackles for a loss that a player produced in their final two years in college.  Player's who attended non-Division I schools will be penalized with a 25% deduction from their actual results.  How did we settle on 25%?  Well, we kind of pulled that number out of our ass.  Remember our goal of being stupid?  We're achieving that quite nicely.  Regardless of the questionable (half-assed) scientific merits of this 25% figure, it only seemed fair to say that players at a lower level of competition should have their statistical production downgraded a bit.

Okay, so we have to combine this athletic potential, with the player's statistical production in some way.  Once again, we're just going to go with the method suggested near the end of the post on Explosive Pass Rushers, even though it was intended to be used somewhat differently.

Players Over 0.500 Players Over 0.300
Avg. TFL Round Avg. TFL Round
15                                 1st N/A N/A
14                                 2nd N/A N/A
13                                 3rd 15                                  3rd
12                                 4th 14                                  4th
11                                 5th 13                                  5th
10                                 6th 12                                  6th
9                                  7th 11                                  7th

Let's give an example of how this works.  If a player had a Total Score over 0.500 (for his athletic ability), and averaged 12 tackles for a loss in his final two college seasons, the computer would give him a 4th round draft grade.  The computer would not be allowed to select him any higher than this.  At the same time, a player with a Total Score of 0.300, but who averaged 15 tackles for a loss, would be given a 3rd round draft grade, which would put them slightly ahead of the previously mentioned, but more athletically gifted player.  I've always had a policy of not drafting players with sub-0.500 total scores before the 3rd round, which is why you see the 0.300 group having no proposed draft grade that could have them selected in the 1st or 2nd round, which we've carried over to this game, even though it is irrelevant for our purposes, since none of the computer's picks will come in the first two rounds anyway.

In the end, this might seem fairly complicated to some people, but it really isn't.  It's just a very basic system of balancing potential versus actual proven performance.  The less athletically gifted a player is, the more the computer will demand to see evidence of exceptional production.  The more freakishly gifted an athlete is, the more the computer is going to be willing to gamble on potential and upside.  I suspect my description of the process the computer will use makes it sound much more complicated than it really is. Let me see if I can simplify things.

We want freakish athletes with a lot of tackles for a loss.

There, that sounds a lot better, doesn't it?  Oh well, I tried.  Let's just move on to the list of players that the computer would have selected over this 10 year span.

Mike Catapano
207th Pick

Even with a 25% reduction to his average of 12.75 TFL (since he came from Princeton), Catapano still winds up with a result of 9.56 TFL.  This in combination with his Total Score of 0.789, would result in the computer giving him a 7th round grade, which is in fact where he was selected by the Chiefs.  While his contributions have been minimal so far, his Kangaroo Score of 1.176, and Agility Score of 0.402 are very intriguing.  Only time will tell what he will become.

Miles Burris
129th pick
San Diego State

At just 246 pounds, Burris barely qualifies for this list.  Still, the rules are the rules, so he is the computer's pick for 2012.  At his current playing weight of 240 pounds, the computer wouldn't have been able to select him at all.  Averaging 19.5 TFL, with a Total Score of 0.783, the computer would give him a 1st round grade, which is obviously a bit insane.  It's perhaps fortunate, in this case, that the computer is barred from taking him any higher than the 3rd round, which is one round ahead of where Burris was actually selected.  That's a price I would have been perfectly happy to pay for someone with his physical gifts (a 0.319 Kangaroo Score, and a 1.246 Agility Score).  I have to admit that I am fairly interested in how Burris' career progresses, as I've briefly mentioned before, though I do question the way in which the Raiders have chosen to utilize him.

Justin Houston
70th Pick

Averaging 16.75 TFL, with a Total Score of 1.043, the computer would have given Justin Houston a 1st round grade.  Fortunately, NFL teams didn't feel the same way about this, and let him fall to the 3rd round, where the computer would have happily and quickly picked him up.  In retrospect, I think people would agree that the computer's evaluation was probably the more correct one in this case.  It really does seem odd to me that NFL teams would have let a player with a 1.581 Kangaroo Score and a 0.506 Agility Score  slide this far... until you consider that none of these numbers actually matter to them.

Austen Lane 
153rd Pick
Murray State

When we adjust Lane's 20.75 TFL to 15.56 TFL, due to his level of competition, and combine it with his barely passable Total Score of 0.304, he becomes the computer's pick for 2010.  The options for the computer were fairly slim in this year, so we sort of have to write this one off as a failure, as Lane did very little in his brief time in the NFL.

Michael Johnson
70th Pick
Georgia Tech

Depending on whether or not we round up Johnson's 11.75 TFL,  and combine it with his Total Score of 0.732, the computer would have given Johnson something in the range of a 3rd or 4th round grade.  Since I can't find another 2009 prospect that the computer would have even viewed as draftable, we're sort of forced to go with the round-it-up-to-12 TFL option, and say that the computer would take Johnson in the 3rd round, which is indeed where he was actually selected.  I'm not really a huge fan of Johnson, but he's been relatively productive, so we can live with this result.

Trevor Scott
169th Pick

With an average of 14.25 TFL, and a Total Score of 0.608, Scott would have been the computer's pick in 2008.  Injuries, and perhaps a lack of opportunity, seem to have slowed Scott's career.  He had 12 sacks in his first two NFL seasons, but has rarely been asked to start many games since then.  He's still bouncing around the league, and I have to wonder if he may be a bit underutilized.

Brian Robison
102nd pick

This is a fairly interesting situation.  With Robison's average of 12 TFL, along with his Total Score of 1.154, the computer would have given him a 4th round grade, which is in fact, precisely where he was selected.  While his athletic ability is spectacular (with a 1.383 Kangaroo Score, and a 0.926 Agility Score), it's only his college production that held him back from getting a much higher grade.  While his NFL career started off a bit slowly, this seems to largely be due to playing behind Jared Allen and Ray Edwards in his first few years.  Since becoming a regular starter in 2011, by which point he was unfortunately already 28 years old, Robison has averaged 8.5 sacks per year, from 2011-2013.  I strongly suspect that the Vikings might have wasted a lot of the best years that Robison had to offer.

Chris Gocong
71st Pick
Cal Poly

Chris Gocong never became the sort of pass rusher that I would have hoped for, but he seems to have still been a serviceable player.  What makes this even more disappointing, is that the computer's 2md and 3rd choices for 2006 would have been Rob Ninkovich and Mark Anderson, who were both more successful.  Still, I suppose things could have been worse.

Justin Tuck
Notre Dame

With an average of 16.5 TFL, and a Total Score of 0.528, the computer would have given Justin Tuck a 1st round grade.  Fortunately for us, he actually fell to the 3rd round, where the computer would have happily pounced on him.  While Tuck has exclusively played as a 4-3 defensive end, I think his 1.060 Kangaroo Score and -0.004 Agility Score might have also made playing 3-4 OLB a real possibility.  I guess we'll never know for sure.

Roderick Green
153rd Pick
Central Missouri

This is a perfect example of why I am hesitant to dumb down our little system for identifying pass rushers.  Based on our normal method of doing things, where we give more weight to some combine drills than we do for others, Green would be rated much lower.  He also only weighed 245 pounds at his weigh in, so he would just barely qualify for this list.  In reality, there are numerous factors which would have made it unlikely that this would have been our actual pick, and it's particularly unfortunate since the computer's next two options would have been the vastly more successful Shaun Phillips and Jared Allen.  Still, the rules are the rules.  Roderick F***ing Green it is!

Okey-dokey, the computer has made it's ten picks, so let's see what sort of picture this presents to us.  Depending on whether we are considering the average or median pick at which these players were actually taken at, the results would be around the 119th or 115th pick, which places most of them solidly within the 4th round area.  So, these players generally didn't cost much to acquire.  When you consider how low most peoples' expectations are for players who are selected at that point in the draft, our expectations should probably be quite modest.

When we consider how productive these players have been, I want to focus not just on their sacks, but also on how many games they've played in, and how many games they were listed as a starter.  For this, we're going to refer to Potential Games Played.  Instead of just saying how many games a player actually played, we're going to count how many they could have theoretically played.  If a player was drafted 2 years ago, they could have potentially played in 32 regular season games.  We want to use Potential Games Played, because it will actually give us a measuring stick that is a bit crueler and less forgiving of injuries, or players who briefly played, but were booted out of the league.  We'll just abbreviate this to POTGP.  The results listed here will just reflect what they have done from the time they were drafted, through the end of the 2013 NFL season.

Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Mike Catapano 16 15 0 1 0.062 93.75 0
Miles Burris 32 22 15 1.5 0.046 68.75 46.87
Justin Houston 48 43 37 29.5 0.614 89.58 77.08
Austen Lane 64 30 17 3 0.046 46.87 26.56
Michael Johnson 80 79 45 26.5 0.331 98.75 56.25
Trevor Scott 96 76 18 16.5 0.171 79.16 18.75
Brian Robison 112 110 54 39 0.348 98.21 48.21
Chris Gocong 128 79 67 9.5 0.074 61.71 52.34
Justin Tuck 144 127 90 61.5 0.427 88.19 62.5
Roderick Green 160 54 0 12 0.075 33.75 0

Total 880 635 343 200 0.227 75.15 38.97

I think it's fairly safe to say that Justin Houston, Brian Robison, and Justin Tuck would probably be viewed by most people as successes, and Michael Johnson could arguably fall into that category as well.  Those four players alone account for 156.5 of the computer's 200 sacks.  So, let's say that the computer has been successful about 40% of the time.  Then, we have a couple of oddballs like Miles Burris, Chris Gocong, and perhaps another debatable player of your choice, where you might not consider them successes, but you probably wouldn't call them failures either.  So, depending on how you look at things, the computer might be edging somewhat closer to a 50% success rate.

However you slice it, these results are clearly below the 65- 82% success rate that I have suggested might be possible (numbers which I still believe are attainable).  You have to remember though, we're dealing with a computer that views all the combine drills as being of equal importance in this scenario, which I believe is a fairly huge mistake.  The restriction banning the computer from taking players in the first 2 rounds is also a rather significant handicap.  So, all things considered, I would say that the computer did a fairly decent job, under the circumstances, though how this compares to the results of an actual NFL teams is something we will get into later.

Now, let's address some obvious questions. 

Isn't judging a player solely on their sack production a bit stupid?  Shouldn't they also be judged on how they perform against the running game?

It's not that I believe defending against the running game is completely unimportant, but people get carried away with praising such versatility.   I think if you had a player (defensive end or 3-4 OLB) who was good against the run, but only producing about 3 sacks a year, you'd probably be perfectly willing to trade them for a somewhat one dimensional pass rusher who was capable of putting up 10 sacks a year.  Hell, Dwight Freeney made a career out of it.  While I was never a huge fan of Freeney (Robert Mathis on the other hand, I like quite a bit), he was very good at rushing the passer.  With the way the rules in the NFL are changing to make it nearly impossible for defensive backs to properly cover someone without drawing a penalty flag, I think the only real option is to murder the quarterback.  I also tend to think that a good pass rush benefits the players in coverage, much more than the coverage is likely to benefit the pass rush, though that is an argument that is difficult to resolve.

Also, pass rushers are simply the rarer commodity.  If you just want somebody who is solid against the run, well, you can find someone like that in the 7th round without too much difficulty.

Isn't it cheating to draft players without giving any thought as to whether they are suited for a 4-3 defense, or a 3-4?

I think there tends to be a fairly fine line that divides a 4-3 defensive end from a 3-4 OLB, and a lot of these distinctions get exaggerated.  It's not that I don't believe that there is a difference, it's just that I think the difference probably doesn't matter too most cases.  People also might underestimate the degree to which NFL teams could already be employing players in defensive schemes that might not really suit them.  Take Brian Orakpo, for example.  With his 1.979 Kangaroo Score, he has the sort of explosive pass rushing power that makes me salivate.  On the other hand, his somewhat below average Agility Score of -0.312, could suggest some problems dropping back into coverage.  To some extent, I feel playing in a 4-3 might have been a better fit for him, but overall, he's still been quite successful a getting to the quarterback, averaging about 7.9 sacks per year from 2009-2013, despite missing nearly a full season's worth of games due to injuries.  On the other hand, Jason Babin might fit better in a 3-4, yet continues to play in a 4-3.  With a middling 0.492 Kangaroo Score, but a 0.988 Agility Score, I think he's probably better suited to playing with a bit more space. 

While I would agree that Michael Johnson and Austen Lane probably fit better in a 4-3, and Miles Burris in a 3-4, I'd say that the rest of the computer's picks might be surprisingly flexible.  You also can't underestimate the value of adjusting your team's defensive scheme based on the talent that is available at the time, though most organizations seem to loathe doing so...because they stubbornly think their playbook matters ore than the talent implementing it.

Is it actually sensible to be drafting players like this, every single year?

Well, that all depends on whether you actually care about having a successful pass rush.  If a team is unwilling to invest in the position, then they certainly can't complain about being bad at rushing the passer.  I will say though, that this level of investment in pass rushers really isn't that different from what a lot of NFL teams actually do (though this comparison is going to have to wait until the next post).  In reality, I would probably be less rigid about selecting a player every single year, but might double down on the position in certain years where the talent appears to be stronger.  In the end, this would still probably result in a similar total number of players being taken though.

The biggest problem would probably be continuing to draft pass rushers, even when you already have good ones on the team.  Personally, I think teams should do precisely that, though I can understand how they might feel less incentive to do so.  The goal, from my perspective, isn't just to draft talented players.  It's to draft talented players, that push other talented payers out the door, so that you don't have to re-sign them to the sort of costly contracts that veterans demand.  I think the impact of quality pass rushers is obviously quite high for most teams.  Unfortunately, retaining a high quality veteran pass rusher can be unreasonably expensive, especially when the chances of their talents declining with age seems inevitable.  When you factor in the relative ease with which it appears that replacement talent can be acquired in the draft, I think there's a lot of incentive to not grow too fond of the bird in the hand.

Do I really think an outcome like this is actually likely, or possible, in reality?

Hmm, probably.  I think it is certainly doable, but it would really depend on a lot of difficult to assess.factors, that are outside of the scope of this little test.  I'm really only doing this to see what results might have been possible, if we unleashed a brain damaged computer operating on auto-pilot.  Personally, I think that significantly better results would be quite attainable, and some of the computer's more questionable decisions could have been avoided.  There are additional bits of information the computer is simply unaware of in this game.  A pinch of common sense wouldn't have hurt either  Still, even if the real world results were only half as encouraging as the ones seen here, I believe they would easily beat the results we see from actual NFL teams, but that is something we'll explore in the next post.