Friday, October 10, 2014

Pass Rushers...Now Lobotomized!

For reasons that are probably related to an undiagnosed mental disorder, our post on the Lobotomy Line was among the ones that I enjoyed the most.  Speculating on "what could have been", if a computer drafted offensive linemen based purely on measurable physical traits, gave me a bit of a chuckle.  For the most part, the computer had no problem matching, or simply outperforming, the draft picks that the Ravens made.  So, I thought we would play a similar game with pass rushers, to see how the computer could have theoretically done, in comparison to a handful of NFL teams.  The goal, just as it was when we looked at offensive linemen, was to go about this in as stupid and straightforward a manner as possible. 

When the NFL calendar rolls around to draft season in the spring, I frequently see people bemoaning the lack of quality pass rushers that are available.  Often, I hear people suggesting that if a team wants to improve their pass rush, they are pretty much required to take a player in the first round.  It's simply about supply and demand, so you have to pounce on the highly touted prospects, since there are so few quality players available.

To some extent they might be right.  The general quality of players taken in the first few rounds is a bit better, on average.  Unfortunately, I think a lot of this perception is also guided by the fact that teams just do a horrible job of identifying talent.  I suspect that there is a higher availability of talent than some may realize, though it sometimes gets overlooked.

When we discussed Reilly's views on drafting pass rushers (in the not very creatively titled Explosive Pass Rushers), we suggested a method that we believe should produce a successful pick between 65 - 82% of the time.  Of course, this suggested success rate hinged upon always drafting the best available pass rushers, in the eyes of the computer.  This doesn't necessarily mean taking somebody in the 1st round, as the computer's view on who the best player in a particular draft class can sometimes be surprising.  Still, it does frequently involve making a substantial investment of draft capital.  It's entirely possible that a team might not feel inclined to do that on a regular/annual basis.  When you start targeting lower tier pass rushers, your success rate will obviously go down a bit, though I think there are still some surprising opportunities out there.  Those less obvious opportunities are what we are going to explore today.

The rules for this game, however, will be slightly more complicated than when we were picking offensive linemen.  Just like we did before, the computer will pick one player per year, from the 2004-2013 draft classes.  We will discuss the computer's method of prioritizing these players in a second, but for now will just say that it is only looking at players whose weight ranges from 245-285 pounds (based on their combine weigh in), which is roughly the range in which you find most 4-3 defensive ends and 3-4 outside linebackers.  The computer will not be able to pick anyone who was taken before the 3rd round.  Even when we reach the 3rd round, the computer will be blocked from taking players if they wouldn't have been available to them according the additional rules which will follow.  The computer will also be barred from selecting players who went undrafted.  In the end, the computer will simply be looking for the best combination of athletic ability and statistical production in college.

First of all, let's discuss the athletic ability portion of this game.  Now, in the approach we took in the previously mentioned Explosive Pass Rushers post, we gave a bit more weight to some combine drills than others.  We're going to foolishly throw that idea out the window, for the moment.  Instead, we're just going to give equal weight to a player's Kangaroo Score (based in equal parts on their vertical jump and broad jump), and their Agility Score (based in equal parts on their short shuttle and 3-cone drill).  By simply combining these two scores together, we will arrive at a Total Score, which gives equal weight to all four of these drills.  Personally, I think this is a stupid thing to do, but stupid is what we are aiming for.  In the end, we are looking for players that are at least 0.500 standard deviations above average, relative to their peers, and when they are in short supply we will lower our standards to 0.300.  These thresholds were simply chosen because they are the ones we have used in the past, though they really weren't intended to be used in this way.  Again, stupidity and a lack of effort are our goals.  The 40-yard dash will almost completely be ignored in all of this, except that we will have a cutoff of 4.90 seconds for all of the prospects, which really isn't setting the bar very high at all.

Now, we get to the statistical production portion of the computer's judgment process.  All the computer will consider here is the average number of tackles for a loss that a player produced in their final two years in college.  Players who attended non-Division I schools will be penalized with a 25% deduction from their actual results.  How did we settle on 25%?  Well, we kind of pulled that number out of our ass.  Remember our goal of being stupid?  We're achieving that quite nicely.  Regardless of the questionable (half-assed) scientific merits of this 25% figure, it only seemed fair to say that players at a lower level of competition should have their statistical production downgraded a bit.

Okay, so we have to combine this athletic potential, with the player's statistical production in some way.  Once again, we're just going to go with the method suggested near the end of the post on Explosive Pass Rushers, even though it was intended to be used somewhat differently.


Players Over 0.500 Players Over 0.300
Avg. TFL Round Avg. TFL Round
15                                 1st N/A N/A
14                                 2nd N/A N/A
13                                 3rd 15                                  3rd
12                                 4th 14                                  4th
11                                 5th 13                                  5th
10                                 6th 12                                  6th
9                                  7th 11                                  7th


Let's give an example of how this works.  If a player had a Total Score over 0.500 (for his athletic ability), and averaged 12 tackles for a loss in his final two college seasons, the computer would give him a 4th round draft grade.  The computer would not be allowed to select him any higher than this.  At the same time, a player with a Total Score of 0.300, but who averaged 15 tackles for a loss, would be given a 3rd round draft grade, which would put them slightly ahead of the previously mentioned, but more athletically gifted player.  I've always had a policy of not drafting players with sub-0.500 total scores before the 3rd round, which is why you see the 0.300 group having no proposed draft grade that could have them selected in the 1st or 2nd round, which we've carried over to this game, even though it is irrelevant for our purposes, since none of the computer's picks will come in the first two rounds anyway.

In the end, this might seem fairly complicated to some people, but it really isn't.  It's just a very basic system of balancing potential versus actual proven performance.  The less athletically gifted a player is, the more the computer will demand to see evidence of exceptional production.  The more freakishly gifted an athlete is, the more the computer is going to be willing to gamble on potential and upside.  I suspect my description of the process the computer will use makes it sound much more complicated than it really is. Let me see if I can simplify things.

We want freakish athletes with a lot of tackles for a loss.

There, that sounds a lot better, doesn't it?  Oh well, I tried.  Let's just move on to the list of players that the computer would have selected over this 10 year span.



2013
Mike Catapano
207th Pick
Princeton

Even with a 25% reduction to his average of 12.75 TFL (since he came from Princeton), Catapano still winds up with a result of 9.56 TFL.  This in combination with his Total Score of 0.789, would result in the computer giving him a 7th round grade, which is in fact where he was selected by the Chiefs.  While his contributions have been minimal so far, his Kangaroo Score of 1.176, and Agility Score of 0.402 are very intriguing.  Only time will tell what he will become.

2012
Miles Burris
129th pick
San Diego State

At just 246 pounds, Burris barely qualifies for this list.  Still, the rules are the rules, so he is the computer's pick for 2012.  At his current playing weight of 240 pounds, the computer wouldn't have been able to select him at all.  Averaging 19.5 TFL, with a Total Score of 0.783, the computer would give him a 1st round grade, which is obviously a bit insane.  It's perhaps fortunate, in this case, that the computer is barred from taking him any higher than the 3rd round, which is one round ahead of where Burris was actually selected.  That's a price I would have been perfectly happy to pay for someone with his physical gifts (a 0.319 Kangaroo Score, and a 1.246 Agility Score).  I have to admit that I am fairly interested in how Burris' career progresses, as I've briefly mentioned before, though I do question the way in which the Raiders have chosen to utilize him.

2011
Justin Houston
70th Pick
Georgia

Averaging 16.75 TFL, with a Total Score of 1.043, the computer would have given Justin Houston a 1st round grade.  Fortunately, NFL teams didn't feel the same way about this, and let him fall to the 3rd round, where the computer would have happily and quickly picked him up.  In retrospect, I think people would agree that the computer's evaluation was probably the more correct one in this case.  It really does seem odd to me that NFL teams would have let a player with a 1.581 Kangaroo Score and a 0.506 Agility Score  slide this far... until you consider that none of these numbers actually matter to them.

2010
Austen Lane 
153rd Pick
Murray State

When we adjust Lane's 20.75 TFL to 15.56 TFL, due to his level of competition, and combine it with his barely passable Total Score of 0.304, he becomes the computer's pick for 2010.  The options for the computer were fairly slim in this year, so we sort of have to write this one off as a failure, as Lane did very little in his brief time in the NFL.

2009
Michael Johnson
70th Pick
Georgia Tech

Depending on whether or not we round up Johnson's 11.75 TFL,  and combine it with his Total Score of 0.732, the computer would have given Johnson something in the range of a 3rd or 4th round grade.  Since I can't find another 2009 prospect that the computer would have even viewed as draftable, we're sort of forced to go with the round-it-up-to-12 TFL option, and say that the computer would take Johnson in the 3rd round, which is indeed where he was actually selected.  I'm not really a huge fan of Johnson, but he's been relatively productive, so we can live with this result.

2008
Trevor Scott
169th Pick
Buffalo

With an average of 14.25 TFL, and a Total Score of 0.608, Scott would have been the computer's pick in 2008.  Injuries, and perhaps a lack of opportunity, seem to have slowed Scott's career.  He had 12 sacks in his first two NFL seasons, but has rarely been asked to start many games since then.  He's still bouncing around the league, and I have to wonder if he may be a bit underutilized.

2007
Brian Robison
102nd pick
Texas

This is a fairly interesting situation.  With Robison's average of 12 TFL, along with his Total Score of 1.154, the computer would have given him a 4th round grade, which is in fact, precisely where he was selected.  While his athletic ability is spectacular (with a 1.383 Kangaroo Score, and a 0.926 Agility Score), it's only his college production that held him back from getting a much higher grade.  While his NFL career started off a bit slowly, this seems to largely be due to playing behind Jared Allen and Ray Edwards in his first few years.  Since becoming a regular starter in 2011, by which point he was unfortunately already 28 years old, Robison has averaged 8.5 sacks per year, from 2011-2013.  I strongly suspect that the Vikings might have wasted a lot of the best years that Robison had to offer.

2006
Chris Gocong
71st Pick
Cal Poly

Chris Gocong never became the sort of pass rusher that I would have hoped for, but he still seems to have been a serviceable player.  What makes this even more disappointing, is that the computer's 2md and 3rd choices for 2006 would have been Rob Ninkovich and Mark Anderson, who were both more successful.  Still, I suppose things could have been worse.

2005
Justin Tuck
74th
Notre Dame

With an average of 16.5 TFL, and a Total Score of 0.528, the computer would have given Justin Tuck a 1st round grade.  Fortunately for us, he actually fell to the 3rd round, where the computer would have happily pounced on him.  While Tuck has exclusively played as a 4-3 defensive end, I think his 1.060 Kangaroo Score and -0.004 Agility Score might have also made playing 3-4 OLB a real possibility.  I guess we'll never know for sure.

2004
Roderick Green
153rd Pick
Central Missouri

This is a perfect example of why I am hesitant to dumb down our little system for identifying pass rushers.  Based on our normal method of doing things, where we give more weight to some combine drills than we do for others, Green would be rated much lower.  He also only weighed 245 pounds at his weigh in, so he would just barely qualify for this list.  In reality, there are numerous factors which would have made it unlikely that this would have been our actual pick, and it's particularly unfortunate since the computer's next two options would have been the vastly more successful Shaun Phillips and Jared Allen.  Still, the rules are the rules.  Roderick F***ing Green it is!



Okey-dokey, the computer has made it's ten picks, so let's see what sort of picture this presents to us.  Depending on whether we are considering the average or median pick at which these players were actually taken at, the results would be around the 119th or 115th pick, which places most of them solidly within the 4th round area.  So, these players generally didn't cost much to acquire.  When you consider how low most peoples' expectations are for players who are selected at that point in the draft, our expectations should probably be quite modest.

When we consider how productive these players have been, I want to focus not just on their sacks, but also on how many games they've played in, and how many games they were listed as a starter.  For this, we're going to refer to Potential Games Played.  Instead of just saying how many games a player actually played, we're going to count how many they could have theoretically played.  If a player was drafted 2 years ago, they could have potentially played in 32 regular season games.  We want to use Potential Games Played, because it will actually give us a measuring stick that is a bit crueler and less forgiving of injuries, or players who briefly played, but were booted out of the league.  We'll just abbreviate this to POTGP.  The results listed here will just reflect what they have done from the time they were drafted, through the end of the 2013 NFL season.


Player   POTGP        GP        GS      Sacks  Sack/POTGP      % GP      % GS
Mike Catapano 16 15 0 1 0.062 93.75 0
Miles Burris 32 22 15 1.5 0.046 68.75 46.87
Justin Houston 48 43 37 29.5 0.614 89.58 77.08
Austen Lane 64 30 17 3 0.046 46.87 26.56
Michael Johnson 80 79 45 26.5 0.331 98.75 56.25
Trevor Scott 96 76 18 16.5 0.171 79.16 18.75
Brian Robison 112 110 54 39 0.348 98.21 48.21
Chris Gocong 128 79 67 9.5 0.074 61.71 52.34
Justin Tuck 144 127 90 61.5 0.427 88.19 62.5
Roderick Green 160 54 0 12 0.075 33.75 0








Total 880 635 343 200 0.227 75.15 38.97



I think it's fairly safe to say that Justin Houston, Brian Robison, and Justin Tuck would probably be viewed by most people as successes, and Michael Johnson could arguably fall into that category as well.  Those four players alone account for 156.5 of the computer's 200 sacks.  So, let's say that the computer has been successful about 40% of the time.  Then, we have a couple of oddballs like Miles Burris, Chris Gocong, and perhaps another debatable player of your choice, where you might not consider them successes, but you probably wouldn't call them failures either.  So, depending on how you look at things, the computer might be edging somewhat closer to a 50% success rate.

However you slice it, these results are clearly below the 65-82% success rate that I have suggested might be possible (numbers which I still believe are attainable).  You have to remember though, we're dealing with a computer that views all the combine drills as being of equal importance in this scenario, which I believe is a fairly huge mistake.  The restriction banning the computer from taking players in the first 2 rounds is also a rather significant handicap.  So, all things considered, I would say that the computer did a fairly decent job, under the circumstances, though how this compares to the results of an actual NFL teams is something we will get into later.

Now, let's address some obvious questions. 

Isn't judging a player solely on their sack production a bit stupid?  Shouldn't they also be judged on how they perform against the running game?

It's not that I believe defending against the running game is completely unimportant, but people get carried away with praising such versatility.   I think if you had a player (defensive end or 3-4 OLB) who was good against the run, but only producing about 3 sacks a year, you'd probably be perfectly willing to trade them for a somewhat one dimensional pass rusher who was capable of putting up 10 sacks a year.  Hell, Dwight Freeney made a career out of it.  While I was never a huge fan of Freeney (Robert Mathis on the other hand, I like quite a bit), he was very good at rushing the passer.  With the way the rules in the NFL are changing to make it nearly impossible for defensive backs to properly cover someone without drawing a penalty flag, I think the only real option is to murder the quarterback.  I also tend to think that a good pass rush benefits the players in coverage, much more than the coverage is likely to benefit the pass rush, though that is an argument that is difficult to resolve.

Also, pass rushers are simply the rarer commodity.  If you just want somebody who is solid against the run, well, you can find someone like that in the 7th round without too much difficulty.

Isn't it cheating to draft players without giving any thought as to whether they are suited for a 4-3 defense, or a 3-4?

I think there tends to be a fairly fine line that divides a 4-3 defensive end from a 3-4 OLB, and a lot of these distinctions get exaggerated.  It's not that I don't believe that there is a difference, it's just that I think the difference probably doesn't matter too much...in most cases.  People also might underestimate the degree to which NFL teams could already be employing players in defensive schemes that might not really suit them.  Take Brian Orakpo, for example.  With his 1.979 Kangaroo Score, he has the sort of explosive pass rushing power that makes me salivate.  On the other hand, his somewhat below average Agility Score of -0.312, could suggest some problems dropping back into coverage.  To some extent, I feel playing in a 4-3 might have been a better fit for him, but overall, he's still been quite successful a getting to the quarterback, averaging about 7.9 sacks per year from 2009-2013, despite missing nearly a full season's worth of games due to injuries.  On the other hand, Jason Babin might fit better in a 3-4, yet continues to play in a 4-3.  With a middling 0.492 Kangaroo Score, but a 0.988 Agility Score, I think he's probably better suited to playing with a bit more space.

While I would agree that Michael Johnson and Austen Lane probably fit better in a 4-3, and Miles Burris in a 3-4, I'd say that the rest of the computer's picks might be surprisingly flexible.  You also can't underestimate the value of adjusting your team's defensive scheme based on the talent that is available at the time, though most organizations seem to loathe doing so...because they stubbornly think their playbook matters ore than the talent implementing it.

Is it actually sensible to be drafting players like this, every single year?

Well, that all depends on whether you actually care about having a successful pass rush.  If a team is unwilling to invest in the position, then they certainly can't complain about being bad at rushing the passer.  I will say though, that this level of investment in pass rushers really isn't that different from what a lot of NFL teams actually do (though this comparison is going to have to wait until the next post).  In reality, I would probably be less rigid about selecting a player every single year, but might double down on the position in certain years where the talent appears to be stronger.  In the end, this would still probably result in a similar total number of players being taken though.

The biggest problem would probably be continuing to draft pass rushers, even when you already have good ones on the team.  Personally, I think teams should do precisely that, though I can understand how they might feel less incentive to do so.  The goal, from my perspective, isn't just to draft talented players.  It's to draft talented players, that push other talented payers out the door, so that you don't have to re-sign them to the sort of costly contracts that veterans demand.  I think the impact of quality pass rushers is obviously quite high for most teams.  Unfortunately, retaining a high quality veteran pass rusher can be unreasonably expensive, especially when the chances of their talents declining with age seems inevitable.  When you factor in the relative ease with which it appears that replacement talent can be acquired in the draft, I think there's a lot of incentive to not grow too fond of the bird in the hand.

Do I really think an outcome like this is actually likely, or possible, in reality?

Hmm, probably.  I think it is certainly doable, but it would really depend on a lot of difficult to assess.factors, that are outside of the scope of this little test.  I'm really only doing this to see what results might have been possible, if we unleashed a brain damaged computer operating on auto-pilot.  Personally, I think that significantly better results would be quite attainable, and some of the computer's more questionable decisions could have been avoided.  There are additional bits of information the computer is simply unaware of in this game.  A pinch of common sense wouldn't have hurt either  Still, even if the real world results were only half as encouraging as the ones seen here, I believe they would easily beat the results we see from actual NFL teams, but that is something we'll explore in the next post.


3 comments:

  1. You didn't do horrible for a brain damaged computer man.

    ReplyDelete
  2. I don't know if you know or not but charles johnson (browns dude u wrote about) had 6 catches for 85 yards...and with the vikings WR issues (they all suck ecept for patterson) and no Number 3 WR...he'll get playing time and bridgewater trusts him so...hello proudction.

    ReplyDelete
    Replies
    1. Oh, I've definitely been paying attention to Charles Johnson. It will be interesting to see what happens if he continues to get an opportunity. So far, he has caught 85.7% of the passes thrown his way, compared to a mere 47.5% for Cordarelle Patterson, who I've always had some doubts about.

      Delete