Reilly and I generally prefer to avoid the subject of quarterbacks. We've already found plenty of other ways to prove that we are fools, without dipping our feet into the peculiar and dangerous arena of QB speculation. Still, we feel as if this year's crop of QB prospects is sort of odd, especially when you look at Jared Goff and Carson Wentz, who are supposedly the top prospects at their position. The more we looked at the data related to these players, the more interesting we felt their individual stories were. I can't say that we feel terribly confident about our ability to present our arguments regarding these two players in a way that will be sufficiently persuasive, but hopefully you can bear with us.
Because quarterbacks don't really fit within the mold of what we normally discuss, which often relates to our obsession with highly productive athletic freaks, Reilly and I have never been uncertain as to how we should pursue this topic. It's one of those odd positions where what is going on between a player's ears probably matters more than how superior an athlete they are. We also tend to feel a bit uncomfortable with purely subjective opinions of a player's performance, but with quarterbacks we sometimes have to wonder if that sort of analysis might be more valid. Still, that really isn't our way of doing things.
So, what we decided to do was to analyze the college production for about 20 of this year's QB prospects. Our methods were half-assed, and highly questionable, as you would probably expect if you have been here before. First, we wanted to calculate what percentage of an individual's pass attempts resulted in a touchdown, interception or a sack, in each season that a prospect played. We also kept a close eye on their YPA (yards per attempt), since we feel this has significant value when it comes to a prospect's willingness to be more aggressive with the ball. Then, we ran their game to game numbers, and attempted to compare their results to the touchdown rates, interception rates, YPA and sack rates of the different opponents they faced. We were basically trying to find out which games the player might have performed above or below the expectations one might have, relative to the opponent they were facing.
Now, we have to admit that we still aren't certain about whether this is a method we will pursue in the future. It's just something we are kicking around, and have gradually been trying to improve over the past few years. Maybe we'll come up with something better, or maybe we'll become frustrated and give up on pursuing the highly frustrating subject of quarterbacks. Either way, we were kind of amused to see what the computer had to say about this little endeavor.
When we reached the end of our calculations, we were able to produce a rough sort of score that possibly suggested which players were most consistently performing above expectation, relative to the opponents they faced. The interesting thing was, the computer's most highly rated players were also this year's presumed favorites, Carson Wentz and Jared Goff (what a fortunate coincidence!). Unfortunately, there was still a lot of data hidden in all of this, and our investigation took us in some weird directions.
Let's start off by simply comparing the most superficial results for these two players.
On the surface, these two appear to have a fair bit in common, and look rather impressive. They both show a high rate of touchdown production, with a seemingly respectable ability to avoid interceptions. When it comes to pushing the ball deeper down the field, and being aggressive, they both seem to get good marks in that area as well. They also seem to potentially display a gradually improving ability to avoid sacks, from year to year.
Unfortunately, we think a lot of these similarities are bullshit. The more we looked at the data, the more we started to see some glaring differences in their numbers, which caused us a fair amount of concern. It also made us start to suspect that one of these players might be a fraud, while the other one could be fairly interesting. I suppose the title of this post might be tipping our hand as to which prospect our conclusions ended up pointing towards as the safer investment.
Okay, let's get our geek on!
Absence Makes The Heart Grow Fonder
One of the potentially interesting questions that occurred to us, was to ask whether these results are purely a product of the quarterback, or a reflection of their team's passing offense as a whole. This could be summed up as "Are Matthew Stafford's statistics really a product of his skill, or a result of playing with Calvin Johnson?" You can insert your own scenario where a player might be benefiting from surrounding talent here. "Was Ringo Starr a great drummer, or was he just benefiting from playing with The Beatles?", we could apply this idea to pretty much anything.
Now, here was one other interesting issue that these players had in common. At no point did either one of them have a receiver who accounted for a terribly high percentage of his team's receiving yards. Wentz might have leaned a tad bit more on individual receivers, but not by much. In 2015, 23.9% of North Dakota State's receiving yards went to R.J. Urzendowski. For Goff, the most he ever leaned on a receiver was in 2013, when 21.4% of his team's receiving yards went to Chris Harper. Either way, these sorts of low percentages from a group of receivers would most likely eliminate the possibility that either quarterback was benefiting from having a particularly dominant receiver helping them out.
One possible difference that is worth noting here is the rate at which their respective receivers were dropping passes. With Wentz, it would appear that his receivers dropped about 5.5% of the balls thrown their way, in 2015. In Goff's case, this number climbs to 9.8%. While we can't validate this information ourselves, we are instead using data provided by Ian Wharton. If this information is accurate, then we would probably have to say that Goff's receivers potentially presented quite a bit more of a disadvantage than the ones that Wentz played with.
Then, we come to a different sort of question. While the passing results for both quarterbacks appear to be impressive, how do these results differ from the quarterbacks that preceded them at their respective schools? After all, you would expect a truly superior and unique talent to produce results that clearly separate them from some other bozo who was placed in a similar situation. Let's take a look.
Okay, so what happened when Goff took over at Cal, just a few weeks before his 19th birthday? Well, you might not notice much here initially, beyond immediately reducing the rate at which the team was producing interceptions, and perhaps lowering the rate at which the QB was getting sacked. There's a bit more going on in these numbers, which we'll get to as we proceed, but we can say that Goff gradually progressed to produce results in virtually every category that were significant improvements better than anything his predecessors had ever done. The team's touchdown rate kept going up. The interceptions kept going down. Though his 2015 interception results are a bit higher, that is mainly the result of just one game against Utah, which we might want to dismiss as an anomaly You can make up your own mind about that possibility. If that game was excluded, his interception rate would have been 1.65%, which is probably more in line with his other seasons. The rate at which the QB was getting sacked also kept improving (which we'll eventually try to show was a product of Goff's presence). The team's YPA similarly kept moving ever higher, possibly suggesting that Goff was becoming a more aggressive passer with each season. Goff's presence appeared to matter.
What we see with Wentz, is a bit different. The quarterback who preceded Wentz at North Dakota State, Brock Jensen, seemed to have steadily been making progress. In general, the rate at which Jensen was producing touchdowns and interceptions was fairly exceptional, especially by his 2013 season. Jensen's YPA and sack rate also didn't differ greatly from what we see with Wentz. If anything, Wentz merely appeared to be picking up where Jensen left off. Wentz was also only becoming a starter just a few weeks before his 22nd birthday, which is about 3 years older than when we saw Goff entering games.
Of course, Wentz's 2015 season only consisted of a mere 8 games in which he played. When he was out, Easton Stick took over (with a name like that, he should have been a lacrosse player). Strangely, Stick seemed to end up producing touchdown and interception rates that didn't differ significantly from what we saw from Wentz, despite being a freshman with no prior starting experience. The areas where Stick fell short of Wentz, largely came in Stick's YPA and sack rate results, but we think we can explain at least some of that as we get further along. Now, admittedly, we do have some ways to suggest that Wentz was still performing a little better than Stick. That largely comes down to comparing their results to the individual opponents that they faced. Still, you would probably hope to see a greater divide between these two players, all things considered. In the 8 games in which Stick filled in for Wentz, North Dakota State continued to pummel their opponents, and still won every single game This all makes it rather difficult to tell whether Wentz was the driving force of this offense, or if his presence even mattered.
If a player is actually superior, you would tend to think their presence or absence would be felt by their team. Which of these two players, Wentz or Goff, appears to have had a greater effect on their team's passing results?
Generally speaking, I think we can say that we want to see a quarterback keep their interception rate below 2.5%. For the most part, Wentz and Goff would appear to meet this criteria. Of course, when and how a quarterback produces interceptions, is also something we have to consider.
We thought it would be interesting to eliminate all the games in which either player threw an interception, and only examine the games in which they might have been making mistakes. The funny thing was, with neither player was there any significant correlation between the rate at which they were getting sacked, and the rate at which the were producing turnovers. At best, our regressions only produced an R^2 value of 0.018, which is absolutely insignificant.
The more interesting question might relate to how playing with a lead, versus playing from behind, affected their individual interception rates. Let's consider the very different situations these players found themselves in, despite their seemingly similar ability to produce gaudy passing statistics. For Wentz,, his team was operating with an average score differential of 17.34 points, over the past 2 seasons. For Goff, his team was operating with an average score differential of just 3.04 points, also over the past 2 seasons. They were clearly operating in very different circumstances.
So, once again, we eliminated all games in which these players were interception free, and only looked at their worst days. In Goff's case, there was an R^2 value of just 0.059, very slightly suggesting that as his team gained the lead, he became less likely to throw interceptions. This is still a very insignificant statistical relationship. In Wentz's case, his R^2 value leapt to 0.347 in this regression, which is a somewhat relevant correlation. The odd thing was, we couldn't attribute this to him being more aggressive in order to win. Wentz's results actually suggested that the bigger his team's lead was, the more likely he was to throw an interception. It was a bit peculiar. Wentz seemed to become more reckless in games that were blowouts.
But let's get back to the idea of having a quarterback keep his interception rate below our 2.5% mark. While Goff might have thrown interceptions more steadily from game to game, possibly due to the need to catch up to opponents, he rarely put up truly terrible numbers. In his past 2 seasons (26 games), Goff only had his interception rate fall below 3% in 6 games (23% of his games in that span of time). Throwing out the games in which Goff threw no interceptions, his average interception rate would have still only been 3.8%. With Wentz, during the past 2 seasons (23 games), his interception rate exceeded 3% in 10 games (43.4% of his games in that span of time). When we eliminate games in which Wentz threw no interceptions, his average interception rate was 6.07%.
So, this is a bit odd. Despite having less incentive to be aggressive, because of the leads his team operated with, Wentz would appear to more frequently become a potential interception machine. Wentz's bad days were possibly both more frequent than those of Goff, and also more extreme in their negative outcomes.
Sack Rates And Rushing Attempts!
We discussed this a bit before, when we were making fun of Pro Football Focus, but the way a quarterback can influence our perception of a team's pass protection, is something that greatly interests us. When looking at quarterbacks like Tom Brady, Peyton Manning and Drew Brees, there does appear to be something to suggest that they can make an offensive line look significantly better than it would with a lesser quarterback. Maybe this is a trait that is worth seeking out, just as much as you would a good completion rate or a high TD to INT ratio.
We also wanted to explore this idea that Wentz brings something significant to the table as someone who can occasionally scramble with the ball. This is something that people feel that Goff may lack, and perhaps might give some potential edge to Wentz. Since this sort of crosses over with what we are going to say about sack rates, we're going to bundle these two thoughts together, and probably create an unnecessary amount of added confusion.
Let's start this off fairly simply, and just look at Wentz's rushing production, and compare it to the other quarterbacks who have played at North Dakota State in recent years.
Once again, we seem to find ourselves in a very weird situation. Despite the praise that Wentz has received for his ability to carry the ball, there seems to be little evidence to separate him from the quarterbacks who preceded him (mainly Jensen), or the man who filled in for him when he was injured for much of 2015 (Stick). If anything, Wentz's rushing TDs appeared to drop significantly from what Jensen was producing, both in total, and on a per attempt basis. Wentz's rushing averages also come in a hair below what Jensen was producing in his final season. With Stick, who filled in for Wentz in 2015, we also see his replacement rushing for a significantly better average gain, and an arguably similar amount of touchdowns.
There is another peculiar trend here. On offensive plays that weren't designed to be handoffs to the running back, North Dakota State quarterbacks have chosen to run the ball, on average, about 26.3% of the time during the past 6 seasons. That is well above the 14% rate we see from Cal. The interesting thing about this is that with North Dakota State, there wasn't any significant correlation between these rushing attempts, and the pressure that was being applied to the quarterback. The R^2 value for sack rates to percentage of plays in which the quarterback scrambled with the ball was a measly 0.098. When we look at Wentz's individual game to game results, there is slightly more evidence to say that he is choosing to run to avoid pressure, but the R^2 value still only rises to 0.209. For the most part, the majority of Wentz's runs still don't seem to be a product of escaping pressure, but a characteristic of his team's offense. Regardless of who the quarterback was at North Dakota State, they were going to be trying to rush for yards. But, let's look at this another way.
Let's take a look at the trajectory for North Dakota State quarterbacks from the past 6 years, when it comes to their rushing averages. We've highlighted Wentz's two seasons in green, in order to help separate his results. If anything, it looks like Wentz might have potentially been performing a tad below expectations, relative to the other quarterbacks who have played for his team. Either way, I have a hard time being persuaded by the idea that Wentz was a unique runner, and not a product of a team's offensive system, and their desire to see the quarterback scramble.
Ah, but we promised earlier that we were going to discuss how North Dakota State's sack rate appeared to worsen in the eight games in which Wentz was out in 2015. This is kind of interesting, if also a bit obvious. In 2015, the North Dakota State's sack rate did increase from 3.7% to 5.1%, when Easton Stick took over at quarterback. This 3.7% rate that was also notable for being the best sack rate for the team, during the past several years. Unfortunately, this probably wasn't entirely a product of Wentz having gained an exceptional sense of pocket presence. As we already said, the correlation between the percentage of plays that aren't handoffs to the quarterback, but where the quarterback scrambles with the ball, produced an insignificant R^2 value of 0.098. On the other hand, when we look at the pure volume of plays in which a North Dakota State quarterback ran with the ball, we get an R^2 value of 0.442, which is quite a bit more significant.
So, a fair amount of the improvement in Wentz's 2015 sack rate probably stems from the fact that he was simply running with the ball less frequently. When Easton Stick, came into the game, he actually ended up running the ball far more frequently than any quarterback who has played for North Dakota State in the past 6 years, which was almost inevitably going to end up making the team's sack rate look significantly worse under his leadership.
Okay, let's turn this back to the subject of Jared Goff. With Cal, we have a radically different situation. Among the 5 different quarterbacks who have played for the team in the past 6 seasons, none of them produced significant rushing results. Of course, we also don't think Cal intended to have their quarterbacks run with the ball the way that we think North Dakota State might have. In fact, we would say that Cal quarterbacks almost exclusively ran to escape pressure. They were there to pass the ball, not run with it. So, remember how we said that with North Dakota State there was almost no correlation between the team's year to year sack rate, and the percentage of snaps where the quarterback ran with the ball, with only a 0.098 R^2 value. What do you think the results were for Cal? Well, under these same circumstances, the R^2 value for Cal was 0.881, which is highly suggestive of the idea that their QBs were mainly running simply to preserve their lives.
Now, let's consider something for a moment. As we said much earlier in this post, there did appear to be some suggestion that Cal's sack rate gradually improved during the Goff years. There are also many people who would suggest that Cal's offensive line remained fairly inept during Goff's time with the team. The problem is in deciding whether we are underestimating improvements that the line made, of whether this is something we should be crediting Goff with. It's a bit of a tricky question.
This is where we stumble into a realm of magical discussion related to the difficult to quantify term "pocket presence". In the chart above, We have shown the percentage of plays in which Cal quarterbacks have chosen to scramble with the ball (highlighted in blue). As we have said, the rate at which they do this appears to be almost exclusively linked to feeling pressure, and not out of an actual desire to run with the ball. We have also listed the sack rate for Cal's offense during this same period of time (highlighted in green).
While we are mainly concerned with understanding Goff, I think it is interesting to take note of what was occurring prior to his arrival in 2013. In the three years from 2010-2012, we can see the team's sack rate gradually getting worse, and the team's quarterbacks were taking flight at an increasingly alarming pace. Then, somehow, with the arrival of some goofy 18 year old named Goff, the team's sack rate begins to stabilize, and gradually improve. More noticeably, the rate at which Goff was fleeing the pocket dramatically dropped, and remained fairly stable.
I think we have two possible ways to view this. One theory would be that the Cal offensive line radically improved just as Goff took the field, and we all owe them a great apology for the criticism their linemen have taken. The second possibility, is that Goff handles pressure way better than his predecessors did. If it is option number two, we might be seeing evidence of a characteristic we frequently associate with some of the NFL's better quarterbacks.
So, What Does It All Really Mean?
I think the debate over who a team should select, whether Jared Goff or Carson Wentz, is kind of interesting. The criticism and praise you frequently hear for these two players is so incredibly different, that it makes examining people's perspectives and biases kind of entertaining.
If you are just looking at their respective stat sheets, it's easy to say that they both put up rather impressive numbers. When you poke around a bit more, the picture seems to change quite a bit.
1. The data suggests that Wentz's receivers were probably more reliable pass catchers, relative to their level of competition.
2. While Goff might have thrown interceptions more steadily than Wentz, when Wentz did so, he might have went off the rails more spectacularly. Wentz also appeared to have less of a clear reason for throwing interceptions, considering the leads his team routinely operated with.
3. When comparing these quarterbacks to other quarterbacks who have played for the same team, Goff's presence seemed to significantly improve his team's statistical output. With Wentz, the evidence seems to suggest that a number of other North Dakota State's quarterbacks have produced similar results.
4. While Goff probably isn't a huge rushing threat, Wentz might not be either. Wentz's rushing results appear to be no different than what we would expect of any North Dakota State quarterback.
5. While there seems to be substantial evidence to support the idea that Goff has what people call "pocket presence", or an ability to make his offensive line look better than they might really be, with Wentz, the results are just murky at best. Maybe Wentz has some skill in this area, but we can't say that the data clearly revealed significant or unexpected changes in his team's sack rate.
6. While it might be a minor issue, we're also interested in the fact that Goff is so much younger (he turns 22 this October), while Wentz is significantly older (turning 24 in December). The potential upside in investing in youth, and continued development, seem like a significant factor.
Despite all of that, people will continue to debate which quarterback is the better prospect. Right now, much of the support for Wentz appears to revolve around his larger and heavier frame. With Goff, people seem intensely focused on his smaller than normal hand size. (around 9"), despite a lack of evidence to suggest that this has actually impeded his progress. Some anonymous scouts have also criticized Goff for not winning more games, as if that is entirely a product of the quarterback. This strikes us as the sort of old and tiresome "he's not a winner" analysis that probably lacks much real thought behind it, but we can't really debate this perspective either. Many of these sorts of discussions seem to revolve around the idea of "who looks the part of an NFL quarterback", which also isn't something we really want to discuss.
While people will have their own subjective opinions, based on watching these two individuals play, that is also something we don't want to get caught up in debating. Everybody has their own opinions, and that's swell. We're merely curious about finding evidence to support the idea that either of these two players has elevated the performance of the players that surround them. With Goff, we think this is possibly the case. With Wentz, we feel there are quite a few reasons to be concerned. Wentz has a number of issues to suggest that he might have merely been the product of North Dakota State's offensive system. In Goff's case, when we consider our doubts about the receivers he played with, as well as Cal's offensive line, he looks like someone who might have been making chicken salad out of chicken shit. That sort of impresses us.
At the end of the day, we would never want to say that any quarterback is guaranteed to succeed or fail. The teams that a player winds up on, and the circumstances they find themselves in, will probably dictate our eventual opinions, just as much (and probably more) than the talent they actually possess. All we can really suggest, is that if we were having to make such a decision, at the top of the 1st round, we would be terribly nervous about ignoring a lot of the data that is buried in these player's statistics. While some people will suggest that Wentz has more potential for continued development and improvement, that is also something we can't argue, one way or another. It simply appears to us that, given the current data, Goff is the safer and smarter selection to make, and presents less risk. Avoiding risk and the unknown, is something we generally choose to support.