Establishing production expectations for Joe Burrow’s rookie season

Bengals fans know Joe Burrow at his absolute best...so far, that is.

Burrow was an incredibly productive high school quarterback in Athens, Ohio, who experienced a three-year hiatus before starting for a football team once more. His first year at LSU in 2018 was, by all accounts, pedestrian, and ultimately made his 2019 season even more impressive than it already was.

That level of success will not be sustainable in the NFL. When focusing on just his rookie season, it shouldn’t even be in the discussion.

Ever since the collective bargaining agreement of 2011 changed the values of rookie contracts, building a team around a young quarterback became the blueprint to creating a winning franchise. What often gets overlooked is that the window of Super Bowl-contending for these teams doesn’t usually begin until the quarterback’s second year in the league.

Even in today’s pass-happy NFL, true rookie quarterbacks struggle more times than not. But what does that actually look like?

Take Andy Dalton’s 2011 season. As a rookie thrusted into a starting job for a struggling franchise, Dalton put together a season pleasing to eyes. 3,398 passing yards, 20 passing touchdowns, 13 interceptions, and a 9-7 record is what we all witnessed.

Conventional wisdom leads us to believe that his season was impressive for him being a rookie starter. When we compare it to other rookie seasons in recent years, it is indeed admirable. But teams hoping to contend with a rookie quarterback aren’t desiring success relative to other rookie quarterbacks, they need success relative to veteran quarterbacks.

Dalton’s 2011 numbers put him below the average compared to other starters from that season. And his yards, touchdowns, interceptions, and win-loss record are not the numbers I’m referring to.

---

In the year 2020, it’s time we all recognize the single-best metric for measuring anything in football: Expected Points Added per play (EPA/play). The Athletic’s Ben Baldwin (@benbbaldwin on Twitter) is one of the industry’s leaders in researching trends with EPA/play and wrote a piece that included Pro Football Focus’ Timo Riske’s (@PFF_Moo on Twitter) findings on how effective EPA/play is at predicting team success.

Baldwin’s article on The 10 commandments of numbers-based football analysis further boasts EPA/play’s superiority when assessing quarterback production. EPA/play is the base of it all, but Baldwin kept digging.

Completion Percentage Over Expectation (CPOE) is a metric that utilizes Completion Probability and Expected Completion Percentage data generated by the workings at NFL.com’s Next Gen Stats. CPOE is the end result of calculating Expected Completion Percentage from Completion Probability and comparing that to a passer’s actual completion percentage.

Completion Probability is defined by Next Gen Stats as:

The probability of a pass completion, based on numerous factors such as receiver separation from the nearest defender, where the receiver is on the field, the separation the passer had at time of throw from the nearest pass rusher, and more.

Expected Completion Percentage is defined as:

Using a passer’s Completion Probability on every play, determine what a passer’s completion percentage is expected to be.

Using those two, we get CPOE, or as Next Gen Stats coins it, Completion Percentage Above Expectation (why Baldwin refers to it as CPOE instead, I do not know).

A passer’s actual completion percentage compared to their Expected Completion Percentage.

Logically, CPOE gives us added context to go along with EPA/play. A passer who generates a lot of EPA on a per-play basis should be commended, but the quality of each pass can be further examined by how difficult it was to complete it.

Baldwin decided to combine both metrics into a composite form, which he lists as EPA + CPOE composite (we will abbreviate this as ECC). The purpose was to see if combining the two variables would allow us to predict a quarterback’s EPA/play the following season more effectively.

His regression analysis found that not only does EPA/play and CPOE carry nearly equal value, the ECC resulted in strikingly similar variance to PFF offense grades for quarterbacks.

Now, why is that important? Well, Baldwin later discovered that PFF offense grades carry the highest correlation in predicting EPA/play for a quarterback in the following season (he later found that Sports Info Solutions’ Total Points recently took the top spot, but since PFF carries a larger sample size, we will focus on that).

Both EPA/play and PFF offense grades account for rushing as well, which is important now that most quarterbacks are mobile threats. Quarterback rushing is quarterback production and should be factored into everything at the end of the day.

Essentially, Baldwin calculated a metric that tells us as much information as PFF’s offensive grading does, and gives us actual context of what a grade looks like from a data perspective. And if PFF offense grades sit near the top—or at the top—from a year-to-year stability perspective, this allows us to better identify how good and sustainable quarterback seasons truly are.

Here’s how easy it is for us simpletons to visualize this truth. In the early process of studying this data, I noticed a conspicuous trend with ECC and PFF offense grades. Since 2006, there were seven examples of a quarterback who finished with an ECC in the top 10 and a PFF offense grade that ranked in the bottom 10. For five of the quarterbacks, their ECC ranking either regressed the following year while their PFF offense grade ranking remained relatively stable, or they just didn’t play the next year because honestly, they weren’t true starters anymore.

For two of them, they managed to avoid regression the following year. They also happen to be the best quarterbacks on the list: Ben Roethlisberger and Chad Pennington.

They were not The Frauds that the other five were.

If there is a large enough discrepancy between the two rankings, the likelihood of that EPA/play being legitimate and carrying over into the following year is not great. Regression tends to not miss those who deserve it.

---

Alright, that was a lot of background info, but it was all necessary. This article is about Joe Burrow, don’t forget, so what does this all mean for him and the Bengals?

What I did is utilized Baldwin’s website RBSDM.com (the acronym may upset Joe Mixon) to identify every starting rookie quarterback since 2006. That year is important because from 2006 on, PFF has graded every NFL season. It’s also two years after the NFL changed its pass interference rules and created a boom in passing efficiency.

A quarterback qualified to be a part of this study by having recorded 250 plays in a given regular season while the win probability for his team was within a range of 10-90%. What that means is if a quarterback play occurred when his team had a < 10% or > 90% chance of winning at that given moment, the play was not included in the data. This does two things:

1. It eliminates quarterbacks who weren’t full-time starters for that season. The average number of qualifying quarterbacks for each season is 29.3. For example, Ryan Finley took part in 82 plays inside of this window and was not included.

2. It takes out garbage time production almost entirely, as the majority of plays occurring outside this window ultimately don’t matter in terms of winning. That isn’t to say all production outside of this window doesn’t matter, but when looking at it from a macro-perspective, most of that data comes at unimportant times in games. Admittedly, the 250 play cutoff was fairly arbitrary, but it provided a solid threshold for what can be considered “full seasons”.

Unfortunately, while Baldwin’s site allows us to factor in for win probability, PFF Elite does not. But I still wanted to use PFF offense grades as a control when viewing every quarterback’s ECC calculated by Baldwin under the 10-90% parameter. Most of a quarterback’s production occurs within this window during an average game, so the PFF offense grade still largely matches a quarterback’s performance.

So, we have everything we need. 410 quarterback seasons qualified for the study, including a total of 36 rookie seasons. By rookies, I mean true first-year players, so Patrick Mahomes’ 2018 season did not count as a part of the rookie sample.

Here is how every rookie qualifier fared with their ECC and PFF offense grade ranked in each season and ordered by descending ECC in descending years.

What’s the first thing you notice? Is it that most of them were bad? Because that’s extremely evident.

Out of the 36 qualifiers, only six had an ECC ranking above the average ranking of that season. Here are the six.

That’s right. When looking at the metrics that really matter, one in every six rookie quarterbacks produced above average for that season. Of these six, a couple things stand out.

1. For starters, of the five most recent ones in the 2010s, Mike Glennon was the only white guy. Does that mean anything? Depends on what your priors were before reading this. Glennon was essentially average during the 2013 season anyways. Going by just his PFF grade, it wasn’t even that promising.

2. Glennon’s ECC and PFF rankings carry the biggest discrepancy and considering he’s the objectively the worst quarterback on here, that makes sense. In an AMATEUR attempt to further explore the correlation between ECC and PFF from year-to-year, I charted the averages of each metric from year to year.

3. Cam Newton also barely made the above average group with his 2011 campaign. Both Kyler Murray in 2019 and Jameis Winston in 2015 missed the cut by one spot, and should be credited as honorable mentions.

4. In the years where there were a Summer Olympic Games (2008, 2012, and 2016), a rookie quarterback finished with an ECC in the top five. Matt Ryan even finished FIRST in 2008 when you set the win probability window to 15-85%. Both Russell Wilson and Robert Griffin III were obviously amongst the best quarterbacks during the 2012 season, and Dak Prescott surprised everyone back in 2016.

This upcoming season provides a chance to extend that trend, but it seems that Burrow will be the only one who can do it. Not because he’s perceived as the best rookie quarterback in the league right now, but he may be the only one who will play 250 meaningful snaps.

But even though it’s been done before, the numbers tell us it’s not to be expected. This study is obviously not perfect, but it does tell us a lot.

Since Wilson and RGIII, 20 rookie quarterbacks qualified for this study and only two were able to join their company. Prescott and Glennon each had impressive rookie seasons, but there’s a reason one is vying for a market-setting contract and the other turned into a backup well before he turned 30. The more context we have, the better off we are.

---

We know that rookie production for quarterbacks won’t ultimately define their career paths, but this should be a wakeup call to those who are putting all their faith into Burrow lighting the league on fire during his first season. If he does, he’ll be the exception to the rule. It’s not an indictment on him, his ability, or his future outlook. It’s simply important to set reasonable expectations for those in historically unforgiving situations. And we didn’t even bring up the current COVID-19 pandemic.

Comparing surrounding talent and coaching for the previous rookie quarterbacks may be something you feel inclined to do. If that’s your prerogative, that may add in context. The issue is that it’s hard to quantify both of those factors and isolate them away from the influence of the quarterback. The impact a quarterback has on the rest of his team is objectively humongous.

Joe Burrow remains a great college quarterback who projects to do very well in the NFL. There are precedents for him thriving in his first year with the Bengals and he should still be able to play well at times. In all likelihood, though, this season will represent a giant hurdle that he will struggle to get through relative to the rest of his peers.