Why PECOTA projections were so wrong about the Cubs
Before the 2019 season began, opinions on how the Chicago Cubs would rebound from a disappointing finish to 2018 were varied — but one stood out from the rest. PECOTA, Baseball Prospectus’ math-based system, pegged Chicago for 79 wins and a last-place finish in the NL Central. The same Cubs who had won 90-plus games in five consecutive seasons and were coming off a 95-win season. As you can imagine, this did not go over well in the Cubs clubhouse or throughout the Windy City.
Now flash forward a few months and the Cubs are sitting at 27-18 and on pace for a 97-win season, a far cry from that 79-victory campaign that PECOTA projected. So what gives? How could a projection system be this far off? Does this early-season success mean Chicago is going to outperform the formula all year or is a big-time pitching regression on the way? We enlisted ESPN’s Sam Miller and Jesse Rogers to look into it all.
Why PECOTA was so low on the Cubs: Here’s what a team projection really is: PECOTA runs individual projections for every player, based mostly on recent performance (weighted for how recent) with some small adjustments for age. Then, it adds up how many runs a lineup of those players “should” score and how many runs a staff of those pitchers “should” allow. This run differential is used to estimate the team’s record, since teams that score more runs than they allow tend to win more games than they lose.
Put that way, team projections are much less controversial than we regard them as, and in the large majority of cases — maybe 27 or 28 teams a year — they’re not even surprising, plus or minus a couple of wins. PECOTA is not a hot-take columnist or a fortune teller trying to divine which great player will suddenly turn bad. It mostly just applies consistent arithmetic to do what we all do when we decide which teams we like in the NL Central this year. That’s how, for instance, PECOTA projected the Cubs — who finished sixth in the NL in scoring in 2018, narrowly behind the two teams ahead of them — to finish fourth in scoring this year. The Cubs were sixth the year before, in a virtual tie with the two teams immediately above them. No Cub could complain too much about that.
With that said: There are some extra steps that are a little more interpretative, and those added up to a surprising projection for the Cubs. One extra step is putting more weight on some player attributes (strikeout and walk rates, especially) over others. The idea is that K and BB rates (to simplify things) are more stable, less influenced by luck or circumstances and more predictive. Projections generally use ERA estimators (like FIP, or in PECOTA’s case, deserved run average) instead of ERA to assess pitchers. To give one example, here is Jon Lester‘s performance over the past two years:
2018: 3.32 ERA, 4.39 FIP
2017: 4.33 ERA, 4.10 FIP
PECOTA (which doesn’t use FIP, but is philosophically more aligned to it) thinks Lester is actually the low-4s pitcher. It doesn’t even really see that 3.32. So it projected a 4.42 ERA for the 35-year-old in 2019.
That’s not a controversial position to take on a single pitcher. Most analysts, fans and fantasy managers do some calculation of this sort when forced to predict the future. But the Cubs’ pitchers complicated it by almost all outperforming their FIPs last year. They were, as a staff, second in the NL in ERA but 12th in FIP. That was the largest gap between those two measures in baseball, and it accounts for about 80 runs of difference in how PECOTA assesses the 2018 Cubs and what the standings said.
Notice anything different about Kris Bryant? Here’s how a mid-at-bat change to an ax-handle bat helped ignite the Cubs slugger. Jesse Rogers
Furthermore, because the members of the Cubs’ staff are old, PECOTA figured they were more likely to be getting worse in their careers than getting better. This is why PECOTA thought the Cubs would allow more runs this year than they did last year.
The other extra step is trying to estimate defense with some precision. For the Cubs projection, this showed up especially in catcher framing. The Cubs, according to Baseball Prospectus, have had very poor framers recently, and PECOTA expected that would continue. Most projection systems don’t incorporate framing, which can have a substantial effect on a pitching staff. This is why PECOTA thought the Cubs would allow more runs than some other projection systems did.
The bottom line is that PECOTA expected the Cubs to basically have the same offense that won 95 games last year, and the same behind-the-pitcher defense that won 95 games last year, but a much worse pitching staff, even as it brought most of last year’s 95-win staff back. That’s a fairly bold take, centered mostly on mediocre projections for Cole Hamels, Jon Lester, the bullpen and Willson Contreras‘ framing abilities. — Sam Miller
Why the Cubs were so ticked off by that: This one is simple. When someone — much less a computer program or algorithm — states a player is too old to perform at a high level, said player is going to push back on the math that produced said analysis. That’s human nature, so the Cubs took it personally. Even if history has proved PECOTA more right than wrong, they felt you can’t apply a formula without taking into account the individual. Hamels probably summed it up best when he explained that what made him and Lester special in the first place is still going to make them special even at 35. In other words, it’s not about their stuff, it’s about them. Of course, Father Time has never lost, but we’re not talking about two 40-somethings here.
The Cubs also believed their staff could only improve considering both Yu Darvish and Tyler Chatwood gave them nothing a season ago. With Hamels in tow for a full year and a healthy Darvish — even if he was just average — the math just did not add up to 79 wins. Regression was and is a possibility, but that was a little too much to swallow for the Cubs. And they said so. — Jesse Rogers
The math says: The Cubs’ pitching has been fantastic. (Their catchers’ framing has not been, but no matter.) Of the nine Cubs pitchers who’ve thrown the most innings, all but one is pitching much better than his projected ERA:
This is partly because the Cubs’ pitchers are again outperforming their peripherals, but the more so because they’re collectively pitching better this year than they did last. They’re on pace to strike out 150 more batters than they did last year. Lester, to give one example, is striking out almost two more batters per nine innings than he did last year, while also walking batters at a career-low rate. He is, in fact, so far, having the best year of his career.
PECOTA — which, again, is basically averaging the past to come up with a projection — will almost never project a player to have his best year, and especially not a 35-year-old. It’s virtually impossible for the system to spit out such a projection, limited as it is by the boundaries of the player’s past performance. It is not, of course, at all impossible for a 35-year-old to have a career-best year, because the past is no restriction on him. PECOTA mostly leaves the surprises to surprise us, and they inevitably do. — Miller
The view from the ballpark: This is where watching how things develop becomes important, and if the Cubs hadn’t bottomed out during their season-opening road trip, perhaps they would not have gotten the bounce they’ve experienced. The year began with a lot of walks, from their starting staff as well as their bullpen. The results followed suit. The team came home with a 2-7 record before ever stepping foot in Wrigley Field. But as the pressure began to mount on the mound, the veteran starters took it upon themselves to stop the free passes. They said as much publicly.
Perhaps, this is where PECOTA comes up short as it can’t take into account the brains and fortitude age brings you. Hamels led the way, going on a 22-inning walk-less streak. Jose Quintana, Lester and Kyle Hendricks followed suit. Those four pitchers have walked a total of 45 batters all season over the course of 36 starts. Do the math on that. They’re making the opposition earn it — to the point where they don’t care about giving up home runs. Everyone gives up home runs in this era, but the key is making them solo shots, not three-run jobs. The Cubs seized on that mantra and it has permeated the staff — with one exception. Darvish leads the league in walks, so it’s no surprise his ERA is 5.06 after Monday’s loss to the Phillies, in which he walked three. — Rogers
The math says: Well, look, projections can be real jerks. One of their jerky habits is persisting in their assumptions about a team for a while, even when the team in question thinks it has whupped PECOTA. So, if you go to Baseball Prospectus, you won’t find PECOTA’s letter of apology. Rather, it just keeps spitting out its forecasts, day after day, and each day it still basically distrusts the Cubs’ pitching staff. It still thinks they’re a .500 team, maybe slightly worse. Its projected ERA for Jon Lester, going forward, is only slightly better than it was two months ago: 4.32.
If you think that the system was always wrong about the Cubs, this shouldn’t bother you. Cast it aside. It’s got a bug. Who cares? If you think that the system was on to something, with all that stuff about the pitchers being old and outperforming peripherals in a hard-to-repeat kind of way, then the bad news for the Cubs is that six weeks of games aren’t that predictive. As Jeff Sullivan, now of the Rays, wrote at FanGraphs, even first-half records, and first-half run differentials, do a much worse job of forecasting the second half than projections do.
But the good news for Cubs fans, regardless of how seriously they take PECOTA’s concern, is that all these Cubs wins are already in the bank. Today’s PECOTA projection might think the Cubs are a .500 team going forward, but a .500 team with a game-and-a-half lead and a 10-game head start on .500. That’s good enough to make the Cubs at least co-equal favorites for the NL Central, and better-than-50 percent bets to make the playoffs. Even PECOTA says so. — Miller
The view from the ballpark: This is where it would be too easy to simply say PECOTA was wrong and move on. If it is right, it’s very possible the regression shows up in the second half when Hamels and Lester, for example, have more innings under their belts. Age is more likely to “catch up” to a pitcher later in the year as opposed to in April, right?
The other issue is how things went down for the Cubs last season. Their second half showed warts in their offense that never appeared early in the season. It’s a reminder that a team can really have two seasons, before and after the All-Star break. A month from now, we all might be able to declare PECOTA dead wrong — it looks that way now — but regression doesn’t have to show up on day one. Time will tell the story. Even the Cubs were pleasantly surprised by the start Lester had, for example, so it’s possible some guys are over-performing. It’s just not likely. — Rogers