Man vs. PECOTA: Should teams actually care about their projections?
Imagine walking into the Chicago Cubs clubhouse with a Magic 8 Ball, giving the thing a shake and asking it whether the Cubs will make the playoffs in 2019. An answer floats into view: “My reply is no.” It is unlikely the Cubs would care.
Imagine walking into the Chicago Cubs clubhouse with David Ross, Vin Scully and Anthony Rizzo‘s mom and dad. If you were to ask them whether the Cubs will make the playoffs in 2019, and they were all to say “no,” it’s likely the Cubs would care a lot. Maybe too much!
So that’s the spectrum: On one end you have an impersonal toy, and on the other you have the people they love and care most for in this world. Where does a website’s statistical projection system fall on this spectrum?
Last week, the Cubs went to war with one of those. PECOTA, the forecasting system run by Baseball Prospectus — a site I used to edit, but whose projections I had no role in — issued its annual standings forecast, and among 29 not-that-surprising win totals was one whopper: Chicago, last place in the NL Central, with 80 wins. (Now 79 wins, after updates to the depth charts.) Various Cubs have smirked and scoffed, and one Cub used it for bulletin board material:
It is natural to critique or question projection systems, especially in a sport as unpredictable as baseball. It has always seemed a little silly, though, when a team would actually take them personally, considering the things are literally nonpersons. They seem closer to the 8 Ball side of the spectrum.
But should it be silly? To answer that, let’s review two things: The History of Teams Actually Responding to Bad PECOTA Projections, and The Types of Conflict Typically Found in a Narrative.
In the latter category, there are four main conflicts, which you might remember from 11th-grade English class: Man vs. Man, Man vs. Society, Man vs. Nature and Man vs. Self.
Taking a look back through history, we have four classic examples preceding this latest incident with the Cubs:
1. Man vs. Nature: The 2007 White Sox
Chicago was coming off a 90-win season, which had followed a World Series win in 2005, but before the 2007 season PECOTA offered a 72-win forecast. “That’s a good sign for us, because usually they’re wrong about everything regarding our dealings,” White Sox general manager Ken Williams said, and then the White Sox famously won 72 games.
“It’s dumb because they have us in last place.” Winners of an average of 97 games over the past four years, here’s what the Cubs really think about a projection that has them winning 79 games.
Also, why does PECOTA have it out for the Orioles? Something’s not quite right there, no? Here’s your consumer’s guide to Tuesday’s 2017 Baseball Prospectus projections.
At the time, the White Sox had an aging core of stars, and PECOTA saw (among other things) reason to fear steep declines. It was, in other words, simply the forces of nature pulling the White Sox further from their peak and toward the inevitable. But Williams resisted that inevitability: “Maybe we’re a year better then,” he said.
They weren’t, but maybe Williams had a point: We are all in a fight against nature. And we are also, by our very existence on Earth, proof of the durability of our stock. Each of us comes from a long, long, long line of survivors — outliers, each of us, when you think about it. A projection system isn’t made for the outliers.
2. Man vs. Society: The 2015 and 2016 Royals
PECOTA undershot the Royals badly in 2014, when Kansas City went to the World Series, but was at least in crowded company with the miss. The next spring, the system said 72 wins — the Royals won 95 games and the World Series — and the next spring 75 — think pieces abounded, and the Royals finished .500. Probably no team before or since has so regularly mocked the projections as the Royals did.
For example, manager Ned Yost and GM Dayton Moore argued projections didn’t value the same things the Royals did. “The stat people and the people that predict the numbers, they don’t understand our style of play,” Yost said. “We know, as a group, that we never stop fighting, we never quit, we continue to battle. If we’re close in a game from the fifth inning on, we know that we have the ability to put the ball in play and manufacture one or two runs that we need to win a ballgame. We feel like if we’re ahead, our bullpen’s going to lock it down and we’re going to win those games. That’s a great confidence to have, and it’s a teamwide confidence that our offensive players have with our bullpen, knowing, hey, all we’ve got to do is be close by the fifth, and we’re going to lock this game out.”
So the Royals weren’t mad at a computer system. They were mad at the societal values the computer system represented. In particular, they dismissed the notion that a team is merely 25 individuals summed, rather than 25 (or, even, more) parts of one mysterious and unknowable whole.
3. Man vs. Self: The 2016 Diamondbacks
The Diamondbacks, who were coming off a 79-win season, added Zack Greinke and Shelby Miller but were projected to win only 78 games. GM Dave Stewart called it “a joke” and explained, “You know, I just think that without those guys we won 15 more games [in 2015] than we [did] the year before, so we’ve got to be better at least by a couple more games.”
Stewart’s argument, though, was basically the same as PECOTA’s: The Diamondbacks won 15 more games in 2015 than they did in 2014, when they won 64. To PECOTA, that doesn’t mean a team that’s on the incline, or even a 79-win team, but something in between 64 and 79. That’s what it means to have a long memory, as projection systems invariably do. Projections are all based on the idea that a player is who he has shown you he is, and the Diamondbacks of the previous two years had shown themselves to be a 64- and 79-win team. Adding Greinke and Miller made them better, but regression to their normal, more established performance level had already pulled them back. This is a fairly bleak and miserable way of seeing oneself, especially coming off a personal success, so it’s understandable an athlete or team would resist it. Arizona went 69-93 in 2016.
4. Man vs. Man: The 2011 Angels, and to some degree all of these examples
The Angels had a trick, for a while, of winning more games than their run differential would suggest. This is trouble for most projection systems, which estimate how many runs a team will score, how many it will allow, and then convert those into expected wins — so it could theoretically have the player projections all correct but the team projection all wrong. PECOTA was getting walloped by the Angels every year, and — when the system projected them to win only 78 games in 2011 (they ended up winning 86) — the Angels noticed.
“They’ve had us losing 90 games for the last [seven] years, and we managed to win 90 [five times],” manager Mike Scioscia said — an exaggeration, but he’d earned the right to exaggerate. Then, he said this: “I think it’s a young science.”
It’s back! Sign-ups are open for the 2019 fantasy baseball season.
Create or join a league today.
Get rankings and more for your draft.
In other words, Scioscia didn’t care that a computer was saying bad things about the Angels. The computer is a tool, and the tool could do only what it was told, and we could tell it only what the science had advanced to. It was a young science! Scioscia’s concern was with anybody who treated it as anything but a young science.
This is a common theme in teams’ responses: The projections are just whatever. They are what they are and they have an excuse. But the people who treat the projections as more than whatever don’t have an excuse. They should know better.
When a player or executive criticizes a PECOTA projection, he will refer to it almost as though it’s a person (“They want to get some publicity,” the Cubs’ Kyle Schwarber said last week) which seems strange, unless you consider the criticism to be of the person who either oversells the projection or who, upon reading it, takes it too seriously.
Which brings us back to the Cubs. The Cubs were told that a thing they maybe hadn’t even heard of didn’t think they were a very good team. They had to decide how much they cared. They could have said “Magic 8 Ball” and blown it off. Teams often do. The Rockies’ Bud Black, on his team’s mediocre projections last year: “You will see some prognosticators that don’t think we are a contending team, but I think you will also find those people that think we are. But ultimately it doesn’t matter.” (The Rockies made the playoffs.)
But one of the most important things in baseball is to care. It can be hard to care as much as you need to, and a good reason to care is basically free money. Somebody told the Cubs that some … thing didn’t like them, and they found in that thing a reason to run out to the practice field that day and prove the thing wrong. Good for them.
(Now nobody tell them I think they’ll win a lot more than 80 games this year, and so does almost everybody else.)