How PECOTA Projections Could Miss on the Twins (Part One)

According to Baseball Prospectus’s PECOTA system, the Twins have a 76.6-percent chance of winning the AL Central in 2020. FanGraphs pegs them just a bit lower, at 55.0 percent to win the division. FanGraphs gives them a 68.3-percent chance to make the playoffs in any form; BP has them at a gaudy 87.5 percent. Both systems project them for over 90 wins, though fewer than 95. However, as I was preparing the Twins’ team preview for Prospectus last week, I found myself noting ways in which PECOTA (and all projection systems, in fact) is unable to properly capture the risk and the upside of a team like Minnesota.

First, consider injury risk. At Baseball Prospectus, the only way in which those risks are accounted for is through the manual process of building team depth charts. Because Byron Buxton strongly tends to get hurt and miss portions of each season, he’s not projected for as full a share of the playing time in center field as he’ll enjoy whenever he’s healthy. However, the playing time he is projected to receive (75 percent of center field innings, and 490 total plate appearances) isn’t treated as a variable by the system.

The format for team previews at Prospectus this spring involves building out and breaking down the decile projections for each team—that is, the record each club would have given a 90th-percentile outcome, an 80th-percentile outcome, and so on, down to 10th. The idea is to sketch out how PECOTA captures the variance possible at a team level, based on the vagaries of individual performances. The decile projections are created using the individual decile projections PECOTA generates, with each team projection representing the outcome of 500 simulations wherein each team’s players all perform at that decile and the rest of the league is kept at their median projection.

Here are the Twins’ decile projections for 2020, as of last week:

• 90th percentile: 109-53
  
• 80th percentile: 104-58
  
• 70th percentile: 100-62
  
• 60th percentile: 98-64
  
• 50th percentile: 94-68
  
• 40th percentile: 90-72
  
• 30th percentile: 86-76
  
• 20th percentile: 81-81
  
• 10th percentile: 74-88
  

That seems, at first blush, like a fair depiction of the real possibility spectrum for this team. The Twins are likely to be very good, and the only way they won’t at least be a respectable Wild Card contender is if the wheels truly come off.

However, in all nine of those amalgamated projections, and in all 4,500 simulations that underpin them, Byron Buxton goes to the plate 490 times, and is the Twins’ center fielder for 75 percent of their defensive innings. In all of the simulations, José Berríos pitches 184 innings, Michael Pineda pitches 100, and Rich Hill pitches 52. The system is only capturing variance created by players performing in unexpected ways; it doesn’t see injury risk.

For that matter, the system also only reflects variance in offensive performance. In all those simulations, Buxton (with his fixed playing time) is worth 20 runs as a defender in center field. In all of them, Miguel Sanó (whom BP has pegged for 65 percent of the playing time at first base, five percent at third base, and five percent at DH, reflecting some of his own injury history) is worth eight runs as a defender at first base, because typically, third baseman who make the transition to first base while still in their prime tend to be good fielders there.

PECOTA has no engine for systematically scaling defensive production upward or downward, and even if it had one, it would be hard to decide how to use it. Would a player having a bad season at the plate be expected to struggle to the same extent in the field? Would a guy who turned out to be a better-than-expected defender lose something at bat because of his focus on defense and the wear and tear of playing harder in the field?

PECOTA isn’t alone in not being able to answer these questions, and BP isn’t alone in being unwilling to attempt to calibrate all of that manually. Playoff odds and other team-level projections, like the atomic player projections of which they consist, are fraught with oversimplifications and unexamined axes of upside and downside.

This is part one of a two-part exploration of the ways projection systems miss certain elements of the Twins' risk and upside. Check back tomorrow for five specific examples drawn from the Twins' individual projections.