How PECOTA Projections Could Miss on the Twins (Part Two)

[Matthew Trueblood]

Here are five players for whom certain aspects of the advanced system's projection could be wildly wrong, and for which the system doesn't allow flexibility.Yesterday, I broke down the way projection systems can fail to capture the real variance of a forthcoming baseball season, using the Twins’ PECOTA projections as an example. Here, now, are five key projections that, while fixed in all PECOTA simulations, seem highly uncertain, and which would have a huge impact on the Twins if they swung widely in either direction:

 

Byron Buxton’s Playing Time

 

We already discussed this one, but it’s worth touching on again. If Buxton gets just half the number of plate appearances for which he’s projected, he not only forces the team to cover his absence and install a (likely) lesser hitter into the lineup, but also loses the chance to have the overwhelming defensive impact he has when he’s positioned in center field.

 

Josh Donaldson’s Defense

 

PECOTA pegs Donaldson for -3 Fielding Runs Above Average (FRAA). That seems crazy, and it’s probably flat-out wrong. In the past, other defensive metrics have consistently rated Donaldson much more highly than FRAA, and the eye test says those metrics are right. However, defensive statistics remain muddier and harder to assess than offensive ones. If it were to turn out that Donaldson has little defensive impact on the Minnesota infield, the team would have a weakness the public has generally overlooked. If he does save them a bunch of runs with his glove, despite his age and FRAA’s pessimism, then you can mentally tack an extra win or two onto each of the decile projections for the team above.

 

Mitch Garver’s Defense

 

Before he transformed himself under the guidance of catching coordinator Tanner Swanson prior to 2019, Garver had a poor reputation as a pitch framer, and BP’s stats on that skill matches the scouting report. As a result, and even after the stats reflected his growth into an average-plus framer in 2019, PECOTA forecasts -8 FRAA from the Twins’ catcher for 2020. It’s worth noting that other catchers have had good seasons of framing after years of struggle, only to go backward.

 

However, PECOTA is probably wrong again, here, because we know that Garver has made substantial and (presumably) sustainable changes to both the physical and the mental way he receives the ball.

 

This speaks to something else that is worth noting, because Garver also isn’t a darling of PECOTA on the offensive side. Player development has become a much more real, tangible, and replicable practice over the last several years. Projection systems, as currently constructed, have very limited ability to bake that in. Even a perfectly tuned system will miss on players who have undergone a large, fundamental transformation over a short period of time, because one of the primary responsibilities of a projection system is to avoid being fooled by small samples and random fluctuation.

 

Eventually, and particularly in light of the availability of such granular data (exit velocities, launch angles and directions, pitch velocity and movement, spin rate, sprint speed, defensive positioning, and so on), the industry could and should look to move to a first-principles projection system—one founded on everything a player does, from their tendency to chase pitches outside the zone to their hard-hit rate and clustering of launch angles, rather than on every outcome a player produces.

 

In the meantime, however, systems have to contend with the risk that a player who appears to have undergone a legitimate breakout could have done so with smoke and mirrors. As humans who know the humans who build these systems, we can sometimes allow ourselves to acknowledge that the system’s limitations are greater than our own.

 

Jake Odorizzi’s Innings Total

 

In the past, Odorizzi has often had trouble pitching deep into games, and he’s sometimes had trouble staying healthy. For a mixture of the two reasons, the architects of BP’s depth charts have him listed for just 149 innings in 2020. Because the depth charts fix innings pitched, and not batters faced, Odorizzi remains a low-mileage starter even when checking his higher-percentile outcomes.

 

That’s a fair way for the system to solve the complex problem of projecting both performance and health risks, but in practice, Odorizzi has pitched at least 159 innings in five of the last six seasons. That 159 was his total last year, and was held down, in part, by the team’s careful usage of him in September, to ensure he was ready to pitch in the postseason.

 

Given the changes Odorizzi has made to his approach, pitch mix, and mechanics, and given that he’s starting the season with a clean slate, it’s easy to imagine a scenario in which he throws 25 more innings than he’s projected for in 2020. It’s also possible he could pitch 50 fewer, and while the Twins have above-average pitching depth, that would be a major blow. Neither risk is captured by the decile projections.

 

Trevor Larnach’s and Alex Kirilloff’s Playing Time

 

While PECOTA produces projections for thousands of players, only the ones who are listed on Baseball Prospectus’s depth charts go into team record projections. That means that Kirilloff and Larnach, both of whom are off those depth charts entirely, are ignored by the system.

 

From a downside perspective, that doesn’t matter much, because the system also doesn’t reflect the risk of a more serious Buxton injury, or that of a prolonged absence for Eddie Rosario or Nelson Cruz. However, it misses some upside, because both Larnach and Kirilloff are high-level hitters who have succeeded at the upper levels of the minors and have impressed observers this spring. The system likes both hitters, too. Their median projections are for a 102 (Kirilloff) and 103 (Larnach) DRC+, making them above-average hitters. If either were to reach the big leagues and perform at their 80th-percentile level (a 116 and 117 DRC+, respectively), they’d be solid contributors, providing more value even than Marwin González or Rosario are likely to.

 

Projection systems are still more accurate and more interesting than pundits and opinion polls. However, if you really want to see a full picture of the season ahead, you have to look hard at them, and ask yourself what those systems are missing. When you do so, you realize that baseball is even less predictable than we tend to believe.

 

Click here to view the article

[amjgt]

If you were to attempt to build in a playing time projection, what do you think the best way is?

[Trov]

 

If you were to attempt to build in a playing time projection, what do you think the best way is?

I would use player historical data, both majors and minors.  Of course you would have to reflect minors data to limited games.  It is not perfect, but think that would be best.  You can take out the years that 1 injury ended a season or left it out completely, but you could even take other players that had suffered a similar injury and similar recovery to get some date to expect.  Of course each person is different, so again not perfect, but better than assuming they will play 150 games each season when over, say 8 years of a career they only played on average 100.