A deep dive into umpire accuracy and our standings

[Eric Blonigen]

Every sports fan is likely to believe the umps are biased against their team, but there have been several games lately that make it seem like the fates are working against the Twins. Let's look at some examples.

In last night's game against the Pirates, assistant hitting coach Derek Shomon was ejected for arguing balls and strikes. As it turns out, he was right to be frustrated, as HP ump Paul Clemons called a pitch to Correa a strike when it was a ball. Later in the inning, with bases loaded, a ball to Kepler was called a strike, impacting the shape of the plate appearance, the inning, and the game, as the Twins were unable to score a run. After the game, the folks at umpirescorecards.com released their scorecard, and Pittsburgh was awarded 1.17 runs in their favor.

During the Yankees series, there were several questionable calls and most seemed to go against the Twins. That seems to be a trend on the season. Let's take a deeper dive into the data

So far this season, if all pitches had been called correctly, the Twins should have scored an additional 5.22 runs. Their main divisional opponent, the Guardians, should have scored 17.82 FEWER runs than they have. Overall, that 23 run differential means that the two teams should be much closer in the standings than they are. However, not all runs are created equal. So, I looked at the ump scorecards for each game that was decided by two of fewer runs to understand exactly what impact the umpires have had on the two teams results.

For starters, there have been two head-to-head games that went Cleveland's favor that actually should have been Twins victories. Here are the scorecards for the 4/6 and 5/17 games.  

There have been other examples of Twins losses that the outcome should have been wins, or at least may have ended differently. On 4/14 against the Tigers, the final score was 3-4 Tigers, but they scored 1.32 additional runs due to ball and strike calls.

 

On 4/19, also against the Tigers, the 4-5 loss was in part decided by the extra .88 runs awarded to the Tigers. 

That's four losses directly attributed to incorrect ball and strike calls.

There was also a single example of the Twins recording a win that they may not have earned. On 4/23, against the White Sox, the 6-5 score was aided by an extra run in our favor.

Net-net, this likely means we should have at least three more wins than we actually do. 

Earlier, I mentioned Cleveland's 17.82 surplus runs. According to umpirescorecards.com, they lead all of baseball in favorable calls. How has this impacted their win total? In addition to the two-game swing based on the head-to-head matchups discussed earlier, they have had several one-run victories where they scored a half-run due to ball and strike calls. While these may still have ended in Cleveland wins, the outcome may have been different on 4/18, 5/6, 6/1, and 6/7.

Perhaps more importantly for Cleveland, they have only had a single game this season that calls went against them in a meaningful way - 4/13 against the Yankees. With a final score of 3-2 Yankees, the evil empire was granted an extra .55 runs.

.

Assuming that Cleveland would have won half those games anyway, it's fair to think their actual win total should be four wins less than reality.

What's the impact of all this? Well, entering play on 6/8/24, Cleveland is 40-22 and the Twins are 33-30. Based on expected win totals were balls and strikes called accurately and consistently, the ACTUAL win/loss records should be Cleveland at 36-26, and the Twins at...36-27. That reality would feel much different for fans than the one we are living. The good news is we are well-positioned the rest of the way, with one of the easiest remaining schedules in baseball. Cleveland has one of the hardest. With some fair umpires, we just might be able to run away with the division.

Thoughts?

[Vanimal46]

When the games in question have the Twins scoring 4 runs or less, I couldn’t care less about the umpire. The Twins have to generate more runs, and the umpires won’t matter. 

[bean5302]

Consistency in the strike zone is what I care about most, and umpire scorecards doesn't really take that into consideration by quantifying estimated ump zone. If the strike zone was expanded 0.5" or 1.0" into a single shadow zone (in/out/top/bottom), does the ball/strike call suddenly jump to nearly 100%?

Consistency in predicting what will be called a ball vs. strike, and egregious misses are what I care about as, historically, what pitchers have complained about most is not knowing what will be called a strike when they hit a certain location, and catchers and pitchers routinely try work a "ball" into a "strike" if they can by hitting the same spot repeatedly. It's been part of the game forever.

Most of the whining on this site about balls vs. strikes recently is over 2 or 3 calls a game out of the 300 pitches thrown and the plus or minus run value assumes players have no idea what will be called a ball or strike, which is not usually true. For example, Malachi Moore on 6/1 above. Any hitter who is taking pitches just on the right side of the plate in the image is asking for a called strike because the ump had been calling those "balls" strikes consistently throughout the game. Pitchers and hitters historically adjust for that.

Also, the amount of fan engagement on umpiring is huge, and for a sport which desperately needs some fan engagement

[SD happy]

A lot of time and thought put into this. Good stuff, thank you. Things usually seem to even out in baseball. hopefully that will be the case moving forward.

[Linus]

And what is the umpire scorecard based on?  Certainly not the box mlb uses to grade the umpires. So I don’t think the data is reliable enough to make set conclusions. 

[bean5302]

1 hour ago, Linus said:

And what is the umpire scorecard based on?  Certainly not the box mlb uses to grade the umpires. So I don’t think the data is reliable enough to make set conclusions. 

What data do you use, and where does it come from?

MLB releases detailed data for every pitch of every game. Each morning, my program grabs all of this pitch by pitch data from the previous day’s contests. Within the data, each pitch is assigned 89 attributes, from the pitcher's release position to the pitch’s horizontal acceleration. We care about 5 of those 89 values. Two are the pitch’s horizontal (plate_x) and vertical (plate_z) position as it crosses the plate. Two are measures of the top and the bottom of the strike zone (sz_bot and sz_top), values that reflect the size of the zone once adjusted for batter height and stance. Finally, we use the resulting call of the pitch. In conjunction, these 5 values can tell us whether a pitch was a strike or a ball, and whether or not it was called correctly.

https://umpscorecards.com/info/

[Linus]

1 minute ago, bean5302 said:

What data do you use, and where does it come from?

MLB releases detailed data for every pitch of every game. Each morning, my program grabs all of this pitch by pitch data from the previous day’s contests. Within the data, each pitch is assigned 89 attributes, from the pitcher's release position to the pitch’s horizontal acceleration. We care about 5 of those 89 values. Two are the pitch’s horizontal (plate_x) and vertical (plate_z) position as it crosses the plate. Two are measures of the top and the bottom of the strike zone (sz_bot and sz_top), values that reflect the size of the zone once adjusted for batter height and stance. Finally, we use the resulting call of the pitch. In conjunction, these 5 values can tell us whether a pitch was a strike or a ball, and whether or not it was called correctly.

https://umpscorecards.com/info/

Interesting. I stand corrected. 

[KirbyDome89]

I'm so tired of the bitching about umpiring on this site. It's the lamest scapegoat ever....

Nobody complained when Nick Gonzales was rung up on ball 4 from Duran in the 8th inning. Where was the outrage? Pittsburgh should've had the bases loaded with 1 out instead of first and third with 2 outs in a one run game. Nobody gives a ****, and it's conveniently forgotten, but we're still holding onto a Cleveland loss from 2 months ago. The selective memory nonsense is out of control. 

 

[tony&rodney]

How many complaints and articles were there about the umpires during the Twins winning streak?

That said, an ABS or challenge system needs to be instituted, particularly with the rise in advertising and promotion for gambling on the games. The intersection of human calls and gambling are on a collision course.

[RpR]

16 hours ago, tony&rodney said:

 

The intersection of human calls and gambling are on a collision course.

Then get rid of the asinine gambling, but then also , gambliing is not fair, it is a game of chance, do not like the odds do not play.

[ChermesZ]

19 minutes ago, RpR said:

Then get rid of the asinine gambling, but then also , gambliing is no fair, it is a game of chance, do not like the odds do not play.

It makes too much money for the mlb.  They’ll never separate from the advertising.

 

I’d like the abs system…but doesn’t the umpires union stand in the way of that?

[tony&rodney]

6 minutes ago, ChermesZ said:

’d like the abs system…but doesn’t the umpires union stand in the way of that?

I believe that is the hangup right now. There are silly statements issued about how to know what strike zone to use and a consistent commentary by MLB announcers that the system isn't ready but AAA has zero issues this year from a a viewer point of view. It does look better run than last year. Watch a couple of dozen games via milb.com and see what you think.

The dollars from gambling were sought out and are catered to by MLB. This isn't something fans can weigh in on. We would have more luck opposing a plumbing ad. It doesn't matter if one is opposed to gambling. I don't gamble but that is irrelevant.

My concern with sports and gambling has already played out with NBA referees and mostly young not well known athletes across sports. It is predictable. I just wonder if the use of an electronic strike zone/ ABS/ challenge system would help avoid issues for baseball.

FWIW, the umpires seem to be better than they ever were in the past. The strike zone, however, seems to remain an issue in MLB while safe or out calls on the bases have seen corrections through the manager's challenge. Ironically, the ABS challenge system is lightning fast (less than 5 seconds) while base challenges have routinely taken several minutes and even much more.

[mnfireman]

3 hours ago, tony&rodney said:

The intersection of human calls and gambling are on a collision course.

And suddenly we are having players (and interpreters) being investigated for gambling on baseball....

[USAFChief]

4 hours ago, KirbyDome89 said:

I'm so tired of the bitching about umpiring on this site. It's the lamest scapegoat ever....

Nobody complained when Nick Gonzales was rung up on ball 4 from Duran in the 8th inning. Where was the outrage? Pittsburgh should've had the bases loaded with 1 out instead of first and third with 2 outs in a one run game. Nobody gives a ****, and it's conveniently forgotten, but we're still holding onto a Cleveland loss from 2 months ago. The selective memory nonsense is out of control. 

 

Concur.

And BTW, the idea that an umpires calls result in a measurable amount of runs, to 2 decimal points, is ridiculous. 

Just because someone invents a measuring system doesn't mean it's measuring anything. 

[cmoss84]

30 minutes ago, USAFChief said:

Concur.

And BTW, the idea that an umpires calls result in a measurable amount of runs, to 2 decimal points, is ridiculous. 

Just because someone invents a measuring system doesn't mean it's measuring anything. 

Do you think it's important to individual players fighting for their jobs? Do you think it's important to teams fighting for playoffs with 20 games to go?

If we can improve professional sports' accuracy of outcome...why not?

[mnfireman]

So, are we going to allow MLB to stop at the electronic strike zone when so many judgment calls can go either way? Check swings and fair/foul calls on balls hit directly over the base have led to at least as many game changing events as missed ball or strike calls, but can not be challenged under the current replay system, but foul tips and hit by pitches can.

[cmoss84]

9 minutes ago, mnfireman said:

So, are we going to allow MLB to stop at the electronic strike zone when so many judgment calls can go either way? Check swings and fair/foul calls on balls hit directly over the base have led to at least as many game changing events as missed ball or strike calls, but can not be challenged under the current replay system, but foul tips and hit by pitches can.

Those things should definitely be challengeable...just speed up the process.

[cmoss84]

RpR...care to elaborate instead of a thumbs down only response?

[KirbyDome89]

On 6/8/2024 at 11:29 PM, USAFChief said:

And BTW, the idea that an umpires calls result in a measurable amount of runs, to 2 decimal points, is ridiculous. 

Or the fact that those runs are being used to shift the W/L records by multiple games....yeah....

[Road trip]

Interesting that from the samples provided it appears far more balls are mistakenly called as strikes, than strikes called as balls.  

If this is indeed true across all umpired games it does provide another factor as to why the league as a whole is well on the way to setting a modern-day low for batting averages.  

 

[Schmoeman5]

Can someone explain to me how an umpire can visually call balls and strikes on balls traveling 100 mph but can't see catchers moving their gloves in attempt to frame pitches. We laud catchers for this ability and chastise umpires for missing what? 2 or 3%. Hell, you have fans and coaches blowing gaskets on pitches where their view is off just from the angle they're viewing from. Ban framing and that box. Problem solved.