Jump to content
Twins Daily
  • Create Account
  • entries
    68
  • comments
    388
  • views
    120,741

Confidence in ABS


Twins Video

The Athletic had an article today that some data I had seen before but couldn’t recall.

According to MLB the ABS system is 95% confident that the ABS is no more than 0.39 inches from its predicted location. It is 99% confident that it is no more than .48 inches from its predicted location.

That makes me wonder if MLB should be overturning calls that are within a half inch or four tenths of an inch. They could be overturning a human call that is actually correct. Perhaps it should be labeled inconclusive in that range and go with the human call. The team keeps their challenge.

Human umps were deemed 92.83% accurate last year and the best were better than 95% accurate. In truth humans were more accurate than 92.83%. That 92.83% assumes that robo umps are 100% accurate. MLB confirms they aren’t. Some of those 7.17% “bad calls” were actually correct and ABS had it wrong.

I don’t think baseball was looking to solve a problem with the ABS system. I think they were looking to manufacture drama. The argument that ABS is consistent doesn’t work on close pitches. The solution to the problem is simple. Acknowledge that it isn’t perfect and then automatically reverse calls that are outside that 1/2 inch (or 0.4 inches) tolerance. Don’t waste time on challenges.

Every time a call is overturned inside that tolerance this year we need to realize that the human may have had it right. Baseball is a great game and doesn’t need this manufactured drama.

12 Comments


Recommended Comments

MN Brian

Posted

"According to MLB the ABS system is 99% confident that the ABS is no more than 0.39 inches from its predicted location. It is 95% confident that it is no more than .48 inches from its predicted location. "

It should be more confident about a larger range.  Do you have the numbers swapped?

jorgenswest

Posted

1 hour ago, MN Brian said:

"According to MLB the ABS system is 99% confident that the ABS is no more than 0.39 inches from its predicted location. It is 95% confident that it is no more than .48 inches from its predicted location. "

It should be more confident about a larger range.  Do you have the numbers swapped?

I do. I will fix. Thanks.

DJL44

Posted

I think 95% confidence is a really high bar to not overturn. One standard deviation is a good threshold (0.2 inches or 0.5 cm).

annismark

Posted

I'm not sure that you can use these statistics to make an argument to ignore certain results--you don't know which pitches were measured inaccurately by ABS in the moment.

Also, the accuracy of ABS could go either direction--a pitch that an umpire called a ball but ABS called a strike by 0.25 inches might have been inaccurately measured by ABS by 0.4 inches, but the inaccuracy could have been either direction--even more a strike 0.65 inches or maybe a ball by 0.15 inches, so I don't know it works from that perspective either.

jorgenswest

Posted

6 hours ago, DJL44 said:

I think 95% confidence is a really high bar to not overturn. One standard deviation is a good threshold (0.2 inches or 0.5 cm).

It should be a really high bar. Umpires were very accurate last year.

Approximately 42-43.5 percent of pitches were 0-0.5 inches away from the edge of the plate. Of those about 90 pitches a game both ABS and humans are going to miss some particularly as it gets closer to 0. No one will know for certain which is correct.

Shouldn’t MLB simply acknowledge that there are some pitches close enough to the edge that they aren’t sure whether the human or ABS is correct? Maybe that is less than 0.4 inches. It should be something. Outside that chosen range send the call down to home plate and fix it. Inside that range acknowledge that ABS is not certain. Show respect to umpires who otherwise will have some correct calls on close pitches overturned by the inaccuracy of ABS.

MN Brian

Posted

3 hours ago, jorgenswest said:

Shouldn’t MLB simply acknowledge that there are some pitches close enough to the edge that they aren’t sure whether the human or ABS is correct?

I think this is the correct way to think about it and I would support it.  But the challenge is that most fans have an overly simplistic view of it - the robo ump numbers are definitive to them.  And, as you noted, this is probably as much to increase fan engagement as it is to "get it right" otherwise we wouldn't have the challenge system.

At a certain level they might be doing something similar already for precision (not accuracy) reasons, although I haven't heard it described.  Is 0.1" the smallest increment that they identify publicly? I presume the precision is at least an order of magnitude smaller, let's just say it is 0.01". 

If they identify 0.0" results than likely anything between +/- 0.05 would get reported as 0.0" which I assume would mean the umpire's call stands.

I would be disappointed in MLB if they forced a rounding to either +/- 0.1" and did not report a 0.0" but I suppose it is possible.

jorgenswest

Posted

This one reversed a strike out. It is simply disrespect to umpires to overturn a call where at this point ABS truly has no idea of the ball is in the strike zone or not. The closer it is to the edge I would bet on the human seeing the movement of the ball over ABS. Image from the Athletic story.

IMG_2026.jpeg

I am sure some are watching and marveling about the precision of ABS. They don’t realize there is a tolerance and ABS isn’t certain of the exact location of the ball.

tony&rodney

Posted

This is why I was opposed to the ABS challenge system. I'm simply not going to get worked up about it.

My solutions were: 1) use the technology full time or 2) let the umpires call everything, giving bonuses for higher accuracy and retraining/reassignment for poor results.

MLB wants the ABS for their own reasons. Fair enough, I accept whatever course of action the bosses  direct even if I'm not in favor of those decisions. It is easy enough to focus on the best aspects of the game. I have already had some sympathy for blue when calls were overturned, which were within microscopic levels according to ABS measurements. So it goes.

 

TJSweens

Posted

Use the technology full time. Even in a year when umpires were pretty accurate, the ABS was still more accurate. The ABS is also a lot more consistent. A close pitch in a given location will be called the same every time no matter who threw it or what inning it is. Pitchers, hitters and catchers will all know the strike zone.

jorgenswest

Posted

3 hours ago, TJSweens said:

Use the technology full time. Even in a year when umpires were pretty accurate, the ABS was still more accurate. The ABS is also a lot more consistent. A close pitch in a given location will be called the same every time no matter who threw it or what inning it is. Pitchers, hitters and catchers will all know the strike zone.

It won’t be called the same because ABS isn’t finding the exact location of the ball and won’t consistently find that pitch in the same location. We can be confident that the ball is no more than 0.5 inches away from where ABS’s prediction of the location. We can be pretty confident that it is no more than 0.4 inches away from actual. Those beautiful graphics shown on games omit the disclaimer that MLB IS 99% confident that the ball is no more than 0.5 inches from where it is shown on the graphic. 

I think the best option is use it all the time to automatically overturn a call when it is outside that 0.5 inches away tolerance. Even 0.4 inches would be OK. Under that I think the umpires seeing the entire path and movement of the ball are more likely to make the correct call.

dxpavelka

Posted

You will notice that far more challenges are made based on the bottom of the strike zone than on the top.  The real issue is that if you look at the rule definition of the top of the strike zone and the top of the ABS box they are not the same.  Of course the top of the strike zone hasn't been called for years so I guess it really doesn't matter all that much.  Same thing with the TV box we've been watching for years.  It's never encompassed the true top of the zone.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...