Balls and strikes called incorrectly 1 out 5 times

[yarnivek1972]

We’ve always been led to believe that the overwhelming majority of calls were right. Boy, was that bull.

 

Robo. Umps. NOW! 20% incorrect is UNACCEPTABLE!!

 

 

Note, I keep trying to make this an active link, but it keeps not working.

 

Try this one:

 

https://theconversation.com/an-analysis-of-nearly-4-million-pitches-shows-just-how-many-mistakes-umpires-make-114874

[Mr. Brooks]

You don't have to convince me.

[Otto von Ballpark]

The 20% number isn't an overall error rate. I think it's just the error rate for a subset of pitches when the batter has a two-strike count ("two strike bias").

 

Here's a link to the actual study findings, rather than a media report about it:

https://www.bu.edu/today/2019/mlb-umpires-strike-zone-accuracy/

 

See point #5 "Bad call ratio by year". The overall error rate was only 9.21% last year, and it has improved every year since 2008. So perhaps PitchF/X and Statcast are helping human umpires improve?

 

I'd like to see the results weighted by egregiousness of the bad calls, rather than a binary good/bad. If a pitch is 2 microns outside the Statcast strike zone, I really don't care if a human calls it a strike where a machine would call it a ball -- my eyes couldn't perceive the difference anyway. The summary says the Statcast measurement margin of error is claimed to be within one inch -- how many of these "bad calls" were within that?

[yarnivek1972]

9.21% is still WAY too high.

[Otto von Ballpark]

 

9.21% is still WAY too high.

Well, it's been going down every year for the past 10 years, so that rate may improve further.

 

But without any clue about their magnitude, I'm not sure the raw percentage of "errors" is that meaningful anyway.

 

I mean, some percentage of pitches are going to be right on the edge. (Maybe a significant percentage, if pitchers are aiming there.) Umpire "errors" on that edge, as compared to Statcast, are going to be virtually imperceptible to the human eye (if they are even umpire errors at all -- they could be Statcast calibration/measurement errors). Why would you care if Statcast thinks a pitch is 2 microns outside, and the umpire calls it a strike?

 

Hypothetically, if those edge cases made up half of the "errors", then you're really looking at a meaningful error rate of 4.6%. Is that still too high?

 

(Keep in mind, these "error rates" are just among called pitches. 46.6% of pitches were swung at last year and thus did not require a call and are excluded from these percentages. If you are considering total pitches, you'll want to cut these error rates in half.)

[Riverbrian]

I think the umpires do an amazing job calling balls and strikes but with the job they are tasked with, perfection is impossible. 

 

However there is serious potential gravity on any missed call. 

 

Case in point: 2017 Wild Card Game: Twins vs. Yankees. Twins jump out to a 3-0 lead. Ervin Santana has two strikes on Didi... He throws a pitch that is clearly a strike that should have rung him up and it wasn't even that close. It was undoubtedly a strike... Ump calls it a ball. Next pitch... Didi parks a 3 run shot to RF and the game is tied 3-3. If the ump makes the right call on that pitch... the Twins could have walked out of the first inning with a 3-0 lead and possibly advanced. It was a big moment decided by a wrong call. 

 

Automate it... give the umps a break. 

[nicksaviking]

Yeah, we already had the referendum on the merits of “the human element” in umpiring when they went to review with the neighborhood play at 2B and hit by pitch. Finish the job and rip the band aid off. If losing the theatrics of the umpires’ strike three calls is too hard of a pill to swallow, compromise and let the umps be the intermediary between the robots and the fans. It should be split second automation.

.

[Jim Hahn]

I think a meaningful question is how often is Statcast wrong? If statstat is only accurate to within 1 in., does that mean Statcast sometimes turns a 16 plate into 14 in. plate? Or maybe an 18 in. plate sometimes? What about the top and bottom of strike zone? Is it the same for every hitter? What about the depth of the plate? Is Statcast set a the front Or the back of the plate ? Either way, how many times does a breaking ball go over the plate but miss the box?

 

I will be more inclined to support automated strike zones when I am convinced they aren't likely making nearly as many bad calls as the umpires do.

[Danchat]

I'd like to see a handful of games get called by the 'robo ump' and see how it goes.

[cardsfan]

I think a meaningful question is how often is Statcast wrong? If statstat is only accurate to within 1 in., does that mean Statcast sometimes turns a 16 plate into 14 in. plate? Or maybe an 18 in. plate sometimes? What about the top and bottom of strike zone? Is it the same for every hitter? What about the depth of the plate? Is Statcast set a the front Or the back of the plate ? Either way, how many times does a breaking ball go over the plate but miss the box?

 

I will be more inclined to support automated strike zones when I am convinced they aren't likely making nearly as many bad calls as the umpires do.

Even though the plate is 14 inches the ball only has to graze the line so it is more like 17-18 inches?

[johns]

Why would you want umpires to be perfect?  There's an entire mythology of umpires stinking at times, it's part of the game.  The umpire is one of the players on the field, just as a shortstop botches the occasional play, so too the umpire.  

 

At what point is the human element acceptable? Where is the line?

[wsnydes]

I find it interesting that so many just assume that the robots would be right 100% of the time. Who's to say that they're correct now? Do we have the ability to know that? The ball is moving at 90+ mph with movement in multiple directions. Frankly, I'm impressed that they get as much as they do correct.

[nicksaviking]

 

Why would you want umpires to be perfect?  There's an entire mythology of umpires stinking at times, it's part of the game.  The umpire is one of the players on the field, just as a shortstop botches the occasional play, so too the umpire.  

 

At what point is the human element acceptable? Where is the line?

 

When the human is wearing a glove, running the bases or holding a bat.

 

I think half of the problem is that some of the umpires think they ARE one of the players in the game.

[Respy]

I generally believe that the error here is complicated, but if we have enough evidence that robocalls are more accurate than real umpires, let's use the robocalls.

 

Several people have already asked what the real error of umpires and the real error of statcast is. Again, that's complicated. If we claim that umpires have 10% error, that's in reference to what? What's determining with close to 100% accuracy whether pitches actually are balls or strikes? Are we determining that 10% error using PitchFX data? Well, what if the PitchFX has 10% error? What's determining that PitchFX has 10% error? 

 

"One-inch accuracy" for PitchFX is nice, but that likely means that it's 99% accurate outside of 1-inch (I'm just speculating), and much less accurate at less than 1 inch. As other have stated, even using the best technology, a pitch that is 1/16 inch outside the strike zone is going to be a coin flip.

[nicksaviking]

 

I think a meaningful question is how often is Statcast wrong? If statstat is only accurate to within 1 in., does that mean Statcast sometimes turns a 16 plate into 14 in. plate? Or maybe an 18 in. plate sometimes? What about the top and bottom of strike zone? Is it the same for every hitter? What about the depth of the plate? Is Statcast set a the front Or the back of the plate ? Either way, how many times does a breaking ball go over the plate but miss the box?

I will be more inclined to support automated strike zones when I am convinced they aren't likely making nearly as many bad calls as the umpires do.

 

Perhaps I'm wrong, but I'd assume any of us in favor of automated balls and strikes calls is basing it on the assumption that the veracity of the technology is vetted and perfected to acceptable terms.

 

If it's not 100% accurate it would be easy to find out. I doubt any switch would happen without significant testing to the technology that would be used.

 

And I'd guess Statcast wouldn't be given this golden ticket; the new system would probably be based on a similar but different tech proprietary to the MLB.

[luckylager]

The game has survived just fine with human umpires for 150+ years, mistakes and all. Perfect is the enemy of good.  

[adjacent]

I guess the issue is not only error but bias. The error that the robot could  make is related to the limitations of the optics,and presumably there is no bias there (unless somebody rigs the machines, and that can be tested). Humans, all of  us, have bias, most of the time unintentional. But all of us pre-judge at some point, because what we thought we saw is different of what we saw. Besides we get tired, annoyed, etc. That doesn't happen with a robot. That is why I am for robocall of balls and strikes. 

[ScrapTheNickname]

 

I think a meaningful question is how often is Statcast wrong? If statstat is only accurate to within 1 in., does that mean Statcast sometimes turns a 16 plate into 14 in. plate? Or maybe an 18 in. plate sometimes? What about the top and bottom of strike zone? Is it the same for every hitter? What about the depth of the plate? Is Statcast set a the front Or the back of the plate ? Either way, how many times does a breaking ball go over the plate but miss the box?

I will be more inclined to support automated strike zones when I am convinced they aren't likely making nearly as many bad calls as the umpires do.

Well said!

[ashbury]

I will be more inclined to support automated strike zones when I am convinced they aren't likely making nearly as many bad calls as the umpires do.

What would convince you?

[biggentleben]

Everyone does understand that "roboumps" are still going to be a program sent down to human umpires who are standing behind home plate, right?

 

This is not eliminating an umpire behind the plate. If for some reason, the technology were to go out during the game, you still have the umpire behind the plate. When Eric Byrnes tried this out 2 years ago in one of the Indy Ball leagues, he said it was much more like getting confirmation of what he had already intended to call and then helping with the 1-2 pitches in the entirety of the game that were so close he just wasn't sure. Umpires very well may LIKE this, and it could really serve as more of a backup to what they're already seeing rather than a complete overhaul.

[cardsfan]

What are they using? I mean technology?

[RedBull34]

The math in that article was not well done.  It stated there was a 20% error rate and yet only 1.6 missed calls an inning.  While that's still high, that definetly doesn't add up at all.  That'd suggest only 8 pitches an inning (not even half inning!)

 

Maybe I'm missing something, but it doesn't check out to me.

[Shaitan]

I'd rather see 100% robo umps than some of the goofy "time saver" rules proposed for next year.

 

But I also 100% think the human element is essential. Otherwise I might as well just play video game baseball.

[yarnivek1972]

Well, it's been going down every year for the past 10 years, so that rate may improve further.

 

But without any clue about their magnitude, I'm not sure the raw percentage of "errors" is that meaningful anyway.

 

I mean, some percentage of pitches are going to be right on the edge. (Maybe a significant percentage, if pitchers are aiming there.) Umpire "errors" on that edge, as compared to Statcast, are going to be virtually imperceptible to the human eye (if they are even umpire errors at all -- they could be Statcast calibration/measurement errors). Why would you care if Statcast thinks a pitch is 2 microns outside, and the umpire calls it a strike?

 

Hypothetically, if those edge cases made up half of the "errors", then you're really looking at a meaningful error rate of 4.6%. Is that still too high?

 

(Keep in mind, these "error rates" are just among called pitches. 46.6% of pitches were swung at last year and thus did not require a call and are excluded from these percentages. If you are considering total pitches, you'll want to cut these error rates in half.)

 

If there are 100 to 125 pitches called in a game, the umpire is missing 10-12. More than one an inning. So, yeah, I think that’s too many.

 

As for the accuracy of robo umps, I would imagine whatever system they eventually use will trim accuracy to one centimeter or maybe even less.

[TheLeviathan]

 

The game has survived just fine with human umpires for 150+ years, mistakes and all. Perfect is the enemy of good.  

 

Merely because something has "survived" doesn't mean it can't be improved.  

 

Ask my wife if you don't believe me.

[sloopjont]

The most important part of automated umps isn't whether they make mistakes or not; it is because it would be the same strike zone for all players, both pitchers and both teams.  Take a look at some of the calls on Garver last night to see how his at bats were affected.  That made a big difference on a hot hitter and how he approached his at bats..

[Otto von Ballpark]

If there are 100 to 125 pitches called in a game, the umpire is missing 10-12. More than one an inning. So, yeah, I think that’s too many.

 

Again, some percentage of those "misses" are edge cases, effectively just coin flips even if called by robots. Are "misses" that are undetectable to human eyes still "misses" that we should care about?

[Otto von Ballpark]

The math in that article was not well done. It stated there was a 20% error rate and yet only 1.6 missed calls an inning. While that's still high, that definetly doesn't add up at all. That'd suggest only 8 pitches an inning (not even half inning!)

 

Maybe I'm missing something, but it doesn't check out to me.

I mentioned that in my post upthread (and shared a link directly to the study) -- the 20% error rate is just on 2 strike pitches.

 

The overall error rate is 9.21% on all called pitches.

[Otto von Ballpark]

I guess the issue is not only error but bias. The error that the robot could make is related to the limitations of the optics,and presumably there is no bias there (unless somebody rigs the machines, and that can be tested). Humans, all of us, have bias, most of the time unintentional. But all of us pre-judge at some point, because what we thought we saw is different of what we saw. Besides we get tired, annoyed, etc. That doesn't happen with a robot. That is why I am for robocall of balls and strikes.

Not all bias is bad. Expanding the zone to hurry a game through bad weather, or in a blowout. Or if two pitches in a row come in right on the edge, is it really that bad if the ump splits the difference and calls the first one a ball and the second one a strike, even if the second one was still 2 microns off the plate?

 

Also, this new system wouldn't eliminate bias. Most obviously, there would be bias in how the upper and lower bounds of the zone are calibrated for each hitter.

[twins_89]

I'm an advocate of automating the strike zone. I really like the idea of balls and strikes being called consistently every game. I would also like to see catcher pitch framing become nothing more than a historical oddity.