Unfavorable strike zones have been costly to the White Sox

In 2021, Ethan Singer has compiled and made available a feature known as Ump Scorecards which have become quite popular on Twitter. The account creates visualizations to evaluate the performance of home plate umpires at accurately calling balls and strikes. Using the unique run expectancy of the 288 baserunner, amount of outs, and count combinations, Ethan’s program determines the cost or benefit in terms of runs of each pitch that is called incorrectly. You can read more about the process and access leaderboards on the Ump Scorecards website.

A secondary outcome to the analysis is that we can see which teams have benefited and which teams have been hurt by home plate umpires. According to Singer’s work, the White Sox are getting crushed by ball/strike calls. The White Sox are fourth-to-last in baseball with 10.0 runs lost, which is roughly equivalent to one win by the various WAR metrics.

Before jumping to any conclusions about the root cause of the damage, it’s important to understand the noise that goes into the the net runs lost number. The number includes pitches called on both offense and defense, so it’s a a comparison of how two teams fare with the same umpire on the same day. Further, the context of the situation is very important to the calculation; a blown call with the bases loaded and one out moves the needle far more than a missed call with two outs and the bases empty.

Of course, it’s natural to try to view these results in the context of pitch framing, as the way the catcher receives the ball is the only piece of the puzzle over which the White Sox have any control. However, Baseball Prospectus’ framing runs for White Sox catchers doesn’t tell a story anywhere near as bleak:

OK, so Collins has been pretty bad. But Baseball Prospectus’ model is only attributing about one run lost by the White Sox due to framing, which is nothing compared to the total of 10.0 lost on umpire calls. There’s several possible reasons for the discrepancy.

  1. BP’s model is context-neutral and seeks to estimate the number of strikes gained or lost, and assigns an average run value to strikes. It’s possible that the Sox have had unlucky timing with regards to when calls go against them.
  2. Because the 10.0 runs lost includes runs lost on offense, the Sox may have run into opposing catchers that have framed well.
  3. There could be some noise from unlucky umpiring in just a half-season of play.
  4. BP’s model might dampen outliers so as to not overreact to a small sample like Collins’ 1,700-ish framing chances (yes, this constitutes a small sample for framing).

This season, I’ve been logging the Ump Scorecards data by game and observing how the strike zone battle has gone for the White Sox under each of the two primary catchers. Here are the splits through the end of June:

CatcherTotalHomeAwayPer GamePer HomePer Away
Grandal0.252.40(2.15)0.010.09(0.10)
Collins(10.26)(4.72)(5.54)(0.35)(0.30)(0.43)
Total(10.01)(2.32)(7.69)(0.13)(0.05)(0.22)

Two observations leap to mind:

  1. Though it’s probably not all that surprising, the home team seems to get more favorable calls than the away team.
  2. Nearly the entirety of strike zone runs lost are attributable to games started by Collins, regardless of the degree to which this is his fault.

There’s undoubtedly plenty outside of Collins’ control underlying this split. Unlucky calls could be happening against White Sox hitters in his starts, which would distort the number. He might have run into an above-average slate of framing catchers (though he’s “lost” all four of his starts against baseball’s worst framer, Salvador Perez). It’s also possible that umpires have simply missed some calls for reasons unrelated to how Collins receives pitches.

The thing in common to all of those Collins-absolving conjectures is that they’re all rooted in unfortunate things that are outside of his control. Since no catcher can sustainably draw unlucky umpiring or outstanding opposing framing catchers, the natural conclusion is that if the strike zone runs the White Sox have leaked in Collins’ starts aren’t primarily due to his receiving, they won’t continue at this untenable rate.

For that reason, I’ll be keeping a close eye on the “Per Game” numbers in the table above as we move further along into the season. The most likely outcome is that they’ll get better due to “positive regression” and we’ll be able to conclude that Collins is bad-but-playable behind the plate. However, if they don’t improve, there might need to be an honest look at where he fits within the team’s long-term plans with the logjam of 1B/DH types currently at their disposal.

Take a second to support Sox Machine on Patreon
7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Michael Kenny

Great article Jim.

ParisSox

Who dis?

Last edited 2 years ago by ParisSox
karkovice squad

I still think this analysis is stretching data from an undergrad stats project way past its usefulness. Their methodology just isn’t that rigorous for more than saying what happened in the game with called strikes at a top line level.

One issue is that the model is designed so that it really only grades obviously bad calls, ones that should’ve gone the other way 90% of the time. Setting the universe that narrowly might meaningfully skew the results.

Since no catcher can sustainably draw unlucky umpiring or outstanding opposing framing catchers, the natural conclusion is that if the strike zone runs the White Sox have leaked in Collins’ starts aren’t primarily due to his receiving, they won’t continue at this untenable rate.

There is also some selection pressure on his sample as a backup catcher. It’s not impossible that he could draw the short end of the stick at a disproportionate rate because of that. Plus, along with the other caveats you flagged, there are also some lineup and pitcher effects to called strikes. It’s part of the reason BP uses a multi-factor model and that it grades called strikes by probability.

You’re right that by making its model context neutral it might undestate how costly Collins’ defense has been in actual games. But it’s probably a better measure of true talent and what we should expect from him going forward.

BP’s model might dampen outliers so as to not overreact to a small sample like Collins’ 1,700-ish framing chances (yes, this constitutes a small sample for framing).

BP’s model starts to stabilize at 10% of a season. Collins has 270 innings behind the plate so far, about 20% of a season. We shouldn’t expect all that much variance in his CSAA.

As Cirensica

Insightful from you and Jim…I mean Patrick. Very difficult topic to cover that took me long time to appreciate. I understand Collins is not a great framer, but I have leraned to accept that as a back up catcher, we are lucky to have Collins. He is adequate, still have upside in his hitting tools, and he can eventually learn to become a better framer.

When compared with other teams backup catchers, offensively, the White Sox are actually doing pretty well. Again, offensively, Collins is even better than some main catchers in some teams.

Trooper Galactus

Collins is already better than I thought he’d be. And even if that’s not very good at all, it’s a lot better than running Dioner Navarro out there again.

soxygen

Any insights in the data as to whether there is a bigger home field umpiring advantage than in 2020? It would be interesting to get a sense of how much of that is due to fan noise.

Last edited 2 years ago by soxygen
burning-phoneix

This is all a conspiracy by the Commishs office to keep the White Sox down.