Jump to content

Providing independent coverage of the Minnesota Twins.
The same great Twins Daily coverage, now for the Vikings.

The Store


Photo

New Catcher Framing Values

  • Please log in to reply
34 replies to this topic

#1 markos

markos

    Senior Member

  • Members
  • 219 posts

Posted 03 March 2014 - 11:10 AM

The folks at Baseball Prospectus have released a new method for calculating value of catcher pitch framing.

http://www.baseballp...articleid=22934

It seems very thorough, as it adjusts for all of the following factors:

1) They created multiple unique ball-strike probability maps to account for a number of different factors: each separate ball-strike count, 5 separate pitch types: fastball, curveball, slider, offspeed and knuckleball, batter height and handedness, and pitcher handedness.

2) Run values based on specific ball-strike count. For example, getting a strike call on a 3-2 pitch is much more valuable than on a 3-0 pitch.

3) Adjusted for pitchers. So less value for catching a good control pitcher like Kyle Lohse than a wild pitcher like Edwin Jackson.

4) Adjusted for umpires.

The also created a similar model for passed-balls and wild-pitches to determine values for blocking pitches.

Edited by markos, 03 March 2014 - 11:44 AM.
typo


#2 tobi0040

tobi0040

    Senior Member

  • Members
  • 1,290 posts

Posted 03 March 2014 - 11:25 AM

[FONT=arial]I think the quants have created a monster here. I just don't buy framing plays anywhere near the role these studies suggest.[/FONT]

#3 ashburyjohn

ashburyjohn

    Twins Moderator

  • Twins Mods
  • 4,727 posts
  • LocationLake Tahoe, Nevada

Posted 03 March 2014 - 01:45 PM

[FONT=arial]I think the quants have created a monster here. I just don't buy framing plays anywhere near the role these studies suggest.[/FONT]


I don't see how the method presupposes anything. It sets up a framework, and if patterns emerge from it, we can go from there. It is written in terms of the value observed from situations; not the value of framing as an assumption going in. If anything, it stands a chance of debunking framing, if things turn out to cancel out.

#4 tobi0040

tobi0040

    Senior Member

  • Members
  • 1,290 posts

Posted 03 March 2014 - 02:18 PM

I don't see how the method presupposes anything. It sets up a framework, and if patterns emerge from it, we can go from there. It is written in terms of the value observed from situations; not the value of framing as an assumption going in. If anything, it stands a chance of debunking framing, if things turn out to cancel out.


Do you believe McCann was worth 21 fewer runs a season from 2008-2013 as a result of him framing pitches?

My belief is that this type of analysis makes some good arguments and is valuable at identifying the better catchers. I just don't think the difference between McCann and Doumit is 42 runs a year.

Let's not forget the Braves have had a slightly better pitching staff than the Twins and Pirates during this stretch.

Edited by tobi0040, 03 March 2014 - 02:24 PM.


#5 ashburyjohn

ashburyjohn

    Twins Moderator

  • Twins Mods
  • 4,727 posts
  • LocationLake Tahoe, Nevada

Posted 03 March 2014 - 03:08 PM

Let's not forget the Braves have had a slightly better pitching staff than the Twins and Pirates during this stretch.


And maybe a study like this points to a reason why?

#6 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 03 March 2014 - 03:17 PM

Turkenkopf did a study and found there was a difference by inning and home team as to when and who got the benefit of the call. There were some other anomalies he found. He kept the sample size to a half season to keep the study for entertainment value only.
For the pitch frame enthusiasts, what is the correlation factor between team ERA and close called pitches?

#7 tobi0040

tobi0040

    Senior Member

  • Members
  • 1,290 posts

Posted 03 March 2014 - 03:32 PM

And maybe a study like this points to a reason why?


I guess we will see this year, Mccann is with the Yankees and Doumit may actually catch a bit for the Braves.

I think Teheran, Medlan, Minor, and Hudson are just better pitchers than Worley, Blackburn, Pelfrey, Devries, Walters, and team.

Since 2010 the Twins have rotated Mauer, Butera, Doumit, Pinto, and Fryer with similar results.

Do you really think Doumit catching for the Braves and McCann for the Twins would have swung 42 runs a year? Doesn't that seem extreme?

#8 Willihammer

Willihammer

    ice cream correspondent

  • Members
  • 2,924 posts
  • LocationSaint Paul

Posted 03 March 2014 - 04:17 PM

My belief is that this type of analysis makes some good arguments and is valuable at identifying the better catchers. I just don't think the difference between McCann and Doumit is 42 runs a year.

Let's not forget the Braves have had a slightly better pitching staff than the Twins and Pirates during this stretch.


The formula specifically adjusts for pitcher impact.

We empirically determined each pitcher's value—to isolate it from each catcher's value—by performing a WOWY ("With or Without You") analysis. We note that we also compared these values to a linear regression model that included pitcher and catcher as separate factors; the high correlation between these measures suggested a good degree of ability to correctly assign credit (or blame) to individual players. The WOWY adjustments provide a viable and modular means of assessing the impact of pitchers on framing.
The adjustments derived from the WOWY analysis reflect two aspects of our approach. First, pitchers who throw a pitch that may not fit the norm for a given pitch group may show some difference in the WOWY results (such as hard cutters in the slider/cutter group). Second, pitchers with better command of a pitch than their peers (or the unqualified respect of the umpire) will seem easier to frame.
The WOWY analysis created adjustments ranging from +/- .1 called strikes per opportunity and from +/- .01 runs per opportunity. The largest gross beneficiary of easy-to-frame pitchers was—Yadier Molina. The perennial gold glove winner started the analysis with 127 runs added before giving 60 back to his pitchers. This reflects the command contributions of teammates of the class of Chris Carpenter and Adam Wainwright and is no knock on Molina, who still ranks high overall.


Looks to me like they've made an honest effort to try and address the most common concerns about putting values to pitch framing. Its remarkable to me how little the results waivered from earlier more basic estimates.

#9 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 03 March 2014 - 04:47 PM

If pitch framing is a repeatable skill then why are the numbers so variable for the high value catchers from year to year?

#10 Wookiee of the Year

Wookiee of the Year

    Senior Member

  • Members
  • 135 posts

Posted 03 March 2014 - 06:42 PM

If pitch framing is a repeatable skill then why are the numbers so variable for the high value catchers from year to year?

I haven't dug in too far yet, but I can think of three potential answers:

1) What counts as "so variable"? From 2007 to present, Joe Mauer's batting averages were:
.293, .328, .365, .327, .287, .319, .324
From 2007 to present, Mike Pelfrey's ERAs were:
5.57, 3.72, 5.03, 3.66, 4.74, 2.29, 5.19
Those all look pretty variable, but I don't doubt batting average and ERA are repeatable skills. How does pitch framing's variability compare to the typical stat?

2) We'd have to look at the sample sizes for each of the players in each year. Anyone who's a back-up or anyone who spent part of the year injured is going to have more variability because they caught fewer games--whether it's a counting stat (Total Framing Runs) or rate stat (Framing Runs per 7,000 Chances). Counting stats are obviously driven by playing time, but the rate stat's less accurate, too, if the sample's not large enough. How does consistency compare to annual sample size?

3) Some if it could just come down to seasonal opportunity. When I think about the year-to-year variability in defensive stats, I always think of any given center fielder and the number of chances he has every year to rob a home run. Some years you're just served up more opportunities. How many borderline calls are we looking at in a given year?

Again, without digging into the study yet, I think these are ranked in order of their impact--#3 the least, because even if borderline pitches aren't particularly common, catchers are seeing so many pitches every year that I'd think, given a season of playing time, you'd have a good sample size. #1 first because it does look to me like the same names keep showing up at the top and bottom each year.

Edited by Wookiee of the Year, 03 March 2014 - 06:45 PM.


#11 jorgenswest

jorgenswest

    Senior Member

  • Members
  • 1,611 posts

Posted 03 March 2014 - 06:50 PM

If pitch framing is a repeatable skill then why are the numbers so variable for the high value catchers from year to year?


I don't see the same variability. The range in total framing runs is 23-33. Home runs has ranged 36-54 in same time span.

The same players are at the top. Molina made the leaderboard in spite of three different teams and pitching staffs. The run numbers on the negative are getting smaller, but that could be a result of team awareness of the skill.

I do notice three players on the leaderboards happened to be available by free agency or trade this winter. The Twins found a guy on the wrong side of that leaderboard. Maybe the Twins are correct and the Rays, Pirates, Yankees and others are wrong in seeking and acquiring players who do well in this measure.

#12 biggentleben

biggentleben

    Senior Member

  • Members
  • 1,000 posts

Posted 03 March 2014 - 07:13 PM

Do you believe McCann was worth 21 fewer runs a season from 2008-2013 as a result of him framing pitches?

My belief is that this type of analysis makes some good arguments and is valuable at identifying the better catchers. I just don't think the difference between McCann and Doumit is 42 runs a year.

Let's not forget the Braves have had a slightly better pitching staff than the Twins and Pirates during this stretch.


Really? You know you're referencing some pretty poor staffs in that time frame. The Braves have built up their pitching, but the 2008 team's pitching was putrid outside of Tim Hudson before he was injured. The Braves pitching from 2009-2012 could have been characterized as mix and match. One excellent season from Javier Vazquez versus three putrid Derek Lowe seasons. Yes, the pitching was better, but McCann is actually tremendously good at framing from the naked eye. Picking out McCann won't help your argument, as he's one of the more underrated defensive catchers in baseball because he's not exceptional at any metric, but he is very good in nearly everything to do with defensive catching, even when he has had some injuries recently. He also had one of the best teachers at framing that one could possibly have in David Ross.
Staff Writer for Tomahawktake.com, come check it out!

#13 markos

markos

    Senior Member

  • Members
  • 219 posts

Posted 03 March 2014 - 07:47 PM

Looks to me like they've made an honest effort to try and address the most common concerns about putting values to pitch framing. Its remarkable to me how little the results waivered from earlier more basic estimates.


This is a big deal for me. It seems like everyone is working hard to figure out this part of the game, and this is another step forward.

#14 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 03 March 2014 - 08:00 PM

I don't see the same variability. The range in total framing runs is 23-33. Home runs has ranged 36-54 in same time span.

The same players are at the top. Molina made the leaderboard in spite of three different teams and pitching staffs. The run numbers on the negative are getting smaller, but that could be a result of team awareness of the skill.

I do notice three players on the leaderboards happened to be available by free agency or trade this winter. The Twins found a guy on the wrong side of that leaderboard. Maybe the Twins are correct and the Rays, Pirates, Yankees and others are wrong in seeking and acquiring players who do well in this measure.


The factors that go into hitting a pitched ball are far different than catching a ball near the plate. The ability to frame a pitch is supposedly not pitcher dependent. The numbers should then show a consistency for a full time catcher. They do not for the positive full time catchers. 44.2, 38.2, less than 32.9, less than 28.3 per 7000 pitches. Lucroy's numbers are not consistent. The range in runs rer season would have more to do with games caught, which varies. Going by pitchers per 70000 makes it so you are comparing against the same.

Edited by old nurse, 03 March 2014 - 08:03 PM.


#15 Willihammer

Willihammer

    ice cream correspondent

  • Members
  • 2,924 posts
  • LocationSaint Paul

Posted 03 March 2014 - 08:20 PM

Its a fair question. A couple possibilities come to mind

-fluctuations in repertoire year to year, and in pitch calling sequences.
-fluctuations in number of counts and changes in swing% in those counts (esp. 3-2), and pitches called/thrown after getting to those counts.
-park factor? Game time? Weather? Extra innings?
-different trends by base/out/score?

#16 tobi0040

tobi0040

    Senior Member

  • Members
  • 1,290 posts

Posted 03 March 2014 - 09:36 PM

Really? You know you're referencing some pretty poor staffs in that time frame. The Braves have built up their pitching, but the 2008 team's pitching was putrid outside of Tim Hudson before he was injured. The Braves pitching from 2009-2012 could have been characterized as mix and match.


Yes I was serious. Their staff was flat out much better than ours over that time period. It is not even close.

2013 Braves 3.18 ERA 1st in MLB
2012 Braves 3.42 ERA 5th in MLB
2011 Braves 3.42 ERA 4th in MLB
2010 Braves 3.56 ERA 5th in MLB
2009 Braves 3.57 ERA 3rd in MLB
2008 Braves 4.46 ERA 21st in MLB


2013 Twins 4.55 ERA 29th in MLB
2012 Twins 4.77 ERA 28th in MLB
2011 Twins 4.58 ERA 29th in MLB
2010 Twins 3.95 ERA 11th in MLB
2009 Twins 4.50 ERA 23rd in MLB
2008 Twins 4.16 ERA 11th in MLB

#17 biggentleben

biggentleben

    Senior Member

  • Members
  • 1,000 posts

Posted 03 March 2014 - 09:42 PM

Yes I was serious. Their staff was flat out much better than ours over that time period. It is not even close.

2013 Braves 3.18 ERA 1st in MLB
2012 Braves 3.42 ERA 5th in MLB
2011 Braves 3.42 ERA 4th in MLB
2010 Braves 3.56 ERA 5th in MLB
2009 Braves 3.57 ERA 3rd in MLB
2008 Braves 4.46 ERA 21st in MLB


2013 Twins 4.55 ERA 29th in MLB
2012 Twins 4.77 ERA 28th in MLB
2011 Twins 4.58 ERA 29th in MLB
2010 Twins 3.95 ERA 11th in MLB
2009 Twins 4.50 ERA 23rd in MLB
2008 Twins 4.16 ERA 11th in MLB


First, NL vs. AL, but looking at the team ERA hides the job the coaches did with some pretty crappy pitchers and the job McCann and David Ross did handling the pitching staff. The 2013 Braves were excellent, and the bullpens have been amazing recently, but there have been a mix of hodge-podge types in the rotation.
Staff Writer for Tomahawktake.com, come check it out!

#18 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 03 March 2014 - 10:08 PM

Its a fair question. A couple possibilities come to mind

-fluctuations in repertoire year to year, and in pitch calling sequences.
-fluctuations in number of counts and changes in swing% in those counts (esp. 3-2), and pitches called/thrown after getting to those counts.
-park factor? Game time? Weather? Extra innings?
-different trends by base/out/score?


Those reasoning would then lead me to believe there are influences other than the catcher's pitch framing contributing to the call

#19 jorgenswest

jorgenswest

    Senior Member

  • Members
  • 1,611 posts

Posted 03 March 2014 - 10:11 PM

The factors that go into hitting a pitched ball are far different than catching a ball near the plate. The ability to frame a pitch is supposedly not pitcher dependent. The numbers should then show a consistency for a full time catcher. They do not for the positive full time catchers. 44.2, 38.2, less than 32.9, less than 28.3 per 7000 pitches. Lucroy's numbers are not consistent. The range in runs rer season would have more to do with games caught, which varies. Going by pitchers per 70000 makes it so you are comparing against the same.


There will be inconsistencies in any measure. Is the argument that the Twins are wise in ignoring the data in making roster decisions? Perhaps they see the same inconsistencies pointed out above.

#20 tobi0040

tobi0040

    Senior Member

  • Members
  • 1,290 posts

Posted 03 March 2014 - 10:14 PM

First, NL vs. AL, but looking at the team ERA hides the job the coaches did with some pretty crappy pitchers and the job McCann and David Ross did handling the pitching staff. The 2013 Braves were excellent, and the bullpens have been amazing recently, but there have been a mix of hodge-podge types in the rot
ation.


hodge podge? they have been top 5. in five of six years. al-nl maybe a bit, but we are talking about a very wide gap.

I do find it interesting. mccann missed 60 games last year and their era was the best. He missed a total of 50 in 2011 and 2012. Those were the best three ERA's they had. He played 130-140 games from 2008 to 2010.

Edited by tobi0040, 04 March 2014 - 10:08 AM.


#21 biggentleben

biggentleben

    Senior Member

  • Members
  • 1,000 posts

Posted 04 March 2014 - 10:11 AM

hodge podge? they have been top 5. in five of six years. al-nl maybe a bit, but we are talking about a very wide gap.

I do find it interesting. mccann missed 60 games last year and their era was the best. He missed a total of 50 in 2011 and 2012. Those were the best three ERA's they had. He played 130-140 games from 2008 to 2010.


Review by who the pitchers were, not the stats they had. I do think the pitchers matter some in pitch framing, but he has worked with good pitchers, but with pretty poor framability types. Jurrjens and Hanson may have had good results in those seasons, but their pitches were not ones that'd be good framing guys. The best argument for "pitchers aid in framing" is a guy who is more Radke/Maddux with control using fastball and changeup rather than movement using sliders and curveballs.
Staff Writer for Tomahawktake.com, come check it out!

#22 LaBombo

LaBombo

    Senior Member

  • Members
  • 2,989 posts

Posted 04 March 2014 - 03:18 PM

It's interesting that in a thread about quantifying what is essentially the skill of deceiving umpires, the only mention of umpires is that they are 'accounted for' in the methodology.

If catcher framing is exerting a measurable (let alone significant) effect on ball and strike calls, then MLB either needs to either completely overhaul the umpire selection, training, and review process, or else take the primary responsibility for ball and strike calls away from them.

I realize I'm in the unvast minority on this one, but I'd much rather know whether a pitch is actually a ball or a strike than what the vivid imagination of a Joe West or a C.B Bucknor happens to think of it.

Edited by LaBombo, 04 March 2014 - 03:22 PM.


#23 tobi0040

tobi0040

    Senior Member

  • Members
  • 1,290 posts

Posted 04 March 2014 - 03:24 PM

It's interesting that in a thread about quantifying what is essentially the skill of deceiving umpires, the only mention of umpires is that they are 'accounted for' in the methodology.

If catcher framing is exerting a measurable (let alone significant) effect on ball and strike calls, then MLB either needs to either completely overhaul the umpire selection, training, and review process, or else take the primary responsibility for ball and strike calls away from them.

I realize I'm in the unvast minority on this one, but I'd much rather know whether a pitch is actually a ball or a strike than what the vivid imagination of a Joe West or a C.B Bucknor happens to think of it.


I actually agree. Just like the NBA as well where the star pitchers and hitters get better calls and rookies get hosed. Like the NBA, whining and acting like a child does seem to pay off as well.

#24 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 05 March 2014 - 06:49 AM

There will be inconsistencies in any measure. Is the argument that the Twins are wise in ignoring the data in making roster decisions? Perhaps they see the same inconsistencies pointed out above.


I would believe it is likely that the Twins would think they could coach a player into being better. I once read that the best number of net calls per game was about 2.5. Maybe on a game by game basis the Twins feel the framing effect is negligble
In pitch framing there are data showing differences for each pitcher, umpire, home vs away, inning, pitch type and location, ball and strike count, as well as catcher. There have been no studies that I could find that say who the batter is matters, ie are there more close pitches thrown/called on a Joey Votto versus David Blarney or a N.L. pitcher. There are enough AB where there is a pitch that is called a ball that was a strike, a strike that was a ball, and every pitch correct that there should be a set of data for the OPS for each condition. You could do it for each pitcher. What would the data show?

#25 Oxtung

Oxtung

    Senior Member

  • Members
  • 1,486 posts

Posted 05 March 2014 - 10:13 PM

I am stunned by the amount of effort and analysis that went into this. Great job by these guys. I think they've taken the critiques head on here. Good for them.

#26 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 06 March 2014 - 05:54 AM

Public pressure must work
http://www.baseballa...ver-before.html

#27 Oxtung

Oxtung

    Senior Member

  • Members
  • 1,486 posts

Posted 07 March 2014 - 12:22 AM

Public pressure must work
http://www.baseballa...ver-before.html


Perhaps. It's also possible the umpires, just like hitters and pitchers, want to be the best that they can be and now that there is a tool that allows them to accurately dissect the way they call games they are trying to improve themselves regardless of public pressure.

Either way it's a win. :)

#28 old nurse

old nurse

    Member

  • Members
  • 1,711 posts

Posted 07 March 2014 - 03:13 AM

Is there a more recent numbrt than Turkenkopfs for the correlation between pitch framing and runs allowed? In his 2008 article he found the correlation to be .30. That would be weak at best for the effect of pitch framing.

#29 Wookiee of the Year

Wookiee of the Year

    Senior Member

  • Members
  • 135 posts

Posted 07 March 2014 - 06:45 AM

Is there a more recent numbrt than Turkenkopfs for the correlation between pitch framing and runs allowed? In his 2008 article he found the correlation to be .30. That would be weak at best for the effect of pitch framing.

Hold on--weak? That sounds almost unbelievably strong to me. Wouldn't that mean 30% of a team's Runs Allowed is determined by one dimension of one player?

I don't know, but I have to think if you looked at the correlation between Centerfield UZR and Runs Allowed, or between closer ERA and Runs Allowed, or even between lead-off man OBP and Runs Scored, they'd all be significantly less than .30. And that's even with all of these being known skills, so you would expect an investment in them when a team is in "win-now" mode and a disinvestment during rebuilding.

As a potentially undervalued skill, I'd expect catcher framing to be much more divorced from where a team is in the winning cycle than these other skills. And .30 still sounds massive to me.

Am I interpreting that wrong?

#30 biggentleben

biggentleben

    Senior Member

  • Members
  • 1,000 posts

Posted 07 March 2014 - 08:21 AM

Hold on--weak? That sounds almost unbelievably strong to me. Wouldn't that mean 30% of a team's Runs Allowed is determined by one dimension of one player?

I don't know, but I have to think if you looked at the correlation between Centerfield UZR and Runs Allowed, or between closer ERA and Runs Allowed, or even between lead-off man OBP and Runs Scored, they'd all be significantly less than .30. And that's even with all of these being known skills, so you would expect an investment in them when a team is in "win-now" mode and a disinvestment during rebuilding.

As a potentially undervalued skill, I'd expect catcher framing to be much more divorced from where a team is in the winning cycle than these other skills. And .30 still sounds massive to me.

Am I interpreting that wrong?


With correlation comparisons, a perfect 1 is a perfect correlation, indicating causality (doing A causes B to happen) while a negative one is a perfect negative correlation, meaning there is a completely adverse relationship in causality (doing A causes the opposite of B to happen). In most statistical analysis, you want a correlation of 0.5 or better to show some sort of causality between variables. For instance, you can say that the consumption of tea is higher in the summer than in the winter. If your data showed a correlation equivalent of 0.3, you'd likely conclude that tea and summer seem to go together, but there doesn't seem to be strong causality. Basically, if you look at the number it's a measure of how much of 1 is cause and how much is coincidence. A score of 0.3 would mean that 30% of runs allowed and catch framing has something to do with catch framing causing a runs allowed difference and 70% coincidence. Once you get over a 0.5, there's a lot more causality to assign to catch framing changing the runs allowed by a team.

Edited by biggentleben, 07 March 2014 - 10:12 AM.

Staff Writer for Tomahawktake.com, come check it out!