Question about WAR calculation

[TheLeviathan]

I really like your post above spy cake. I'm not a fan of the cumulative WAR usage right now in the metrics community and you laid out a really nice way to look at it.

[biggentleben]

WAR really has one tremendous usage - being able to compare value of players across positions and eras with different skill sets. The back of a baseball card statistics of my youth would have told me that Cal Ripken was a much better shortstop than Ozzie Smith, but in "real" baseball, that wasn't exactly true. I love using it for comparison with my favorite player, Andrelton Simmons, across eras against historically defensively-reputed shortstops.

 

All those comparisons should be viewed through the lens of the comparison being imperfect, but if Andrelton's DRS was 21 last season and the best that Ozzie ever put up was a 13 or Mark Belanger could only put up 15 or Omar Vizquel's best was 11, then that is notable on how great Simmons' 2014 was. If each of the four put up a 20- to 25-run season, it puts that season in an upper-echelon category, but comparing the exact runs and ranking them is where the farce enters in. A WAR of 9.8 shows more value than a WAR of 8.2, but 9.8 vs. 9.5 is fairly negligible due to imperfections in the measures in the system. Once that is more understood by the Average Joe fan, the statistic will start to be used more responsibly and accurately.

[USAFChief]

WAR really has one tremendous usage - being able to compare value of players across positions and eras with different skill sets. The back of a baseball card statistics of my youth would have told me that Cal Ripken was a much better shortstop than Ozzie Smith, but in "real" baseball, that wasn't exactly true. I love using it for comparison with my favorite player, Andrelton Simmons, across eras against historically defensively-reputed shortstops.

 

All those comparisons should be viewed through the lens of the comparison being imperfect, but if Andrelton's DRS was 21 last season and the best that Ozzie ever put up was a 13 or Mark Belanger could only put up 15 or Omar Vizquel's best was 11, then that is notable on how great Simmons' 2014 was. If each of the four put up a 20- to 25-run season, it puts that season in an upper-echelon category, but comparing the exact runs and ranking them is where the farce enters in. A WAR of 9.8 shows more value than a WAR of 8.2, but 9.8 vs. 9.5 is fairly negligible due to imperfections in the measures in the system. Once that is more understood by the Average Joe fan, the statistic will start to be used more responsibly and accurately.

How would anyone be able to say what Ozzie Smith's DRS was? It's calculated by guesswork.

[biggentleben]

 

How would anyone be able to say what Ozzie Smith's DRS was? It's calculated by guesswork.

 

This is the issue. I used DRS as an example metric, not a specific example. Whatever metric that has some ability to be measured back across eras that you prefer...

[Mike Sixel]

It's no more or less guesswork than any scientific method based on observation, like physics, chemistry, and that fancy computer you are using to type here. Almost all science is based on observation and theory, not on proof of anything.

[USAFChief]

It's no more or less guesswork than any scientific method based on observation, like physics, chemistry, and that fancy computer you are using to type here. Almost all science is based on observation and theory, not on proof of anything.

But that's the thing, Mike.

 

They can't observe Ozzie Smith. There's no video PBP.

[Mike Sixel]

 

But that's the thing, Mike.

They can't observe Ozzie Smith. There's no video PBP.

 

I thought you were also saying judging defense today is by guesswork, apologies....

[drjim]

 

But that's the thing, Mike.

They can't observe Ozzie Smith. There's no video PBP.

 

And even the observation they do now is far from scientific. It is 20 somethings getting paid little money to chart plays. Not exactly the most conducive to repeatable observations. 

[Mike Sixel]

 

And even the observation they do now is far from scientific. It is 20 somethings getting paid little money to chart plays. Not exactly the most conducive to repeatable observations. 

 

you just described nearly every chemistry and cancer lab in the country......20 somethings not getting paid much.

[USAFChief]

 

I thought you were also saying judging defense today is by guesswork, apologies....

Well, I don't have a high opinion of today's defensive metrics, either, but at least there's video PBP.

 

 

[Otto von Ballpark]

 

And even the observation they do now is far from scientific. It is 20 somethings getting paid little money to chart plays. Not exactly the most conducive to repeatable observations. 

I don't think a simple "chart plays" description is accurate anymore.  Zones are smaller, hang time is tracked, everything is confirmed on video, etc.

 

Both UZR and DRS are using data from Baseball Info Solutions.  BIS also provides data to 21 of 30 MLB teams:

 

http://www.mcall.com/sports/baseball/mc-mlb-the-big-shift-20150324-story.html

 

 

I am sure it's not perfect, but given the demands of those customers and that BIS has been in business since 2002, I'd guess their data is pretty reliable.

[Otto von Ballpark]

 

All those comparisons should be viewed through the lens of the comparison being imperfect, but if Andrelton's DRS was 21 last season and the best that Ozzie ever put up was a 13 or Mark Belanger could only put up 15 or Omar Vizquel's best was 11, then that is notable on how great Simmons' 2014 was.

It seems this was a hypothetical example with made-up numbers, but just to clarify: DRS doesn't actually exist for Ozzie Smith or Mark Belanger.  TotalZone (TZ) is the pre-2003 data set / metric.

[biggentleben]

 

It seems this was a hypothetical example with made-up numbers, but just to clarify: DRS doesn't actually exist for Ozzie Smith or Mark Belanger.  TotalZone (TZ) is the pre-2003 data set / metric.

 

Yep, we already went down that road. Sorry for making an example that I knew was fanciful.

[jimmer]

 

I don't think a simple "chart plays" description is accurate anymore.  Zones are smaller, hang time is tracked, everything is confirmed on video, etc.

 

Both UZR and DRS are using data from Baseball Info Solutions.  BIS also provides data to 21 of 30 MLB teams:

 

http://www.mcall.com/sports/baseball/mc-mlb-the-big-shift-20150324-story.html

 

 

I am sure it's not perfect, but given the demands of those customers and that BIS has been in business since 2002, I'd guess their data is pretty reliable.

We all know defensive metrics are just made up by 5 nerds in a closet picking random numbers for no reason :-)

[drjim]

 

I don't think a simple "chart plays" description is accurate anymore.  Zones are smaller, hang time is tracked, everything is confirmed on video, etc.

 

Both UZR and DRS are using data from Baseball Info Solutions.  BIS also provides data to 21 of 30 MLB teams:

 

http://www.mcall.com/sports/baseball/mc-mlb-the-big-shift-20150324-story.html

 

 

I am sure it's not perfect, but given the demands of those customers and that BIS has been in business since 2002, I'd guess their data is pretty reliable.

 

Again, how do you think BIS gets their data? Charting plays. I'm not sure there is a better way to do it.

 

This isn't a critique, I don't know how else they would do it. But to think there is not room for interpretation or problems with replication is to ignore how the data is gathered.

[drjim]

 

We all know defensive metrics are just made up by 5 nerds in a closet picking random numbers for no reason :-)

 

It would be better if it was 5 nerds, would guarantee more consistency.

[jimmer]

I wonder how much consistency we'd get if we took 5 fans from each team and had them rate defensive abilities of all 30 team starters at each position how consistent those rankings would be (and how hilarious the results would be).

 

Or even 5 fans from one team, for that matter.

[Otto von Ballpark]

 

Yep, we already went down that road. Sorry for making an example that I knew was fanciful.

Sorry, I overlooked your follow-up post, didn't mean to pick on you.  I was busy checking Ozzie's TotalZone numbers, I think.

[Otto von Ballpark]

 

Again, how do you think BIS gets their data? Charting plays. I'm not sure there is a better way to do it.

 

This isn't a critique, I don't know how else they would do it. But to think there is not room for interpretation or problems with replication is to ignore how the data is gathered.

If by "charting plays" you mean, someone is charting shifts, triangulating hit location to precise zones/vectors, and timing the ball between contact and ground/glove, and someone else is double-checking it all on video, then sure, "charting plays."

 

Like any data collection set, I am sure it's not perfect, but 21 of 30 MLB teams are paying for it, so I am guessing its imperfections aren't too detrimental to its usefulness.

[drjim]

 

If by "charting plays" you mean, someone is charting shifts, triangulating hit location to precise zones/vectors, and timing the ball between contact and ground/glove, and someone else is double-checking it all on video, then sure, "charting plays."

 

Like any data collection set, I am sure it's not perfect, but 21 of 30 MLB teams are paying for it, so I am guessing its imperfections aren't too detrimental to its usefulness.

 

Who said it wasn't useful? I think it is the best method we have.

 

But if you want to go down that road, I would say that the teams that are using this data are using it for something much more sophisticated (and probably not even all that related) than the current defensive metrics that are publicly available.

[biggentleben]

 

Sorry, I overlooked your follow-up post, didn't mean to pick on you.  I was busy checking Ozzie's TotalZone numbers, I think.

 

Yeah, though the comparative metrics we do have do show Simmons to be exceptional thus far in comparison to even the best of the best. I just hope he keeps it up and doesn't go the way of Rey Ordonez.

[Otto von Ballpark]

Who said it wasn't useful? I think it is the best method we have.

 

But if you want to go down that road, I would say that the teams that are using this data are using it for something much more sophisticated (and probably not even all that related) than the current defensive metrics that are publicly available.

I wouldn't be surprised if some of the teams weren't using it at all except to double-check some of their own data.

 

Still, picking on the "poor 20-somethings charting plays" seems like a petty (and misleading) grievance with defensive metrics. There are plenty of real issues with the metrics which are freely admitted (notably the small samples and the sometimes unpredictable human subjects ).

[drjim]

 

I wouldn't be surprised if some of the teams weren't using it at all except to double-check some of their own data.

Still, picking on the "poor 20-somethings charting plays" seems like a petty (and misleading) grievance with defensive metrics. There are plenty of real issues with the metrics which are freely admitted (notably the small samples and the sometimes unpredictable human subjects ).

 

In my defense it is far from my only grievance, just the most petty.

 

But I have worked for a similar data company and think about how they gather their data. Perhaps BIS is much better than I could ever anticipate.

[USAFChief]

Again, how do you think BIS gets their data? Charting plays. I'm not sure there is a better way to do it.

 

This isn't a critique, I don't know how else they would do it. But to think there is not room for interpretation or problems with replication is to ignore how the data is gathered.

The article you linked shows how teams are using BIP data to position fielders.

 

That's an entirely reasonable use for that data. "Hey look...Joe Mauer never hits the ball in the air to right field...why are we putting a right fielder out there?"

 

But that has nothing to do with using that data to assign run prevention data to individual players, which is the claim made by proponents of defensive metrics.. Whether or not thats being done accurately, or whether that is even possible is a different argument.

[USAFChief]

In my defense it is far from my only grievance, just the most petty.

 

But I have worked for a similar data company and think about how they gather their data. Perhaps BIS is much better than I could ever anticipate.

It was a few years ago, and it was at the AAA level, but I spent several summers working as a stringer for BIS. Greatest part time job ever...paid to watch baseball. At that time, at that level, there was no video, each game had two stringers and we charted each ball in play as it happened live. As far as I know, we all put some effort into getting it right, but it was still pretty haphazard. I'm sure at the big league level it's much more accurate.

 

Sadly, I had to give it up when my real job got in the way, and now that I could do it again, we no longer have a AAA team in Tucson.

[sane]

WAR! huh yeah!

What is it good for?

Absolutely nothing! say it again y'all

WAR! huh good God

What is it good for?

Absolutely nothing!

[drjim]

The article you linked shows how teams are using BIP data to position fielders.

 

That's an entirely reasonable use for that data. "Hey look...Joe Mauer never hits the ball in the air to right field...why are we putting a right fielder out there?"

 

But that has nothing to do with using that data to assign run prevention data to individual players, which is the claim made by proponents of defensive metrics.. Whether or not thats being done accurately, or whether that is even possible is a different argument.

This is pretty much thought too. There is certainly valuable uses for this data and makes sense teams would utilize it. Spray charts filtered by pitches/situations/etc. Best part of that is precision is of much less importance.

[Otto von Ballpark]

 

This is pretty much thought too. There is certainly valuable uses for this data and makes sense teams would utilize it. Spray charts filtered by pitches/situations/etc. Best part of that is precision is of much less importance.

Nobody is claiming any great precision for defensive metrics here either.  Just that it's better to include them when evaluating a player's overall performance over a large enough sample, rather than to use offense only and some ambigious eye test adjustment for defense.

 

I seriously doubt that BIS stringer accuracy has much of an impact on the current DRS or UZR of Fielder and Hosmer (which, lo and behold, are pretty much in line with their career numbers and usage -- i.e. Fielder at DH).

[drjim]

 

Nobody is claiming any great precision for defensive metrics here either.  Just that it's better to include them when evaluating a player's overall performance over a large enough sample, rather than to use offense only and some ambigious eye test adjustment for defense.

 

I seriously doubt that BIS stringer accuracy has much of an impact on the current DRS or UZR of Fielder and Hosmer (which, lo and behold, are pretty much in line with their career numbers and usage -- i.e. Fielder at DH).

 

Perhaps some people think we shouldn't use metrics for defense, but I'm not one of them. I am perfectly willing to use defensive metrics, I consider them directionally relatively accurate and useful, especially in your extreme example of Hosmer being better than Fielder. But even the ambiguous eye test could probably catch that one!

 

My point, in the broader discussion of WAR, is how it is converted into "runs" and how that is combined with offensive (and to a lesser extent running) numbers into one number. And I especially don't buy the values of replacement level value of players on the low end of the defensive spectrum and how that inflates WAR value of the better defenders of the best defenders.

[Otto von Ballpark]

 

My point, in the broader discussion of WAR, is how it is converted into "runs" and how that is combined with offensive (and to a lesser extent running) numbers into one number.

Again, if you want the offensive numbers plus your eye test to tell you that Fielder and Hosmer are roughly equal overall contributors, you're going to have to convert your defensive evaluation to a runs scale somehow (or some kind of scale where it can be directly compared with your offensive evaluation, it can be called "runs" or "player points" or "baseball bucks" or "scouting scale" or whatever).

 

I'll chose runs as my scale, let DRS and UZR do their work, and then use the eye test to gauge small samples, correct for outliers, etc.

 

And I especially don't buy the values of replacement level value of players on the low end of the defensive spectrum and how that inflates WAR value of the better defenders of the best defenders.

Can you clarify?  I don't know what this means.