Question about WAR calculation

[jimmer]

This post is about using stats of all kinds to tell us a player's true talent level versus telling us what a player did in a single year.

 

Lets start with a traditional stat and let's use 3 seasons

 

Hypothetical Joe Smith played for the same team from 2012 to 2014 and will continue to play for that team in 2015. He posted an OPS of .840 in 2012, .719 in 2013, and .855 in 2014.  We'd look at that and say Joe Smith is probably a .830-.840 OPS guy more or less right? That's probably his true talent level, roughly. But does that mean they .719 year didn't happen?  Does that mean that in 2013 he didn't actually have a .719 OPS?  When our goal is to look at how Joe Smith played in 2013 compared to his peers, do we care about his true talent level in that discussion?If 

 

Let's take hypothetical pitcher Jim Johnson,.  Same years, same team situation. He posts an ERA of 3.25, then 4.37, then 3.18.  His true talent level is probably around 3.25-.3.35 ERA.  But if we talk about the year he had a 4.37 ERA, regardless of whether or not his true talent level is 4.37 or not, it still happened.  That's still what he did that year.

 

We've known this for a long time how to figure out what the true talent level is in these kind of stats but we also clearly accepted that the year to year data told us what the player did that year regardless of whether or not he was really that good or that bad.

 

Now, just because the SABR community reminded us that stats are best used with more info to determine a players true talent when they explain the new stats, that doesn't mean we don't do the same thing with new metrics that we did with traditional stats above.  Yearly stats still count when talking about how a player did that year compared to others regardless of whether or not it's his true talent level.

 

 

[jimmer]

This post is about using stats of all kinds to tell us a player's true talent level versus telling us what a player did in a single year.

 

Lets start with a traditional stat and let's use 3 seasons

 

Hypothetical Joe Smith played for the same team from 2012 to 2014 and will continue to play for that team in 2015. He posted an OPS of .840 in 2012, .719 in 2013, and .855 in 2014.  We'd look at that and say Joe Smith is probably a .830-.840 OPS guy more or less right? That's probably his true talent level, roughly. But does that mean they .719 year didn't happen?  Does that mean that in 2013 he didn't actually have a .719 OPS?  When our goal is to look at how Joe Smith played in 2013 compared to his peers, do we care about his true talent level in that discussion?

 

Let's take hypothetical pitcher Jim Johnson,.  Same years, same team situation. He posts an ERA of 3.25, then 4.37, then 3.18.  His true talent level is probably around 3.25-.3.35 ERA.  But if we talk about the year he had a 4.37 ERA, regardless of whether or not his true talent level is 4.37 or not, it still happened.  That's still what he did that year.

 

We've known this for a long time how to figure out what the true talent level is in these kind of stats but we also clearly accepted that the year to year data told us what the player did that year regardless of whether or not he was really that good or that bad.

 

Now, just because the SABR community reminded us of what we know when just traditional stats were around, that stats are best used with more info to determine a players true talent, that doesn't mean we don't do the same thing with new metrics that we did with traditional stats in the above two scenarios. That doesn't mean we can't use stats compiled in a year, traditional and newer, when talking about how a player did in that year compared to others regardless of whether or not it's his true talent level.  His true talent level isn't relevant when discussing how a player did in one individual year.

 

So, if a guy had a WAR of 5.0 in 2012, 2.7 in 2013, and 5.1 in 2014, his true talent is around probably around 5.0, but in 2013 he WAS a 2.7. That still happened. He under-performed his true talent level in 2013. Same thing if someone had a 6.5 UZR, a 1.7 UZR and a 6.2 UZR.  He's probably a 6.1 or so UZR guy but that one year, he WAS a 1.7.  

 

If the conversation is about what kind of talent does this player really have, look at the last three year with more emphasis on the more recent seasons (do that with traditional stats as well as newer stats), but if asking how a player performed during a single year, you can look at the stats for that year (traditional and newer stats) and say, regardless of true talent level, THIS is what he did that year.

 

 

 

[jimmer]

As far as TEAM stats, using three years worth of data to determine a team's true talent level with ANY stat, traditional or newer, is worthless. Team rosters change too much year to year to do that.  Does it matter what the Padres team DRS was in 2013 or 2014 if I want to know what their true talent level on defense is this year?  Look how much that roster changed. Same thing with the Red Sox and so on. Prior to the 2015 season, why would I have looked at the Padres' 2012, 2013, and 2014 team DRS as a way to try and gauge what the true defensive talent level of the 2015 Padres team would be?  Makes zero sense. Just like it would make no sense in 2016 to look at the Angels team OPS this year if, say, Trout and Pujols somehow were moved. Or the Tigers teams OPS if Cabrera and JD Martinez were moved in the offseason. In both situations, the true talent level for the teams would have been drastically changed.

 

Because of the constant roster changes year to year on teams, yearly stats, traditional and new, are all you can use..  Even that gets shaky the more teams rotate their 25 man rosters in-season.  Certainly most team's talent level fluctuate throughout the year, but at least the players played for the team that year. Still tells us how the team did, as a whole, that year. 

 

 

[USAFChief]

If I want to know the "true" value of anything, I have to have something to measure it against.

 

The more valid the constant I use as a measurement basis, the better my estimate of the "true" value of whatever I'm trying to measure.

 

If I go to a car show comprised of only 1979 Ford Pinto station wagons, one of them is going to be the " most valuable."

 

But that doesn't tell me much about the "true" value of 1979 Ford Pintos, relative to cars in general.

 

Likewise, if I want to know how the Twins defense is performing, it seems to me I will have a better chance of arriving at a good answer if I use more data than less.

 

Either way, I'm comparing the Twins defense to a constant. It would HELP improve my estimate if that constant was arrived at using a bigger sample. Otherwise, I can't know if I'm at a Ford Pinto car show, or a Maserati car show, or a car show with both.

 

Like using three years of OPS rather than one for a player.

[ashbury]

I get dejected anytime I see mention that 3 years of anything is needed.

 

Time frames like that, or larger, are appropriate for looking back - gauging someone's HoF credentials and the like.

 

But for forecasting, which for me is 90% of the interest in coming up with numbers, I need something that is a little more nimble than waiting 3 years to see how a guy's major league career is already panning out.

 

Not to mention, as jimmer states, it's too difficult to hold talent level constant for 3 years, whether of an individual player himself or of some aggregate like a whole team.

 

Or, to extend Chief's automotive analogy a little, if I want to look forward to the 0-60 times for an array of cars, I probably will learn something from their respective 0-10 times, even if I acknowledge it suffers from Small Sample Size and will have to wait for the more reliable numbers. And I'll probably be able to rule out that it's a Maserati if it's really a Pinto.