82 replies to this topic

[Brock Beauchamp]

[quote name='USAFChief'][quote name='SpiritofVodkaDave'][quote name='Brock Beauchamp']
I'm not admitting that Span's UZR was doubled because of Revere. Ben almost certainly had an impact but the bulk of Span's "improvement" likely came from being healthy.[/QUOTE]

Dude what on earth are you talking about? We are talking about how he doubled his UZR in 2011/2012 vs what he was putting up yearly in 2008-2010. In 2008-2010 he was healthy almost the entire time, 2011-2012 is when he had injury issues. Unless you think his concussion issues made him magically have a doubled UZR over his career average it is easy to assume that his UZR yearly average doubled 95% because of the fact that Revere was manning the OF with him.[/QUOTE]

Perhaps he went 2 years without being challenged.[/QUOTE]

Derp.

[old nurse]

By the numbers Valencia played well for a short stretch of time. .

We know that.

What we don't know is if "by the numbers" = "actually played well."

The talent is better assessed when you have more data. That is very basic, first concept, simple concept of statistics.

I think the very basic, first concept, simple concept of statistics is, "garbage in, garbage out."

Garbage in, garbage out is for computer programming. Understanding what is being measured is statistics. Clearly it is useless to explain it to you.

[kab21]

i think it's awesome that we are still arguing about UZR long after BYTO has died.

I've seen enough from UZR to feel confident when defensively ranking players as the following (with multiple seasons of data):
Excellent
Above Average
Average
Below Average
Awful

It's clearly not as good as offensive stats but it does give a pretty good idea of a player's talent level. And it is 100x better than using gold gloves or relying on random defensive evaluators that at best get to see all of the players in the MLB a couple of times/season and mostly on TV.

[TheLeviathan]

So chief predicates all his nonsense on the idea that small sample = innaccurate rather than inconclusive and we waste time with him? Seems anyone with that flimsy of a grip on the subject oughta be just dismissed immediately no?

[Brock Beauchamp]

Would it then be fair to say that in the following season, when UZR says Jeter was below average, that there is an equal chance there weren't enough "determining" statistics?

That in fact, there is no logical reason to say season "b" is any more accurate than season "a"?

That you're just guessing that somehow one season presented him with "conditions that were favorable" while the following season presented an accurate representation of these very same conditions?

And that in fact, nobody can say with any degree of accuracy what, if anything, UZR actually measures?

Which is why you use three seasons with all numbers combined to get an accurate representation of a player's value. Players will have up seasons, players will have down seasons. An extra season means more opportunities. An extra season means a different pitching staff. An extra season means, in all likelihood, a different third and/or second baseman. All of these things affect UZR and therefore, the larger the sample size with different variables, the more likely you are to reach some sort of meaningful conclusion. Season B is no more accurate on its own than Season A but when combined, they start to paint a picture of the player's defensive ability. Again, the metric is flawed and there are still situations not accounted for or not recorded accurately (as Dave mentioned, Field f/x would be a huge step forward). But that doesn't mean we throw the baby out with the bath water. We continue working toward improving it instead of dismissing it entirely.

[old nurse]

Understanding what is being measured is statistics.

Couldn't agree more. I rest my case.

Clearly you need to rest your case until you understand what is being measured

[tmerrickkeller]

I haven't read enough of this thread to know whose side I'm taking, but I do understand that people who follow baseball and watch baseball and coach and manage and scout baseball are way better at telling you whether someone is a "good" or "not good" fielder than UZR. The human eye can tell the difference between routine plays and great plays, botched easy plays and botched tough plays, and also tell you whether a player is more or less likely to boot a baseball or be out of position, or not paying attention, when the pressure is on than during the late innings of a blowout.

I certainly understand statistics (WAR, OPS+, etc.) that attempt to take individual statistics and turn them into something that measures a player's impact on his team and the game. I think those statistics are maturing and are valuable. I think there will also be a measure of fielding at some point, but getting into a discussion about the "value" of UZR as it relates to an ability to tell whether someone is a good or bad fielder (regardless of the sample size, though I do agree that larger sample sizes over several seasons are more valuable than whether someone went a month without an error), ignores the realities of baseball.

JJ was a sure-handed shortstop with limited range (and that's exactly why the Twins got rid of him - they thought they needed more speed on the bases and in the field in order to take that next step, which hindsight tells us was an improvident move); Valencia occasionally made good plays, but too often did not make routine plays, and this failing seemed to increase with the pressure of a situation. He also consistently struck me as a person whose head was not 100% into the game. I don't know what UZR says about either of those guys - I just know by watching a bunch of baseball that I trusted Hardy and didn't trust Valencia.

If someone tells me, then, that UZR says that Valencia is a better fielder than Hardy, or even that he was for a short period of time, I tell them that UZR is an inaccurate measure of fielding.

[Mike Sixel]

Only if those scouts watch hundreds of games, which btw, is what the people wo calculate Uzr do...they watch every play ans score how the defender did. Managers who see 7 games against Valencia have no real idea if he is good or not.

so your argument that Valencia scores better on fielding for 2 weeks proves it is a useless stat? I think you missed the whole part about sample size..

[tmerrickkeller]

Only if those scouts watch hundreds of games, which btw, is what the people wo calculate Uzr do...they watch every play ans score how the defender did. Managers who see 7 games against Valencia have no real idea if he is good or not.

so your argument that Valencia scores better on fielding for 2 weeks proves it is a useless stat? I think you missed the whole part about sample size..

Sorry if my point didn't come across correctly. I think what I was saying is that I don't care if someone scores every play as routine or exceptional and then puts it all into a matrix and comes up with something (UZR) that claims to be an accurate reflection of a player's defensive prowess. I don't think that is nearly as accurate a reflection of a player's defensive skill as "watching the games." And I think that the UZR ratings of players all throughout baseball (whether small or large sample size) bears out that UZR is not an accurate reflection of defense, and that actually watching the games gives someone a better idea of whether a player is good or not good defensively.

I think the mistaken point is to say that if there is a sufficiently large same size, that if player A's UZR exceeds player B's UZR, that player A is a better defensive player. That's a fault in the statistic itself (which was also, I hope, my point).

[tmerrickkeller]

That last post should have said "sufficiently large sample size" not "same size."

[Riverbrian]

I haven't read enough of this thread to know whose side I'm taking, but I do understand that people who follow baseball and watch baseball and coach and manage and scout baseball are way better at telling you whether someone is a "good" or "not good" fielder than UZR. The human eye can tell the difference between routine plays and great plays, botched easy plays and botched tough plays, and also tell you whether a player is more or less likely to boot a baseball or be out of position, or not paying attention, when the pressure is on than during the late innings of a blowout.

I certainly understand statistics (WAR, OPS+, etc.) that attempt to take individual statistics and turn them into something that measures a player's impact on his team and the game. I think those statistics are maturing and are valuable. I think there will also be a measure of fielding at some point, but getting into a discussion about the "value" of UZR as it relates to an ability to tell whether someone is a good or bad fielder (regardless of the sample size, though I do agree that larger sample sizes over several seasons are more valuable than whether someone went a month without an error), ignores the realities of baseball.

JJ was a sure-handed shortstop with limited range (and that's exactly why the Twins got rid of him - they thought they needed more speed on the bases and in the field in order to take that next step, which hindsight tells us was an improvident move); Valencia occasionally made good plays, but too often did not make routine plays, and this failing seemed to increase with the pressure of a situation. He also consistently struck me as a person whose head was not 100% into the game. I don't know what UZR says about either of those guys - I just know by watching a bunch of baseball that I trusted Hardy and didn't trust Valencia.

If someone tells me, then, that UZR says that Valencia is a better fielder than Hardy, or even that he was for a short period of time, I tell them that UZR is an inaccurate measure of fielding.

This post is nearly perfect. The only thing I disagree with... I don't recall ever seeing Valencia make a great play. At least... not as often as I see others make a great play... Everything else is spot on.

This discussion is a case of everyone being kinda right. A big part of my job is stats and one advantage I have over a lot of people who are looking at the same stats is recognizing conclusions based on bad or insufficient data. Others will take the flawed data and let it lead them in the wrong direction. I believe I'm able to toss out the bad stuff,

UZR is just plain bad data. It's based on typical zones and they are assigned to a position. It doesn't take into account that players don't play in the same spot batter to batter. This is just one thing in the many shades of defense that help keep runs off the board. Failing to get that out keeps your defense on the field and the damage can be 5 or 6 runs or it can be nothing if the next batter is simply retired.

Chief is right... You can put as much bad data together and increase the sample size... It's still bad data and the result will be flawed...

Brock is right... Increasing the sample size will stabilize the result and make it more reliable. If you study jackknife replication... You understand that results will replicate at a certain point.

One thing to always keep in mind with baseball metrics... Including offensive metrics. The data is constantly changing. Player improve... Players get worse. They get hot... They get cold... The length of the hotness and coldness goes up and down. Sample Size only matters if the data is consistent. There is nothing consistent about baseball. Baseball Stats are great... Baseball Stats are valuable but there is some bogus stuff out there and making it gospel will ultimately be a mistake. Combo the stats with your eyes and you have a better chance at a solid conclusion.

Defensive metrics will get better because smart people are working on it but if the result rendered today says that Valencia is a defensive friend of the pitchers. It's bad data leading to a bad result.

This post from tmerrickkeller says it perfectly... Just watch Valencia play defense.

[Mike Sixel]

I am arguing that a process that watches every game is more likely to be accurate than a scout that watches a handful of games. Precise? No, I am not arguing about precision.

[TheLeviathan]

Chief is right... You can put as much bad data together and increase the sample size... It's still bad data and the result will be flawed..

The concept of bad data in is not the issue. Chief claimed wrongly that the data was "inaccurate" in small samples. The problem in small samples is not accuracy, but in the ability to draw conclusions from it. In other words, the data is not inaccurate - it is inconclusive. If someone landed on this planet for the first time and met one person in the middle of the desert who had their arm amputated, they might decide that the entire human race has one arm from that experience. Their conclusion was inaccurate because they based it off of a sample size too small to draw larger conclusions. If that same alien waited 20 more people, 100 more, or 500 more - with an increasing degree of accuracy they could make a correct conclusion.

This is a fundamental, elementary statistical concept. That anyone could mistake inconclusive = inaccurate should tell them they should find another topic to debate in. At least until they're done with Stats for Dummies, much less 101.

[Riverbrian]

I am arguing that a process that watches every game is more likely to be accurate than a scout that watches a handful of games. Precise? No, I am not arguing about precision.

It Depends... If the process of watching every game is sitting down and marking a zone and calculating data inside zone... You are not watching much at all. Because Defense is a lot more than simply reaching a ball at X Spot.

Now if someone eyeballing a player watches one or two games and says... I've got this guy figured out... He isn't watching much at all either.

But How About a Manager who watches all 162 games. Who's more valuable... The Coach or guy in front of the screen watching zones.

Just to show I'm not firmly on any side here... The Tricky thing with eyeballing a player and treating that as gospel... Well that isn't an exact science either because I'm willing to bet that a Manager and a 3rd base coach on the same team can watch a player for 162 games and both come up with different conclusions about said player.

It's the beautiful thing about baseball... It can be explained to an extent by stats. It can be explained to an extent by plain ole gut feelings on a guy and those intersections will contradict often.

The only thing I would ever warn against... Is using either as gospel. Draw your own conclusions is what I do. I use stats and I use the eyeball and I read alot and I filter.

AND... I'm wrong as often as anybody else. It's a beautiful game.

[Riverbrian]

Chief is right... You can put as much bad data together and increase the sample size... It's still bad data and the result will be flawed..

The concept of bad data in is not the issue. Chief claimed wrongly that the data was "inaccurate" in small samples. The problem in small samples is not accuracy, but in the ability to draw conclusions from it. In other words, the data is not inaccurate - it is inconclusive. If someone landed on this planet for the first time and met one person in the middle of the desert who had their arm amputated, they might decide that the entire human race has one arm from that experience. Their conclusion was inaccurate because they based it off of a sample size too small to draw larger conclusions. If that same alien waited 20 more people, 100 more, or 500 more - with an increasing degree of accuracy they could make a correct conclusion.

This is a fundamental, elementary statistical concept. That anyone could mistake inconclusive = inaccurate should tell them they should find another topic to debate in. At least until they're done with Stats for Dummies, much less 101.

Yeah I read that... I paraphrased for him and felt that he was trying to express that the data sucked. It may have been my own spin.

[tmerrickkeller]

I think you are trying to reduce what is being said to a semantics argument, and it isn't. Whether data gathered from someone checking zones and seeing if plays are made is "inconclusive" as to a player's defensive ability regardless of sample size, or, whether you consider such data to be "inaccurate" in the first place is the issue. Some people will consider the underlying data based on zones and plays to be inaccurate - and I mean flat inaccurate.

Therefore, accumulating masses of data that don't accurately measure a player's defensive ability IS inaccurate data. It is not simply "inconclusive." Inconclusive suggests that it is headed in the right direction, and just lacks volume and mass (a large sample size). Inaccurate underlying data repeated over and over does not produce accurate conclusions.

I don't pretend to know all the thinking that goes into each rating of a play within UZR, but I know if the shift is on for Thome and he hits a ball right where the 3rd baseman is "supposed to be" and the 3rd baseman, playing a deep shortstop, runs like a madman and tries to make a play on that ball and fumbles it, I don't trust a system that tells me that a 3rd baseman gets a demerit because he didn't make a play that was within his normal zone. I know that a right fielder playing too deep with the go-ahead runner on 3rd and 1 out in the 9th who allows a ball to fall in front of him, or plays too shallow in the same situation, catches a ball going backward, and the throw fails to beat the runner home is not as good a fielder as the player who doesn't do that. And I don't think UZR measures that stuff. The human eye does.