Ervin Santana: Switch back to AL, Suzuki and OF Defense.

[jimmer]

Maybe it comes down to your personal definition of "used correctly".

 

Defensive stats can be used in smaller samples as long as we recognize they're likely to be slightly less accurate as a whole. I wouldn't think that means they can't ever be used as a reference without being incorrect. I also don't think someone referencing them in shorter periods should have to make a qualifying statement every time, which whether you're explicitly saying it or not, is the only middle ground left in your statement.

The funny thing about WAR is that people think of it as exact number.  Like if one guy is a 4.5 and another is a 4.6, the 4.6 guy must be better.  The people who make WAR are consistently telling us that's not necessarily true. Like there isn't enough different between a 4.2 WAR guy and a 4.8 WAR guy. Instead, WAR is more meant to put players into pretty much seven categories of value. Scrub, role player, solid starter, good player, all star, superstar, MVP.

 

Fangraphs WAR uses UZR as opposed to B-R that uses DRS. UZR fluctuates from year to year in part because the talent level at each position fluctuates a lot from year to year.  It's why small sample sizes (like one season) of UZR and by extension WAR, can't be used to evaluate what a true player's talent is over time even if it is a decent way to tell you what his talent was that specific year.

[jorgenswest]

Defensive stats as well many pitching and batting stats need three years for projection.

 

Strikeout rate, walk rate home run rate, groundball/fly ball rate are examples of measures that work well in one full season sample. Most of the rest, like defensive stats, need a bigger sample for projection.

 

That shouldn't stop us from using OPS or FIP in discussions, when we see it in a single year or even a partial year or split we need to understand the variability due to sample. We need to do the same for defensive stats.

 

As for Santana, we can look at his 2013 season with Royals and see the difference between his ERA and FIP and at least suspect that the Royal defense contributed some towards his low ERA for that season. We can also suspect, based on the current Twin outfielder's data over the last three years, that he won't get the same ERA boost from the Twins defense this year.

[Craig Arko]

Statistical sample size determination. The variance and confidence intervals need to be specified for the numbers to be useful.

 

http://en.m.wikipedia.org/wiki/Sample_size_determination

[TheLeviathan]

The funny thing about WAR is that people think of it as exact number.  

 

People think that because it is often poorly argued and employed.  Maybe Fangraphs has never been guilty of that (a claim I'd find a bit dubious tbh), but there are many who have been.  That perception didn't come out of nowhere.  Again, as has been my point all along, that isn't the metrics fault but it is once again being misused in the hands of those using it.

 

Also, I get that almost all stats are bad at projection.  It is, afterall, a guess about the future.  I don't hold a lack of precise prognostication against WAR or anything else.  That's another common theme I've seen when the metric faces some tough questions: we hide behind everything being weak at projection.

 

The problem with WAR is that it's not even all that great at telling us what has already, actually happened.  OPS needs a larger sample to get a fuller sense of value, but it is absolute, objective truth about what actually happened in the given sample.  UZR needs a larger sample to weed out human error.  Those are two VERY different problems - let's not conflate them.

[jimmer]

I try not to use OPS at all. OPS+ only in general terms only with people who don't want to dive deeper. It's a crude tool, IMO. Like it's something I'd discuss with Joe Blow in a bar, or a family member or such.

 

IMO, the problem with OPS is two fold. One it doesn't weigh OBP properly. It assumes SLG% and OBP are equally valuable, when OBP is worth more, almost twice as much, in creating runs. Second, slg% values hits improperly. Like it thinks a double is worth twice of what a single is when it's not.

In other words, it attempts to tell us who the most valuable offensive players are in regards to creating runs by what they do at the plate, but falls short. IMO.

 

I prefer to use wOBA in place of OPS when talking about overall offensive value created by what's done at the plate. I like wRC+ when comparing offensive value across the league.  I make no attempt to convince anyone else to do the same, though.

[TheLeviathan]

Those are all fair criticisms of OPS, some of which I share.  None of them, however, are human error.  None of them are a misrepresentation of what actually happened.  None of them are a criticism that the stat subjectively interprets what happened.

 

It is a measure of what actually did happen that cannot be disputed, whether it propery values a player or not is a different problem.  Whether it is the best tool to measure value is a different problem.  As a measurement of what happened it is indisputable.  WAR and pitch framing are not, but they are often wrongly misrepresented as having the same feature.

[jay]

They are likely to be significantly less accurate in a one year interval.

 

Not really. Each player's valuation is likely to be significantly accurate even in a one year interval.

 

You used the most extreme example possible to highlight the error range. Just like a bell curve, most are going to be closely centered around the average with a few outliers.

 

When the vast majority are going to be extremely accurate, I think it is both fair and reasonable to use one-year figures in many contexts short of saying Player A is definitively better than Player B because he had 1 more WAR last year.

[Craig Arko]

Significance; statistical and other kinds.

 

https://www.statpac.com/surveys/statistical-significance.htm

 

Oh, and less than one hour to go.