Ervin Santana: Switch back to AL, Suzuki and OF Defense.

[ashbury]

 My criticism is in the interpretation of the metrics.  Somehow through an equation

You believe framing metrics are from "an equation"?

 

Framing and Blocking Pitches: A Regressed, Probabilistic Model

[TheLeviathan]

You believe framing metrics are from "an equation"?

 

Framing and Blocking Pitches: A Regressed, Probabilistic Model

 

I have repeatedly, and clearly, been taking a general tone in my comments.

 

In any case, just because you've derived a metric from a complicated set of calculations doesn't really change my point.  There is still way too many subjective assignments of value going on for me not to be skeptical.

[FattCrapps]

Personally, I think framing tends to help certain teams (like the Yankees) and it has little of anything to do with who is behind the dish.

 

Don't forget about the Cardinals.

[h2oface]

My conclusion on framing is that they need to automate ball/strike calls and take away human error. That would take this whole thing out of the equation.

 

I think the idea that framing is a real thing is probably true. It's impact on the game is what's up for debate. Personally, I think framing tends to help certain teams (like the Yankees) and it has little of anything to do with who is behind the dish.

Absolutely baseball needs to automate the ball/strike calls. It is a change that already is years late. Of course pitch framing comes into play.  Umpires are easily fooled.  They get it wrong on their own often enough, even without a catcher's trick to cheat the game and the strike zone.

 

So what does one use to establish the pitch framing stat? They use the automated pitch f/x.  May as well just finally have a real and true consistent strike zone, and use it to call the balls and strikes.  That is what the umpires have always supposedly tried to do, by rule of the game. It is just it is humanly impossible to have a consistent accurate zone with humans calling it.  Both the pitchers and batters deserve the strike zone to be real, and not a fantasy. Then there will be no more talk about pitch framing.

[jay]

I've made plenty of "pro-framing" posts in the past. I've tried to keep them more conversational than definitive. However, I'm concerned by what seems to be a group of smart folks that have rallied around discrediting metrics because they aren't as precise or objective as counting runs that crossed the plate.

 

I get that it's much more comforting to say you can count with exact precision how many runs crossed the plate. I also don't believe there's a big contingency here at TD that cram metrics at people and proclaim them to be gospel. I further believe that most posts here on the topic recognize framing as a thing.

 

Parochial tones on using stats rarely helps, but when a good portion of the posts questioning a metric can be refuted with some very standard principles of statistics, I wonder where the level of understanding is at on how these metrics are actually calculated. I really don't understand why we can't utilize the significant efforts and data available in an attempt to put some type of a value on these things without having to defend and justify the calculation methods and perceived level of precision just about every time it comes up. Maybe every post that uses any type of estimated metric needs to be qualified with something like "these figures are calculated with some level of subjectiveness and the values have a margin of error"... ??

[Otto von Ballpark]

Pitch framing has the largest data sample of any event on the field, and thanks to pitchfx, it is also the most precisely measured right now too. I am sure it is not perfect, but not sure if it is fair to lump it in with DRS/dWAR.

[jimmer]

Sorry for the length, but bare with me

 

The reason I like metrics/advanced stats is because it helps me rate players against players at the same position. Whether one wants to wrestle with the pin-point accuracy of it (like did he really save 20 runs could it have been more like 15), I feel they have a good grasp of being able to do that, because even if there’s subjectivity in it (and how much subjectivity is debatable), it’s at least applied evenly across the board. 

 

Let’s look at defense as an example.  I like to use four categories when talking about how a player plays defense.  He’s great, he’s good, he’s bad and he’s horrible.  Now, I feel I have a good enough eye to at least go that far. I played at a pretty competitive level, and I’ve always watched a good chunk of baseball, so I feel pretty comfortable in that.  What I don’t do is watch all of the games of all the teams with the sole purposes of evaluating players defense like the people who do the metrics. No one fan can do that because none of us watch enough games of other teams and when we do watch other teams, it’s certainly not with the sole purpose of evaluating the defensive ability of every player on the field and comparing against their positional peers.

 

So, I may be comfortable in my belief that Plouffe was a good defender last year (and, not shockingly, metrics backs that up), I don’t watch enough of every team play while focusing just on 3B play to see where he rated 1-30. Even if I magically could watch that many games, I certainly can’t be sure my bias wouldn’t show itself, as much as I’d like to think it wouldn’t.  So that is why, for the purpose of ranking player’s abilities against their peers, I have no problems with looking to organizations that do it for a living.  These people have been doing these things for years and years.

 

For me, metrics get better and better but aren’t perfect.  The SABR guys will even tell you that, including guys like Dave Cameron. I don’t know anyone who takes it for gospel to include the people who do them for a living. They continuously look for ways to improve.  Heck, Fangraphs is adjusting its pitcher WAR next year to include inducing pop ups. Nobody is making them do that, they saw an issue and they openly admitted it and are fixing it.

 

What I think is important to note that, while not perfect,  it’s certainly better than any one person’s evaluation. Many detractors don’t seem to want to admit that. They talk about the issues with metric, but blow off the inherent problems with fan bias.  I wouldn’t feel more comfortable saying something like ‘I bet player X great defense only saved 5 runs as opposed to 20 last year, or I bet he saved more than 20.’  I don’t look at every play of every team’s games and see how that non-strike call (or a strike 3 called when it should have been ball 4) ended up affecting a rally like the guys who do this do. I don’t see every play of every game to see how Juan Ligare’s unbelievable catch turned what would have been a bases clearing double into out #3. I think defense affects games way more than some believe and the really great defenders save a lot of runs, likely even more than metrics currently says (as speculated also by the people who do them).

 

I do find it interesting how some fans and GMs use metrics in an argument that agree with their opinion but so easily toss it aside as hogwash when it doesn’t.  If one always does that, there is literally no reason to ever reference them. Even better, is when people think they know how to use them when they don’t and then, because of that, find fault in them. And the best, is how they think certain traditional stats tell a much better picture, not admitting how badly these stats we grew up on really are at giving us a true evaluation of even the thing they are supposedly telling us about a player.  These stats get a pass because they aren't new, they aren't a 'metric'. 

[jimmer]

I've made plenty of "pro-framing" posts in the past. I've tried to keep them more conversational than definitive. However, I'm concerned by what seems to be a group of smart folks that have rallied around discrediting metrics because they aren't as precise or objective as counting runs that crossed the plate.

 

I get that it's much more comforting to say you can count with exact precision how many runs crossed the plate. I also don't believe there's a big contingency here at TD that cram metrics at people and proclaim them to be gospel. I further believe that most posts here on the topic recognize framing as a thing.

 

Parochial tones on using stats rarely helps, but when a good portion of the posts questioning a metric can be refuted with some very standard principles of statistics, I wonder where the level of understanding is at on how these metrics are actually calculated. I really don't understand why we can't utilize the significant efforts and data available in an attempt to put some type of a value on these things without having to defend and justify the calculation methods and perceived level of precision just about every time it comes up. Maybe every post that uses any type of estimated metric needs to be qualified with something like "these figures are calculated with some level of subjectiveness and the values have a margin of error"... ??

Love this post.

[Mike Sixel]

Metrics =/= 100% accurate, ergo, metrics are bad/evil/wrong......ugh. It's almost like people don't realize that's how all statistics that are predictive work, all of them. Including all the ones about medicine, and flight, and computers, and pretty much everything you use in your life. 

[TheLeviathan]

Metrics =/= 100% accurate, ergo, metrics are bad/evil/wrong......ugh. It's almost like people don't realize that's how all statistics that are predictive work, all of them. Including all the ones about medicine, and flight, and computers, and pretty much everything you use in your life. 

 

I believe a series of posters missed the point.  Never once did I suggest we get rid of framing stats or that they're evil or they need a caveat in every post or any of that.

 

The problem isn't the metric or the practice of developing metrics.  The problem is the people employing them.  The borderline zealotry with which they are idolized and the abhorrence of skepticism.  

That's the problem, not the metrics.  The metrics are fine and valuable when used appropriately and fairly and many people are able to do that.  But because there has been a rise in anti-metrics people (of which I'm not a part), the pro-metrics people have started taking a similarly polarized stance.  Personally, I prefer to keep my options open to both traditional measures and new measures and I never discount anyone's skepticism.  Certainly not when major components of a metric are not as objective as they are portrayed to be.  

 

Baseball is beautiful because, at times, it can produce statistics without variables that are totally objective and measurable.  I appreciate how many have tried to take the game when it is not so precise and find similar value.  But pitch framing, defense, and other statistics step outside of those strict measurables and start introducing concepts with uncontrolled variables and subjective analysis.  It is so fundamentally important that isn't forgotten when you're arguing the merit of a particular metric.  There is valid room for significant skepticism.  If you're someone that can argue your metric with that in mind, you're doing fine.  But my perception is that pro-metric zealotry is rising every year, it's been really noticeable since Mike Trout came into the league to put a time-table on it.

[jay]

The borderline zealotry with which they are idolized and the abhorrence of skepticism.

That's the problem, not the metrics.

 

I think we've all witnessed some of that across the interwebs, but I don't personally perceive that to be much of a frequent issue here at TD.

 

Even jimmer's posts here in this thread were pretty cautious and didn't proclaim exact precision. The quote of FG run values was only used to highlight an ordering of who is good and who is bad.

 

Do others perceive there to be significant advanced metric gospelization taking place here?

[USAFChief]

I think we've all witnessed some of that across the interwebs, but I don't personally perceive that to be much of a frequent issue here at TD.

Even jimmer's posts here in this thread were pretty cautious and didn't proclaim exact precision. The quote of FG run values was only used to highlight an ordering of who is good and who is bad.

Do others perceive there to be significant advanced metric gospelization taking place here?

No, I don't think so.

 

Nor is there a great deal of anti-metric vitriol. Just some honest reluctance to jump to firm conclusions based mostly on what some consider theory, not fact.

[JB_Iowa]

"Do others perceive there to be significant advanced metric gospelization taking place here?"

 

I try to stay away from the metrics threads. The endless bickering tends to give me a headache.

 

But yes, there is more worship at the sabremetric altar than in the early days of TD. That's okay but the fervency of the "discussion" (and often the failure to recognize that the opposing view is not "all or nothing") is off-putting.

 

I'm waiting for games to start so we can have some fun and talk about reality instead of some hypothetical statistical universe. And strong hint: game threads are usually not the place for metrics.

[TheLeviathan]

I think we've all witnessed some of that across the interwebs, but I don't personally perceive that to be much of a frequent issue here at TD.

 

No, I think we're generally ok here.  I hope it stays that way.

[USAFChief]

One minor quibble, JB...everyone is welcome in game threads.

 

Even those who want to talk about baseball! ;-)

[JB_Iowa]

One minor quibble, JB...everyone is welcome in game threads.

Even those who want to talk about baseball! ;-)

Everyone is welcome but if game threads start getting bogged down in metrics, I'm running away just like some of those who have already fled.

[Craig Arko]

The little countdown timer now says "1 day, 2 hours."

 

That's the metric that really matters.

[jay]

No, I think we're generally ok here. I hope it stays that way.

I do too, but that leaves me a bit confused on why this thread stirred up such a negative response to the concept of framing.

 

The ESPN article largely cites multiple years of data and makes references to those who rated well or rated poorly. Those are pretty generic terms that seem to have built in plenty of skepticism in an attempt to use some of the data to talk about who might be influenced by it.

 

It leads me to believe that part of the crowd here isn't open to the idea of using any of it for anything at all but is trying to pass that off as openness with some skepticism.

[jay]

That's okay but the fervency of the "discussion" (and often the failure to recognize that the opposing view is not "all or nothing") is off-putting.

 

I'd venture to state that happens here with a much broader brush than this topic. I see the all-or-nothing rebuttal on both sides of just about every debate here (and across the interwebs).

 

You're obviously welcome to enjoy baseball however you like. For me and a good number of others, that includes more stats and math in an attempt to increase our understanding of the beautiful game. For as much as you wouldn't want me to tell you how to enjoy the game, your strong hints don't lead me to believe you're open to letting others enjoy the game however they wish.

[stringer bell]

I'll weigh in on this.  I think that pitch framing happens, but it isn't as clear cut as the statistics might indicate.  Further, there are other skills in being a catcher--first and foremost is developing a rapport with the staff, so that they feel comfortable pitching.  Of course, if you hit like Mike Piazza, you can catch like Mike Piazza and be a Hall of Famer.

[bird]

I'm one of our number who appreciates the new statistics a lot, but who tends to refrain from discussing them per se. There are rare times when an avid aficionado of the new metrics uses them in blunt force in an argument to make a point, and in the process, condescends and/or ignores "unmeasurable" factors. It helps when both "sides" acknowledge as valid the existence of both measurable and unmeasurable factors.

 

Catcher performance, to me, is perhaps the best example of how stats and unmeasurable factors collide in a discussion about it. Pitch-framing is measurable. Game-calling and being a pitcher whisperer is not. Could it be that managers almost invariably first cite the latter qualities when discussing a catcher's prowess because they don't understand the importance of pitch-framing? Is it also possible that stat lovers first cite pitch-framing and proclaim its importance because it's measurable and information about pitcher whispering is anecdotal and second-hand?

[TheLeviathan]

I do too, but that leaves me a bit confused on why this thread stirred up such a negative response to the concept of framing.

 

It leads me to believe that part of the crowd here isn't open to the idea of using any of it for anything at all but is trying to pass that off as openness with some skepticism.

 

Absolutely not.  Jimmer's initial response was about framing stats being "facts".  As people talked about the subjectivity in the metric of pitch framing, ashbury and jimmer suggested people were questioning the existence of framing by questioning the metric used to measure it.

 

That's the slippery slope to what I have an issue with.  I don't question the value of pitch framing - I think some catchers are good at that and it's a nice skill to have.  I also don't question the idea that players are more valuable than others or have varying defensive abilities if I'm questioning a metric.  I am merely questioning how factual the metric is as opposed to one way to approximate it.  This happens frequently in these debates - I say something like "I'm not sure I really buy UZR here" and the response is "So you don't think defense is important?"  Or "I'm not so sure about framing stats" becomes "So you don't think framing is a helpful thing?"

 

Of course I understand that, but it feels like people are willing to fall on any sword to defend their metric as long as they don't have to admit "hey, the metric isn't perfect".

[JB_Iowa]

You're obviously welcome to enjoy baseball however you like. For me and a good number of others, that includes more stats and math in an attempt to increase our understanding of the beautiful game. For as much as you wouldn't want me to tell you how to enjoy the game, your strong hints don't lead me to believe you're open to letting others enjoy the game however they wish.

 

My post wasn't random.  It was in response to a question specifically posed above. In general I don't post much in the metrics discussions so you are certainly free to "include more stats and math in an attempt to increase our understanding of the beautiful game."  I'm not trying to interfere with that.  It would be nice if these threads didn't come down to the same arguments over and over.  I have often commented that I learn more from Parker's articles than anything else on this site -- he has the ability to use statistical analysis in a comprehensible, non-argumentative way.

 

My strong hint was for metrics discussions to be kept out of game threads.  In the past, game threads have generally been reserved for light hearted discussions of the current game.  The invasion of metrics into those discussions would certainly put a damper on that.  It's one thing if there are only a few still watching the game.  It's wholly different if the group is lively.

 

(This discussion should probably be in a different discussion.  Sorry for going off topic.)

[jimmer]

Absolutely not.  Jimmer's initial response was about framing stats being "facts".  

 

 

Actually, the part you specifically pointed out early in this thread that I was claiming as a fact was this part 'There's been enough analysis on this to know framing is a real thing and there are catchers better at it than others.'

 

That's a generic comment. That doesn't even talk about the 'framing stats' in any way, just that framing itself is real. Those who have really issues with the actual metrics in framing acknowledge that it's a real thing (which is my first point in that quote).  So if it's a real thing, does it not stand to reason that some are better at it  than others (which is the second part of that sentence).

 

So basically what I said was a 'fact' (by using the quote you said I was saying is fact) has two parts.

 

The first part is that framing is real. By reading this thread, whether or not it's real isn't up really for debate just whether or not it's measured properly.

 

The second part is that some are better at it than others. Is that the part we don't think is for sure true?  Since it seems everyone agrees in the basic idea that framing is a real thing, are there people who aren't convinced there aren't some better at it than others?

 

The problem lies with how well it's actually measured and I never said how it's measured is perfect by any means either in that quote that is being pointed out as me claiming as fact, nor in any subsequent post. And even the biggest SABR people aren't saying it's perfect.  I asked the guy who created ZIPs about this yesterday in his Fangraphs chat and he said the same thing.

 

I asked, 'How comfortable are you with saying framing stats are pretty much accurate? For example, strikes saved/given equals runs saved?'

 

He replied, 'I’m comfortable with their value, but there’s still enough that we don’t know that we shouldn’t go crazy about the magnitude.'

[jorgenswest]

I am confident that Santana won't have anynwhere near the same defense from his catcher and outfield that he enjoyed in Kansas City or Anaheim.

 

As for Suzuki, he had two season that were split between Washington and Oakland. It was a rare opportunity to compare his performance against several other catchers receiving the same pitchers. I didn't compare framimg stats, I compared the two things catchers might be able to assist with either the framing or whispering skills. Strike outs and walks.

 

It is mostly unreadable now with the change in blogging software and lack of tables

http://twinsdaily.com/blog/36/entry-4580-kurt-suzuki-signed-to-be-the-starter/

 

I wrote this in summary

 

"Over three years, the other 12 catchers have a 23% better strikeout walk ratio, a strikeout rate that is 9% better and a walk rate that is 11% lower.

 

Every year over the last three and on every team, pitchers who happen to be throwing to Suzuki strike out fewer batters and walk more batters. Small sample size? It is over 12000 batters. Maybe he has had teammates that are very good catchers. It was 12 different guys on two different teams."

 

Suzuki does one thing well. He blocks pitches. If he is good at managing a pitcher, the other 12 guys must have been very good or those skills didn't matter as much and his poor framing numbers drove the strikeouts down and walks up.

 

Santana performed at an ERA+ of 92 last year. He can't afford any decline in that result whether it is due to aging or defense or pitch framing or bad luck. That is number 4 starter territory and it isn't going to cause a great change in the Twins quest for a contending team.

[jimmer]

In the last five seasons, Santana ERA has been worse than his FIP just once, last year.  The others, his ERA was worse.  Last season seems to have been an outlier for him with having a lowish FIP  and a high ERA (which seemed even weirder having Heyward and Simmons behind him) and I don't think it's repeatable

 

IMO, and for his career, he's likely more of a very high 3/low 4 FIP guy and, with our defense, he's likely looking at an ERA in the low-mid 4s (see the FIP ERA differences for our starters last year).  A serviceable pitcher for sure, but likely still having an ERA some will look at at be upset about.

[jorgenswest]

In the last five seasons, Santana ERA has been worse than his FIP just once, last year. The others, his ERA was worse. Last season seems to have been an outlier for him with having a lowish FIP and a high ERA (which seemed even weirder having Heyward and Simmons behind him) and I don't think it's repeatable

 

IMO, and for his career, he's likely more of a very high 3/low 4 FIP guy and, with our defense, he's likely looking at an ERA in the low-mid 4s (see the FIP ERA differences for our starters last year). A serviceable pitcher for sure, but likely still having an ERA some will look at at be upset about.

I agree. It won't be reasonable to be upset with Santana who if performs as expected will have an ERA significantly above league average. Numbers like that still may help the team get below the 90 loss bar.

 

It might be reasonable to be upset with management committing a great deal of resources over 4 years to a pitcher whose 50th percentile projection starts his Twin career as a number 4 starter at age 32 and will likely decline in the seasons following.

[TheLeviathan]

Actually, the part you specifically pointed out early in this thread that I was claiming as a fact was this part 'There's been enough analysis on this to know framing is a real thing and there are catchers better at it than others.'

 

The problem lies with how well it's actually measured and I never said how it's measured is perfect by any means either in that quote that is being pointed out as me claiming as fact, nor in any subsequent post. And even the biggest SABR people aren't saying it's perfect.  

 

Then maybe I took your post more strongly then you meant it in this case, but it's a phenomenon that is quite frequent in these discussions in the larger statistical community.

 

I hear all the time that "the guys that invented it said..." and yet that does nothing to dissuade the adherents to misuse it.  As an example, I can't recall a single time WAR has been used correctly on this forum given it incorporates a statistic knowingly inaccurate but for larger 3-5 year sample sizes.  Yet if you say "Eh...not sure about WAR in this case" you get treated like you just claimed the earth was flat.  

 

TD has been pretty good about not getting preachy about advanced metrics, it just rubbed me the wrong way when skepticism was expressed and then slammed down.  I'm bothered about that in the larger debate and would hate to see that start happening here.

[jay]

As an example, I can't recall a single time WAR has been used correctly on this forum given it incorporates a statistic knowingly inaccurate but for larger 3-5 year sample sizes.

 

Maybe it comes down to your personal definition of "used correctly".

 

Defensive stats can be used in smaller samples as long as we recognize they're likely to be slightly less accurate as a whole. I wouldn't think that means they can't ever be used as a reference without being incorrect. I also don't think someone referencing them in shorter periods should have to make a qualifying statement every time, which whether you're explicitly saying it or not, is the only middle ground left in your statement.

[TheLeviathan]

Maybe it comes down to your personal definition of "used correctly".

 

Defensive stats can be used in smaller samples as long as we recognize they're likely to be slightly less accurate as a whole. I wouldn't think that means they can't ever be used as a reference without being incorrect. I also don't think someone referencing them in shorter periods should have to make a qualifying statement every time, which whether you're explicitly saying it or not, is the only middle ground left in your statement.

 

They are likely to be significantly less accurate in a one year interval.  We use WAR every year as a comparative tool to judge what players accomplished.  Even though in Fangraphs own description of UZR it recommends always taking three years worth of data.  Apparently when we imbed UZR in another stat that recommendation somehow vanishes?  

 

When did we finally see some recognition of that problem?  When Alex Gordon forced some recognition last year.  So no, I'm not sure it's my opinion of "used correctly", the problem is the metric isn't being employed based on the recommendations of the people that designed and computed it.  Hell, those same people are seemingly forgetting that in their own evaluations sometimes.  

 

So it's simple to me, if you want to employ the metric in a not-entirely proper manner without caveat that's fine.  Just make sure you have a healthy respect for skepticism, because it's well earned.