Evaluating Deron Johnson's Drafts

Yeah, completely agree with these points. 2012 in particular could turn out to be an absolute monster year.

The counter argument to that is that you'd better not miss on a top 5 pick. As of right now, Stewart looks like a total bust and the jury is still out on Buxton. 

Allow me to use a different angle--were these results so good that if Johnson were replaced we would expect future results to regress?  Making replacements in management is quite typical (expected) when a new top executive is hired. Shaking an organization up shows that past results weren't good enough and that's why a change was made in the first place!

Should anyone lose their jobs?  This team has been horrible for years.  Has it all just been bad luck? Ryan didn't deserve to lose his job, Johnson doesn't deserve to lose his job. Antony doesn't either?  I just keep reading how no one deserves to be fired (opinion is not always the OPs)

Can we all at least agree MOLITOR should be fired?  Can we all get behind that? :-)

I think Deron blew it by passing on Benintendi For Jay. This decision will probably haunt us for the rest of this decade and all of the next.

Yeah. Development plays a role.

How can a guy with Buxton's speed go through the our system for 3.5 years and never learn how to bunt? That is not on Deron Johnson. I think our biggest issue is the fascination with "athletes" and high school players.

They worked with Byron ALOT in spring on his bunting abilities. With his speed bunting should definitely be part of his game still needs work.

https://www.youtube.com/watch?v=WSHBF6ElHeA

I'd be happy if they worked on Buxton's HITTING, period.  The least of his batting problems is he can't bunt.  This guy has been a top 2 prospect for three years and strikes out roughly 40% of the time while looking clueless at the plate at a much higher %.

I get these points.  But about half of the teams that drafted below the Twins from 2008-11, actually did better than the Twins.  There was an interesting similar comment arguing that the teams that drafted higher in each draft (like the Nats early) are favorites.  So I looked at the top 5 or so individual WAR contributions from each of the early drafts vs. overall draft pick and looked like that:

 

5 (or so) Highest WARs in draft by drafted position:

2008: 5 (31.6), 135 (18.3), 117 (17.5), 16 (15.2), 96 (14.2)
2009: 25 (44.4), 246 (27.4), 82 (21.9), 63 (18.3), 1 (18.3), 59 (18.0)
2010: 13 (29.4), 3 (22.3), 1 (21.5), 70 (19.2), 23 (12), 272 (11.5)
2011: 172 (13.1), 14 (13.1), 18 (10.3), 11 (9.6), 1 (9.4), 6, (9.1),

 

Funny enough no number 1 pick has the highest draft.

 

As far as the second point goes, this really is part of the drafting strategy and should weigh in the evaluation.  The Twins chose to take preps who will take 5 years or so to develop vs. College players who are ready in 2-3.  It was a choice and it hurt them.  And it is fine to do that if you are willing to fill the gaps with free agents in between, which is not what the TR & Co Twins do

 

Fun thing to speculate:  Would the 2015 Twins that nearly missed a post-season berth be there if they chose Wacha over Buxton in 2012?

The real point of this post is that there are plenty of good to excellent players past the first few selections. There is talent out there--find it! The Twins "found" Morneau in the 3rd round. 

This is a hard thing to judge.......because I think development plays a role also, and as John pointed out, so does draft position. I do appreciate the hard work....and will have to give some thought on how to interpret it.

I also appreciate the effort that goes into such an ambitious attempt to analyze the relative skill of these organizations. As you might predict however, I find the analysis to be terribly flawed, almost to the point of complete uselessness. The primary reason has to do with the absurd connection that gets made between the success (as measured by WAR, which is its separate problem) of a given selection and an opinion that the skill of the selecting team is what is at play here. Think about it for a moment. Let's take a hypothetical 15th round selection in 2008 that generates 20 WAR in his 5 years of MLB when this study goes to press. Every team in baseball would select that player in the first round if he was available to them in a redraft. However, all 30 teams lacked the foresight to draft the player in the first round. They all lacked the skill to project success for that player 15 times!!! So, adding up a bunch of WAR numbers and declaring one team to be more skillful than the next is folly.

The second reason, which is an equally fatal flaw, is the failure to adjust for draft order. The studies are too numerous to even mention regarding how favorably skewed collective WAR numbers get due to the inclusion of those premium early picks. 

Here's a thought: let's take the 2008 draft as our example. Now, since this was a weaker draft than maybe the next, perhaps the players selected #5 through #15 in this year are regarded as a "weak crop", and the same group a year later is regarded as stellar. Because of this important nuanced reality, it makes sense to compare teams who are faced with similar talent options, right?

So, in Rd1 of 2008, I would compare the success of this group of selections, #9-#20: Aaron Crowe, Jason Castro, Justin Smoak, Jemile Weeks Brett Wallace, Aaron Hicks, Ethan Martin, Brett Lawrie, David Cooper, Ike Davis, and Andrew Cashner. This gives us a window into the selections of 11 teams in a single round. 

In Rd2, you'd compare Destin Hood, Robert Ross, Tyson Ross, Shane Peterson, Joseph Austin,Tyler Ladendorf, Joshua Lindblom, Thomas Adams, Kenneth Wilson, Robert Stovall, and Aaron Shafer.

And on, so maybe a team looks bad compared to the teams that picked Dee Gordon and Jason Kipnis 5 picks apart in the 4th, and Chris Herrmann gets compared to Tommy Milone in the 10th.

It would be a much more meaningful analysis, especially if one avoided 2010 and later at this juncture, where the WAR accumulation is close to meaningless right now.

I don't know how the Twins would stack up in this methodology. My guess is better. In the example above, I'd guess that the combo of Aaron Hicks and Tyler Ladendorf would compare favorably.

I feel like this argument is analogous to arguing that park factors don't matter because Mike Trout has a higher OPS than anyone on the Rockies. Just because there are good players scattered throughout the draft doesn't negate the fact that the vast majority of the value in the draft is skewed toward the top picks. Just looking at your comment here, 7 of the 20 best WAR players (35%) were top-10 picks, even though they make up only 3% of your draft pool. Draft position matters, and any draft analysis that doesn't take take it into account is not telling the full story.

Yes.

If not WAR, what measure would you propose?

In a perfect world, I'd ask the teams to rank the players according to the order in which they would select them today for their own team, staying blind to positional need. But, from a practical standpoint, we're kinda stuck with WAR. We just need to be more judicious about how much credence we give it.

Again, I wouldn't use anything in the way WAR was used in the analysis thrylos did, because it isn't measuring what it intends to measure in reality.

A college player drafted in 2008 is not even through their peak years. A high school player drafted then is just starting their prime. That is the problem with trying to figure out how good your draft guy is.

You could very easily improve this study by findng an index WAR value for each pick in the first round, or the whole draft for that matter, and then comparing the teams drafted results to that index.

You can also take into account time from draft pick by using a cumulative WAR to date index for each of the years.  

So, in 2008 the Twins drafted Aaron Hicks with the 14th pick.  The average MLB 14th pick 8 years out has an average WAR of X.  Hicks has a WAR of Y.  The ratio of X to Y would demonstrate how well the GM did to date.  One would be average.

You could very easily improve this study by findng an index WAR value for each pick in the first round, or the whole draft for that matter, and then comparing the teams drafted results to that index.

 

You can also take into account time from draft pick by using a cumulative WAR to date index for each of the years.  

 

So, in 2008 the Twins drafted Aaron Hicks with the 14th pick.  The average MLB 14th pick 8 years out has an average WAR of X.  Hicks has a WAR of Y.  The ratio of X to Y would demonstrate how well the GM did to date.  One would be average.

This was the thought I had to improve the analysis as well. Weighting for the draft strength of each year would help as well. Adjust the final result based on total WAR produced compared to the average draft in your sample period, for example. If the draft over/under-performed by 20% then it seems reasonable to adjust each draft slot's expected result by the same amount.