Article: Evaluating Deron Johnson's Drafts

[Thrylos]

As the Minnesota Twins are evaluating their front office, I would like to take the opportunity to evaluate the performance of certain parts of their front office for which there might be appropriate objective evaluation criteria. Deron Johnson, the Twins scouting director since 2008, has been primarily responsible for the Rule 4 amateur draft. There is a feeling that the Twins do not draft and develop players well. But drafting and developing are two different things. It's hard to evaluate drafting departments without comparing them to the rest of the league.

 

How has Johnson done against the league?-----------

Here is the data: I looked at the overall WAR for each team for each draft for all players selected by that team in the first 10 rounds of that draft. Calculated the average WAR for each draft and the better teams are indicated with green. The teams are listed alphabetically and the Twins are bolded. Averages in yellow indicate small sample size, because simply not enough players made it to the bigs in the last few seasons to make any conclusions, but the ones who made it, count to the total team WAR numbers.

 

 

Long story short:

 

During the time that Johnson has been in charge of the Twins draft the Twins did better than only six other teams in the league: The Red Sox, Yankees, Phillies and the Rangers and Dodgers who all are willing to open their wallets and buy players drafted and developed by others and, surprisingly, the Rays, who are more than willing to sell everyone high to restock their system with players that other teams drafted to develop. The Twins do neither, so that it a pretty big problem. By this criteria, Deron Johnson has been performing way below average, and D students should go home...

 

Few notes:

• This is pretty interesting data about where the Twins can look to stock their front office. Surprisingly, the Diamondbacks', Padres', Jays', White Sox', and Nationals' systems might be good sources of scouting talent to replace at least Deron and some of his scouts.
• If it were not for 2009 where Dozier and Gibson account for most of the Twins' WAR, the numbers would be really pathetic.
• If you look at teams like the White Sox that have had the fourth best draft WAR in the period and really nothing to show for it in the majors, you can make easy conclusions about problems with their development and management systems.
• Yes, there are up and downs in most teams, but with the Twins, other than 2009, it has been mostly downs, and the other teams have done something about it (replacing under-performing front office pieces.) Will the Twins do the same?
• Originally published at The Tenth Inning Stretch

 

Click here to view the article

[John Bonnes]

I'll start this off...

This is a hard thing to evaluate. I appreciate the methodology that was applied and the work involved. But I also would say there are at least a couple of significant factors that certainly don't work the Twins way.

1. The Twins drafted pretty low from 2008-2011 - and those are the players that are most likely to be producing WAR right now. I would expect them to be in the bottom third for those years, because they were in the bottom third of teams drafting those years.

2. The Twins top draft choices in the most recent years have been prep players as opposed to college players. The fact that Byron Buxton, Jose Berrios and Kohl Stewart are all 22 years old or younger has far more to do with weak WAR scores than their overall quality. This list might look a lot different in a few years as they gain WAR.

I published this because I appreciate the attempt to quantify this stuff, and maybe Thrylos or some other members of the community can take this data as a starting point for even more analysis. But I don't know that it suggest that Deron Johnson lose his job.

[Mike Sixel]

This is a hard thing to judge.......because I think development plays a role also, and as John pointed out, so does draft position. I do appreciate the hard work....and will have to give some thought on how to interpret it.

[nytwinsfan]

 

I'll start this off...

This is a hard thing to evaluate. I appreciate the methodology that was applied and the work involved. But I also would say there are at least a couple of significant factors that certainly don't work the Twins way.

1. The Twins drafted pretty low from 2008-2011 - and those are the players that are most likely to be producing WAR right now. I would expect them to be in the bottom third for those years, because they were in the bottom third of teams drafting those years.

2. The Twins top draft choices in the most recent years have been prep players as opposed to college players. The fact that Byron Buxton, Jose Berrios and Kohl Stewart are all 22 years old or younger has far more to do with weak WAR scores than their overall quality. This list might look a lot different in a few years as they gain WAR.

I published this because I appreciate the attempt to quantify this stuff, and maybe Thrylos or some other members of the community can take this data as a starting point for even more analysis. But I don't know that it suggest that Deron Johnson lose his job.

Yeah, completely agree with these points. 2012 in particular could turn out to be an absolute monster year.

[Mike Sixel]

 

I'll start this off...

This is a hard thing to evaluate. I appreciate the methodology that was applied and the work involved. But I also would say there are at least a couple of significant factors that certainly don't work the Twins way.

1. The Twins drafted pretty low from 2008-2011 - and those are the players that are most likely to be producing WAR right now. I would expect them to be in the bottom third for those years, because they were in the bottom third of teams drafting those years.

2. The Twins top draft choices in the most recent years have been prep players as opposed to college players. The fact that Byron Buxton, Jose Berrios and Kohl Stewart are all 22 years old or younger has far more to do with weak WAR scores than their overall quality. This list might look a lot different in a few years as they gain WAR.

I published this because I appreciate the attempt to quantify this stuff, and maybe Thrylos or some other members of the community can take this data as a starting point for even more analysis. But I don't know that it suggest that Deron Johnson lose his job.

 

John.....if being in the bottom third means you expect them to be bottom third in results, aren't you implying that the scouting department is no better than average, and that's ok? if your team doesn't spend big money, don't you need to be better than that to succeed over the long run?

 

I'm pretty sure St. Louis picked in the bottom third in this time frame......

[diehardtwinsfan]

You said it in your opening, drafting and developing are two different things.  The problem is that by using WAR of the guys that make it, you've combined the two.  Not sure how easy it is to separate things out, and the only real metric you have is that the industry generally likes are farm system, but for some strange reason, said players seem to be taking a while to get it or don't get it at all. 

[d-mac]

 

Yeah, completely agree with these points. 2012 in particular could turn out to be an absolute monster year.

 

The counter argument to that is that you'd better not miss on a top 5 pick. As of right now, Stewart looks like a total bust and the jury is still out on Buxton. 

[Kwak]

Allow me to use a different angle--were these results so good that if Johnson were replaced we would expect future results to regress?  Making replacements in management is quite typical (expected) when a new top executive is hired. Shaking an organization up shows that past results weren't good enough and that's why a change was made in the first place!

[tobi0040]

Yeah. Development plays a role.

 

How can a guy with Buxton's speed go through the our system for 3.5 years and never learn how to bunt? That is not on Deron Johnson. I think our biggest issue is the fascination with "athletes" and high school players.

 

We have had four picks in the top 6 from 2012 to 2015 and it doesn't look like we have any stars.

[jimmer]

Should anyone lose their jobs?  This team has been horrible for years.  Has it all just been bad luck? Ryan didn't deserve to lose his job, Johnson doesn't deserve to lose his job. Antony doesn't either?  I just keep reading how no one deserves to be fired (opinion is not always the OPs)

 

Can we all at least agree MOLITOR should be fired?  Can we all get behind that? :-)

Edited July 28, 2016 by jimmer

[gunnarthor]

My thoughts are that this should be divided a second time.  The 08-11 period under Johnson and Smith and then the 12-16 period where Ryan and Radcliff had more input.  And the draft pool thing came into effect in 2012, as well.  I'm not sure how much budget affected the 08-11 drafts but we know there were pretty strong rumors it affected previous drafts.

 

That said, Johnson certainly has some differences than Radcliff - we mainly see that with the type of pitchers he takes.  He prefers fireballers whereas Radcliff focused more on control (this is an imperfect comparison as Johnson did draft Wimmers and Radcliff did take Garza and Durbin).   But right now, the 08 draft (3 first rounders) was a big miss.   And 11 was pretty rough, too.  But 2012 could be an all-timer.

 

I'm not sold on Johnson but the last few drafts have been a lot more exciting than his first few.  Maybe he's learning, maybe there's more than we know.  In a recent chat, Klaw had good things to say about the Twins scouting department so maybe there's more to know.  I won't complain if the new GM replaces Johnson but I won't complain if he keeps him.  I think there's just too much to this that I don't know.

Edited July 28, 2016 by gunnarthor

[operation mindcrime]

I think Deron blew it by passing on Benintendi For Jay. This decision will probably haunt us for the rest of this decade and all of the next.

[Bob Sacamento]

 

Yeah. Development plays a role.

How can a guy with Buxton's speed go through the our system for 3.5 years and never learn how to bunt? That is not on Deron Johnson. I think our biggest issue is the fascination with "athletes" and high school players.

They worked with Byron ALOT in spring on his bunting abilities. With his speed bunting should definitely be part of his game still needs work.

 

https://www.youtube.com/watch?v=WSHBF6ElHeA

[jimmer]

I'd be happy if they worked on Buxton's HITTING, period.  The least of his batting problems is he can't bunt.  This guy has been a top 2 prospect for three years and strikes out roughly 40% of the time while looking clueless at the plate at a much higher %.

[gunnarthor]

 

I think Deron blew it by passing on Benintendi For Jay. This decision will probably haunt us for the rest of this decade and all of the next.

That seems pessimistic.

[Thrylos]

 

I'll start this off...

This is a hard thing to evaluate. I appreciate the methodology that was applied and the work involved. But I also would say there are at least a couple of significant factors that certainly don't work the Twins way.

1. The Twins drafted pretty low from 2008-2011 - and those are the players that are most likely to be producing WAR right now. I would expect them to be in the bottom third for those years, because they were in the bottom third of teams drafting those years.

2. The Twins top draft choices in the most recent years have been prep players as opposed to college players. The fact that Byron Buxton, Jose Berrios and Kohl Stewart are all 22 years old or younger has far more to do with weak WAR scores than their overall quality. This list might look a lot different in a few years as they gain WAR.

I published this because I appreciate the attempt to quantify this stuff, and maybe Thrylos or some other members of the community can take this data as a starting point for even more analysis. But I don't know that it suggest that Deron Johnson lose his job.

 

I get these points.  But about half of the teams that drafted below the Twins from 2008-11, actually did better than the Twins.  There was an interesting similar comment arguing that the teams that drafted higher in each draft (like the Nats early) are favorites.  So I looked at the top 5 or so individual WAR contributions from each of the early drafts vs. overall draft pick and looked like that:

 

5 (or so) Highest WARs in draft by drafted position:

2008: 5 (31.6), 135 (18.3), 117 (17.5), 16 (15.2), 96 (14.2)
2009: 25 (44.4), 246 (27.4), 82 (21.9), 63 (18.3), 1 (18.3), 59 (18.0)
2010: 13 (29.4), 3 (22.3), 1 (21.5), 70 (19.2), 23 (12), 272 (11.5)
2011: 172 (13.1), 14 (13.1), 18 (10.3), 11 (9.6), 1 (9.4), 6, (9.1),

 

Funny enough no number 1 pick has the highest draft.

 

As far as the second point goes, this really is part of the drafting strategy and should weigh in the evaluation.  The Twins chose to take preps who will take 5 years or so to develop vs. College players who are ready in 2-3.  It was a choice and it hurt them.  And it is fine to do that if you are willing to fill the gaps with free agents in between, which is not what the TR & Co Twins do

 

Fun thing to speculate:  Would the 2015 Twins that nearly missed a post-season berth be there if they chose Wacha over Buxton in 2012?

Edited July 29, 2016 by Thrylos

[Thrylos]

 

 

You said it in your opening, drafting and developing are two different things.  The problem is that by using WAR of the guys that make it, you've combined the two.  Not sure how easy it is to separate things out, and the only real metric you have is that the industry generally likes are farm system, but for some strange reason, said players seem to be taking a while to get it or don't get it at all. 

 

The thing here is that, unlike the Twins, lots of teams trade their prospects who get developed by other teams, which is the equalizing factor in this situation...

 

[Thrylos]

 

They worked with Byron ALOT in spring on his bunting abilities. With his speed bunting should definitely be part of his game still needs work.

 

https://www.youtube.com/watch?v=WSHBF6ElHeA

 

Not only bunting. 

 

I just looked at his July numbers.  He had a .335 OBP.  And had 11 hits, 8 of which were singles, 2 doubles plus 5 walks.  So 15 SB opportunities.   Guess what?  He tried to steal twice and was thrown out once.  Pretty ridiculous with that speed.  What would Rickey do? (and Buxton has Rickey speed and at least Rickey intellect.)  Something is wrong here big time.

 

[Kwak]

 

I get these points.  But about half of the teams that drafted below the Twins from 2008-11, actually did better than the Twins.  There was an interesting similar comment arguing that the teams that drafted higher in each draft (like the Nats early) are favorites.  So I looked at the top 5 or so individual WAR contributions from each of the early drafts vs. overall draft pick and looked like that:

 

5 (or so) Highest WARs in draft by drafted position:

2008: 5 (31.6), 135 (18.3), 117 (17.5), 16 (15.2), 96 (14.2)
2009: 25 (44.4), 246 (27.4), 82 (21.9), 63 (18.3), 1 (18.3), 59 (18.0)
2010: 13 (29.4), 3 (22.3), 1 (21.5), 70 (19.2), 23 (12), 272 (11.5)
2011: 172 (13.1), 14 (13.1), 18 (10.3), 11 (9.6), 1 (9.4), 6, (9.1),

 

Funny enough no number 1 pick has the highest draft.

 

As far as the second point goes, this really is part of the drafting strategy and should weigh in the evaluation.  The Twins chose to take preps who will take 5 years or so to develop vs. College players who are ready in 2-3.  It was a choice and it hurt them.  And it is fine to do that if you are willing to fill the gaps with free agents in between, which is not what the TR & Co Twins do

 

Fun thing to speculate:  Would the 2015 Twins that nearly missed a post-season berth be there if they chose Wacha over Buxton in 2012?

The real point of this post is that there are plenty of good to excellent players past the first few selections. There is talent out there--find it! The Twins "found" Morneau in the 3rd round. 

[Hosken Bombo Disco]

Not only bunting. 

 

I just looked at his July numbers.  He had a .335 OBP.  And had 11 hits, 8 of which were singles, 2 doubles plus 5 walks.  So 15 SB opportunities.   Guess what?  He tried to steal twice and was thrown out once.  Pretty ridiculous with that speed.  What would Rickey do? (and Buxton has Rickey speed and at least Rickey intellect.)  Something is wrong here big time.

in Rochester this season:

 

22 singles, 11 walks, 2 hit-by-pitch, unknown number of reached by error, four stolen base attempts. That doesn't seem like much base stealing work for a guy with Buxton's speed and inexperience.

[bird]

 

This is a hard thing to judge.......because I think development plays a role also, and as John pointed out, so does draft position. I do appreciate the hard work....and will have to give some thought on how to interpret it.

 

 

I also appreciate the effort that goes into such an ambitious attempt to analyze the relative skill of these organizations. As you might predict however, I find the analysis to be terribly flawed, almost to the point of complete uselessness. The primary reason has to do with the absurd connection that gets made between the success (as measured by WAR, which is its separate problem) of a given selection and an opinion that the skill of the selecting team is what is at play here. Think about it for a moment. Let's take a hypothetical 15th round selection in 2008 that generates 20 WAR in his 5 years of MLB when this study goes to press. Every team in baseball would select that player in the first round if he was available to them in a redraft. However, all 30 teams lacked the foresight to draft the player in the first round. They all lacked the skill to project success for that player 15 times!!! So, adding up a bunch of WAR numbers and declaring one team to be more skillful than the next is folly.

 

The second reason, which is an equally fatal flaw, is the failure to adjust for draft order. The studies are too numerous to even mention regarding how favorably skewed collective WAR numbers get due to the inclusion of those premium early picks. 

 

Here's a thought: let's take the 2008 draft as our example. Now, since this was a weaker draft than maybe the next, perhaps the players selected #5 through #15 in this year are regarded as a "weak crop", and the same group a year later is regarded as stellar. Because of this important nuanced reality, it makes sense to compare teams who are faced with similar talent options, right?

 

So, in Rd1 of 2008, I would compare the success of this group of selections, #9-#20: Aaron Crowe, Jason Castro, Justin Smoak, Jemile Weeks Brett Wallace, Aaron Hicks, Ethan Martin, Brett Lawrie, David Cooper, Ike Davis, and Andrew Cashner. This gives us a window into the selections of 11 teams in a single round. 

 

In Rd2, you'd compare Destin Hood, Robert Ross, Tyson Ross, Shane Peterson, Joseph Austin,Tyler Ladendorf, Joshua Lindblom, Thomas Adams, Kenneth Wilson, Robert Stovall, and Aaron Shafer.

 

And on, so maybe a team looks bad compared to the teams that picked Dee Gordon and Jason Kipnis 5 picks apart in the 4th, and Chris Herrmann gets compared to Tommy Milone in the 10th.

 

It would be a much more meaningful analysis, especially if one avoided 2010 and later at this juncture, where the WAR accumulation is close to meaningless right now.

 

I don't know how the Twins would stack up in this methodology. My guess is better. In the example above, I'd guess that the combo of Aaron Hicks and Tyler Ladendorf would compare favorably.

[markos]

 

I get these points.  But about half of the teams that drafted below the Twins from 2008-11, actually did better than the Twins.  There was an interesting similar comment arguing that the teams that drafted higher in each draft (like the Nats early) are favorites.  So I looked at the top 5 or so individual WAR contributions from each of the early drafts vs. overall draft pick and looked like that:

 

5 (or so) Highest WARs in draft by drafted position:

2008: 5 (31.6), 135 (18.3), 117 (17.5), 16 (15.2), 96 (14.2)
2009: 25 (44.4), 246 (27.4), 82 (21.9), 63 (18.3), 1 (18.3), 59 (18.0)
2010: 13 (29.4), 3 (22.3), 1 (21.5), 70 (19.2), 23 (12), 272 (11.5)
2011: 172 (13.1), 14 (13.1), 18 (10.3), 11 (9.6), 1 (9.4), 6, (9.1),

 

Funny enough no number 1 pick has the highest draft.

I feel like this argument is analogous to arguing that park factors don't matter because Mike Trout has a higher OPS than anyone on the Rockies. Just because there are good players scattered throughout the draft doesn't negate the fact that the vast majority of the value in the draft is skewed toward the top picks. Just looking at your comment here, 7 of the 20 best WAR players (35%) were top-10 picks, even though they make up only 3% of your draft pool. Draft position matters, and any draft analysis that doesn't take take it into account is not telling the full story.

 

 

[Hosken Bombo Disco]

So, adding up a bunch of WAR numbers and declaring one team to be more skillful than the next is folly.

If not WAR, what measure would you propose?

[bird]

 

I feel like this argument is analogous to arguing that park factors don't matter because Mike Trout has a higher OPS than anyone on the Rockies. Just because there are good players scattered throughout the draft doesn't negate the fact that the vast majority of the value in the draft is skewed toward the top picks. Just looking at your comment here, 7 of the 20 best WAR players (35%) were top-10 picks, even though they make up only 3% of your draft pool. Draft position matters, and any draft analysis that doesn't take take it into account is not telling the full story.

 

Yes.

[bird]

 

If not WAR, what measure would you propose?

 

In a perfect world, I'd ask the teams to rank the players according to the order in which they would select them today for their own team, staying blind to positional need. But, from a practical standpoint, we're kinda stuck with WAR. We just need to be more judicious about how much credence we give it.

 

Again, I wouldn't use anything in the way WAR was used in the analysis thrylos did, because it isn't measuring what it intends to measure in reality.

Edited July 29, 2016 by birdwatcher

[old nurse]

A college player drafted in 2008 is not even through their peak years. A high school player drafted then is just starting their prime. That is the problem with trying to figure out how good your draft guy is.

[mlhouse]

You could very easily improve this study by findng an index WAR value for each pick in the first round, or the whole draft for that matter, and then comparing the teams drafted results to that index.

 

You can also take into account time from draft pick by using a cumulative WAR to date index for each of the years.  

 

So, in 2008 the Twins drafted Aaron Hicks with the 14th pick.  The average MLB 14th pick 8 years out has an average WAR of X.  Hicks has a WAR of Y.  The ratio of X to Y would demonstrate how well the GM did to date.  One would be average.

 

[Taildragger8791]

 

You could very easily improve this study by findng an index WAR value for each pick in the first round, or the whole draft for that matter, and then comparing the teams drafted results to that index.

 

You can also take into account time from draft pick by using a cumulative WAR to date index for each of the years.  

 

So, in 2008 the Twins drafted Aaron Hicks with the 14th pick.  The average MLB 14th pick 8 years out has an average WAR of X.  Hicks has a WAR of Y.  The ratio of X to Y would demonstrate how well the GM did to date.  One would be average.

 

This was the thought I had to improve the analysis as well. Weighting for the draft strength of each year would help as well. Adjust the final result based on total WAR produced compared to the average draft in your sample period, for example. If the draft over/under-performed by 20% then it seems reasonable to adjust each draft slot's expected result by the same amount.

[nicksaviking]

How about instead of comparing the teams, a comparison of the draft position is made? So if the Twins drafted 10th and selected a HS kid, that pick is compared in WAR to the next 10-20 HS kids taken and the same is done separately for the college guys.

 

Then WAR is added for each team and divided by the number of draft picks, then we'd have WAR per player and only for guys the Twins had a shot at drafting.

[Mike Sixel]

Why compare to other HS kids......who would you rather have right now, Trea Turner or Nick Gordon?