Article: Monitoring The Bullpen

[Seth Stohs]

2015 was a positive season for the Minnesota Twins. Yet, when the season ended, it was clear that there were needs in the organization that needed to be addressed. Foremost among them was the bullpen.

 

In Parker’s interview for the Twins Daily Offseason Handbook, he asked Terry Ryan, “Where would you prioritize the bullpen in general this offseason?”

 

Ryan responded with, “High.”

 

Parker probed, “Highest priority?”

 

Ryan said, “It’s pretty close up there. Pitching is always the most prioritized area of any team.”

 

That interview was in October. The offseason came and went and the Twins signed exactly zero pitchers to major league contracts, starters or relievers.Granted, the Twins signed MLB veteran lefty reliever Fernando Abad to a minor league contract. There was never really any question that he would make the Opening Day roster.

 

Abad was one of several pitchers signed to minor league deals. Besides Abad, the only pitcher who really had a chance at cracking the Opening Day roster was right-hander Brandon Kintzler, another veteran with several years of big league service time. Dan Runzler and Buddy Boshers were nice stories for spring training, and maybe one of them will perform at AAA and eventually get a promotion back to the big leagues.

 

There were not many bullpen jobs available. Glen Perkins is signed for another year. Kevin Jepsen, after what he did for the Twins after last year’s trade deadline, was coming back, and that’s a good thing. We can debate bringing back Casey Fien, but once he was re-signed, he was a given. Trevor May got some opportunity this spring to start, but we all knew that he would go to the bullpen. Michael Tonkin was out of options. Ricky Nolasco was a possibility for a bullpen spot depending upon spring training performance. Again, there were only one or two jobs to be competed for.

 

In my opinion, I was always comfortable with Twins not wanting to go beyond one year with any relief pitcher for a few reasons. First and foremost, there are very few relievers who are good for multiple years, especially free agent pitchers who are already 31 years old, or older. Most likely, during a two-year contract, the pitcher would be good one of the years. In a three-year contract, you could hope beyond hope that you get two decent years. Of course, we can follow the next three years of Tony Sipp and Antonio Bastardo to see how they pan out.

 

Secondly, the Twins are loaded with relief pitching prospects, guys that we hope are ready sometime in 2016. Guys that we don’t want to see blocked by mediocre veterans.

 

In the same Offseason Handbook Terry Ryan interview, Ryan was asked about the power arms in the system and if they could surface in 2016. He said, “Yeah, we do. I do. I would expect some of those guys to surface this year. I was hoping maybe last year but it was maybe one year premature. Some of those guys had a few struggles, and that’s not a bad thing. Alright, now you know what you’ve gone through. Now you can take a step back. A few of them are out in that Arizona Fall League which is good. I would think that some of those guys are going to surface this year which would be well received here. We can use some of that influx of people. I would like to see some of those relief pitchers there.”

 

In 2016, the following pitchers could come up and contribute to the Minnesota Twins. (Note-That is not saying all of them will, just that they are at a point in their career and development that it is possible)

 

40-Man Roster

Alex Meyer

JR Graham

JT Chargois

Taylor Rogers

Ryan O’Rourke

Mason Melotakis

Pat Dean

 

Non-Roster

Nick Burdi

Jake Reed

Logan Darnell

Brandon Peterson

Alex Wimmers

 

Now that’s 12 names. Most likely no more than two to four will actually come up in 2016, but by the end of 2017 several more of them will and potentially other names like Luke Bard, Trevor Hildenberger and Yorman Landa will be ready.

 

All of that is well and good, but for a team that expected to compete in 2016, performance matters. And through one week - a very small percentage of the season - the bullpen has been one of the biggest issues contributing to the Twins 0-7 loss. It’s not the only contributing factor. The complete lack of offense and run scoring has pushed the bullpen issues into the spotlight a lot this first week.

 

They have had three one-run losses and two-two run losses. In a couple of the games, the Twins lost leads in the late innings.

 

I’ve always wanted to establish a statistic of sorts to help measure the effectiveness of a relief pitcher. I think it’s fair to say that ERA and even WHIP are not the best statistics to measure the reliability of a reliever. Because relief pitchers generally pitch one, and maybe two, innings once or twice a week, one or two really bad outings can affect how the pitcher’s numbers look for much of the season.

 

To me, I want to know how often a reliever came into a game, into a situation and got the job done. All pitchers are going to have a few clunkers, so I’m going to try something new this year. I’m going to look at each and every appearance by relief pitchers throughout this season and determine whether or not the pitcher did what he was brought in to do.

 

Someone else can name this stat, if it’s worthwhile. Frankly, the reality is that this is subjective. Pitching well or getting the job done can mean different things to different people. For instance, if Trevor May comes in to a situation where there are runners on 1st and 3rd and nobody out and gets out of that inning with just one run scoring, I think he got the job done. If Michael Tonkin comes in with runners on and the Twins already down 8-0 in the 2nd inning, but he leaves the game with the Twins down 11-0 after the fifth or sixth inning, I think he did his job. If Glen Perkins comes in to a game with a 2-run lead and give up just one hit but no runs in the inning, he got the job done. If he comes in to a game with a three-run lead, gives up two runs on three hits and two walks but gets the save, I can’t say that he did his job.

 

So using my opinion, along with box scores and often watching on TV, here are how the members of the 2016 Twins bullpen grade out by this method through the way-too-small-of-a-sample-size of seven games. This stat may be more valuable in six to eight weeks, but here is the introduction:

 

Pitcher Y N Success

Glen Perkins 0 2 0.0%

Kevin Jepsen 1 2 33.3%

Trevor May 0 3 0.0%

Casey Fien 1 2 33.3%

Ryan Pressly 3 1 75.0%

Michael Tonkin 0 1 0.0%

Fernando Abad 3 0 100.0%

 

Y=Yes, they got the job done.

N=No, they didn't

Success = percentage

 

Aside from Fernando Abad and Ryan Pressly, it’s been a tough go for the Twins bullpen so far this season.

 

As I would say to everyone after a poor seven-game start to the season, it is a long season, and things will (most likely) normalize over the course of the next couple of months.

 

I don’t know what is good or bad with these percentages. Is 85% good, or is 70% good? I think we can agree from the start that 50% and lower would not qualify as good.

 

The bullpen was said to be a focus in the offseason. Little significant was done to address it in the offseason which has made it a large focus in the team’s slow start. It is certainly something that warrants monitoring throughout the season.

 

Click here to view the article

[h2oface]

J.T. Chargois? I would think he is in the mix sooner than later.

[sane16]

 

J.T. Chargois? I would think he is in the mix sooner than later.

Chargois is returning from Tommy John Surgery performed in September of 2013.

 

Only in baseball!.......

Instead of naming a surgical procedure after the Doctor who first designed and performed it, they named it after someone who slept through it.

[theJemmer]

Watched JT in Florida. He can really bring it but, on the day I watched he was having trouble getting on top of his breaking pitch which led to giving up tie run. He was very despondent after the outing. I remember thinking that his arm slot was lower than I expected. I can see why he is one to watch.

[Seth Stohs]

I'll add JT. Amazing that he's the name I left out since I think he'll be the first one called up.

[Vanimal46]

I actually like this exercise a lot, Seth. Instead of deep diving into numbers and leverage points and holy batman! useless stats, it can be boiled down to did x reliever do their job or not.. Sucks that the consensus top 3 relievers in the fan's minds (May, Perk, Jepsen) have not got the job done yet.  

[Blackjack]

In order for those young arms to be called up -- the ones on the current roster need to get injured - or fail.  So if 3-4 of them get called up this years it means the Twins aren't going to do very well.....

[Jeremy Nygaard]

I like what you're doing here Seth.  I like an exercise so simple... but I'd also like to see the games broke down - in your opinion - into high-, mid-, and low-leverage situations... maybe something to think about anyway. Especially if you can see a trend of how pitchers are being used and if it changes.

[HitInAPinch]

I'm still hopeful that the plan was to keep a couple 'place holders' until some of those bullpen prospects pan out.

 

Seth, check this out: 

 

http://www.fangraphs.com/library/pitching/sd-md/

 

 

[Seth Stohs]

 

I'm still hopeful that the plan was to keep a couple 'place holders' until some of those bullpen prospects pan out.

 

Seth, check this out: 

 

http://www.fangraphs.com/library/pitching/sd-md/

 

WPA can be interesting. For me, it's about what the biggest plays were. In many cases, these Shutdown or Meltdowns might be the same. I think iit speaks to Jeremy's leverage comment/idea. I think it'll be fun to compare this stat to the fangraphs stat and see how they look side-by-side.

[Taildragger8791]

Fangraphs' shutdown/meltdown statistic (based on win probability added) linked to by HitInAPinch seems to be a reasonable metric to track relievers. Even with the SSS that is the 2016 season, it so far seems to agree with the eyeball test as well as Seth's first draft at the "unnamed stat" in this article. I imagine there will be a lot of movement over the next few weeks.

 

http://i.imgur.com/zVZICTa.jpg

 

Link to image:

http://i.imgur.com/zVZICTa.jpg

 

 

[hybridbear]

 

WPA can be interesting. For me, it's about what the biggest plays were. In many cases, these Shutdown or Meltdowns might be the same. I think iit speaks to Jeremy's leverage comment/idea. I think it'll be fun to compare this stat to the fangraphs stat and see how they look side-by-side.

Can you get this SD/MD data for the bullpen for last season? I'd be curious to see what it shows for a full season of the poor bullpen performances that we witnessed. I don't really know how to use Fangraphs to search for data... Thanks!

[TheLeviathan]

I'd have felt better about the "do nothing in FA because we have young guys we like" thinking if any of the young guys had made the team.  

 

You don't have to just sign FAs to 3 year deals - you can always trade too.  

[DJL44]

If the Twins had signed a relief pitcher to a 3 year deal that would give them all of one relief pitcher signed to a 3 year deal. They need 7 relief pitchers.

[Minny505]

Here I thought the 'two-two' was for stage plays and the guy on your slow pitch softball team who strikes out.  Appropriate that it is now making its way into Twins team culture.

[Brandon]

you can also look at the hold statistic.  Its like a save for middle relievers.  but that only counts in games where the Twins have the lead when the reliever is pitching. 

[d-mac]

 

Fangraphs' shutdown/meltdown statistic (based on win probability added) linked to by HitInAPinch seems to be a reasonable metric to track relievers. Even with the SSS that is the 2016 season, it so far seems to agree with the eyeball test as well as Seth's first draft at the "unnamed stat" in this article. I imagine there will be a lot of movement over the next few weeks.

 

http://i.imgur.com/zVZICTa.jpg

 

Link to image:

http://i.imgur.com/zVZICTa.jpg

 

Interesting that there are exactly 4 relievers with more than one meltdown so far this year. The Twins have two of them.

[Pardon My Dinger]

 

you can also look at the hold statistic.  Its like a save for middle relievers.  but that only counts in games where the Twins have the lead when the reliever is pitching. 

Well, la tee da...

[Taildragger8791]

 

Can you get this SD/MD data for the bullpen for last season? I'd be curious to see what it shows for a full season of the poor bullpen performances that we witnessed. I don't really know how to use Fangraphs to search for data... Thanks!

 

No problem. Last year's data seems to mostly agree with expectations as well, although there were a few curious results mixed in.

 

This was the first time I'd used Fangraphs as well btw. I suggest you give it a try, it wasn't too tricky and there's a lot of cool information to sort through.

 

http://i.imgur.com/64A2eKE.jpg

 

Link:

http://i.imgur.com/64A2eKE.jpg

 

[Otto von Ballpark]

 

Can you get this SD/MD data for the bullpen for last season? I'd be curious to see what it shows for a full season of the poor bullpen performances that we witnessed. I don't really know how to use Fangraphs to search for data... Thanks!

And here's a direct link to the SD/MD chart image posted above, if you want to sort it, change teams/years, etc:

 

http://www.fangraphs.com/leaders.aspx?pos=all&stats=rel&lg=all&qual=0&type=3&season=2015&month=0&season1=2015&ind=0&team=8&rost=0&age=0&filter=&players=0&sort=14,d

[alskntwnsfn]

I have been a very vocal critic of this team's bullpen going back to the 2014/2015 offseason. I thought the bullpen would be one of the worst in the league last year, but to my surprise they were actually about average (in terms of Win Probability Added, and ranked between 11-25 in several other relevant statistics): 

 

http://www.fangraphs.com/leaders.aspx?pos=all&stats=rel&lg=all&qual=0&type=3&season=2015&month=0&season1=2015&ind=0&team=0,ts&rost=0&age=0&filter=&players=0&sort=2,d 

 

Further, we had 4 relievers in the top 120 WPA leaders in baseball last year. Even in terms of fastball velocity, we ranked 16/30. A thorough average bullpen, which given the expectations of many fans could be considered as huge success.

 

This year's squad is essentially the same group as last year. Given the results of that squad from last year, the relative lack of relief depth in the FA pool, and our crop of young relievers... I can better understand TR's inactivity. 

 

Whether we started 0-7 or 7-0, I do wish that we would have brought in a few more relievers on NRI or minor league contract flyers, but this is basically the group the front office expected to have in the pen to start 2016 all along. A strong spring from an unknown (whether they came from inside/outside the system) probably wasn't going to change that.  

[alskntwnsfn]

I was curious about the importance of bullpen performance. So I looked at playoff teams from the last five years and averaged their WPA and WPA/LI performance... basically how well did they do situationally and how good were they irregardless of the situation. A mix of results and "true ability". 

 

The average WPA rank for playoff teams was 11.8 and the average WPA/LI was 11.7. If the best bullpens always made the playoffs, the average would be 4.0 and if bullpen performance (in terms of ranking) made no difference, then the average would have been 15.0. 

 

Basically, playoff teams tend to have better than average bullpens. However, there's certainly many clubs who made the playoffs in spite of their bullpens. Further, World Series teams had a lower average ranking than all playoff teams, so at least for this 5-year sample, it suggests that overall bullpen quality/performance during the regular season wasn't a determining factor once teams were in the playoffs. 

[Darius]

Meh.

 

The bullpen arms in the minors do nothing for this team. Terry Ryan won't call one up until the season will be longneck over. Also, the most promising one, IMO, is injured (Burdi).

 

May should be starter. To me, that's a no brainier. He may not be a great one, but he has some of the best stuff in the organization. Also, I think he's showing he can't handle the bullpen mentally. He unravels mentally in tough situations. Not the guy I want trying to slam the door on someone else's base runners in the 7th/8th inning

 

I don't know how anyone can think going into a season with guys like Fien, Abad, etc. in a major league bullpen is a good idea.

 

Kevin Jepsen isn't a major league set-up guy, and Glen Perkins is no longer a major league closer. Not good ones, anyway.

 

This is a bad bullpen top to bottom. Ryan Pressley is the only guy who suits his role at this point. Typical Twins way to sit and wait for thing to "normalize." I hate to break it ya, but this is "normalized."

[Brock Beauchamp]

 

May should be starter. To me, that's a no brainier. He may not be a great one, but he has some of the best stuff in the organization. Also, I think he's showing he can't handle the bullpen mentally. He unravels mentally in tough situations. Not the guy I want trying to slam the door on someone else's base runners in the 7th/8th inning.

As evidenced by May holding opponents to a .581 OPS in close and late situations last season?

[Taildragger8791]

I know it was a tough offseason to find BP help, but I have to believe there were better options than this. We actually went into a season with a bullpen strategy of waiting for guys to play themselves out of a job, then replacing as we go with unproven prospects. We set up a minefield for the first chunk of the season, and if the rookies don't pan out then it is going to be a minefield all year. There's no sure anchor to this staff right now to hide those performances.

 

I have high hopes for some of our minor league RPs, but my confidence has been shaken based on injuries and declining performances at AA. I wouldn't want to bank my season on having multiple rookies provide average or better performance when those same guys are barely holding their own at AA.

Maybe somebody can help me out here. Who was the last rookie reliever the Twins called up that became a strong bullpen stalwart? I just looked back 4-5 years and no names really jumped out. And we're to assume we'll now get 2-3 in one year? It's such a big jump, and I hope it happens, but I'm not terribly comfortable with that strategy.

[Dr. Evil]

Free agent arms this offseason were arguably sparse. However, trade partners are always available. Ryan could have worked something. He didn't. Holy Krap. Make a friggin move or two. Show some desire to get better. Take a risk. This $hit is just boring.

[DocBauer]

I like your theory in general terms, Seth, but as you said, it will take time to work out the measureable and equate what is a good numer, what is bad. In a sense, where is the break point in all of this. And there many be more than one break point.

 

For instance, while there is more to a player's measured worth than simply their BA, the fact is .250 is a break point. .250 or higher is good, universally speaking, and above is better with .300 being elite. Conversely, below .250 is obviously not so good, while .200 has always been the "inept" mark, for lack of a better term.

 

Considering the somewhat volatile nature of the life of a RP, and in your examples given, even allowing runs doesn't necessarily mean a reliever failed, I almost wonder if a 50% "success" rate might not end up being the average?

[Seth Stohs]

 

As evidenced by May holding opponents to a .581 OPS in close and late situations last season?

... or by his terrific job there last year... 

[Mike Sixel]

Ryan said it was a high priority to fix, and he added Abad........so, either it wasn't a high priority to fix, or he did not fix it, unless you think adding Abad fixed it. 

[Brock Beauchamp]

 

... or by his terrific job there last year... 

Yeah, that was pretty much my point. I didn't use leverage situations because May spent so many innings as a starter but in case people are interested...

 

High leverage: .713

Med leverage: .761

Low leverage: .763

 

It seems to me May has done just fine with the game on the line.