Article: Gambling On The Relief Market: Simply Folly

[jorgenswest]

The Minnesota Twins have said that improving the bullpen is a very high priority this offseason. However, it is difficult to know which relievers were will be successful, will be worth their contract. Today, Jorgenwest provides some details about the success rates of multi-year contracts to relievers.Signing relievers to multi-year free agent deals is a risky proposition.

 

Dave Cameron had these numbers in a study from 2010.

As a group, teams have paid for premium production and instead received the same level of performance that they could have expected if they had signed minor league free agents. The evidence couldn’t be any stronger: signing guys like Guerrier and Crain to three year deals is just throwing money away. It’s not that they’re bad pitchers; it’s that relief pitchers are so prone to huge swings in performance that trying to project the long term future of any of these guys is simply folly.

I wondered if anything had changed since the study and the new free agency rules. There were 11 non-closer relievers signed prior to the 2013 season to multi-year deals.

 

Jeremy Affeldt (2013-2015), 3 years, $18 million: $-6.6 million(Fangraphs)

Mike Adams (2013-2014), 2 years, $12 million: $0.1 million(Fangraphs)

Jason Grilli (2013-2014), 2 years, $6.75 million: $14.3 million(Fangraphs)

Joel Peralta (2013-2014), 2 years, $6 million: $8.8 million(Fangraphs)

Sean Burnett (2013-2014), 2 years, $8 million: $-0.5 million(Fangraphs)

Randy Choate (2013-2015), 3 years, $7.5 million: $6.6 million(Fangraphs)

Tom Gorzelanny (2013-2014), 2 years, $5.7 million: $7.3 million(Fangraphs)

Jonathan Broxton (2013-2015), 3 years, $21 million: $4.9 million(Fangraphs)

Kyuji Fujikawa (2013-2014), 2 years, $9.2 million: $1.1 million(Fangraphs)

Brandon League (2013-2014), 3 years, $22.5 million: $-3.3 million(Fangraphs)

Joakim Soria (2013-2014), 2 years, $8 million: $12.3 million(Fangraphs)

 

Teams spent $124.65 million for 26 years of relievers. Their return on the investment was $45 million. Peralta and Gorzelanny were solid modest investments. Grilli had an all-star season as a closer. Soria pitched well when healthy, averaging 34 innings a season.

 

As a group these relievers provided little value. They averaged 38 innings per year of mediocre relief. The same kind of relief the Twins have been able to find among the likes of Burton and Boyer. Do we have confidence the Twins can find the reliever that will by among the minority in this year's group and provide good value the next two or three season? I am not. I am not confident any team has the skill to find that value.

 

The Twins need relievers. That is without question. The question is whether signing 30-something relievers in the free agent market is the answer. I think the Twins' dollars are better spent elsewhere. Go with the young players. Continue to sign guys like Boyer, Burton and Stauffer who will take a one-year deal. The Twins success rate with those guys is no worse than the success rate in Cameron's study or the 2013 group. It is also much easier to cut ties with a guy on a one-year deal than a guy just signed to three years and $15 million. If it doesn't work out, solid relievers can be found every July.

 

Click here to view the article

[HitInAPinch]

Relief Pitchers are a tough call.  More of a gamble, really.  One really good year and they cash in.  I'd be interested in Darren O'Day.  33 yrs. old, yes.  But after 4 amazing years.  The Baltimore Sun speculates O'Day could get a contract like Luke Gregerson got:  three years and $18.5 million.  As a side-armer, could he perform at a high level for those 3 years?

 

More than anything, I'd like to see a bunch of those hard-throwing MiLBers breakthrough and take the job.

[Seth Stohs]

There's really no way to know how a reliever will perform when signed. Tony Sipp could get 3 years and $21 million (or so), and he was pretty non-good for 5 years in the big leagues before becoming terrific last year... Which is real? There are a million examples of this over time. 

[nicksaviking]

Affeldt, Adams, Burnett, Broxton and League all performed poorly.  Fujikawa was an unknown international signing who was a real crap shoot going in.  It looks like from this list it's about 50/50 that it works out (45/65 if Fujikawa is weighted the same).

 

The one thing that seems to be common though, is that the busts have either been injured or were not good strikeout pitchers.  Meanwhile, Grilli, Peralta, Choate, Soria and Gorzlanny all seem to be able to manage at least a 8.0K/9.

[ThejacKmp]

The Twins have such an overabundance of back of the rotation starters, it's time to start converting some of those guys to relievers. Logan Darnell, Taylor Rogers, Pat Dean - these guys are unlikely to get a chance starting in the bigs so it's time to see what they can do in shorter stints. If the Twins can't unload Ricky Nolasco, it's time to see what he can do from the pen too. I'd also like to see the Twins go after the veteran guys who don't profile as starters anymore - I hate to use the P word but the Pelfreys. These guys are unlikely to start in the majors again but a guy like Pelfrey could be a nice middle reliever out of the pen.

The nice thing for the Twins is that they have two of the three 8th and 9th inning guys you need* in Jepsen and Perkins so they don't need dominance, just guys who can soak up those innings between starters and the elite pen arms.

 

* Note that I'm being realistic - as much as we'd like to see Perkins used in the 6th with the bases loaded, it's not going to happen. So we have to think like Molly with 8th and 9th inning guys. Sadly.

 

[DocBauer]

There's really no way to know how a reliever will perform when signed. Tony Sipp could get 3 years and $21 million (or so), and he was pretty non-good for 5 years in the big leagues before becoming terrific last year... Which is real? There are a million examples of this over time.

 

Yeppers. FA RP probably a bigger crapshoot than any other position. But the Twins DO have a need. Especially, IMHO, one more big late inning arm to join Jepsen and Perkins. Trading for one is probably safe, probably more cost effective, but it will cost players. You keep players/prospects in the FA route, but you almost certainly spend more and increase your risk. The key, obviously, is being smart enough to make the right choice/investment.

 

Sipp is a great example. I think I'd absolutely target success over time, or, that successful over time player coming off a down year, not the other way around.

[Vanimal46]

Is it better to acquire a more proven bullpen arm through trade? Or is it a necessary evil to gamble on the FA arms and hope for the best?

[ken]

The Twins have spent draft picks on college relief pitcher and then try to convert them to starter.  Now they have a surplus of back of the rotation starter and lack relief pitching.  It's time to start using those arms in the bullpen.  Veteran relief pitchers end up being just as much as a gamble as the young players.  So go young with your bullpen, use the money to fill other needs - maybe stop signing mid-level starter and go big on a front of the rotation starter.

[nicksaviking]

 

The Twins have such an overabundance of back of the rotation starters, it's time to start converting some of those guys to relievers. Logan Darnell, Taylor Rogers, Pat Dean - these guys are unlikely to get a chance starting in the bigs so it's time to see what they can do in shorter stints.

 

None of those guys can strike anyone out though, and I think that's pretty imperative for a guy in the pen.

 

At least it's been the biggest weakness of the recent relievers this team has assembled.

[Kwak]

Or, they could just promote from within (Meyer) and put their reputations on the line.  GMs will gladly spend somebody else's money on an insurance policy (veteran from another team) than promote the prospect they drafted or acquired in a trade.

[Mike Sixel]

 

None of those guys can strike anyone out though, and I think that's pretty imperative for a guy in the pen.

 

At least it's been the biggest weakness of the recent relievers this team has assembled.

 

Rogers could dominate lefties, correct?

Meyer? He should be up and in the bullpen.

That's 2 right there.

 

Now try converting some of these number 8 starters to the bullpen, and see what they can do. Or, just keep them in AAA, and never really use them for anything......How'd Liam Hendriks look last year?

[Shane Wahl]

 

None of those guys can strike anyone out though, and I think that's pretty imperative for a guy in the pen.

 

At least it's been the biggest weakness of the recent relievers this team has assembled.

 

Darnell and Rogers have struck out enough guys to be successful, potentially. Rogers would likely see an uptick in velocity, and if used more simply against lefties, he could be a high strikeout guy.

[Shane Wahl]

The Liam Hendriks situation is pretty illustrative. Stop throwing risky money around and start cycling through marginal starters.

[Seth Stohs]

 

* Note that I'm being realistic - as much as we'd like to see Perkins used in the 6th with the bases loaded, it's not going to happen. So we have to think like Molly with 8th and 9th inning guys. Sadly.

 

 

To be fair, that's Molly and every other manager in baseball... 

[Seth Stohs]

 

Alex Meyer, 

Taylor Rogers,

Logan Darnell,

JT Chargois,

Jake Reed,

Nick Burdi...

 

All guys who could come up this year. Chargois and Burdi can tough triple digits while Meyer and Reed are generally 95-97... 

 

Also, Tonkin's out of options, so they need to make a call there or open up a spot and let him sink or swim

 

And assuming May isn't given a starter's job, he'll be sitting 94-97 out of the bullpen.

 

They're close.

[diehardtwinsfan]

I have no problems with some minor league contracts on some guys that could be good, but yeah, some of hte marginal starters should be cycled into relief help.

[dbminn]

Twins need a power LH RP. There isn't much in the MiLB cupboard, even when you include back of the rotation types. IMO, a trade would be best.

[Thrylos]

This is a flawed argument, by Cameron and Co, in several levels:

 

A. They are using fangraphs "value", which is fWAR-based, and if WAR is bad in one thing, is in evaluating reliever performance.

B. They are throwing a whole bunch of players in a bucket and they are looking into an average imaginary player.  That player does not exist.

C. They are ignoring stellar performances

D. They are ignoring the factors that lead to bad performances.  It definitely was not the contracts they signed...

E. They do not normalize for anything, including age, defense, team record etc, for example.

F. They use a unicorn sample of middle relievers who signed with other teams, instead of all relievers out there (re-signed, arb-eligible, non-eligible; closers, mop up guys etc.  )

 

 

Yuck.

 

Edited November 13, 2015 by Thrylos

[dbminn]

 

* Note that I'm being realistic - as much as we'd like to see Perkins used in the 6th with the bases loaded, it's not going to happen. So we have to think like Molly with 8th and 9th inning guys. Sadly.

 

 

There are a many more high-leverage situations as you enter the late innings. Bullpen depth is a better solution than simply using the closer in the 6th, IMO. Specialists or very good set-up RP would be a better bet.  

 

If all the team has is a good closer, I would agree with you.

[Otto von Ballpark]

 

Teams spent $124.65 million for 26 years of relievers. Their return on the investment was $45 million.

Free agents are overpaid as a group because of baseball's service time structure, and this kind of analysis is always going to make the group look bad due to a few bad eggs.  It's particularly bad with relievers and WAR, because great relief seasons often only tally 1 WAR anyway.

 

Looking at your list, the first guy (Affeldt) had an average season, a stellar season, a postseason not included at all in your tally (with Affeldt pitching 11 games/innings without allowing a run or an inherited run to score), and a bad final season when they finished 8 games out of the playoffs anyway.  Do the Giants really think they "lost" $24.6 million on that transaction?

 

Also, Cameron concludes that you might as well sign minor league free agents, but he doesn't actually perform any analysis of minor league free agent relievers.  Getting a minor league free agent reliever that costs nothing and produces nothing indeed seems better than a negative performance on paper, but what are the odds you actually get a plus performance like Grilli or Soria from a minor league free agent, or even a mixed bag like Affeldt?

 

It only really matters if you think the money could have been invested elsewhere, and given the extremely modest amounts of money we're talking about here relative to MLB payrolls, that's far from clear.  $6 mil AAV over 2-3 years is very difficult to better allocate.

Edited November 13, 2015 by spycake

[dbminn]

 

This is a flawed argument, by Cameron and Co, in several levels:

 

A. They are using fangraphs "value", which is fWAR-based, and if WAR is bad in one thing, is in evaluating reliever performance.

I agree 100 percent - Yuck!

[Otto von Ballpark]

 

Continue to sign guys like Boyer, Burton and Stauffer who will take a one-year deal. The Twins success rate with those guys is no worse than the success rate in Cameron's study or the 2013 group.

Analysis please?

 

A guy like Maloney or Stauffer who fails and gets cut doesn't count much on the negative side of the ledger... but then who replaces them?  A guy like Gray or Roenicke who struggles but sticks around doing low-leverage work to keep his WAR from dropping too low -- who takes the higher leverage innings?

 

The Twins bullpen the last 4 years has seen a notable lack of plus performers, which was highlighted in this year's pennant race.  I don't know if the Twins current recipe is really working all that well.  Targeting a good, high strikeout reliever on a 3/15 type deal seems like it could be a really smart play for the modest commitment involved, unless you know a better way to invest that money.

[nicksaviking]

 

Rogers could dominate lefties, correct?

Meyer? He should be up and in the bullpen.

That's 2 right there.

 

Now try converting some of these number 8 starters to the bullpen, and see what they can do. Or, just keep them in AAA, and never really use them for anything......How'd Liam Hendriks look last year?

 

I'd give Rogers a shot as a LOOGY, I have no problem there.  If we're talking about fixing the bullpen though, I'm talking about getting the shut down guys for end of the game.

 

I'm all for throwing Meyer, Burdi, Chargois, Reed, Cole Johnson and Zach Jones into the water and seeing who floats.  Sounds like a great idea to me actually.  If the club isn't willing to take that gamble however, I'd prefer free agents who can miss bats over the Deans, Darnells and Milones who should only be used as long-man/swing-man/mop-up duty.

[woody knows all]

The bullpen is always better when the starters are good.  The last 3 years the bullpen has been fine in May-July.  But when the bullpen has to pitch 3+ innings in 80% of the games that is too much.  Guys get overused and all of a sudden we look at guys and wonder what happened to them.  As soon as the starters can consistently get 18 outs every day then the Twins should spend money on top of the line relievers.  Otherwise it will always come down to Aug trades and hoping that guy in AAA can come up and give the twins 3-4 good weeks.

[Mike Sixel]

 

I'd give Rogers a shot as a LOOGY, I have no problem there.  If we're talking about fixing the bullpen though, I'm talking about getting the shut down guys for end of the game.

 

I'm all for throwing Meyer, Burdi, Chargois, Reed, Cole Johnson and Zach Jones into the water and seeing who floats.  Sounds like a great idea to me actually.  If the club isn't willing to take that gamble however, I'd prefer free agents who can miss bats over the Deans, Darnells and Milones who should only be used as long-man/swing-man/mop-up duty.

 

i'm with you on all of that, not surprisingly.

[Mike Sixel]

 

The bullpen is always better when the starters are good.  The last 3 years the bullpen has been fine in May-July.  But when the bullpen has to pitch 3+ innings in 80% of the games that is too much.  Guys get overused and all of a sudden we look at guys and wonder what happened to them.  As soon as the starters can consistently get 18 outs every day then the Twins should spend money on top of the line relievers.  Otherwise it will always come down to Aug trades and hoping that guy in AAA can come up and give the twins 3-4 good weeks.

 

I think this is, and I don't mean this as an insult, "old school thinking". The data is clear, the third time through a lineup isn't good for most SP, the 4th time is a disaster. 

 

We, MLB, needs to change its thinking about bullpen/starter usage.

 

4 starters, 2 long men (with 1-2 in AAA waiting to come up), and 2 1-2 inning guys, plus your 3 "regular" bullpen guys will be tried by a team in the next 3-5 years. And they'll win a lot of games......if they do it as a plan, not because their pitching stinks.

[Darius]

 

There's really no way to know how a reliever will perform when signed. Tony Sipp could get 3 years and $21 million (or so), and he was pretty non-good for 5 years in the big leagues before becoming terrific last year... Which is real? There are a million examples of this over time. 

 

Not disagreeing with the sentiment at all, in fact, I agree with you completely that its a crap shoot.

 

But, Sipp is curious case because he always had a better than average repertoire (including a heavy fastball, a splitter, slider, and sinker) from the left side.  Sometimes, just having the ability to get the ball over the plate with your left arm gets you paid (Duensing).  He's always had high K/9, and has strung together two very impressive seasons of FIP and WHIP.  I don't think its overly strange for him to get a nice contract.  But, in sticking with the theme of the thread, and your sentiment, I wouldn't be surprised to see him struggle again, as you mentioned.

 

I'm curious as to your opinion on the current, possible high-end, relievers in the Twins system (Burdi, Reed, Jones, Cederoth, Melotakis, Peterson - is Chagrois still around? - possibly Meyer, etc), and how that relates to the upcoming season.  Any chance any of these guys crack the bullpen in 2016, so the Twins don't have to participate in the RP free-agency roulette for a few years?  Any guys in particular that you like?  It seems like Burdi, considering where he was drafted as a pure reliever, should be progressing a little faster.  Any opinions on how he's coming along, now that his first full year as a pro is over?

[Darius]

 

None of those guys can strike anyone out though, and I think that's pretty imperative for a guy in the pen.

 

At least it's been the biggest weakness of the recent relievers this team has assembled.

 

Agree.  Its nice to have that guy who can come in with a runner on 3rd with less than two outs, in a tie game in the 7th, for example, and give you a real shot at sitting down a quality big league hitter.

[BK432]

 

Free agents are overpaid as a group because of baseball's service time structure, and this kind of analysis is always going to make the group look bad due to a few bad eggs.  It's particularly bad with relievers and WAR, because great relief seasons often only tally 1 WAR anyway.

 

Looking at your list, the first guy (Affeldt) had an average season, a stellar season, a postseason not included at all in your tally (with Affeldt pitching 11 games/innings without allowing a run or an inherited run to score), and a bad final season when they finished 8 games out of the playoffs anyway.  Do the Giants really think they "lost" $24.6 million on that transaction?

 

Also, Cameron concludes that you might as well sign minor league free agents, but he doesn't actually perform any analysis of minor league free agent relievers.  Getting a minor league free agent reliever that costs nothing and produces nothing indeed seems better than a negative performance on paper, but what are the odds you actually get a plus performance like Grilli or Soria from a minor league free agent, or even a mixed bag like Affeldt?

 

It only really matters if you think the money could have been invested elsewhere, and given the extremely modest amounts of money we're talking about here relative to MLB payrolls, that's far from clear.  $6 mil AAV over 2-3 years is very difficult to better allocate.

 

I have nothing to add. Spot on.

[DJL44]

 

4 starters, 2 long men (with 1-2 in AAA waiting to come up), and 2 1-2 inning guys, plus your 3 "regular" bullpen guys will be tried by a team in the next 3-5 years. And they'll win a lot of games......if they do it as a plan, not because their pitching stinks.

 

I'm on board with 4-man starting rotation but I see a 13 man pitching staff with 9 relievers, not 11 and 7 like your plan. Starters never, ever go more than 6 innings and 5 is common. That limits your starters to roughly 200 innings (40 starts x 5 innings). Starting pitching is the scarcest commodity in baseball. It is much easier to find a reliever who will be as good as your SP the 3rd time through the order. Blaine Boyer types are everywhere. The best relievers are as good as a SP the first time through or sometimes better.