One can clearly see the reason why we’ve focused on this area: Milwaukee didn’t accomplish anything in terms of hitting, fielding, or starting pitching last season. Only the club’s relief pitching garnered positive recognition, and correctly so. With the fourth-lowest DRA in baseball, the bullpen alone prevented the Brewers from compiling 100 defeats (although it couldn’t help them avoid the 90-loss threshold).

Of course, what matters here is how well the bullpen will perform in the future. What chance does it have of a repeat performance? If the position players and rotation improve, will the relievers continue their elite play and make the club great? As we’ll see, the history of similar squads makes for a murky forecast.

The majors expanded to 30 teams in 1998, with the addition of the Rays and Diamondbacks. Since then, there have been 17 pairs of consecutive seasons (year n and year n+1), which we’ll make our sample. Of the 85 teams who placed in the top five for relief DRA in the first campaign, 19 made it back in the second. On average, those squads finished 12th in the following year — a much more mediocre level. By contrast, 40 of the 85 top-five clubs in starting DRA sustained their rank in season n+1, placing an average of ninth.

If we expand our threshold to the top ten bullpens in baseball, we see the same trend, albeit to a lesser extent. We notice that 77 of the bullpens in the top third of the league one year repeated the next year, while ranking 12th overall on average. A total of 88 of the best rotations, meanwhile, made it to the top ten in year n+1; they placed 11th on average. In simpler terms, great starting pitching tends to stick around, whereas superb relief pitching can come and go.

Could this fate befall the Brewers? Looking at their roster, we can certainly imagine it coming to pass. Francisco Rodriguez, who earned a 2.24 DRA in 2015, will pitch for the Tigers in 2016. Prior to last year, Michael Blazek owned a 6.12 DRA in 17.1 frames, casting some doubt on the 2.46 mark he posted in his full-season Milwaukee debut. The same applies to Will Smith and his pre-2015 DRA of 5.06. Beyond them, the relief options range from passable (Jeremy Jeffress, Corey Knebel) to shaky (Tyler Thornburg, David Goforth). If Blazek and Smith fall off their pace and Yhonathan Barrios doesn’t take the next step, the late innings could get ugly for the Brewers.

This shouldn’t detract from Milwaukee’s relief performance in 2015, nor should it cast unnecessary doubt on the bullpen’s 2016 prospects. Virtually every club has these potential holes in their pitching staff. Rather, this should serve as a reminder of the volatility of relief pitching, even on a team level. A decline to some degree is likely in 2016. As J.P. argued, building around a strong bullpen can pay off, but perhaps a formidable rotation would make for a wiser — and longer-lasting — investment.

In the modern era, relief pitching matters more than ever. The 2016 offseason has seen many clubs pay exorbitant prices for late-inning players, many of whom would have gone for much cheaper just a few years ago. This is likely because, as my colleague J.P. Breen observed back in August, teams with the best bullpens will often succeed, while those who lack in this area may struggle. When you consistently hold onto leads, you will generally net a lot of close wins, leading to more wins overall. In other words, you’ll outperform your run differential.

As we know by now, the 2015 Brewers didn’t collect many victories, despite possessing one of the best bullpens in baseball. The cause of this? Their atrocious starting rotation, which sapped any advantages that their relievers might have offered, helped to sink their record to 68-94. While top-notch players such as Will Smith, Michael Blazek, and Francisco Rodriguez ensured that most games ended well, the likes of Kyle Lohse and Matt Garza stunk up the beginnings of too many contests. The starters didn’t act alone, though; we’ll soon discover another factor that held them back.

In any given year, a fair amount of clubs will feature starting pitching that can’t live up to the relievers’ standards. The 2015 Brewers’ 4.79 rotation DRA and 3.87 bullpen DRA yield a run difference of 0.92 per nine innings. In the 30-team era (i.e. over the past 18 years), only 24 squads have surpassed that gap:

The Brewers begin to separate from the pack when we add some context. Take, for instance, the 2007 Cardinals, who narrowly top them in this regard. That team certainly had an awful rotation — placing 29th in the majors in starting DRA — but its relievers didn’t fare much better, with the 15th-best DRA among their competitors. By comparison, Milwaukee’s starting and relief pitching ranked 26th and 4th, respectively, meaning the former truly excelled and the latter truly stumbled.

When we look for teams with similar finishes — in the bottom five among rotations and in the top five among bullpens — we get a much smaller sample, of just 12 total:

You might notice something about this group, especially the most recent member before Milwaukee: As a whole, they didn’t play that poorly. The 11 non-Brewer clubs averaged 76 wins, eight more than the 12th club accumulated. Interestingly enough, they also pitched much better overall — whereas the Brew Crew ranked 25th in total DRA last season, these teams placed 19th on average. How did they make it to respectability without a decent rotation?

It’s simple: They gave their relievers more time to shine. Milwaukee’s bullpen accounted for 36.9 percent of its 2015 innings, whereas these teams allotted 38.0 percent of their frames to non-starters. This meant that those 11 clubs ranked an average of sixth in relief innings, compared to eighth for the Brewers. They didn’t make monumental changes, but by capitalizing on their strong relief corps and pulling starters earlier, they managed to comparatively succeed.

The bad-rotation-good-bullpen combo doesn’t necessarily spell doom. If a club adjusts its strategy accordingly, it can go far — perhaps even win a World Series. Obviously, the 2015 Brewers lacked the talent to accomplish that, or to even post a winning season; still, they could have notched a few more victories by turning their bullpen loose. Hopefully, as the rest of the league begins to value relief pitching, Milwaukee will follow suit.

*****

One of the major topics discussed on this site has been the Brewers bullpen. It has been discussed in depth and at length since BP Milwaukee first launched. While the Brewers had an awful season, the one bright spot was the relief corps. It was a way to focus on the positives of the season.

The interesting part is that evaluation of the bullpen has recently been a big topic in baseball. It started a few years ago when the Oakland A’s traded for Jim Johnson. Most people, including myself, thought they were nuts, the belief being it was a huge overpay, especially for the A’s who took on all of the 10 million dollars remaining on the deal. The strangest part was that this was the A’s; they were supposed to be one of the “smart” teams. Common sabermetrics orthodox suggests that spending money on closers and relievers is a bad idea.

Then last season, the Astros signed Luke Gregerson and Pat Neshek. This season, the Red Sox traded two top-100 prospects for Craig Kimbrel, while the Astros traded a number of talented prospects (including the former number-one-overall pick, Mark Appel) for closer Ken Giles. The A’s also signed Ryan Madison to a three-year deal.

So what gives? All of these teams are highly regarded in the sabermetric sphere. Why are they the ones making these types of deals? If they would’ve been made by different teams, they would have been heavily mocked.

Perhaps we’ve been undervaluing relievers all along. Interestingly enough, the Lord Commander of this website J.P. Breen wrote an article about this very notion entitled “Contention & Building Through the Bullpen” back in August. In the article, Breen discussed the notion of market inefficiency in baseball, pondering what would be the next inefficiency to bubble to the surface. He eventually landed on the use of the bullpen. But his point was that teams were not looking to uncover it. Teams had already uncovered it. The revolution was already underway.

Breen had already seen the trend, that teams were utilizing the shutdown bullpens to, “effectively compensate mediocre or below-average starting rotations.” With the combination of the Royals super bullpen, the Yankees addition of Andrew Miller, the Orioles pair of super relievers (Zach Britton and Darren O’Day), and even the Blue Jays young reliever core, those teams were able to mitigate subpar starting rotations with quality bullpens.

His main point was that the Brewers should take note of this during their rebuild. The fact that contracts for relievers still seem cheap means the Brewers should be able to buy quality relief pitching when the time is right. More importantly, the Brewers now have a healthy farm system. They should be able to use their new and improved system to fill in these important holes in the bullpen. What is especially encouraging, as Breen mentioned, is the Brewers currently have useful bullpen pieces.

These pitchers can be used in a number of ways. They can by used as building blocks or trade pieces. If the Brewers feel as though they can get a high return for one of their relievers, they should definitely make a move and look to acquire more prospects. But, while I’ve mentioned that the Brewers bullpen has been good and while it’s been discussed a number of times on the site, the question still remains: just how good was the Brewers bullpen in 2015?

A few months ago, I would have probably simply looked at one of the WAR metrics to get an idea of the situation. But, as Russell Carleton mentioned in his article, “The Kimbrel Gambit,” WAR may not be the best way to measure or look at relievers: “WAR, by its very nature, seeks to strip out the context out of a player’s results, although the major WAR indices are all aware that for relievers, that’s a little silly. In general, we find that in WAR for relievers, there is an adjustment made so that a pitcher’s WAR is inflated by a factor that is halfway between 1.00 (average leverage) and the average leverage that he faced in the games that he threw. If he normally faced a leverage value of 2.00, his WAR (for his pitching components) would be inflated by a factor of 1.5. The problem here is that while closers do pitch in 40-50 save situations each year, they also pitch in games where they are just getting work in or are filling an inning. (Here’s Kimbrel’s game log for last season. There are a few decidedly non-save situations.) Those ‘extra’ innings aren’t really what teams are paying closers for, and they are generally low-leverage, but they mean that the pitcher’s ‘average’ leverage will decrease. In other words, WAR under-values closers, even with its adjustment and here I don’t think it’s a good metric for what we really want to measure.” Russell concluded in that article that Win Probability Added (WPA) was a better way of looking at relievers. I would agree with this to a certain extent.

In August, I engaged in an exercise and asked the question of who was the Brewers true closer? I made a couple of conclusions. One was that we can take the term closer literally and define closer as someone who “closes” games. The other point was that we can look at closers through the lens of clutch. I defined clutch as the pitcher who pitched the highest leverage innings because that’s the point of the closer mentality. It’s the ability to pitch in high-leverage situations. The point is not that closers are getting paid for saves. Maybe they are, in part, but I think teams are smart enough to recognize that the save is a silly stat. What closers are really being paid for is getting outs in high-leverage situations. That success can, therefore, be looked at through WPA.

Win Probability Added may sound complicated but it’s not. What it does is it “credits or debits” a player based on how his performance affects the probability of winning a game. If a relief pitcher comes in with the bases loaded and one out and gets out of the jam, that has more value than a pitcher who came in and got two outs with nobody on. The problem is that most pitching metrics won’t take that context into consideration. This, to a certain extent, is fine for a starter or a position player, but for a relief pitcher it’s not. This is primarily because so much of his role revolves around the ability to get outs in clutch or high-leverage situations.

So now, let’s look at how the Brewers 2015 bullpen ranks throughout their history with WPA. (I went back to 1974 because that’s as far back as the results go.)

With WPA, the Brewers 2015 season was the fifth-best in franchise history. It also ranked seventh among all Major League teams for the 2015 season. Even though the team would have probably ranked higher by other metrics, it’s still a welcome sight to see the Brewers bullpen this high in the rankings after a number of years of poor bullpen performance.

Another way to look at this information is to analyze how the team ranks every year, compared to the rest of the league. While the fifth-best WPA in franchise history is good, in some years the Brewers reliever WPA might have ranked higher as compared to the rest of the league, even though their overall reliever WPA was lower.

With this ranking, the Brewers 2015 season ranks seventh in its history. While in 1974 and in 1981 the Brewers reliever WPA was lower, they ranked higher compared to the rest of the league. In 1981, the Brewers had the highest reliever WPA in all of baseball. (They also ranked 235th out of 1168 teams in reliever WPA since 1974.)

I, obviously, am not the first to look at WPA. Russell Carleton as mentioned before has already noted that WPA is a good way of judging relievers. Jeff Sullivan of FanGraphs has also used this tool. In his article, he noted that the Pirates have beaten their projections the last couple of years while having a high WPA. He noted that the bullpen was a primary factor in beating the projections.

My next question then was this: Is there a relationship between WPA and beating projections? I, therefore, looked at old PECOTA projections dating back to 2008. From there, I looked at the difference in the preseason projections and the actual end of year wins, and how the WPA ranked within those differences. Due to the issue of small sample sizes, I had to split the variables into different buckets. So I put teams who beat their projections from 0-to-5 wins in the same bucket and looked at the average WPA within that bucket. I mainly created buckets of five, with some exceptions due to sample size issues.

I’m sure you can tell that these aren’t the best sample sizes, but they’ll have to do for now. With that caveat, there is a positive relationship between beating projections and WPA. Basically, the more a team outperforms its projections, the better the team reliever WPA was. The more the team underperformed its projections, the worse the WPA. This doesn’t mean that the bullpen was the sole cause of the effect on the difference in projections and actual wins. It merely means that it was probably a factor in it.

On the whole, 114 teams have beaten their projections and those teams averaged a WPA of 3.56. For the counterpart, 118 teams performed worse than their projections and averaged a WPA of -0.15. For the ones who equaled their projections, well, that you can tell from the table above. It’s the one with zero difference.

Beating projections aren’t easy, but teams who have a better bullpen seem to have a better chance of accomplishing that goal.

While WPA is a good stat for judging relievers, it’s not perfect. The biggest problem with WPA is it’s highly dependent on how a manager uses a reliever. If a manager has a misconception about a specific relievers skill set, then it is unlikely that that reliever will be used in high-leverage situations. Closers will naturally get the highest-leverage situation, simply from the virtue of pitching in the ninth inning.

Take Jeremy Jeffress, for instance (I know he’s not a “closer” but it’s the same principle). His WPA was 1.66, meaning that he performed successfully when one takes the context of the game in consideration. While someone like Michael Blazek has a 0.13 WPA even though he performed just as well and even times better than Jeffress in other context neutral metrics. The difference was that Jeffress had more opportunities to pitch in high leverage situations then Blazek.

Looking at gmLI (A pitcher’s average LI when he enters the game), Jeffress has a 1.35 gmLI while Blazek had a 0.66 gmLI. Jeffress also had a 1.44 pLI (A player’s average LI for all game events), while Blazek had a 0.58 pLI. Jeffress, therefore, had many more opportunities to pitch in high-leverage innings. The problem here is determining which one had the better season. I’m still unsure of the answer here. On the one hand, Jeffress’ pitched well in the leverage situations he was put in. On the other hand, Blazek performed well but simply didn’t get the same opportunities.

For teams, the answer is still the same, it’s not about what a pitcher did in the past, but what a pitcher will do projecting forward. When it comes to relievers, teams may then still be able to find bargains by getting pitchers who haven’t pitched in many high-leverage situations, and who performed well even without the context. Those pitchers might be undervalued by their current teams, and if the Brewers stay vigilant, they might be able to us it as an advantage.

Whatever the goal may be, having a strong and quality bullpen is a very real asset. As Carleton mentioned, it can add more than a couple of win’s worth of value, and as Breen has pointed out, it seems that teams are in the know when it comes to this effect. This is no longer the next market inefficiency but now a real strategy in baseball.

It seems many in the media believe sabermetrics to be static. This is simply not the case. It’s actually quite the opposite. Sabermetrics, as Bill James once said, is the “search” or the pursuit towards objectivity. The key word being pursuit. Sabermetrics is always evolving and trying to find better objective ways of making the best decisions. Now, objectivity is a rather complicated concept. Some believe that it is impossible to be completely objective and that very well might be true. But, that is why sabermetrics is pursuing that objective truth.

The point for the Brewers is not to remain static. To keep searching for that next “market inefficiency.” If they want to compete with the likes of the Cubs, Pirates, and Cardinals, they will need to keep an open mind on all things. The bullpen is currently a strength or, at least, was a strength. Building through the bullpen is a smart idea. They currently have the assets for the bullpen to be a strength, but considering the fact that relievers are so volatile, it may be better to move them this winter. (Maybe not someone like Will Smith who cFIP really likes and could have even more value once he’s given more of a chance in high-leverage situations next year). If the Brewers are able to get more quality young players for their relievers, then it might be well worth it.

There isn’t a perfect transition from starter to top-notch reliever. But, most relievers have flaws. Most of them only have two pitches. Some of them have command problems. For the Brewers, the next step will be to identify the best young pitchers who can’t stick it as a starter and put them in the bullpen. Don’t try and force anything. Put the players where they belong and allow them to strive in that role.

Much is made of the next market inefficiency. The true market inefficiency is having a smart and open mind. It is constantly pursuing that objective truth, even if you never reach it.

Thanks to Rob McQuown for research assistance. 

Of course, this is the case with the 2015 Milwaukee Brewers. With the ninth-best bullpen in the major leagues by fWAR, the Brewers are also the only team with three relievers (among pitchers who have never started even one game this season) in the top 25 by PWARP. In fact, you have to go all the way down to 32nd place by PWARP to find a set of three players on the same team (the Yankees’ Dellin Betances, Justin Wilson, and Andrew Miller). All this is to say, Will Smith (ninth in PWARP among relievers), Michael Blazek (17th), and Francisco Rodriguez (23rd) have been outstanding.

But you knew that already, so what’s the point? During my last article (which I’m still sorry about) I made a quick-and-unfounded comment:

“Sometimes I wonder if bad teams have better bullpens because there are fewer high-leverage opportunities, but that’s for another time.”

Well, welcome to “another time.” I don’t think I’m necessarily alone in thinking this to be true, but how best to measure something built only on narrative? I started by looking at leverage index, to see if the first assumption—that the Brewers’ bullpen has pitched  ‘less intense’ innings—was true. Sure enough, the Brewers bullpen as a whole is 27th in pLI and ahead of only the Athletics, Phillies, and Indians. That four teams with sub-.500 records are all clustered at the bottom there bodes well so far. If this theory is to be true however, one would assume we should see all four of these bullpens in the top half of the league in overall performance. Alas, despite the Brewers and Indians nearing the top (first and eighth, respectively), the Phillies and Athletics sit 17th and 24th. I could stomach one team being barely below the median, but the Athletics’ woes are hard to reconcile.

That doesn’t necessarily mean that the narrative is wrong though. After all, pitching fewer high-leverage innings wouldn’t necessarily make all your bullpen pitchers measurably better. One could easily imagine a scenario in which a starter on a bad team gives up a ton of runs, doesn’t make it very far into a game, and the bullpen has to just tidy it up. Those innings, although very low leverage, would still be very taxing. So let’s assume a new scenario: only a team’s best relievers reap the rewards of an under-performing team. In saying this, I’m assuming that a team’s best reliever almost never comes in for mop-up scenarios. In fact, I’ll hazard a guess that I’m definitely right in that assumption. After all, the long reliever is never the closer.

So, how does that stand up to testing? Looking only at the best two relievers by xFIP in each bullpen, we can draw some conclusions. However, these conclusions would quickly be rendered useless because the best relievers in a bullpen are not necessarily the ones deployed in the highest leverage spots either. The reasons for this are two-fold: first, a manager may not pay attention to xFIP when he decides to use a certain pitcher in a certain situation; and second, a pitcher who doesn’t pitch in high-leverage situations might just actually have a better xFIP—the pressure never gets to them.

Now we are starting to understand how difficult it is to test narrative. Let’s try to find another way to look at this. Let’s look at every team’s highest individual pLI out of the bullpen and see which one of those has the best xFIP. This way, we’re narrowing it down definitively to the player who has been used in the highest leverage situations for his team.

Sorting by pLI alone gives some pretty strange results. You wouldn’t necessarily expect a team that is the best in their division to be near the bottom. However, Yimi Garcia of the Dodgers sits in a distant last place, pitching in just 0.16 higher than average leverage situations. But that’s just because Kenley Jansen doesn’t qualify yet.

So, we now have the pitcher that is deployed in the highest leverage situations for his respective club for all 30 teams. Who has performed the best overall? The Brewers’ Francisco Rodriguez sits fifth in xFIP! Just behind the constantly worked Craig Kimbrel of the Padres and David Robertson of the Chicago White Sox. Three sub-.500 teams. Seems like we’re on a roll. Who are the top two? The Orioles’ Zach Britton sits second and the Yankees’ Dellin Betances sits unsurprisingly first. A .500 team and a division-leading team. While this still doesn’t prove anything, maybe it’s a start.

Where are the Athletics, Phillies, and Indians? The Athletics’ Tyler Clippard is dead last, the Phillies’ Luis Garcia (yes, Garcia has apparently pitched in the most high-leverage spots, not Papelbon or Giles) sits 21st, and the Indians’ Cody Allen is 13th.

Of course, what this narrative suggests is that their high-leverage relievers will pitch better in those situations because there are fewer of them and they are better-rested because of that. Having looked at the results, a few things have become evident to me. First, it’s near impossible to actually measure this. Second, great pitchers perform well and poor pitcher perform poorly regardless of rest. It shouldn’t be a surprise that Craig Kimbrel or David Robertson top a list of xFIP leaders, regardless of rest. Third, in order to actually prove this, I think you’d have to narrow it down over multiple years to only teams that are in the bottom percentile for runs scored. That Clippard is at the bottom of the list and also plays for a bad team doesn’t prove anything because he—or the pitching staff in general—could conceivably be the problem for the woeful Athletics.

I’m truly sorry to pose so many questions and offer so few answers, but I think discussions like this that challenge narrative can still be helpful even if they don’t draw conclusions. One conclusion is certain though: K-Rod, Blazek, and Smith are quite the tandem. So it goes.

Lead photo courtesy of Benny Sieu-USA TODAY Sports