What’s Runs Prevented? Flexible Elite Roles ||  Exploring Runs Prevented || Aces Do Not Exist || Rotation Spots || Third-Time Charmers

Here’s how the Brewers pitching staff looks when ranked by Average Runs Prevented. For additional context, each pitcher’s Games, Games Started, and Innings Pitched statistics are included.

Stats: 

Average Runs Prevented is the average of park-adjusted, league-adjusted estimates of a pitcher’s actual runs allowed compared to their expected runs allowed.

DRA Runs Prevented is the difference between a pitcher’s expected runs allowed and their DRA performance. 

Direction is the “Direction of Change” between a pitcher’s 2018 Average Runs Prevented and 2018 DRA Runs Prevented.

G is “Games” (total appearances); GS is “Games Started” (total starts); IP is “Innings Pitched.”

Why is this important? DRA is a pitching statistic that estimates each pitcher’s performance based on numerous contextual factors. DRA is a statistic that can describe a player’s performance on the field by correlating Runs Allowed per 9 IP (RA9) to DRA; it is modeled to consistently assess a player’s performance year-to-year; and it is modeled to predict next year’s RA9. Runs Prevented, on the other hand, is a purely descriptive statistic, simply aiming to measure the extent to which a pitcher compares to their park and league environments.

Before we get into the extended analysis, if you’d like to know why this topic is important, consider the following questions; for fun, the exercise could also end here, as there’s a lot to think about with this staff.

….which of these pitchers would you expect to improve in 2019?

….which of these pitchers would you expect to improve in 2019?

….which of these pitchers would you expect to improve in 2019?

By describing Runs Prevented and DRA Runs Prevented statistics year-over-year, it is possible to understand the absolute volatility of pitching performance. DRA is also potentially a tool that can be used to set someone in the right direction for analyzing statistical profiles in order to project improvement or decline.

Let’s take a look at MLB pitchers that worked in 2017 and 2018:

The value of using a statistic such as DRA is that the year-to-year Runs Prevented performance by MLB pitchers is absurdly volatile. The table above demonstrates the absolute value of change in several key statistics for pitchers that worked in both 2017 and 2018. 639 MLB pitchers worked in both 2017 and 2018 seasons. On the whole, this group was quite volatile, with the average change in runs prevented moving by nine runs prevented (positive or negative); a pitcher that worked in both 2017 and 2018 also saw their innings pitched total fluctuate between 33 and 34 innings, and their average games started fluctuate by four. Focusing specifically on starters (i.e., pitchers who started a game in 2017 and pitchers who started a game in 2018), the fluctuations are even wider.

These fluctuations would be the equivalent of Jhoulys Chacin becoming a slightly below average, slightly smaller workload pitcher in 2018, or improving steadily into “ace” territory; Freddy Peralta expanding into a more regular rotation role, or stepping back into a smaller replacement role; Corey Knebel fluctuating to a below average reliever or recovering his excellent high leverage form; or Josh Hader becoming “just” an average reliever or taking the next step in his high leverage ace development. These are just a few examples of the real impact that typical run prevention fluctuations can cause to a team. Each of these pitchers are likely to remain under Brewers contractual control in 2019, so it matters how their performances change.

Let’s dig deeper into that group of 639 pitchers that worked in both 2017 and 2018 to assess the descriptive value of DRA and Runs Prevented. A couple of caveats are in order. First, this is a biased analysis, insofar as I am expressly limiting my search to players that worked in both 2017 and 2018, which excludes a “true talent assessment” of players that missed either of those seasons for a multitude of reasons (from player development, such as Freddy Peralta, to injury, such as Jimmy Nelson). Second, since I will be describing the general direction of DRA, I am not using statistical methods to assess the significance of DRA’s predictions. With these caveats in mind, I think it remains useful to see how DRA assesses players within a single season, and across two seasons.

Now that we’ve discussed the average absolute value of Runs Prevented change between 2017 and 2018, let’s take an overview of this group of pitchers in terms of improvement or decline. Excluding pitchers with Runs Prevented totals between -2 and 2 in 2017 and 2018, which represents a relatively minimal range of fluctuation that could simply be explained by park factors or league environment, more pitchers declined than improved between 2017 and 2018. In many cases, these changes were quite major, as 132 pitchers declined by 10 or more Runs Prevented, while 53 pitchers improved by 10 or more Runs Prevented. The overall magnitude of major declining performances ensured that this group of 639 pitchers was -439 Runs Prevented (!!!) between 2017 or 2018; this means that if each 2017 team retained these pitchers, on average they would have been expected to lose approximately 44 more games (as a group) in 2018, all else held equal.

Based on 2017 performance, could anyone have predicted these directions of change among these pitchers? Once I assembled an Average Runs Prevented analysis of the 2017 MLB season, and isolated pitchers that worked in both 2017 and 2018, I analyzed several aspects of each player’s performance:

• I analyzed the 2017 Direction of Change, which is the change between 2017 Runs Prevented and 2017 DRA Runs Prevented, in order to assess whether a player overperformed or underperformed their DRA.

• I analyzed the 2017 Direction of Change and the difference between 2018 Runs Prevented and 2017 Runs Prevented, in order to assess whether a player’s between-seasons change (2017 to 2018) matched their 2017 underperformance or overperformance. Focusing on 2017 Direction of Change and between-seasons change is one way to describe the types of projections made by DRA.

• I assessed 2017 DRA Runs Prevented and 2018 DRA Runs Prevented in order to determine whether the statistic consistently estimated a pitcher’s contextual performance.

First and foremost, in terms of 2017 and 2018 consistency, DRA consistently assessed 389 pitchers as either Above Average or Below Average in both 2017 and 2018. Since the “Other Pitchers” group is quite a set of outliers, I provided a couple of key statistics about their DRA Runs Prevented.

In this case, the “Other” group is comprised of outliers, including pitchers like Wade Miley and Chase Anderson, as well as Lucas Giolito, Kyle Freeland, and Derek Holland, among others. This is an area where the biased selection of this group of pitchers could impact analysis, as developments such as a new pitch (by Miley) or backed-up stuff and command (by Giolito) create role discrepancies that would be difficult to predict without granular scouting information. Of course, these are precisely the types of uneven player development facts that teams attempt to exploit. Wade Miley was not a particularly good pitcher in 2018, indeed he could have reasonably been replaced (which is partially why he was available for a minor league contract entering 2018); his development to an average pitcher was worth 44 DRA Runs Prevented between 2017 and 2018, a massive improvement that is going to skew nearly any sample of players.

On the whole, it is worth noting that DRA Runs Prevented tracked better than Average Runs Prevented, in terms of absolute value of change, between 2017 and 2018. Among pitchers that worked in both seasons, DRA Runs Prevented fluctuated by approximately 8 runs, compared to approximately 9 runs by Average Runs Prevented. Not bad!

How does DRA work with this group of pitchers in terms of predicting the general direction of change between 2017 and 2018? Based on a pitcher’s internal 2017 difference between DRA and Runs Prevented, that pitcher’s typical improvement or decline between 2017 or 2018 matched the overperformance or underperformance (in terms of 2017 DRA versus 2017 Runs Prevented). DRA correctly assessed a pitcher’s expected performance change in 79 percent of cases:

Within this group of pitchers, DRA performs quite well in terms of assessing the actual size of the Runs Prevented change, as well as the direction. Once I categorized pitchers into groups of players that had Predicted Improvement, Predicted Decline, or some Other Prediction, I compared the change between 2017 and 2018 DRA Runs Prevented to 2017 and 2018 Actual Runs Prevented:

It should be underscored that this is a descriptive account of DRA’s predictions, rather that a statistical test of the significance of DRA’s predictions. Still, what is incredibly impressive about DRA is just how strong the statistic is in anticipating the shape of the run environment, and understanding the wide variance that can occur year over year.

What is interesting is that, according to DRA, the Brewers pitching staff was indeed better than average in 2018. However, there are 22 pitchers from that staff that might reasonably be expected to post a notable improvement or decline in 2019, if one assesses the extent to which they outperformed or underperformed their 2018 DRA. Thus, it is worth repeating the questions about who might be expected to improve in 2019, for even if the overall direction of the club’s pitchers may be expected to stay the course, their shape and distribution of Runs Prevented can all but expected to look quite different.

On Method:
When I run Twitter chats from BPMilwaukee, one of the most curious things to my mind is that followers of BPMilwaukee will not necessarily support general BP stats work. No concerns there, really; it’s not necessary to “brand” MLB stats analysis, and indeed when one begins supporting stats-as-brands, that’s just as problematic as how so-called Old School stats like Runs Batted In or Earned Run Average are used in orthodox baseball discussions. No, what I find curious is the general idea that a stat like WARP or DRA is faulty because it is “made up,” which is presumably a concern because the BP stats team are extremely transparent about how the stats are constructed and also how (and why) they are changing. So folks actually know that DRA changes…which is different than how the vast majority of websites present baseball stats. What is problematic about this attitude about DRA is that it ignores how other statistics are merely “constructs” in the very same way that DRA is merely a construct, and it also trades in the murky waters of false certainty.

For the past two years, I have worked in Community Development and Economic Development positions while completing a professional urban planning and policy degree. I used to believe that I was a “stats” guy or an “analytics” guy, but I never quite understood the importance of what actual statistical analysis means until I was forced to reckon with my biases while training for economic analysis. Before I learned and studied stats, and was required to use them on the job, I thought the “numbers” were most important. While fields aligned with statistics are concerned in some sense with “numbers” and thus with producing “numbers-oriented results” (i.e., sometimes your boss really wants the results of your analysis), by far the most important elements of statistical analysis are “concept validity,” methodology, and uncertainty. What is most important about statistical analysis is process, it turns, out: how an analyst reaches a conclusion is much more important than the concluding numbers on their own, for it is only in light of outlining methodology, and explaining what is at stake with a certain measurement, that anyone (including a consumer of those numbers) could understand the numerical results of statistical analysis.

It’s ironic that many victories of the so-called “analytics movements” are now enshrined in their own dangerous orthodoxy, for what everyone seems to have forgotten is that even if the debate was about numbers, the original controversy was to convince the “Guards of Baseball Knowledge and Value” that there were legitimately different ways of thinking about the game and that that meant there were legitimately different measurements that could be presented. Somewhere along the line, we became obsessed with those measurements, rather than the process-oriented creed of focusing on how to think about baseball. This extends to statistical analysis, then, too: it is as though when many fans were convinced of the merits of WARP and other stats, they simply turned over the box containing ERA, RBI, etc., dumped out those contents, and stuck the new measurements into the box. That was never the point, and to the extent that many of us did not communicate the significance of process-oriented thinking about baseball stats, that was our problem (and I place myself in this camp, having only realized the significance of this issue over the last few years).

Anyway, “concept validity” is the most important thing that I have learned about statistical analysis, aside from clearly stating your uncertainty in proper terms. “Concept validity” is basically the extent to which the phenomena you’re trying to measure match the methods that you’re using to measure the phenomena. What should be inherent in this process is an understanding that as an analyst’s approach to measuring phenomena changes, so too should their results change; one need not hold the numerical results of analysis sacred, for if new empirical evidence emerges, methodological research unearths a better way to measure something, or a literature review reveals a better way to define a concept, there is nothing wrong with the analytical results changing.

So, keep this in mind when you’re thinking about why DRA has “changed.” DRA doesn’t “hate” anyone on your team, or love them. It is not a mark against DRA, or WARP, that the stat is consistently updated and changed, because that is a sign that its authors are attempting to reach that mark of “concept validity.” If it is the goal of DRA “to tease out the most likely contributions of pitchers to the run-scoring that occurs around them” and updated methodological approaches, or an updated understanding of pitching-related data, helps to accomplish that goal, revising the stat is a methodological strength. That said, I can understand that within a statistics field, one may have disagreements with some of the particular methodological approaches; but I don’t take any substance of that type of disagreement to dismiss the value of the overall methodological process of DRA.

This is why the new DRA is so important: it continues Baseball Prospectus’s commitment to presenting uncertainty (as has been done on Brooks Baseball, as one example) in publishing baseball statistics. Embrace this approach: so far as DRA is “made up,” it is made according to a methodologically sound process that upholds honest and transparent thinking about uncertainty.

DRA Values
One of the approaches to constructing DRA is to valuate the Run-value of pitching outcomes, and those outcomes are published by Baseball Prospectus. These elements are arguably more important than the DRA output itself, for these outcomes show the balance of a pitcher’s performance: is a pitcher saving runs during hits, balls not in play (e.g., Home Runs, strike outs, walks, etc.), or outs on balls in play?

My favorite Brewers pitcher, Zach Davies, is a “casualty” of the new DRA (h/t to Kyle Lesniewski for beating me to this realization). But we’re not going to say, “DRA hates Zach Davies.” On the contrary, it is possible to see that from Davies’s Out Runs (-1.4), Not In Play (NIP) Runs (1.9), Hit Runs (1.4), and Framing Runs (-0.1) that Davies is not getting the job done in terms of limiting runs when the ball isn’t in play, and he’s not limiting runs that occur on hits, either. Here’s how the 2018 Brewers look, sorted by NIP Runs (Josh Hader is real!):

These run elements help to define DRA. At this point in the season, however, it’s important to note just how large the Standard Deviation appears for DRA. For example, Davies’s DRA is currently published at 6.02, but with a standard deviation of 1.00, approximately 70 percent of the time, Davies could be expected to land between 5.02 DRA and 7.02 DRA. Tracking DRA with RA9 (Runs Allowed per 9 IP), something like a 5.02 RA9 gets Davies into respectable rotation territory, and there’s no telling that the righty could also prevent runs to a greater extent (i.e., serve as an even greater outlier).

Here are Brewers starters by variation, sorted by lowest Standard Deviation.

Now, let’s repeat this measurement with WARP, which should help to underscore the extent to which fans should quote Replacement Level stats with certainty. Doesn’t this make you wonder what the error bars might be on Baseball Reference or FanGraphs WAR? Hopefully those websites follow suit and publish WAR error bars where possible.

What I find extremely interesting about this exercise is the extent to which the Brewers starting pitchers exhibit variation in their potential WARP production. Almost to a man, the Brewers remaining rotation (after Brent Suter was moved to the bullpen to make room for Wade Miley) could range anywhere from replacement level to solid rotation piece (for reference, among 149 pitchers with 17.0 IP or higher, 0.34 WARP is a median 2018 performance thus far). This will be a stat worth watching for the remainder of 2018.

Finally, the last stat worth watching is whether the Brewers can continue to out perform their DRA. For my last publication on Runs Prevented, the Brewers as a pitching staff were approximately 18 runs better than their DRA suggested. My hypothesis here is that the Brewers groundball efficiency machine is leading this charge, but that could be one of many explanations including random luck.

These statistics provide a wide range of tools for Brewers fans and analysts. Ranges of DRA and WARP can be compared in order to assess both uncertainty and potential overlapping fields of value. To my mind, the best aspect of this new presentation is that fans and analysts no longer need to feign false certainty over WARP, and this is great; one shouldn’t need to say “Zach Davies has a 6.02 DRA” right now, when one can say “Davies’s DRA ranges from 5.02 to 7.02.” This exercise can be repeated throughout the season, and perhaps through embracing uncertainty we can find better hypothesis about how and why a team is under-performing (or over-performing) their peripheral stats or DRA estimates.

Photo Credit: Patrick Gorski, USA Today Sports Images

After many years in what Brian Kenny refer likes to refer to as “the dark ages” came the sabermetric movement, accompanied by a slew of new metrics. One of the more recent and popular discoveries is DIPS, by Voros McCracken, which brought us Fielding Independent Pitching (FIP), created by Tom Tango.

FIP was modeled after ERA. FIP looks at what a pitcher can control, walks, strikeouts, and home runs. With FIP, we can see which pitchers benefitted from the most luck, the previous season. The best way to do this is by looking at the difference between ERA and FIP. This also would give one insight into how likely a pitcher is to regress the following season.

As my colleague Ryan Romano points out, in a Beyond the Box Score article, FIP has its flaws, “Pitchers can control other elements of the game — some limit hard contact better than others, and some will melt down with runners on base. Because of pitch framing, fluky umpiring, park factors, and a host of other variables, they don’t necessarily control the three true outcomes, either.”

The good news is that 2015 brought two new pitching metrics. The first was contextualized FIP or cFIP, created by Jonathan Judge. Then came Deserved Run Average from Judge, Dan Turkenkopf, and Harry Pavlidis.

Basically, as Judge put it so eloquently, “The vast majority of a pitcher’s DRA is explained by his average linear weights allowed to opposing batters”. Now, DRA also takes into account what Judge calls “externalities”, which including catcher framing, temperature, team defense, the score, the stadium, and more.

cFIP is very similar to FIP. The biggest difference is that it takes into account the context. This includes the catcher, the umpire, the batter’s handedness, the stadium, and more. The most important thing to note, however, is that DRA is a descriptive metric while cFIP is a predictive metric, even though it has descriptive value. Therefore, we can now look at luck or overperformance in a new and better way.

Mainly, I wanted to look at which Brewers players had the biggest overperformance last season. In order to do this, I created a DNA chart to visually show the difference between DRA- and cFIP. (Minimum 50 innings pitched).

I’ve often raved about Michael Blazek’s 2015 season on this site, and for good reason. His 51 DRA- stands as one of the best in Brewers history and deserves praise.

With that said, there’s a reason that teams aren’t clamoring to trade for Blazek or that Blazek isn’t being considered for the Brewers closers job. Instead, all the rumors seem to be surrounding Jeremy Jeffress and Will Smith. Blazek, you see, way overperformed last season. His cFIP stands at 96 compared to his 51 DRA-, marking the biggest difference between DRA- and cFIP among all Brewers pitchers. Comparing it to the rest of the league, Blazek had the sixth biggest difference between his DRA- and cFIP.

In part, do to a fractured hand which ended his season early, Blazek only pitched 55.7 innings last year. This isn’t uncommon for relievers, but it does increase the chance of variation in performance due to the small sample size. It’ll be interesting to see how he performs next year, but this could be signaling that Blazek will have some regression in his performance. He’s never been a big strikeout guy, including last year, and has had trouble with his walk rate in the minors. If those were to flare up again, we could see a downturn in Blazek’s performance. Conversely, a 96 cFIP is still above average, so even though Blazek probably got lucky, he still was no slouch.

As mentioned above, Jeremy Jeffress and Will Smith, have both been mentioned in a few trade rumors this offseason. There’s also been some debate as to whom the “closers” job would go to. This might be because of a couple of factors. A) Both pitchers provided well above average production last season. B) There’s no reason to think those were flukes. Jeffress had the smallest difference between his DRA- and cFIP and Smith had the second smallest, among Brewers pitchers.

With their great stuff, both pitchers bring the ability to strikeout hitters at an above average clip. Jeffress is well known for his blistering fastball and Romano dubbed Smith’s slider as the Brewers best pitch in 2015, which help him accrue an insane 12.93 K/9 in 2015. If he can keep up that performance, then there’s a good chance Smith will repeat his performance from 2015.

I then did the same thing for the teams. (You’ll notice that the X-Axis starts at 80. I did this so that people could better see the difference between both metrics, but it should be noted that there’s far less variance here than there is among players, in general.)

Unfortunately for the Brewers, there wasn’t a big difference between both metrics. The Brewers had a below average pitching staff last year, and they weren’t unlucky, as a whole. So, their performance wasn’t a product of bad luck but merely not having a great pitching staff.

One final element to note is that the White Sox were particularly unlucky last year. More so than any other team. Meaning that their pitching staff could be even better this upcoming season. This could further emphasize the point that they should go for it this upcoming season. Although, they still seem wishy-washy as to whether or not they truly want to compete.