Grading Trades: Surplus
Translating OFP
Cashing Out OFP
Organizational Logic and Playoff Trades

Over the course of the past year, I have used a WARP-based system to assess transactions. I use a harsh depreciation system to demonstrate the assumption that a player’s value will only decrease in time, which basically attempts to price trades closer to their worst-case scenario rather than their best-case scenario. I have priced prospects by assessing the distribution of 18,848 careers and using that distribution to approximate prospect value. This model employs Baseball Reference Play Index WAR data.

Assessing the careers across the history of baseball produces a clear distribution of talent, and also helps to clarify what a player’s ceiling looks like on the field. For example, by the time a batter reaches 1.1 WAR, they are within the top third of all batters in the game. This is helpful to temper expectations of how prospects should produce, and also to understand whether an MLB player is truly elite. Using this career wide scale to assess transactions ensures that analysts can quickly translate the distribution of talent to assess the likelihood of future player production (and therefore the risk of acquiring a player or prospect).

This scheme works across individual seasons, as well, which can be drawn from Baseball Prospectus CSV functions (for example, I found approximately 29,428 individual pitching seasons with recorded WARP, and thousands more with unrecorded WARP, and 95,790 individual batting seasons with recorded WARP, which can be assembled according to mean and standard deviation). Once the mean WARP for pitchers and batters is identified, one can easily scale nearly every player in baseball history according to their percentile on a season-by-season basis:

These scales can be used to approximate Overall Future Potential (OFP), as well, as the distribution between prospect classes can be compared to the distribution between historical seasons. For example, according to the 2013 Baseball Prospectus Top 10 organizational lists, those 300 prospects (and approximately 150 “just interesting” guys) are distributed as follows: 6.7 percent 70 OFP, 27.8 percent 60 OFP, 32.7 percent 50 OFP, and 33.3 percent 45-50 OFP (“just interesting”). In this scenario, 60 and 70 OFP prospects neatly align with the 1+ and 2+ standard deviation historical WARP seasons, while the 50 OFP prospects wind down to the mean WARP or fall just below replacement level on a single season basis. This should align with what one would expect a prospect to produce once they reach the MLB (for example, it would not be surprising if Mauricio Dubon was a player that accumulated between 0.0 and 0.7 WARP on a seasonal basis, while Josh Hader produced 4+ WARP at his best; we could certainly draw such estimates from their tools and scouting profiles).

By identifying mean and standard deviation for individual WARP seasons, one can assess player value in monetary terms based on the progression of each standard deviation:

I believe this is a useful, if crude, system because it seeks to provide meaning to a statement such as, “if Lewis Brinson is a star prospect, he will be likely to produce at least 28.0 WARP in his career;” alternately, one could reasonably expect Brinson to have a 4.0-to-6.0 WARP ceiling should he reach his optimal OFP. I depreciate this historical value in order to express the risk of Brinson reaching that level. Obviously, GM David Stearns and President Jon Daniels did not price out Brinson as a $196 million player (using one free market assessment of the value of WARP); however, depreciating Brinson’s ceiling to accommodate the risk that (1, at that time) Brinson failed to reach the majors and (2, perhaps more plausibly) Brinson plays closer to his floor than his ceiling in the MLB gets Brinson close to Jonathan Lucroy’s value. Placing Overall Future Potential (OFP), Wins Above Replacement (WAR or WARP), and contracts ($$$) on the same scale produces a solid at-a-glance pricing system that allows fans, analysts, and writers to quickly consider risk and reward. A similar price emerges if one moves from a historical career evaluation model to a model that assesses players based on their likely ceiling of seasonal WARP.

One benefit of assessing more than 18,000 baseball careers and scaling those seasons to prospect expectations is that the different parts of these systems speak to each other easily and clearly. We can literally test our assumption that the Lucroy trade was in fact a pretty good deal for both sides on the day of the trade. Obviously, post hoc analysis is necessary each and every year following a trade to test those assumptions. As in Benefit-Cost Analysis, it’s not simply enough to drop things the day of the trade, and adding analysis on an annual basis can help to fine tune assumptions about value, as well (or solidify trade deadline trends). In the case of the Rangers trade for Jonathan Lucroy and Jeremy Jeffress, depreciation analysis shows the rapid decline in surplus value that follows poor production:

Using WARP, OFP, and $$$ to assess trades is inherently problematic insofar as it (a) incorporates biases involving the Replacement Player Model, (b) only assesses players according to marginal value, and (c) assumes that player value can be expressed in one particular figure (be it cash, future potential, or current production). Yet, pushing back on (c), I don’t think it’s entirely problematic to say that an analyst can express player value at one point in time while also understanding how that value can change very quickly, on a seasonal basis, or over the course of a career. Jimmy Nelson is a fine example of this type of issue; the Brewers’ righty struggled with command and mechanical adjustments throughout his first couple of seasons, but working through adjustments has helped him produce notably above average runs prevention in 2017. It’s not wrong to assess Nelson in such a manner now (notably better than average), nor was it wrong to previously assess Nelson (struggling rotation depth). The narrative can connect to the statistics, and one can use a transactional model to assess risk and value in order to judge trades and perhaps understand how value is allocated within a given organization; one could even use such a system to analyze how an organization acquires risk (whether they are risk averse, or neutral, or aggressive).

It should also be clear that players can produce well beyond their OFP. Nolan Arenado is an example of such a player, a 50 OFP top prospect in 2013 who nevertheless entered 2017 with 21.9 WARP over four MLB seasons under his belt. But fans and analysts should be wary of the lesson of Arenado; for one Arenado, the 2013 Top 10 organizational prospects included 35 players with 0.0 or lower career WARP within the class of 50 OFP prospects (the same class as Arenado), and this is prior to considering the forty-five 50 OFP prospects from that class that had yet to reach the MLB (like Tyrone Taylor, for example). Arenado is a valuable lesson about how players can exceed their OFP, but one should understand that developing a single Arenado cost 80 players who have yet to reach the majors or are producing replacement level careers. Incidentally, the 2013 Top 10 prospects rated 50 OFP entered the 2017 season with 190.8 WARP over 287 seasons, which corresponds quite well to the mean seasonal 0.46 WARP produced above.

Both WARP and OFP have their respective imperfections as measurement systems, but their benefits also allow them to serve as solid transactional assessment tools despite their shortcomings. In the case of the Brewers, one can literally price out the value of the club’s extra cash, surplus of prospects, and the depreciated (or maximum) surplus of any intended trade target in order to understand whether a trade is worth the risk. Absent databases full of proprietary scouting, mechanical, and health information, this type of at-a-glance measurement system can approximate transaction prices and help one understand whether teams made an advantageous trade, or simply a good baseball deal.

This isn’t a new phenomenon. In the offseason a couple of trades involving relief pitchers had the internet shaking their heads. Those were the Craig Kimbrel and Ken Giles trades. (Kimbrel was sent from the Red Sox to the Padres for Manuel Margot, Javier Guerra, Logan Allen, and Carlos Asuaje. Giles and Jonathan Arauz were sent from the Phillies to the Astros for Vincent Valasquez, Brett Oberholtzer, Thomas Eshelman, Mark Appel, and Harold Arauz.)

Not only were these trades weird, for some they were unfathomable, especially by the Astros. The Red Sox had just hired Dave Dombrowski, who’s garnered quite the reputation for trading his prospects. Luhnow is the antithesis of that narrative. He’s the young and progressive GM: the one with the huge database, the one who knows that relievers aren’t that valuable. Yet, Luhnow paid a hefty price for Giles.

Some, therefore, started suggesting that there was a divide between the way front office members and we the public value relievers, also noting that WAR may not be the best measurement of the trade. Russell Carleton wrote about this in an article entitled, “The Kimbrel Gambit”. Carleton stated that WPA was a better way to evaluate relief pitchers.

Then the 2016 trade deadline arrived and, as a number of people noted, the reliever market was nearly unbelievable. Deals for Aroldis Chapman, Will Smith, and Andrew Miller were all seen as overpays. I mean, Miller was traded for Clint Frazier, Justus Sheffield, Ben Heller, and J.P. Feyereisen. A huge haul, but perhaps most surprising is that the Indians gave up a bigger hall to get Miller than the one for Lucroy.

Sure is something when teams have to pay more for elite relievers than elite catchers.

— J.P. Breen (@JP_Breen) July 31, 2016

Now, Lucroy ended up vetoing the deal, but it was still surprising and befuddling to see this.

All of this culminated into the idea that there is a disconnect between the way the public and the front office values relief pitchers.

I'm not even really disagreeing that the price for relievers seems really high. I'm just saying it's pretty clear there's a disconnect here.

— Sahadev Sharma (@sahadevsharma) July 29, 2016

But does this make sense? More precisely, does it make sense that we are basically re-thinking the way we evaluate relief pitchers based on trades?

The answer to this is not entirely. Yes, if teams didn’t think relievers had some value, then they probably wouldn’t trade valuable assets for them. WAR, as Carleton demonstrated, is also probably not the best way to evaluate relief pitchers. Each team also has their own analytics department, and it’s very possible that they have different and better metrics to value relievers.

But, the biggest problem is that we are only looking at one element in these trades, and that is the value of the relief pitcher.

The reason, a number of people get befuddled by these trades is because of the return. But, maybe we need to reevaluate the value of the return. In all of these takes and analysis, no one has stopped to ponder on the value of prospects. Maybe it’s not that relievers are netting necessarily a higher return, but that prospects in general aren’t being valued as highly. More precisely, over the past year, teams seem to have been more willing to part with their prospects. It’s possible that teams, in general, are realizing that holding onto prospects is a risky proposition. This can work out very favorably, but can also bite you in the butt especially if the prospects don’t work out.

Let’s use the Red Sox as an example. When Ben Cherington was in charge, the media went after him hard for his unwillingness to part with his prospects. And, in some cases, Cherington was right: just look at Jackie Bradley Jr., Xander Bogaerts, and Mookie Betts. These three prospects swarmed the baseball sphere in trade rumors, but they became very valuable pieces to this year’s team.

Then, however, there’s the other side. The Red Sox held onto prospects such as Deven Marrero, Henry Owens, Brandon Workman, Allen Webster, Garin Cecchini, and more. None of these players worked out, at least not as they’d hoped, and basically went from highly touted prospects to busts who don’t have a lot of value. This is basically the risk.

It highlights that for teams who are looking to acquire big league talent, it’s not necessarily about keeping or trading prospects, but knowing which prospects to trade. Knowing that you should hold onto Betts, Bogaerts, and Bradley, but at the same time, know to trading Webster, Cecchini, Owens etc. Maybe teams are starting to figure that out, which is why we are seeing more prospects being traded.

Changing the way we value relief pitchers based on trades also ignores the market. If we simply assume that since relievers are garnering a greater return than before, then relief pitchers are more valuable than before, then we must assume that Andrew Miller is, in fact, more valuable than Jonathan Lucroy. The Indians traded for both players, and in many scout’s eyes, the return for Miller was better than the one for Lucroy.

But, this ignores the external factors of these deals. The market for relief pitchers seemed absurd because many teams needed pitching. In fact, every contending team needed pitching. That’s the thing about trading pitchers. It’s that no matter the market, teams can always use more pitching because there are twelve pitchers on a team compared to only two catchers. Lucroy might not have provided a big upgrade for some contending teams, but Andrew Miller would have provided a big upgrade for every team. Because at the end of the day, Miller is much better than the twelfth best pitcher on your team. While Lucroy isn’t necessarily that much better than the best catcher on your team.

The Brewers also seemed to have more urgency than the Yankees. Even after the deal fell through Rosenthal wrote, “Oh, he will be traded, most likely to the Rangers, a team that can acquire him without his permission.” The Brewers could have kept Lucroy until the offseason, but that would have hampered his value. The Yankees on the other hand were in no rush to trade Miller. Meaning that they could sit back and wait until a team met their price. The Yankees could be irrational with their demands, while the Brewers had to be more reasonable.

Finally, we assume that the people making these deals are acting like rational beings when in reality emotion and competitiveness play a factor. The deal for Chapman was probably an overpay. But, the Cubs haven’t won the World Series in more than 100 years. No living member of the organization has seen a Cubs World Series and this might be their best chance. The Cubs decided to give up some of the future, future that is unknown, to improve the one spot that needed to be improved, the closer role. Some of the Cubs front office members won’t be there when Torres reaches the majors. Some of them won’t be there next year. The Cubs move, while being an overpay, was done to win now because the Cubs are in a great position to win now, a position that isn’t necessarily going to re-occur.

The same thing can be said for other clubs. These were trades being made by humans, and even though humans are very smart, they are also often driven by emotions.The idea isn’t that we are underselling or overselling relievers. The idea is that coming to that conclusion based on trades is problematic and ignores other factors that could be influencing a team’s decision in making a trade.