2013 Top 10 Lists 1: Value and Risk

Lately, two separate research questions have converged in my mind:

• How do Overall Future Potential (OFP) valuations based on historical MLB performances compare to valuations drawn from actual scouting rankings?
• What is the short-term performance value that the Brewers can come to expect from their 2017 Baseball Prospectus Top 10?

These are two crucial questions. The first question is crucial because it is worth determining the percentage of elite players that were elite prospects, and the percentage of elite players that have come from less stellar OFP backgrounds. If a 50 OFP prospect can become a 20.0 WARP player with regularity, that essentially turns prospect rankings on their heads. In this regard, the focus would be placed on player development departments targeting particular tools that they can polish into MLB productivity, regardless of OFP. If the opposite is true, that elite MLB players are elite OFP prospects, then there is some merit to the idea that player development departments should indeed focus on stockpiling as many 60-70 OFP prospects as possible, perhaps even regardless of risk. Obviously I am writing in generalities here, as player development departments will fight to acquire as many potentially elite players as possible and fight for every 1.0 WARP out of the late round and non-elite prospects as well. It matters if the Brewers can translate Corey Ray’s tools into an MLB performance that approximates his OFP value and that they produce Jacob Barneses along the way.

The second question matters because there is some time-linear thinking about rebuilding cycles that I do not believe is fully warranted. Ideally, the Brewers (or any other club, for that matter) tear down their MLB rosters when they reach the end of a competitive cycle, acquire and develop as much talent as possible, and then that talent is graduated to the MLB to produce wins. This is an obviously appealing line of thinking because there is a certain type of order to this; rebuilding has a certain type of religiosity that is necessary to function, because if fans don’t believe that the rebuild will produce winners due to rebuilding efforts, then MLB teams will produce a cynical line of thinking that severs their relationship between fanbases and front offices during rebuilds. The Brewers need their fans to believe that Lewis Brinson, Josh Hader, and Corey Ray will win games in Milwaukee, because it’s hard to sell “the future” as “check out these future trade pieces and player development misses.” But this latter potential is something that analysts must take seriously, because player development is not a linear process (cf. Jake Odorizzi or Junior Guerra or Carlos Gomez or Scooter Gennett or Tyler Thornburg or Orlando Arcia, etc.).The ability of Milwaukee to achieve any sort of linearity with their rebuilding-to-contending cycle will depend on how their prospects perform immediately upon reaching the MLB.

If the Brewers are to be competitive in 2018 and 2019, and contend in 2020 and 2021, they will need to accurately assess the odds of return elite value for their elite prospects at the MLB level (via development or trade), and they will need to determine the likely production of early career prospects as they gain their sealegs and establish (or fail to establish) MLB roles.

To address these questions, I assessed the 2013 Baseball Prospectus organizational Top 10 lists, as 2017 marks the fifth season since those rankings. I will publish my findings in thee features: first, on risk and value, then on organizational performance, and finally, on individual players. This is an interesting class to assess because Baseball Prospectus began the shift to an OFP ranking system, rather than a “Four Star / Three Star / Two Star / etc.” assessment system in 2013. There are significant differences between the 2013 and 2017 approach to OFP, so this study should be taken with a grain of salt, as the prospect team defined roles differently and did not grade potential MLB floor as clearly as the current (especially 2017 rankings) did. It is a fair argument to state that BP has shifted to a more nuanced approach of prospect ranking by balancing floor and ceiling, as that approach provides a broader range of assessments (compare Josh Hader’s 55-60 to Lewis Brinson’s 55-70, or Brett Phillips’s 45-55). Using this comparison, the 2013 prospect class is more or less divided into three tiers, where the 70 OFP highlights prospects land within the top percentage of minor leaguers, 60 OFP outlines the top two percent of minor leaguers, and 50 OFP prospects round out the top five percent (roughly).

It must be stressed that OFP itself is an imperfect measure. First, effectively using the measure requires analysts to balance a player’s “ceiling” (best possible outcome) and “floor” (realistic minimum MLB outcome). But these aspects are not necessarily the same for any two prospects; certainly Lewis Brinson has a different risk equation between his ceiling and floor than Brett Phillips, as one example (and I mean that without picking on Phillips). The distance between ceilings and floors can also be significantly different, as some players quickly morph into their developmental floors (consider the Brewers’ treatment of Michael Reed’s role after his 2015 breakout) while others are given every chance possible to reach their ceiling (Wily Peralta might be an interesting example here).

Furthermore, there is something of a disconnect between MLB value and prospect value. MLB players do not have static careers, and some players can indeed make adjustments to become fantastic players, as classic non-prospects like Khris Davis and Corey Kluber show (in fact, the non-prospect Khrush has produced more MLB WARP than the entire 2013 Brewers Top 10 thus far). In terms of WARP, Davis now solidly approaching the 90th percentile of all MLB position players, and Kluber is nearing the 97th percentile of all MLB pitchers despite the fact that neither cracked a Baseball Prospectus Top 10. Even ranked prospects can surpass their projected ceilings, as 2013 rankers such as Nolan Arenado and Jake Odorizzi show; making adjustments at the MLB level, or learning a new pitch, etc., can wildly realign OFP.

With caveats in mind, the 2013 Baseball Prospectus Top 10 lists quite effectively separated talent into three distinct categories. Taking the macro-view and setting aside intriguing hits and misses on an individual prospects, the systemwide summary clearly shows that the OFP rankings rather accurately assessed talent level (in terms of translation into WARP) and risk (in terms of prospects reaching the MLB level).

Here’s how the 2013 prospect grades have fared entering the 2017 season (WARP totals and “seasons played” totals are through 2016):

Whatever the shortcomings of BP’s first foray into OFP ranking systems, their rankings were perfectly set up not to miss the cream-of-the-crop, even if 60 OFP and 50 OFP prospects are not as clearly differentiated. Yet, why should 50 and 60 prospects be as clearly differentiated as 70 OFP prospects from everyone else? If I said, “Lewis Brinson and Mauricio Dubon have close odds to produce at the same level upon entering the MLB,” that would be much more difficult to believe and substantiate than if I said “Mauricio Dubon and Lucas Erceg have close odds to produce at the same level upon entering the MLB.” Neither statement may be true in the strictest sense, but it is easier to believe that early career 60 OFP prospects could struggle to find MLB roles as much as early career 50 OFP roles, while 70 OFP prospects more easily step into the limelight.

Yet, the differences in MLB graduations and WARP per season should not be undersold, even between early 50 and 60 OFP prospects from the 2013 rankings class. There is quite a significant difference between a near-70 percent and near-75 percent MLB graduation rate for a particular talent class, especially when that early career MLB pay off is 0.5 WARP higher for the more-likely MLB graduates. That difference arguably reduces the early career value of a 50 OFP prospect from the 2013 class to 63 percent of the early career 60 OFP class. Again, MLB teams are fighting for each step to the big leagues; the difference between graduating an additional prospect or not, and potentially grabbing an additional win, is not insignificant even beyond a “mere” marginal roster analysis.

I followed this investigation by comparing three classes of 2013 prospects (70, 60, and 50) with historical value rankings from my previous Benefit-Cost Analysis model. In terms of reaching their potential career value, 50 OFP prospects are much closer to their expected career outcomes early in their careers than either 60 OFP or 70 OFP prospects. This should not be surprising, as one would reasonably expect the higher ranked prospects to have better odds at sustaining MLB careers beyond their first few years:

Comparing this 2013 OFP survey to my historical OFP valuation, it is interesting to see that 50 OFP prospects may be less valuable than their early career counterparts, but they also potentially return much of their career value earlier than the other prospect classes. In contrast, 60 OFP and 70 OFP prospects are substantially more valuable than 50 OFP prospects (it’s not really that close), but there is considerably more risk that 60 or 70 grade prospects reach their fullest possible MLB career (in terms of sustained starting roles or all-star production). From the 2013 class, Carlos Martinez and Jake Odorizzi, or Wily Peralta and Archie Bradley, might be intriguing comparisons for further investigation into career progression, risk, and prospect grade. MLB teams can use these weights to determine the best combination between trading prospects and developing prospects while attempting to build competitive or contending clubs.

Based on these estimates from the 2013 prospect class, it is possible to estimate a production range for the 2017 Brewers Top 10 prospects between 2017 and 2020. This is a highly speculative exercise, but also one that places the potential impact of a top 10 class in perspective:

22.6 WARP over the course of four seasons is obviously not enough to justify rebuilding, and should provide a good justification for using every available avenue of talent acquisition (in the next installment, it will be shown that even a great prospect class will struggle to produce even 40.0 WARP within their first four seasons). This should also show the importance of sustaining multiple strong prospect classes in consecutive seasons, as an MLB club cannot typically build around one prospect class. Since it takes quite some time for prospects to reach their OFP in terms of MLB production, if they reach it at all, MLB teams understandably cash out prospects for established MLB wins when they have a chance to compete or contend. In Milwaukee’s case, the WARP production of the 2017 Top 10 may be higher due to the number of prospects adjacent to the MLB, but then again, that’s no sure bet for MLB production as the 2013 list shows. A rebuilding effort must include other significant areas of roster construction in order to produce an MLB winner.