System Evaluation · Controlled Study · Analytics · A/B Testing

TWISL: Pitcher Fatigue Test

A controlled A/B experiment: the full roster card, plus the original analysis explaining why certain outputs performed the way they did..

Row color key

Introduction

Unlike the last post on fatigue, this one is both from a league explicitly testing pitcher fatigue and also features more than one pitcher.

I won’t really be looking at in-game fatigue because overall, it wasn’t much of a factor. I will call out the few cases where in-game fatigue was a factor as a side note, similar to where I’ll call out the one case where appearance fatigue was a factor. Otherwise, the fatigue presented below is strictly pitch-based fatigue.

Aside from showing the overall fatigue from the different fatigue levels and comparing the performances from the fatigue teams to the control teams, I’ll also be answering a few questions that came up during the league about why certain teams were performing certain ways. One of the answers to these questions centers on defense, so I’ve also included a few defensive-oriented metrics to highlight those answers. Another answer centers on managerial settings, so I’ve included metrics to highlight those answers.

There were three owners with both a control and fatigue team. For comparison, each pair is highlighted in matching colors in the tables below to easily see the managerial differences and effects.

League setup

Pitching. All Group A (Control) teams and Group B (Test) teams roster identical pitching staffs with one exception: the Test teams remove 1994 Brett Saberhagen from their roster. This leaves all Control teams with 1,355 IP and all Test teams with 1,100 IP. Control teams must employ a 4-man rotation of Maddux, Saberhagen, Halladay, and Lee; Test teams must employ a 3-man rotation of Maddux, Halladay, and Lee. In order to combat early season fatigue, owners may start whomever they like the first two games, but rotations need to be set after that. For Control teams, autorest target is 90–95%; for Test teams, 70–75%. Any micromanaging or bullpen setup is up to the individual owner — if you want to give a reliever a spot start here and there to keep your guys from fatiguing too quickly, you can do so, just please consider the theme goals as you do.

Roster card for all teams:

RolePlayerTW-L-SIP/162ERAOAVWHIPK/9BB/9HR/9K-BBSalary
SP1996 Greg MadduxR15-11-02452.72.2411.036.321.030.40172-28$8,407,161
SP/RP2011 Roy HalladayR19-6-02342.35.2391.048.471.350.39220-35$7,857,245
SP/RP2010 Cliff LeeL12-9-20002123.18.2401.007.840.760.68185-18$6,812,030
SP/RP1994 Bret SaberhagenR14-4-02552.74.2541.037.260.660.66143-13$7,875,590
SP/RP2011 Doug FisterR8-1-2000701.79.2060.847.290.640.5157-5$3,018,490
SP/RP2015 Alex WilsonR3-3-2002702.19.2381.034.891.410.6438-11$1,866,761
SP/RP2014 Scott AtchisonR6-0-2722.75.2271.036.131.750.5049-14$2,110,181
SP/RP2002 Mariano RiveraR1-4-28472.74.2031.008.022.150.5941-11$1,612,112
SP/RP2017 Sean DoolittleL2-0-24512.81.1850.8610.871.750.8862-10$2,092,862
SP/RP1912 Carl WeilmanL2-4-2001512.79.2270.934.470.560.0024-3$1,995,364
SP/RP2008 Billy WagnerL2000-1-27472.30.1850.899.961.910.7752-10$1,920,067

Offense. This is where it gets really interesting. You get to select whatever offense you want. Test teams will have an additional $7,875,590 due to cutting Saberhagen. The only rule is that you may not add another pitcher to your roster. Improve your offense however you like, but don’t change the pitching staff.

Full team stats — offense

TeamGABRH2B3BHRRBIBBSOHBPSBCSAVGOBPSLGOPSType
Well-rested Whompers162606310811849395372141047276841566333.305.340.488.828Control
TWISL- Fatigue Team1626060991179838648214968345845369143.297.337.482.819Fatigue
TWISL- Control Team1626087112618223182822810914547157528.299.354.473.827Control
Tested Fatigue162588295217043313328892841210123319.290.339.504.843Fatigue
Swing in’ 60’s/I Need A Nap1625864821158631222245799326102238311.270.312.457.769Fatigue
So Controlled162599410991898404321691067391621507141.317.361.479.840Control
Running on Empty162610399618904263113296139863829297.310.354.455.809Fatigue
LIFO control16259938521740459381108282396543643.290.321.435.756Control
Im So Tired162575010111573281263359974391156522411.274.330.506.836Fatigue
Illusion of Control162611610181928363391239753735705014323.315.358.448.806Control
Fatigue is Just a State of Mind1626146113719114163113110974417563211438.311.359.453.812Fatigue
Controlled Fatigue16257741050163834228309102547995345229.284.342.513.855Control
Control vs. Kaos1626145984197838853449452755774510463.322.354.424.778Control
Control Switched On16262081021196437943212990289827353028.316.349.494.843Control
Chicks dig the long ball16258249211570269233569063671249352213.270.315.507.822Fatigue
70% Of the Time, We Pitch Every Time.162615710881913354551871051316804539851.311.348.477.825Fatigue
LEAGUE AVERAGE16260101009179836435206980364828445124.299.342.474.816

Defense

Next, the defense — this will help explain how some of the fatigue teams survived and/or thrived, or how some didn’t:

TeamGPPOAEDPFPCT+PBCSSBACS%PKType
70% Of the Time, We Pitch Every Time.16242821679160293.974157591536168.2145Fatigue
So Controlled16243701558130279.97913126840132.3030Control
Running on Empty16242711688113310.9817441974160.4630Fatigue
LIFO control1624351150785267.9861958130130.2312Control
Im So Tired16242441702144348.97663381062199.3121Fatigue
Illusion of Control1624372151394275.9844257534176.1932Control
Fatigue is Just a State of Mind16243321778159341.9752072163050.6001Fatigue
Controlled Fatigue16243551637122297.9809223654188.2872Control
Control vs. Kaos16243981559135275.97811311938101.3762Control
Control Switched On16244451558120262.9801180948102.4711Control
Chicks dig the long ball16242011738120410.9803695946158.2911Fatigue
Well-rested Whompers16243931599233280.963115285946140.3292Control
TWISL- Fatigue Team16243451645135283.978148322654157.3440Fatigue
TWISL- Control Team16243451567143263.97676653132216.1482Control
Tested Fatigue16242841758129387.97912922552174.2992Fatigue
Swing in’ 60’s/I Need A Nap16241841747175447.971128681354116.4663Fatigue
LEAGUE AVERAGE16243231640137314.97896451446148.3082

Pitching — basic

Finally, the pitching stats — I’ve included both basic and advanced here as the advanced data gets used for some of the other metrics later:

TeamGCGSHOWLSVSVOIPHRERHRBBSOOAVOBPSLGWHIPERA
Well-rested Whompers573111134950611464.331464625485942221002.250.281.3591.152.98
TWISL- Fatigue Team66772936930481448.331560771676203260780.269.303.4301.264.20
TWISL- Control Team568721016138531448.3315787246301042251007.271.302.3961.243.91
Tested Fatigue62720679520431428.001800973887215354682.303.346.4841.515.59
Swing in’ 60’s/I Need A Nap550260151475141394.67279723562122399986405.410.486.6962.7113.69
So Controlled649321125046651456.6713696105211342321000.242.277.3691.103.22
Running on Empty612115710515371423.67193411241023278335696.320.359.5311.596.47
LIFO control515100946847561450.3315846866321282191065.272.301.4031.243.92
Im So Tired64441649831551414.67194611321013251438677.321.369.5171.696.44
Illusion of Control659111045833481457.3315446936221342071064.266.294.4011.203.84
Fatigue is Just a State of Mind84600877534481444.001744964844227331638.293.334.4741.445.26
Controlled Fatigue5161231045834481451.671548647575116189958.267.294.3871.203.56
Control vs. Kaos4711431164646601466.0013895945231292141063.243.274.3711.093.21
Control Switched On71300927033521481.6716307416731472651031.273.307.4101.284.09
Chicks dig the long ball6231401814410221400.33280221602018475782528.408.471.7312.5612.97
70% Of the Time, We Pitch Every Time.765005910322591427.33207313481211263561644.332.389.5311.857.64
LEAGUE AVERAGE62561818131481441.0817981009903206364828.299.342.4741.505.64

Pitching — advanced (full 16 teams)

TeamQSIRIRSIBBHBPWPBKGIDPGB/FBBB/9SO/9SO/BBBFPNP/GNP/PAType
Well-rested Whompers11513832635841001.21.46.24.56199124.93.3Control
TWISL- Fatigue Team7023465038651031.11.64.83.06190124.93.3Fatigue
TWISL- Control Team1041835104674931.21.46.34.56207125.53.3Control
Tested Fatigue753011162521631361.22.24.31.96430129.83.3Fatigue
Swing in’ 60’s/I Need A Nap1535316739731271601.26.42.60.47992165.83.4Fatigue
So Controlled11419258104972981.21.46.24.36011122.63.3Control
Running on Empty67291114041811081.22.14.42.16521131.03.3Fatigue
LIFO control94165513633180951.11.46.64.96140126.03.3Control
Im So Tired762709717441231121.22.35.23.16506132.53.3Fatigue
Illusion of Control4929211743401111231.22.84.31.56620134.83.3Control
Fatigue is Just a State of Mind92265711351051001.21.36.65.16120125.93.3Fatigue
Controlled Fatigue274541421461321231.32.14.01.96455129.23.2Control
Control vs. Kaos104209524451141071.31.25.95.16109123.93.3Control
Control Switched On114883103682961.21.36.55.06033124.33.3Control
Chicks dig the long ball101296995135144921.21.66.33.96354129.63.3Fatigue
70% Of the Time, We Pitch Every Time.1043419436532421441.25.03.40.77754160.13.3Fatigue
LEAGUE AVERAGE6442218846431971061.13.54.11.16967142.13.3

Combined: all three starters, by team

TeamTypeAVG %IPBFPPCWLSVRERHHRSOBBWPERAOAVWHIP
70% Of the Time, We Pitch Every Time.Fatigue73.15896.334033130523745055647411219845518294.76.2941.45
Chicks dig the long ballFatigue49.90825.00466815612810401365126917572862794751013.84.4152.71
Control Switched OnControl100.00759.67324210693523313573238325754312553.83.2671.26
Control vs. KaosControl99.65801.00333311104582503392977917057811633.34.2481.13
Controlled FatigueControl99.83817.33341311156653003362968655352610073.26.2611.18
Fatigue is Just a State of MindFatigue68.18923.34417013539445912637557115514141720285.43.2941.47
Im So TiredFatigue76.95895.6639641295439555545479110812847017834.81.2921.44
Illusion of ControlControl99.19777.67328710916642903803428466255111243.96.2661.23
LIFO ControlControl94.89839.34353911645524003853549167557112453.80.2671.24
Running on EmptyFatigue74.06897.6640011299138690609556115413647515645.57.3001.46
So ControlledControl99.98765.00312910308612012972546945450812242.99.2321.07

Individual pitcher data: 1996 Greg Maddux

TeamTypeAVG %IPBFPPCWLSVRERHHRSOBBWPERAOAVWHIPGG<70/99%AVG G<70%LOW %AVG PC
TWISL - Fatigue TeamFatigue74.59332.331405460621150177153337331596204.14.2511.2054169%69%85.30
Tested FatigueFatigue75.72327.671418451217230189173403351294824.75.2941.3854169%69%83.56
Fatigue is Just a State of MindFatigue66.73325.671437464616225216187386441345325.17.2791.35775761%46%60.34
Im So TiredFatigue77.03322.331398452514231186160371481456124.47.2771.3464761%54%70.70
Running on EmptyFatigue74.21317.001405456614280213195401471325625.54.2971.4452169%69%87.81
70% Of the Time, We Pitch Every Time.Fatigue72.02308.331412453614160200166397301297524.85.2971.53551566%58%82.47
Average Fatigue69.30299.501441475911231263238440501579827.14.3291.80561885.55
Average F%-274.69304.171368449414182188168383351746914.96.2961.48591176.82
Average Control98.58267.71113337992110010897276152095003.26.2561.22411092.38
Average Overall81.75283.601287427916161186167358331837415.31.2961.52481488.45

Individual pitcher data: 2011 Roy Halladay

TeamTypeAVG %IPBFPPCWLSVRERHHRSOBBWPERAOAVWHIPGG<70/99%AVG G<70%LOW %AVG PC
Fatigue is Just a State of MindFatigue70.26314.671436470720195203178397421598725.09.2941.54784664%49%60.35
Tested FatigueFatigue77.79308.331353444315180158138359211786024.03.2781.3653071%83.83
Running on EmptyFatigue75.24307.331372447513210199187399381836105.48.3041.5051169%69%87.75
Swing in’ 60’s/I Need A NapFatigue50.08304.0017525789134050245761685107197313.53.3962.67514243%32%113.51
Im So TiredFatigue78.73302.00133744149153164154368311747314.59.2911.4663468%64%70.06
Normalized to 233.67 IP — RL benchmark: .239 OAV, 2.35 ERA, 1.04 WHIP at 107.34 avg PC
Normalized #257.00102637791960716724291713922.36.2431.0732118.09

Per-team analysis: bullpen and managerial differences

A couple of other fatigue teams dealt with bullpen management issues. One (70% of the Time…) was the other team to have a pitcher appear in more than 70% of team games and who had the second lowest share of pitches by the starters at 50.3%, but was the one that publicly acknowledged they were making sure their starters stayed above 70% to provide valuable data and letting their bullpen take the brunt of fatigue. Their starters slashed 4.76/.294/1.45 and yet their overall slash was 7.64/.332/1.85. Which goes to show how painful their late innings must’ve been to watch.

Another that appeared to also deal with bullpen issues was Tested Fatigue, whose starters alone were almost identical to the one fatigue team that was unarguably successful (TWISL - Fatigue), but their overall slash was 25-50% worse. Tested Fatigue also saw its starters share of pitches among the lowest in the league.

On the flip side, the two teams that spiraled (Swing in’ 60’s/I Need A Nap and Chicks dig the long ball) had the highest share of starter pitches and they were the only two teams that dealt with in-game fatigue, often having their starters throw beyond their allotted pitches per game, which only served to increase the rate of fatigue these pitchers encountered.

With fatigue teams, managing work load across the three fatigue types is also crucial to success, as you can see from the results of the teams that treaded into appearance or in-game fatigue on top of their pitch-based fatigue.

Observations

Two Cliff Lee with least IP combined for 81 G, 476 IP, .271 OAV, 1.2457 WHIP, and 4.56 RA/9. Two Cliff Lee with most IP combined for 104 G, 581.67 IP, .272 OAV, 1.2464 WHIP, and 4.75 RA/9. Average fatigue of two with least IP was 100%, while average fatigue of two with most was 74%. That’s 23 extra games and 105 extra IP with no real drop in performance.

The three sets of teams run by a single owner were each nearly identical on offense despite each fatigue version having a top 3 defense and two control versions having a bottom 3 defense and the third a league average defense. The TWISL Fatigue team was also almost identical to the TWISL Control team on pitching, thanks to that extra defense.

The 85-89% bucket, though the smallest sample, isn’t necessarily an aberration — the better stats there over the 90-94% bucket could be explained by defense in that sample since these buckets aren’t controlled by defense or matchups, etc.

With fatigue teams, managing work load across the three fatigue types is also crucial to success, as you can see from the results of the teams that treaded into appearance or in-game fatigue on top of their pitch-based fatigue.

Even though this is linked in the other thread, this particular point about the multiple types of pitching fatigue and how they all interplay is important to think about in terms of how fatigue impacts performance.

Notes on the data

Couple of notes on the above:

When I first started compiling the data there were two teams that clearly had spiraled out of control fatigue-wise and skewed the overall numbers significantly. So you’ll notice on the individual pitchers sections I have a sub-total for the fatigue teams that does not include those two teams labeled “F%-2”. You’ll notice that for the section that combines all three pitchers together that sub-total is now called “F%-3” because as I dug into the data it became clear that there was in fact a third team that had spiraled into out of control. So I also removed them from this data. It was harder to pick up on the individual pitcher sections because of the more limited data, though it did still stand out as an outlier for reasons I’ll get into in the next post.

The made up metrics to the right of each pitcher’s data are the number of games that pitcher pitched below the defined thresholds of 70% and 99%. That column is labeled “G<70/99%”. For those specific games below the threshold, the average fatigue % is displayed in the column labeled “AVG G<70/99%”. Then the lowest fatigue level that pitcher appeared in a game at is in the column labeled “Low %”. Next we have the more straightforward “AVG PC” which is how many pitches that pitcher threw per game — and I highlighted in red the few cases where in-game fatigue played a role.

Legend / key for advanced columns

Starter % — Percentage of pitches thrown per game by starters. Pitchers/Game — Number of pitchers to appear per team game. Defensive XO — Net outs added by defense (includes errors, +/− plays, CS, and double plays). XO/OPP — Extra outs added by defense per opportunity. OAV-+ — OAV with + plays removed. %>RL# OAV — The increase to OAV over the RL# OAV for these pitchers. %>RL# OAV-+ — The increase to OAV with + plays removed over RL# for these pitchers. %>RL# BB/9 — The increase to BB/9 over the RL# for these pitchers. %>RL# HR/9 — The increase to HR/9 over the RL# for these pitchers.

League questions

Question 1: If defense is as valuable as postulated, why with the league’s best defense did “Fatigue is Just a State of Mind” not do better?

In short, defense saved this team from a complete and total out of control death spiral. This is the third team excluded from the combined stats above because it was severely fatigued and so much so that it was skewing the average fatigue numbers so far offline.

Defining success for fatigue teams

There are multiple ways to define success from the fatigue teams here:

First, is by W/L record, of which only two of the fatigue teams managed better than .500. However, a team’s run scoring capabilities has as much to do with winning as their run prevention, and there were a few fatigue teams that performed well enough on the prevention side to have won more games, but didn’t score enough to turn that performance into a winning record.

Second, is by the overall pitching performance relative to expectations (either RL, RL#, the control teams, or some other standard). By this measure, only one fatigue team appears to have succeeded with this test.

Third, is how just the starting pitchers performed relative to expectations (either RL, RL#, the control teams, or some other standard) given that at least one team let their bullpen go to make sure their starters continued to provide useful data (and from the data, intentional or not, at least two other fatigue teams’ bullpen was their weakness that kept them from winning more games). Again, by this measure, only 1-4 fatigue teams appear to have succeeded with this test (and that depends on your standard above).

Given that, depending on how you define success for a fatigue team here, at best half the teams were successful and at worst, only one was.

That said, there is a clear importance in building a fatigue team with the first goal of winning games in mind to be successful, and that includes building a team that can score enough runs, have a defense oriented to overcoming the effects of fatigue as your pitchers tire, and drafting the right kind of pitchers to begin with. Your ballpark selection also goes a long way towards helping your team.

With 1,100 total innings and WIS Park, the pitchers were the right type — low-mid OAV, low BB, low HR pitchers. Most fatigue teams put their team in front of better than average defenses, as can be seen by the 121 net extra outs from the fatigue teams as opposed to the net 36 extra outs from the control teams.

Conclusion

I think it is still quite clear, despite a lack of overall “success” depending on how you’re defining it from above, that fatigue teams can be used successfully (using the same definition). There is definitely a balance that must be obtained between the different types of fatigue, a defense that helps control for hits (+ plays and lack of − minus plays) and helps minimize run scoring opportunities (CS, few errors, DPs, etc), a ballpark that also helps control for hits (-1B) and HR (-HR). However, if you can do the above, you can win because your pitching will be at the top of the league with average offense, or you’ll have average pitching with an offense near the top of the league.

I included the RL, RL#, and control group as ways to compare performance relative to each other, as well as at this particular cap level. At this cap ($85m) we weren’t so far removed from realistic in that we still had a team have their starters cumulatively exceed their RL and RL# stats. However, we were far enough removed that the average control team was roughly 10% worse on OAV, 27% worse on BB/9, and 94% worse on HR/9 than these same pitchers’ RL#. Comparatively, relative to the control teams, the best fatigue team was within the range of the control teams (16%, 47%, 277%), but as a group (F%-3) the fatigue teams were 12% worse than control on OAV, 27% worse on BB/9, and 148% worse on HR/9. You can see this to an extent on the fatigue groups above, and it helps gauge the linear effect of fatigue on these three stats — and, as covered above, how it can be controlled with defense, ballpark, and managerial settings.

I’d love to do this test again, but with the owners now all having a better understanding of the inter-play at work.