System Evaluation · Controlled Study · Analytics · A/B Testing

TWISL: Deadball vs. Modern Pitcher Value Test

A controlled A/B experiment : including the full error-adjusted (OAV+E, WHIP+E, RA/9) comparison that traditional pitching stats miss entirely.

Goals

Here are the things I’d like this league to be able to contribute to:

  1. Increase our understanding of pitcher value — specifically as defense effects overall performance: but more importantly we will look at stats like RA/9 instead of ERA, and we’ll calculate OAV and WHIP with errors counting as hits to get a true feel for how teams are impacted, since the effect of errors doesn’t show up in the traditional pitching stats (ERA/OAV/WHIP).
  2. How many more errors will a team of deadball pitchers allow over a team of modern pitchers?
  3. What effect will the increase of errors (if any) have on team performance?
  4. How big of an advantage (if any) do teams with deadball pitchers have over teams with modern pitchers?
  5. Have fun!

League setup

24 teams (12 owners with 2 teams each). Salary: $80 million · AAA: No · WW: No · Clones: Allowed · Trades: No · DH: No

Team composition. Every owner submits two teams. Offense: both teams must share the same offense — draft whatever hitters you want, but both teams must have identical offenses, with at least 1,000 PA carrying a HR/100# of 5 or greater (to incorporate the HR-suppression value deadball pitchers provide). Pitching: one of your two teams can only draft pitchers from 1888–1919; the other can only draft pitchers from 1985–2016 — keeping people from taking advantage of 2017–2020 not using dynamic pricing and upsetting the value test, while still leaving 31 seasons of pitching options for both rosters.

Ballpark. Any ballpark may be used, but both teams must use the same ballpark.

Alignment. Two groups for the test — Group A (Deadball Pitchers) and Group B (Modern Pitchers), 12 teams each. Each league had 6 of each group, each division had 2 of each group. Each owner had 1 team in each league. Alignment was random and adjusted as needed to meet the conditions above.

Results: traditional stats

First, here are the league-average traditional stats:

MetricDeadballMetricModern
ERA4.58ERA4.76
OAV0.262OAV0.261
WHIP1.51WHIP1.46
HR86HR138
HR/90.53HR/90.86

From this it would appear the deadball pitchers had a slight value advantage. For the same $ spent on pitching staffs, the deadball pitchers had a lower ERA, same OAV, and allowed less HR. Once we add in the effect of errors, though, we have the following:

Results: adjusted stats (including errors)

MetricDeadballMetricModern
E227E163
FLD%0.965FLD%0.974
RA/95.61RA/95.55
OAV+E0.299OAV+E0.288
WHIP+E1.67WHIP+E1.57
W81.42W80.67
W%0.502W%0.498

This result led to the following:

Deadball W: 81.42 · Deadball W%: 0.502
Modern W: 80.67 · Modern W%: 0.498

100+ win teams: 2 modern, 2 deadball. 90–99 win teams: 4 modern, 4 deadball.

Now, there were four teams that dealt with fatigue, two that had severe fatigue and two that had moderate fatigue. As luck would have it, both groups were split deadball/modern and AL/NL, so the overall effect on the league stats was minimal, if it had any impact at all in terms of shifting this. The one aside on that is that of the two severely fatigued teams, the modern team won 9 less games than the deadball version (16 wins and 27 wins). If you balanced out just those severely fatigued teams to 21 each, then the W-L totals shift to:

Deadball W: 80.58 · Deadball W%: 0.497
Modern W: 81.42 · Modern W%: 0.502

Showing how really close this was. Essentially, I think this makes it very clear — there is no value difference between deadball or modern pitchers once you account for errors and XBH/HR rates. The run allowance and W/L records the various pitching eras provide for the price are essentially identical.

The base salary seems to really account for these variances exceptionally well in terms of pricing actual sim value.

For myself, I prefer modern pitchers, as I believe it is easier to build offense/defense/ballpark around them, since you have multiple ways to get their full value — where with deadball pitchers, to get full value, you are limited to a very specific group of defenders, which limits your offense/defense/ballpark options.

Full league data: pitching + defense stats

OwnerTeamGCGSHOWLSVSVOIPHRERHRBBSOOAVWHIPERAPOAEDPFPCTW%OAV+EWHIP+ERA/9HR/9Type
ChargingrynoTWISL: Deadball Fielding Test16220897332421439.001787942761111391378.2971.514.7643171925237446.963.549.3191.685.89.69Deadball
Contrarian23TWISL Old16311917241571450.67159166059950364389.2771.353.724352196484439.987.558.2781.414.09.31Deadball
skinndoggTWISL DOGGS DEADBALLERS ARE ALL DEAD16200758741521455.33156172061134392450.2691.343.7843661903153358.976.463.2821.454.45.21Deadball
jfranco77TWISL Deadballers16230936935491454.00158783063683566454.2701.483.9443621808243441.962.574.2961.655.14.51Deadball
DoctorkzTwisl 1888-1919 Arms162001006241581450.67148268959168416445.2571.313.6743521881188376.971.617.2771.444.27.42Deadball
just4meTWISL - Deadball Pitchers162111867633531446.3317611096819104596390.2891.635.1043391948343509.948.531.3271.876.82.65Deadball
just4meDefense Test: Deadball16210897330481446.33165282369775366294.2801.404.3443391942207453.968.549.3001.545.12.47Deadball
fatguyrdOld Cubs162005111122481427.331667107289681862465.2861.775.6542821879228446.964.315.3071.936.76.51Deadball
teamhydroTalking to Tesla16262907244581461.33144767755931478507.2511.323.4443841833192395.970.556.2721.454.17.19Deadball
schwarzeOld: Full of Krapp & Duryea162851035949591463.67123864453554688589.2231.323.2943911868205429.968.636.2471.463.96.33Deadball
milestDeadball Pitching & Bad Defense @ Target162102713512261408.67217117321420260811495.3402.129.0742261785340509.946.167.3732.3611.071.66Deadball
jtvaugheyOld Cy162103837938521426.00152292368975843560.2631.664.3542781811309344.952.512.3001.885.83.47Deadball
ChargingrynoTWISL: Modern Fielding Test16310927138541465.671774961843168400706.2921.485.1843971781172376.973.564.3071.605.901.03Modern
Contrarian23TWISL New16231837932461449.001719759711107307864.2911.404.424347172773264.988.512.2921.454.71.66Modern
skinndoggDoggs Modern Day Hurlers16242679533441438.671503821733155512894.2631.404.5943161650118295.981.414.2731.485.14.97Modern
jfranco77TWISL Modern-ish16210976547641437.33136168256797378658.2421.213.5543121739176344.972.599.2631.334.27.61Modern
DoctorkzTwisl 1985-2016 Arms16292897339671461.331454714607108465737.2541.313.7443841739137342.978.549.2661.414.40.67Modern
just4meTWISL - Modern Pitchers162111026044651448.331565747617127375726.2671.343.8343451713195344.969.630.2881.474.64.79Modern
just4meDefense Test: Modern162321006243661453.671498680555109300821.2581.243.4443611697164350.974.617.2761.354.21.67Modern
fatguyrdYoung Yu162305710527481427.33174511741015176838874.2951.816.4042821708174375.972.352.3091.937.401.11Modern
teamhydroDriving a Tesla16222946849561452.67135963654991462827.2421.253.4043581668123344.980.580.2541.343.94.56Modern
schwarzeNew: My Johnson Needs Orel16200966643591462.331254715603136595942.2251.263.7143871694167378.973.593.2451.384.40.84Modern
milestModern Pitching & Bad Defense at Target16211161467301388.332358202216972491006558.3632.4211.0041651716290374.953.099.3892.6313.111.61Modern
jtvaugheyYoung Wakefield16200758742621430.3314717716621375581032.2581.424.1742911568172313.971.463.2771.544.85.86Modern
Deadball Totals1945431297796841860217329.3319466108088813102667735416.2621.5144.577519882254727295145.965.502.2991.675.61.53Deadball Total
Modern Totals1945281196897744466117314.9919061106829159166061969639.2611.4594.761519452040019614099.974.498.2881.575.55.86Modern Total
Difference−15−1−992659−14.34−405−126346634−5774223−.001−.055.184−43−2147−768.009−.005−.011−.10−.06.33Modern−Deadball

Full league data: sorted by owner

OwnerTeamOAVWHIPERAEFPCTW%OAV+EWHIP+ERA/9HR/9Type
ChargingrynoTWISL: Deadball Fielding Test.2971.514.76237.963.549.3191.685.89.69Deadball
TWISL: Modern Fielding Test.2921.485.18172.973.564.3071.605.901.03Modern
Contrarian23TWISL Old.2771.353.7284.987.558.2781.414.09.31Deadball
TWISL New.2911.404.4273.988.512.2921.454.71.66Modern
skinndoggTWISL DOGGS DEADBALLERS ARE ALL DEAD.2691.343.78153.976.463.2821.454.45.21Deadball
Doggs Modern Day Hurlers.2631.404.59118.981.414.2731.485.14.97Modern
jfranco77TWISL Deadballers.2701.483.94243.962.574.2961.655.14.51Deadball
TWISL Modern-ish.2421.213.55176.972.599.2631.334.27.61Modern
DoctorkzTwisl 1888-1919 Arms.2571.313.67188.971.617.2771.444.27.42Deadball
Twisl 1985-2016 Arms.2541.313.74137.978.549.2661.414.40.67Modern
just4meTWISL - Deadball Pitchers.2891.635.10343.948.531.3271.876.82.65Deadball
Defense Test: Deadball.2801.404.34207.968.549.3001.545.12.47Deadball
Defense Test: Modern.2581.243.44164.974.617.2761.354.21.67Modern
TWISL - Modern Pitchers.2671.343.83195.969.630.2881.474.64.79Modern
fatguyrdOld Cubs.2861.775.65228.964.315.3071.936.76.51Deadball
Young Yu.2951.816.40174.972.352.3091.937.401.11Modern
teamhydroTalking to Tesla.2511.323.44192.970.556.2721.454.17.19Deadball
Driving a Tesla.2421.253.40123.980.580.2541.343.94.56Modern
schwarzeOld: Full of Krapp & Duryea.2231.323.29205.968.636.2471.463.96.33Deadball
New: My Johnson Needs Orel.2251.263.71167.973.593.2451.384.40.84Modern
milestDeadball Pitching & Bad Defense @ Target.3402.129.07340.946.167.3732.3611.071.66Deadball
Modern Pitching & Bad Defense at Target.3632.4211.00290.953.099.3892.6313.111.61Modern
jtvaugheyOld Cy.2631.664.35309.952.512.3001.885.83.47Deadball
Young Wakefield.2581.424.17172.971.463.2771.544.85.86Modern