Tractable Algorithms for Changepoint Detection in Player Performance Metrics
Abstract
We present tractable methods for detecting changes in player performance metrics and apply these methods to Major League Baseball (MLB) batting and pitching data from the 2023 and 2024 seasons. First, we derive principled benchmarks for when performance metrics can be considered statistically reliable, assuming no underlying change, using distributional assumptions and standard concentration inequalities. We then propose a changepoint detection algorithm that combines a likelihood-based approach with split-sample inference to control false positives, using either nonparametric tests or tests appropriate to the underlying data distribution. These tests incorporate a shift parameter, allowing users to specify the minimum magnitude of change to detect. We demonstrate the utility of this approach across several baseball applications: detecting changes in batter plate discipline metrics (e.g., chase and whiff rate), identifying velocity changes in pitcher fastballs, and validating velocity changepoints against a curated ground-truth dataset of pitchers who transitioned from relief to starting roles. Our method flags meaningful changes in 91% of these `ground-truth' cases and reveals that, for some metrics, more than 60% of detected changes occur in-season. While developed for baseball, the proposed framework is broadly applicable to any setting involving monitoring of individual performance over time.