Beat the Streak: Day 18 - Looking back at 2024

-

2024 was an exciting season for beat the streak, with several streaks appearing unstoppable and one player even breaking to 50 game barrier.  This year was also quite good for me, and while my longest streak was only 20 games, my model was noticeably better than in prior seasons.  In this blog post, I'll share some high level statistics about my model in the 2024 season. 

Number of Games with At Least One Hit

First we will ignore the model and just look at the outcomes over time.  In the figure below, we plot the % Games with Hit across all starting hitters from the year 2000 - 2025.  We see that offensive production is overall not what it used to be, although there was a nice uptick since the all-time low in 2020 and 2021.  With lower offensive production, it is harder to beat the streak, both because it is harder for an individual batter to get a hit, and because each batter gets less opportunities in a single game.  The uptick is caused by the hitter-friendly rule changes that were introduced for the 2023 season, as well as the crackdown on the use of sticky stuff.

Model Calibration

I have a model that predicts the probability that every starting batter will get a hit in every game. This gives about 41000 predictions across the entire season.  In the figure below, we look at how well-calibrated these predictions are.  That is, when the model predicts a hit probability of p, what percent of games do those batters actually end up getting hits. To estimate this, we sort the predictions across the entire 2024 season according to the predicted probability, and taking a rolling average of the predicted hit probabilities and the binary observed hit outcome across a sliding window of size 1000.  A well-calibrated model would be a diagonal line. shown for reference.  The figure shows that the model is pretty well-calibrated, in particular high predicted-probability regime which is the one we care about.  

Average Cumulative Success

The plot above gives a good overall sense of the model calibration, but since it averages over 1000 data points, it provides limited insight into the the high-probability predictions that we care most about for beat the streak.  In the plot below, we look at the "average cumulative success" of the top-predicted picks.  Specifically, we sort the predictions in descending order, and look at the average predicted probability of the top k picks, as well as the empirical frequency of hits among those picks.  We plot the results below.  The blue line shows what the model predicted, while the orange line shows what actually happened (along with a confidence interval to account for the inherent randomness in the process).  The highest-probability picks actually outperformed expectations, as the orange line exceeds the blue line early in the plot.  Later on, the they slightly under-performed, but was still within the bounds of the confidence interval.  The dashed lines are shown as reference points -- I've been trying to get a model with 80% pick accuracy for quite some time now and it appears I may finally have that.  



Top Picks

The table below shows who top picks were across the whole season.  Note that they were all at Coors field in Colorado, a known hitter-friendly ballpark.  As my streak was getting longer last season, I was patiently waiting for favorable matchups at Coors field.  Overall, if I look at the top two picks per day, the model got 78.9% accuracy.  


The table below shows the number of times each batter was in the top-2 predictions for the day:

Conclusion

Overall, I'm pretty happy with my model performance in 2024, and I'm looking forward to the upcoming 2025 season!


Comments

Post a Comment

Popular posts from this blog

Optimal Strategy for Farkle Dice

-
In this post, I will discuss my findings in terms of an optimal strategy for Farkle , which is a dice game of chance.  If you are unfamiliar with the game, I encourage you to skim the linked article so that you can better understand this blog post.  All of my findings are based on sound mathematical methods and a computer program was used to determine the optimal strategy.  Before I begin, let me first state the value of different dice rolls I assumed while developing the strategy. Each 1: 100 points Each 5: 50 points 3 1's: 1000 points 3 2's: 200 points ... 3 6's: 600 points 4 1's: 2000 points ... 4 6's 1200 points 5 1's 3000 points ... 5 6's 1800 points Straight 1-6 1500 points 3 Pairs (e.g., 22 33 44): 750 points Everything else is considered to be a Farkle (0 points). There are many variants to Farkle, but I like to play by this set of rules. The "game state" of Farkle roll can roughly be characterized by 2 values: the curre...
Read more

Markov Chains and Expected Value

-
A few weeks ago, I was using a Markov Chain as a model for a Project Euler problem, and I learned about how to use the transition matrix to find the expected number of steps to reach a certain state. In this post, I will derive the linear system that will help answer that question, and will work out specific an example using a 1-dimensional random walk as the model. Terminology Before I begin the derivation, let me define some terms. The transition matrix is an \( n \) by \( n \) matrix that describes how you can move from one state to the next. The rows of the transition matrix must always sum to 1. States can either be transient or absorbing . An absorbing state is one that cannot be left once reached (it transitions to itself with probability 1). A transient state is a state that is not an absorbing state. In many problems, it is of general interest to compute the expected number of steps to reach an absorbing state (from some start state). Derivation  Let \...
Read more

Automatically Finding Recurrence Relations from Integer Sequences

-
In a lot of the recreational math and computer science problems I solve, I try to derive a recurrence relation that describes the situation. These recurrence relations pop up pretty often in counting problems (how many ways can you tile a 2xn grid with 2x1 tiles?) In some cases, I am unable to derive a recurrence relation directly from the problem statement, but still suspect that there exists some recurrence relation. In these cases, I end up solving the problem usin g brute force methods for small input sizes and analyzing/extrapolating the pattern. However, when the relation is obscure, it becomes hard to find. In order to make this process easier in the future, I created a program that can automatically find recurrence relations for arbitrary integer sequences. To use it, simply type in the first few terms of a sequence you are interested in, and it will try to find a recurrence relation for that sequence. There are some constraints on the allowable inputs/outputs which I di...
Read more