Some of the most exciting applications of data science involve predicting the future: based on the things we know, what is most likely to happen next? The Esports One data science team is taking on that fortune-telling challenge to predict fantasy point scoring and set player salaries, powered at its foundation by a highly customized, Elo-based model.
Setting the salaries for each available player and team is one of the biggest challenges in creating a daily fantasy platform. All of the users playing E1 Fantasy need to feel that the underlying systems are accurately scaling salaries based on how many fantasy points players and teams are likely to score. That places a lot of responsibility on the Esports One data science team to build accurate projection models.
The simplest approach to projecting how many points a player will score is to feed the player’s competitive history into statistical or machine learning algorithms and use those past performances to predict the next performance. The precision of this approach is relatively low, though, and we knew we could do much better.
To improve our projections, the first thing we recognized is that in LoL, players and teams tend to score a lot more points when they win than when they lose—obviously! So we realized that we needed to build our projections on a firm foundation of pre-game win probabilities. We had already built out tech for calculating in-game win probabilities, but pre-game probabilities are an entirely different problem set.
Again, the simplest approach to calculating pre-game win probabilities is to feed teams’ historical results into a set of “naïve” machine learning algorithms and see what comes out the other side. The results were reasonable, but we weren’t happy with “good enough”, so we moved forward with other ideas to see if they could beat the baseline.
Our Elo System
After a lot of exploration, we turned to Elo ratings. If you play competitive online games, you’re probably familiar with Elo, or with similar systems that have spun off from it or iterated on it, sometimes under alternative names like matchmaking rating (MMR). Put simply, an Elo system is used to calculate the relative strength or skill of different competitors, and the gap in ratings between the two competitors can be used to predict the likelihood that each competitor will win. Created several decades ago by Arpad Elo, Elo ratings are relatively simple (by modern data science standards), but they have stood the test of time and continue to be relied on around the sports analytics world, by analytics communities like FiveThirtyEight, among others.
Through weeks of tweaking, iteration, and test runs, we finally arrived at a customized, Elo-based system that considerably outperformed the accuracy and precision of our baseline models. A snapshot of the team ratings for LCS 2019 Summer Split looks like this.
The way we arrived at these ratings is highly custom. The main customization in Esports One’s approach is that, unlike typical team sports Elo systems, our ratings are not attached directly to the teams: instead, our ratings are primarily associated with the players. In other words, the ratings of teams like Team Liquid and Fnatic are based on the combined ratings of their players. When Broxah left Fnatic to join Team Liquid, for example, he carried a high rating from Fnatic’s impressive international results with him, which will bolster Team Liquid’s rating and reduce Fnatic’s going into 2020.
We chose to have our ratings follow the players because of the small size and relatively high volatility of LoL rosters. Since a team has just five players, and those players move between teams quite frequently compared to traditional team sports, it is problematic to allow teams’ ratings to carry over through an offseason. When the ratings move with the players, we get a more reliable portrayal of how the balance of power shifts from split to split.
On top of that crucial adjustment, we also have a variety of smaller customizations that improve the way we calculate both our expected and actual outcomes.
When we’re evaluating a game’s results, we also have custom calculations that affect our K factor (that is the number that establishes how many points are gained and lost after each game). Our base K factor is 20, which we found provided the most accurate overall modeling. But we also have various levers in place that can adjust the K factor for each player/team under different circumstances. For example, brand new players receive higher K values for a certain number of games, so that the system can “calibrate” itself to their true skill level more quickly. We also adjust K values based on margin of victory, with teams receiving more points when they win convincingly and fewer points if they barely eked out the victory. The margin of victory is based on Gold Spent Percentage Difference (GSPD), a statistic I created in 2015. If the winning team had a higher than average GSPD, the K value for that game will be increased in proportion (causing the winning team to take more points of Elo from the losing team), but if the winner’s GSPD was below average, the K is reduced. This allows a team like G2 Esports, for example, to really climb in Elo when they are blowing out all their opponents, which provides a more accurate depiction of their quality relative to the rest of the league.
On the side of expected outcomes, a standard Elo model will simply factor in the ratings of both competitors and use the gap between them to assign a win probability. In our Elo model, we go beyond this with a few additional factors, such as adjusting for which side of the map each team is playing on (when that data is available). Consistently, throughout the history of League of Legends, the blue side of the map has won more often than the red side. As a result, when we know which team is on the blue side, we increase their Elo for that game by 25 points, which is roughly equivalent to four percentage points of win probability.
For all these modifications, we used diagnostic metrics like accuracy, precision, and calibration to measure the effects on the overall strength of the model, and we ultimately selected the customizations that produced the strongest model. We will continue to explore new ideas and implement any that improve the diagnostics!
Better Data Science Means Better Fantasy
The Esports One Elo system is the backbone of our E1 fantasy points projections and salary calculations. Through Elo, we calculate our expectations for which team is favored to win a given game, and we combine that probability with each player’s historical performances in wins and losses to predict how many points they are likely to score against each opponent. Through this system, we can adjust the salaries of each team’s players, providing a reliable, accurate way to balance the game experience for everyone who participates each week.
By applying a combination of cutting-edge machine learning techniques and time-tested classic approaches like Elo ratings, we’re committed to building the most trustworthy, accurate analytics foundations for our fantasy platform as we possibly can.