Roi Solomon-Optimal Strategies for Winning Log-Loss Competitions

Electrical Engineering Systems Seminar

Speaker: Roi Solomon

M.Sc. student under the supervision of Prof. Meir Feder and Dr. Amichai Painsky

Sunday, 18^th February 2024, at 15:00

Room 011, Kitot Building, Faculty of Engineering

Optimal Strategies for Winning Log-Loss Competitions

Abstract

Log-loss is a standard approach for evaluating quality of predictions of binary outcomes, such as sports prediction competitions. It is well known that predicting the true (oracle) outcome probability for each match is optimal for this goal, in the sense that it minimizes the expected log-loss of predictions. However, the goal of participants in prediction competitions (such as those on Kaggle) is often not to optimize the expected score of their submission, but to maximize the probability of winning the competition - that is, attaining lower log-loss than all other competitors. We justify approximating this goal as surpassing the loss of a noisy oracle by a calculable quantity c⋆.
For this goal, we show that the theory of competitive optimality supplies a Markov-inequality bound on the probability of success, and prove that this bound is attainable with simple policies under pretty general conditions. Hence, we can formulate the optimal policy for maximizing win probability, which amounts to selecting an appropriate subset of games and switching their predictions to be zero or one. We demonstrate in simulations and reanalysis of previous competitions that this is an effective practical policy to maximize win probability.

השתתפות בסמינר תיתן קרדיט שמיעה = עפ"י רישום שם מלא + מספר ת.ז. בדף הנוכחות שיועבר באולם במהלך הסמינר