Note: The Tool is Online Here
I have been reading a bit recently about so-called Bayesian Bandits, as referred to by Ted Dunning. This problem involves the challenge of picking a strategy of playing N slot-machines some number of times, where the probability of winning for each slot machine is unknown. This problem has a number of interesting applications in display advertising, news article recommendation, and click-through-rate prediction (Agrawal and Goyal, 2012). As noted by Dunning, any solution must effectively handle the explore/exploit trade-off challenge. The implementation in this case - using Thompson sampling - is straightforward, and I can see how it could smoothly allow for quick updating as more data/information becomes available.
