Active Learning and Thompson Sampling
Active learning is an interesting field in machine learning where the data is actively collected rather than passively observed. That is, the learning algorithm has some influence over the data that is collected and used for learning. The best example of this is Google AdWords, which uses active learning to serve ads to people in order to maximize click-through rate, profit, or some other objective function. In this setting, the learning algorithm used by Google can choose which Ad to show me then it observes whether I clicked it or ignored it. The learning algorithm will then update it's beliefs about the world based on this feedback so it can serve more relevant ads in the future. Since the learning algorithm doesn't know which types of ads are most relevant a priori, it must explore the space of ads to determine which ones produce the highest click-through rate. However, it also needs to exploit it's current knowledge to maximize click-through rate. Thus, a goo