Win-Win Search: Dual-Agent Stochastic Game In Session Search

Session search is a complex search task that involves multiple search iterations triggered by query reformulations. We observe a Markov chain in session search: user's judgment of retrieved documents in the previous search iteration affects user's actions in the next iteration. We thus propose to model session search as a dual-agent stochastic game: the user agent and the search engine agent work together to jointly maximize their long term rewards. The framework, which we term "win-win search", is based on Partially Observable Markov Decision Process. We mathematically model dynamics in session search, including decision states, query changes, clicks, and rewards, as a cooperative game between the user and the search engine. The experiments on TREC 2012 and 2013 Session datasets show a statistically significant improvement over the state-of-the-art interactive search and session search algorithms.