Characterizing and Evaluating Whole Session Interactive Information Retrieval

Prof. Nick Belkin and Prof. Chirag Shah

About the Project
This research addresses a newly important issue in contemporary life. As people become more accustomed to using the Web for finding information, they are increasingly using it for addressing ever more complex and personally important information problems. However, current Web search engines have been developed and specifically tuned to helping people find simple, mostly factual information, usually as a single response list to a single, simple query. But when they try to address the new types of problems, people need to engage in longer information seeking episodes than the one query-one response paradigm assumes. They may also need to engage in many activities other than just clicking on a search result, such as reading, evaluating, comparing and using information. Current Web search engines do not sufficiently support this model of information seeking and use. This research addresses this problem by studying why people engage in such complex information seeking (that is, the reasons that motivate them to do this), and what they try to accomplish during the course of an information seeking episode (their search intentions). The end-goal of this research is to design and evaluate new types of search engines for supporting people in accomplishing the goals that have led them to engage in information seeking. This means, in essence, being able to personalize system support to the individual, and the individual's goals and context. Specifically, this research will establish relationships between people's behaviors during an information seeking episode, the motivating goals that led them to engage in information seeking, and their specific intentions at any point in an information seeking episode. This will enable development of systems that will be able to predict how best to support the individual person in addressing their information problem. For example, the findings from this project could help build a system that automatically identifies that a searcher is shopping for a car, and help him/her compare cost-benefits of new vs. used cars, buying vs. leasing, and eventually making an informed decision. Research will be integrated with educational activities via developing modules to supplement courses in iSchools and library/information science programs, etc. This is important, since a broad range of students would learn about new methods of searching and related user studies and evaluation.
Little is known about the relationships between observable searcher behaviors and the higher-level intentions which a searcher wishes to accomplish at any particular point during an information seeking episode, nor are there good methods or measures for evaluating interactive information retrieval system support for whole information seeking episode evaluation, or for evaluation of support for any particular searcher intention during an information seeking episode. This project addresses the problems of: recognizing searcher intentions during the course of an information seeking episode through observation of searcher behavior; developing methods and measures for evaluation of interactive information retrieval system support for those intentions during the course of an information seeking episode; and, developing measures and methods for evaluating the performance of the interactive information retrieval system in support of the entire information seeking episode. The observed behaviors include, e.g., eye-fixations, mouse movements, clicking, following links, page transitions and query reformulations. The project proceeds in three stages. First, it builds on and extends the results of previous and current research in relating observable searcher behaviors to high-level searcher intentions, with the goal of characterizing and segmenting information seeking episodes according to these behaviors and their associated intentions, and to develop models of information seeking episode behaviors. This is conducted both with previously collected data, and on data derived from a new user studies. Stage 2 collects data on "real" information seeking episodes, through a browser plug-in logging tool, and interviews with participants. These data are analyzed to discover new motivating tasks, to identify new information seeking episode intentions, and to enhance the models of information seeking episode behaviors. Stage 3, including a further study of people conducting complex information seeking tasks, identifies the goals of intentions during the information seeking episode and for the whole information seeking episode, and develops and tests measures of support for those goals, and methods for gathering the data required for applying the measures. Stage 3 also investigates the application of the behavior models for simulation of information seeking episodes. In essence, this project will result in statistical models and algorithms that predict a searcher's intentions, leading to better ways for personalization and recommendations. These outcomes will be novel and significant because they will allow us to address goal/task accomplishment, and not just search improvement, through a user's information seeking episode(s).
Progress and Planned Activities
Summer 2015With newly appointed graduate assistant Matt Mitsui, we are working on designing a new user study to understand people's intentions for different search segments.
Fall 2015Lab study to extract search intentions.