Evaluating Exporatory Search Systems
 Special Issue of Information Processing and Management

  Guest Editors
  Accepted Articles
  Important Dates
  SIGIR Workshop
Guest Editors

Dr. Ryen W. White, Microsoft Research
Dr. Gary Marchionini, University of North Carolina
Dr. Gheorghe Muresan, Microsoft Live Search

Accepted Articles

Evaluating Exploratory Search Systems: Introduction
Ryen W. White, Gary Marchionini, and Gheorghe Muresan

Contextual Factors Affecting the Utility of Surrogates Within Exploratory Search
Ian Ruthven, Mark Baillie, Leif Azzopardi, Ralf Beirig, Emma Nicol, Simon Sweeney, and Murat Yakici

Users Can Change Their Web Search Tactics: Design Guidelines for Categorized Overviews
Bill Kules and Ben Shneiderman

Evaluating WordBars in Exploratory Web Search Scenarios
Orland Hoeber and Xue Dong Yang

An Evaluation of Adaptive Filtering in the Context of Realistic Task-Based Information Exploration
Daqing He, Peter Brusilovsky, Jaewook Ahn, Jonathan Grady, Rosta Farzan, Yefei Peng, Yiming Yang, and Monica Rogati

Model-Driven Formative Evaluation of Exploratory Search: A Study under a Sensemaking Framework
Yan Qu and George Furnas

Important Dates

Submission of articles: Friday, 22nd December 2006
Notification to authors: Monday, 26th March 2007
Final version of accepted papers: Monday, 11th June 2007


Online search has become an increasingly important part of the everyday lives of most computer users. Search engines, bibliographic databases, and digital libraries provide adequate support for those whose information needs are well-defined. However, there are research and development opportunities to improve current search systems to help users succeed more often in situations when: they lack the knowledge or contextual awareness to formulate queries, they must navigate complex information spaces, the search task requires browsing and exploration, or system indexing of available information is inadequate.

In those situations, people usually submit a tentative query and start exploring the information space, selectively seeking and passively obtaining cues about where their next steps lie, i.e., they are engaged in an "exploratory search." Exploratory search is a type of information-seeking that in some respects can be seen as a specialization of information exploration ? a broader class of activities where new information is sought in a defined conceptual area; exploratory data analysis is another example of an information exploration activity. Exploratory Search Systems (ESS) have been developed to support serendipity, learning, and investigation, and generally allow users to browse available information.

Whilst search systems are expanding beyond supporting simple lookup into supporting complex information-seeking behaviors, there are no recognized methodologies for how to evaluate this new genre of search system. This special topic issue will solicit submissions from researchers in communities such as information retrieval, library and information sciences, and human-computer interaction for a discussion of the issues related to the formative and summative evaluation of ESS. The exploratory search focus in recent years has been on the development of new systems and interfaces, not on how to evaluate them. Given the range of technologies available, it is now time to shift the focus of research toward understanding the behaviors and preferences of searchers engaged in exploratory searching, on tasks supported by such systems, and on measuring exploration success.


The special topic issue solicits high-quality manuscripts that focus on the theoretical, empirical and methodological issues surrounding the evaluation of systems to support exploratory search activities. We encourage submissions based on, but not limited to, the following topics:

  • learning
  • system log analysis
  • task-oriented evaluation
  • ethnography and field studies
  • user performance and behaviors
  • searcher simulations
  • biometric data as evidence
  • role of context
  • metrics for ESS evaluation
  • ESS evaluation frameworks
  • mental models for exploratory search processes
  • test collections
  • novel exploratory search interfaces and interaction paradigms


Papers can be up to 12,000 words in length, and should be submitted to the online site for Information Processing and Management (http://ees.elsevier.com/ipm), with the submission box checked off that it is for the special issue on Evaluation of Exploratory Search Systems. Instructions on formatting can also be found at this site. All questions regarding submissions should be directed to the special topic issue Guest Editors at ipm-eess@sakai.rutgers.edu.

 Created and maintained by Ryen White Last modified: September 4, 2007