Dynamic Trade-Off Prediction in Multi-Stage Retrieval Systems release_cwjhgbvynvaptetvvbiwqwr2z4

by J. Shane Culpepper and Charles L. A. Clarke and Jimmy Lin

Released as a article .

2016  

Abstract

Modern multi-stage retrieval systems are comprised of a candidate generation stage followed by one or more reranking stages. In such an architecture, the quality of the final ranked list may not be sensitive to the quality of initial candidate pool, especially in terms of early precision. This provides several opportunities to increase retrieval efficiency without significantly sacrificing effectiveness. In this paper, we explore a new approach to dynamically predicting two different parameters in the candidate generation stage which can directly affect the overall efficiency and effectiveness of the entire system. Previous work exploring this tradeoff has focused on global parameter settings that apply to all queries, even though optimal settings vary across queries. In contrast, we propose a technique which makes a parameter prediction that maximizes efficiency within a effectiveness envelope on a per query basis, using only static pre-retrieval features. The query-specific tradeoff point between effectiveness and efficiency is decided using a classifier cascade that weighs possible efficiency gains against effectiveness losses over a range of possible parameter cutoffs to make the prediction. The interesting twist in our new approach is to train classifiers without requiring explicit relevance judgments. We show that our framework is generalizable by applying it to two different retrieval parameters - selecting k in common top-k query retrieval algorithms, and setting a quality threshold, ρ, for score-at-a-time approximate query evaluation algorithms. Experimental results show that substantial efficiency gains are achievable depending on the dynamic parameter choice. In addition, our framework provides a versatile tool that can be used to estimate the effectiveness-efficiency tradeoffs that are possible before selecting and tuning algorithms to make machine learned predictions.
In text/plain format

Archived Files and Locations

application/pdf  545.0 kB
file_2fespqobrndd5ibza6cyadpdfe
arxiv.org (repository)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article
Stage   submitted
Date   2016-10-08
Version   v1
Language   en ?
arXiv  1610.02502v1
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: b492683a-bd9e-478c-9e9d-be83163a8724
API URL: JSON