Distributed search based on self-indexed compressed text

Diego Arroyuelo, Veronica Gil-Costa, Senén González, Mauricio Marin, Mauricio Oyarzún

Research output: Contribution to journalArticle

9 Citations (Scopus)

Abstract

Query response times within a fraction of a second in Web search engines are feasible due to the use of indexing and caching techniques, which are devised for large text collections partitioned and replicated into a set of distributed-memory processors. This paper proposes an alternative query processing method for this setting, which is based on a combination of self-indexed compressed text and posting lists caching. We show that a text self-index (i.e.; an index that compresses the text and is able to extract arbitrary parts of it) can be competitive with an inverted index if we consider the whole query process, which includes index decompression, ranking and snippet extraction time. The advantage is that within the space of the compressed document collection, one can carry out the posting lists generation, document ranking and snippet extraction. This significantly reduces the total number of processors involved in the solution of queries. Alternatively, for the same amount of hardware, the performance of the proposed strategy is better than that of the classical approach based on treating inverted indexes and corresponding documents as two separate entities in terms of processors and memory space. © 2010 Elsevier Ltd. All rights reserved.
Original languageEnglish
Pages (from-to)819-827
Number of pages9
JournalInformation Processing and Management
DOIs
Publication statusPublished - 1 Sep 2012
Externally publishedYes

Fingerprint Dive into the research topics of 'Distributed search based on self-indexed compressed text'. Together they form a unique fingerprint.

  • Cite this