Conference Proceedings

Supporting Interoperability Between Open-Source Search Engines with the Common Index File Format

Jimmy Lin, Joel Mackenzie, Chris Kamphuis, Craig Macdonald, Antonio Mallia, Michał Siedlaczek, Andrew Trotman, Arjen de Vries

Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval | ACM | Published : 2020

Abstract

There exists a natural tension between encouraging a diverse ecosystem of open-source search engines and supporting fair, replicable comparisons across those systems. To balance these two goals, we examine two approaches to providing interoperability between the inverted indexes of several systems. The first takes advantage of internal abstractions around index structures and building wrappers that allow one system to directly read the indexes of another. The second involves sharing indexes across systems via a data exchange specification that we have developed, called the Common Index File Format (CIFF). We demonstrate the first approach with the Java systems Anserini and Terrier, and the s..

View full abstract

Grants

Awarded by Australian Research Council


Citation metrics