Conference Proceedings

A living lab study of query amendment in job search

B Salehi, D Spina, A Moffat, S Sadeghi, F Scholer, T Baldwin, L Cavedon, M Sanderson, W Wong, J Zobel

The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval | ACM | Published : 2018

Abstract

Errors in formulation of queries made by users can lead to poor search results pages. We performed a living lab study using online A/B testing to measure the degree of improvement achieved with a query amendment technique when applied to a commercial job search engine. Of particular interest in this case study is a clear 'success' signal, namely, the number of job applications lodged by a user as a result of querying the service. A set of 276 queries was identified for amendment in four different categories through the use of word embeddings, with large gains in conversion rates being attained in all four of those categories. Our analysis of query reformulations also provides a better unders..

View full abstract

Grants

Awarded by Australian Research Council


Funding Acknowledgements

This research was partially supported by Australian Research Council Project LP150100252 and SEEK Ltd.