Conference Proceedings

A living lab study of query amendment in job search

B Salehi, D Spina, A Moffat, S Sadeghi, F Scholer, T Baldwin, L Cavedon, M Sanderson, W Wong, J Zobel

Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval | ACM | Published : 2018


Errors in formulation of queries made by users can lead to poor search results pages. We performed a living lab study using online A/B testing to measure the degree of improvement achieved with a query amendment technique when applied to a commercial job search engine. Of particular interest in this case study is a clear 'success' signal, namely, the number of job applications lodged by a user as a result of querying the service. A set of 276 queries was identified for amendment in four different categories through the use of word embeddings, with large gains in conversion rates being attained in all four of those categories. Our analysis of query reformulations also provides a better unders..

View full abstract