Conference Proceedings

User Models, Metrics and Measures of Search: A Tutorial on the C/W/L Evaluation Framework

L Azzopardi, A Moffat, P Thomas, G Zuccon

CHIIR 2021 - Proceedings of the 2021 Conference on Human Information Interaction and Retrieval | Published : 2021


Evaluation is central to Information Retrieval, and is how we compare the quality of systems. One important principle of evaluation is that the measured score should reflect the user's experience with the system. Hence, there should be direct connection between how users interact with the system and the characteristics of the metric. In this tutorial we introduce the C/W/L approach to user modeling and show how different user models lead to different metrics. We then describe the recent innovations and approaches to evaluation that it has facilitated. The tutorial is presented as a mix of on-line synchronous lecture, pre-recorded in-depth videos, and hands-on activities using theC/W/L toolki..

View full abstract