Eliciting structured knowledge from situated crowd markets

Abstract

We present a crowdsourcing methodology to elicit highly structured knowledge for arbitrary questions. The method elicits potential answers ("options"), criteria against which those options should be evaluated, and a ranking of the top "options." Our study shows that situated crowdsourcing markets can reliably elicit/moderate knowledge to generate a ranking of options based on different criteria that correlate with established online platforms. Our evaluation also shows that local crowds can generate knowledge that is missing from online platforms and on how a local crowd perceives a certain issue. Finally, we discuss the benefits and challenges of eliciting structured knowledge from local cr..