Leveraging crowd skills and consensus for collaborative web-resource labeling

In this paper, we propose a three-stage approach called CLabel for enforcing collaborative web-resource labeling in form of a crowdsourcing process. In CLabel, the results of both crowdsourcing and automated tasks are combined into a coherent process flow. CLabel leverages on crowd preferences and consensus, for capturing the different interpretations that can be associated with a considered web resource in form of different candidate labels and for selecting the most agreed candidate(s) as the final result. CLabel succeeds to be particularly appropriate for application to labeling problems and scenarios where human feelings and preferences are decisive to select the answers (i.e., labels) supported by the majority of the crowd. Moreover, CLabel succeeds in providing label variety when multiple labels are required for a suitable resource annotation, thus avoiding duplicate or repetitive labels.