Iterative Learning for K-Approval Votes in Crowdsourcing Systems release_3hvhsxrky5cynnkt4hgek5jgu4

by Joonyoung Kim, Donghyeon Lee, Kyomin Jung

Published in Applied Sciences by MDPI AG.

2021   Volume 11, p630

Abstract

Crowdsourcing systems have emerged as cornerstones to collect large amounts of qualified data in various human-powered problems with a relatively low budget. In eliciting the wisdom of crowds, many web-based crowdsourcing platforms have encouraged workers to select top-K alternatives rather than just one choice, which is called "K-approval voting". This kind of setting has the advantage of inducing workers to make fewer mistakes when they respond to target tasks. However, there is not much work on inferring the correct answer from crowd-sourced data via a K-approval voting. In this paper, we propose a novel and efficient iterative algorithm to infer correct answers for a K-approval voting, which can be directly applied to real-world crowdsourcing systems. We analyze the average performance of our algorithm, and prove the theoretical error bound that decays exponentially in terms of the quality of workers and the number of queries. Through extensive experiments including the mixed case with various types of tasks, we show that our algorithm outperforms Expectation and Maximization (EM) and existing baseline algorithms.
In application/xml+jats format

Archived Files and Locations

application/pdf  2.8 MB
file_a43kqgty7vhypbczzy477efvye
res.mdpi.com (publisher)
web.archive.org (webarchive)
Read Archived PDF
Preserved and Accessible
Type  article-journal
Stage   published
Date   2021-01-11
Language   en ?
Container Metadata
Open Access Publication
In DOAJ
In ISSN ROAD
In Keepers Registry
ISSN-L:  2076-3417
Work Entity
access all versions, variants, and formats of this works (eg, pre-prints)
Catalog Record
Revision: 850c0f37-591e-47ec-97ba-7cfaa3e6d45f
API URL: JSON