Crowd Data Analytics and Optimization

Abstract

Crowdsourcing can be defined as outsourcing with crowd, where crowd refers to the online workers who are willing to complete simple tasks for small monetary compensation. The overwhelming reach of internet has enabled us to exploit crowd in an unprecedented way. Crowdsourcing, nowadays, is considered as a tool to solve both simple tasks (such as labeling ground truth, image recognition etc.) and complex tasks (such as collaborative writing, citizen journalism etc.). Furthermore, it is also used to solve computational problems such as Entity Resolution, Top-k, Group-by etc. While crowdsourcing provides us with plenty of opportunities, it also presents us with a plenty of challenges due to the complex interplay between the tasks and the workers.

In this dissertation, we first study the aspect of collaboration on solving complex task with crowdsourcing. Collaborative crowdsourcing acknowledges as one of the most promising areas of next-generation crowdsourcing. It refers to a specific form of human-based computation involving skilled workers forming groups and solving problems that require advanced skills in different domains in a collaborative manner. A number of emerging applications, such as collaborative document editing, sentence translation, and citizen journalism require human workers with complementary skills and expertise to form groups in order to achieve a complex goal. There are several key challenges to overcome in collaborative crowdsourcing- i) Task Assignment in such applications are primarily self-coordinated to date, or sometimes such assignments are performed manually by the domain experts in a task-specific manner without considering the affinity among the workers and ii) lack of principled solutions for estimating human factors such as skill/cost etc. We first initiate the investigation of the task assignment optimization problem for collaborative crowdsourcing and show how to incorporate team-based factors such as affinity and critical mass. Then, we demonstrate the deployment of our task assignment algorithm in a real-time collaborative system named Crowd4u. Finally, we present comprehensive optimization based formulations for estimation of the skill of workers in collaborative systems.

Then we propose a novel technique for task recommendation in crowdsourcing. There are some notable differences between task recommendation and the area of item recommendation. The main challenge here is that the worker does not explicitly says which task she likes (or dislikes). So, we propose several methods for task recommendation, which consider both the implicit signals and task similarity.

Finally, we propose a probabilistic framework for estimating all pair distance of a set of objects with crowdsourcing. This problem is appropriate for several distance-based machine learning applications such as clustering,k-nearest neighbor etc. We divide the overall problem into three smaller problems- i)Aggregate crowd inputs ii) Estimate distance of the unknown object pairs from the known set of distances iii) Find the next best question to be asked to the crowd.