סמינר מחילקתי

More for Less: Adaptive Labeling Payment for Online Labor Markets”

Dr. Tomer Geva – faculty member, Coller School of Management, Tel Aviv University

Abstract:

Predictive modeling has emerged as integral to the efficient operations and competitive strategies of firms across industries. Because many important predictive tasks require human intelligence to label training instances, online crowdsourcing markets have become a promising platform for large-scale labeling. However, prior research found major quality issues in such markets. In particular, very different tradeoffs arise between payment offered to labelers and the quality of labeling under different settings, and, more broadly, work quality may change over time and with changes in the competitive market settings. Further, determining the effect of labeling quality on the expected improvement in predictive performance is also challenging. Therefore, effective means for dealing with these challenges are essential for a growing reliance on these markets for predictive modelling. In this paper, we propose the new data science problem of Adaptive Labeling Payment (ALP): how to determine and continuously adapt the payment offered to crowd workers, before they undertake a labeling task, so as to produce a given predictive performance cost-effectively. We develop an ALP framework and derive a novel ALP method, which we evaluate extensively over a wide variety of market conditions. We find that our ALP method yields substantial cost savings and robust performance that can be relied on by businesses over a wide variety of settings.

(Joint work with Harel Lustiger and Maytal Saar-Tsechansky)

ההרצאה תתקיים ביום שלישי, 28.03.17 בשעה 14:00 , בחדר 206, בניין וולפסון, הפקולטה להנדסה, אוניברסיטת תל-אביב.