Running Out of Time: The Impact and Value of Flexibility in On-Demand Crowdwork

Artificial Intelligence, Innovation and Economic Growth, Networks, the Internet, and Cloud Computing, Internet and Media and Content

Article Snapshot


Mary L. Gray, Siddharth Suri and Ming Yin


Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’18, New York, NY, April 21-26, 2018


Workers in on-demand digital labor markets often have little control over their schedules. Giving workers more control yields more work product without loss of quality. Workers value control of their pace of work.

Policy Relevance

Allowing workers more time to complete tasks results in completion of more tasks.

Main Points

  • In the past decade, more employment has taken the form of "on-demand" digital online labor, that is, a stream of tasks managed and billed through online platforms like Mechanical Turk (MTurk).
  • Digital on-demand workers may have little flexibility to schedule their own online work.
    • Requesters choose the amount of time the worker has to complete the task before it expires, the "time allotted."
    • Many requesters assign the default time allotted of one hour, or an arbitrary time, without considering how long the job will really take.
  • Studies of traditional workplaces show that increasing a worker's control over her own schedule improves work output and the worker's experience.
  • In this study, MTurk workers were asked to classify consumer reviews as positive or negative; the "time allotted" for workers’ tasks varied from 30 seconds to one hour or one day.
    • Workers accepted and completed more tasks when the time allotted was longer.
    • Workers dwell longer on tasks when more time is allotted, taking breaks or working more slowly.
    • Generally, allotting more time does not decrease worker speed.
  • These changes were caused by changes in the behavior of a small fraction of workers on a small number of tasks; workers who spent more time on MTurk altered their behavior most.
  • Providing an estimate of the time a task will take to complete decreases the number of tasks a worker accepts and does not increase the number the worker completes, but does increase worker accuracy.
  • Workers value the greater control over their own schedules gained by increasing time allotted.
    • Offering 1 cent more per task leads to the completion of more tasks.
    • When the task price is fixed at 4 cents, increasing time allotted from 1 minute to one day lead to completion of more tasks.
    • Workers value the flexibility gained from an increase in time allotted from 1 minute to one day at a rate of about $2.32 per hour.


Get The Article

Find the full article online

Search for Full Article