Amazon's Mechanical Turk service has long provided a cheap source of labor, when the job is simple for humans but difficult for computers. Tasks such as describing a picture, for example, can be completed online by remote, human workers. Programmers already use groups of these workers, called turkers, to do many such tasks at the same time. But Mechanical Turk offers no easy way for programmers developing new software applications to combine and coordinate the turkers' efforts. Now computer scientists at MIT have developed a toolkit that does just that. Called TurKit, the tool lets software engineers write algorithms to coordinate online workers using the Javascript programming language, and create powerful applications that have human intelligence built in. The software can also be debugged like normal code.
"Usually in Javascript, you wouldn't be able to access Mechanical Turk without a lot of work," explains Greg Little, a PhD candidate at MIT's Computer Science and Artificial Intelligence Laboratory, who created TurKit. "This is a bridge for writing code that interacts with the workers on Mechanical Turk, so we can easily explore new methods of human computation."
With TurKit, human input is stored in a database. That way, anytime the software under development crashes, the turkers don't have to start over from scratch. Instead, once the program has been fixed, it can pick right up where it left off. "If you wait an hour for the humans to finish their task, and then the program throws an error, you don't want to wait another hour just to see if your bug fix works," says Little. TurKit also prevents the human input from changing unpredictably during the debugging process. "If I got different behavior every time I ran (a program), I could never debug that moving target," says Michael Bernstein, a PhD candidate at MIT, who used TurKit to create a word-processing application called Soylent. ...more
Adding Human Intelligence to Software
-
aarf