up vote 6 down vote favorite
5
share [g+] share [fb]

I'm a big fan of mechanical turk and often leverage it with smartsheet. I've used mturk for basic writing, gathering content from multiple sources, and some minimal research. However, I don't think I've truly tapped into its potential.

How do you use Mechanical Turk to crowdsource effectively? What are some of the biggest problems or challenges you have solved with Mechanical Turk? Or are there failures you'd like to share?

link|improve this question
75% accept rate
feedback

4 Answers

up vote 2 down vote accepted

Andy Baio has done a bunch of interesting stuff w/ MTurk: http://www.google.com/search?q=site%3Awaxy.org+mechanical+turk

See also: http://stackoverflow.com/questions/193426/have-you-made-interesting-use-of-mechanical-turk

link|improve this answer
Thanks for the great links! – dlynton Oct 16 '09 at 18:28
feedback
up vote 2 down vote

I don't have the rep to add this as a comment. What is a mechanical turk?

link|improve this answer
mturk.com/mturk/welcome – Alexandre Gomes Oct 15 '09 at 23:02
Thanks for the link. That looks pretty cool! – Dennis Palmer Oct 15 '09 at 23:06
upvoted for rep so that you can just comment – Tony Oct 17 '09 at 0:56
feedback
up vote 2 down vote

Check out TurkForce. Isaac Nichols was part of the original team at Amazon developing Mechanical Turk and now helps companies configure services that use it. Plus, he's a great guy.

link|improve this answer
1  
Thanks Hoyt. The "typical tasks" page is particularly useful. TurkForce looks like a good company to consult on any high level mturk project. Their workforce trust graph looks interesting. I didn't realize new workers automatically get a 100% approval rate! – dlynton Oct 16 '09 at 18:31
feedback
up vote 2 down vote

You should look into Crowdflower. They automatically determine the accuracy of workers and correct for bad workers, something that has always been difficult with Mechanical Turks.

You should also see the empirical evaluation of Mechanical Turk called "Cheap and Fast - But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks" by my colleague Rion Snow. There is a paper and easier to read blog post.

link|improve this answer
Thanks Joseph! Great suggestions. I've never heard of crowdspring, but it looks like an excellent resource for graphic design work. The quality appears a lot higher than 99designs. – dlynton Oct 16 '09 at 22:30
Gah, my mistake, I meant Crowdflower, not Crowdspring. Check out Crowdflower. [editing original answer] – Joseph Turian Oct 17 '09 at 8:12
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.