DIVERSIFYING THE POOL OF EXPERIMENTAL SUBJECTS—THE MECHANICAL TURK.
DIVERSIFYING THE POOL OF EXPERIMENTAL SUBJECTS—THE MECHANICAL TURK. One problem with generalizing—or replicating—results from randomized experiments is that the sample may not be representative. In fact, most experiments are done with a pool of American college students. This article in the Economist (May 26) describes how crowdsourcing, in addition to reducing the costs of experiments, is making pools of subjects somewhat more representative. The article begins with a popular acronym for the usual subjects: Western, Educated, Industrialised, Rich and Democratic (WEIRD). The largest source of subjects is the Mechanical Turk, which is run by Amazon and has over 500,000 participants. One third of “Turkers” are from India versus some 40% from the United States, with some 100 countries being represented. The Turkers still represent a biased pool—they are younger and more liberal than the average person (and, the article points out, are set apart from others by signing up for crowdsourcing).