Updated April 23, 2005
(Subject to Revision)
Turing's Original Imitation Game was played for the first time ever at Simon's Rock College of Bard in Great Barrington, MA on Saturday, April 16, 2005. The experiment was organized by Cameo Wood, Melissa Leventhal and Allyson Sgro. Dr. Wallace of the A. I. Foundation was invited to speak after the event and to assist with the chat bot technology.
Because of the deceptive nature of the experiment, the experimenters were forced to observe absolute secrecy about the event until now. Today an announcement can finally be made to say that, after much meticulous planning, the world's first Turing Imitation Game, as opposed to the Standard Turing Test, has been carried out. The organizers are still analyzing the data and will publish more news, data, dialogs and eventually a scholarly paper in the weeks and months ahead. This announcement is just an early heads up to say that the experiment was a procedural success.
The Test that Turing describes on the first page of his 1950 paper "Computing Machinery and Intelligence" (http://www.abelard.org/turpap/turpap.htm) is a subtle experiment involving not, at first, an artificial intelligence, but instead three people: an interrogator chatting with a man and a woman online. The interrogator's job is to determine, in the course of a five minute conversation, which of the two chatters is a man and which is a woman. As a twist, Turing recommended as a strategy, the woman should always tell the truth and the man should always lie.
Dr. Wallace has written before ( http://www.alicebot.org/articles/wallace/lying.html ) how this "Original Imitation Game" (OIG) is quite different from the "Standard Turing Test" (STT) taught in computer science courses and popularized by awards like the Loebner Prize. To be fair, the STT also appears in Turing's 1950 paper, as observed by Susan Sterrett in "Turing's Two Tests for Intelligence" . Sterrett also cointed the terms and the acronyms OIG and STT.
In order to construct Turing's experiment in today's world, the organizers first created a web site, http://www.theguessinggame.net, which announced an opportunity to participate in an online gender-guessing game. The participants were asked to chat with two companions over AOL instant messenger for five minutes, and then to guess which was a man and which was a woman. In order to attract these prospective interrogators, the organizers publicized their web site widely in a number of online communities, but specifically avoided any reference to bots, A. I., the Turing Test, or anything else that might give away the deception. Any prospective interrogators who indicated a suspicion or knowledge of Turing Tests were disqualified.
In fact students from the small Simon's Rock campus were already prohibited from participating by the date of the experiment, because word had already leaked into the local community about the upcoming event. Even in the hours up to and including the playing of the game, during the web site publicity phase, the organizers worried nervously that the link would be made to the "Turing Test" and the gender experiment cover would be blown. Such problems point to the difficulty of repeating the OIG in the future.
Local students provided the labor pool of six human confederates, however. Two males played the roles of the liars and four females the truth-tellers. The chat robot ALICE Silver Edition (http://www.alicebot.org/join.html ) played the role of the computer simulating the lying man. More than 30 interrogators took part chatting with pairs of either real humans, or the female confederates paired with the chat robot.
Some pictures from the OIG event are posted at http://www.alicebot.org/oig/. Hugh Loebner wrote an interesting response to the first draft of this article, and we responded by changing some of the original wording. His insightful reply is archived at http://www.alicebot.org/oig/hugh.html.