|
Abstract
This document was submitted as a position paper for the Human Computer
Conversation workshop being held in conjunction with the January, 1998
Loebner Competition. Hugh Loebner, the founder of the Loebner Prize
in Artificial Intelligence, has received much crticism for his decision
to extend the "Turing Test" to audio-visual media. This essay suggests
groundrules for this "Grand Prize" which attempt to pacify the critics
by reviving the spirit of Turing's original Game.
The Turing Game and the Loebner Prize
The notion that the "Turing Test" may be expanded to audio-visual
media rests on the mistaken, if widely held, assumption that there
is such a thing as a "Turing Test" in the first place. In Turing's
1950 paper "Can Machines Think" he described an elaborate scenario
that he called the "Imitation Game" involving a man, a woman and third
person who may be of either sex. In his original game, the third
person, who is called an interrogator, communicates with the couple
through a restricted medium much like what we now call text-based chat.
The object of the game is for the interregator to guess the gender of
each of the other players. In order to confuse the interegator, Turing
specified that the man should decieve or lie and the woman should tell
the truth. Now the role of the computer, as Turing wrote, is to replace
the man, i.e. the deceiver, in the game. When a machine can imitate
the man in the game, so that the interregator identifies the computer
as the "man" as often as he would the real man, then that computer is
said to be thinking in a restricted sense.
The Imitation Game by Alan M. Turing, as it originally
appeared in jounral Mind in 1950 (Volume LIX).
One wonders what history diluted the original "Imitation Game" into
the caricature called the "Turing Test" in most engineering schools today [4].
Most authors (and teachers) mistakenly understand the "Test" as
a chat-like scenario involving only two players, in which the
experimenters attempt to replace one of them with a computer. This
description of the Turing Test forms the basis of the Loebner competition
rules today [7]. No attempt has yet been made to play a Turing Game as
Turing originally described it.
The critics [1][2] of the Loebner Contest Grand Prize may have a point when
they claim that the Audio Visual competition is far from what Turing
imagined, but then so is the 2-player "Turing Test" itself. Are we to
conclude as some have that the Loebner contest, and especially its
AV extension, is misdirected, a diversion of attention from the
"real issues" of AI?
An Audio Visual Turing Game
No argument will silence all the critics, but here I'd like to advance
some suggestions for the groundrules of the Grand Prize [3] that preserve the
excitement and novelty of the audio-visual experiment but also
rehabilitates the "Imitation Game" as Turing imagined it.
Today we are immersed in the world of e-mail and chat-line communications
and many people are familiar not only with the basic medium but also of
its many pitfalls: among them that the person on the other end may
not be the gender they claim to be. The extension of the Imitation
Game I propose to audio-visual media looks slightly ahead of today's
(mostly) text-based chat world to one where multimedia communications
permits real-time transmission of at least low grade video imagery.
One might be tempted to think that video-based chat would eliminate the
pitfall of gender deception. But there are any number of video and
graphic "tricks" available to deceive the receiver into thinking that
the image of a particular face goes with a particular gender, when in
fact it might not.
The audio-visual imitation game likewise involves three players: a man,
a woman, and an interregator of either sex. The interregator is
involved in a 3-way audio/video chat with the other two players. For
the contest, we should consider limiting the frame rate and the resolution
of the video image so that absolute synchronization of sound and
video is not necessary. When the computer plays the role of the man,
the interregator will see a video image of a man graphically manipulated
as necessary to produce the illusion of the actual male participant.
No restrictions need be placed on the content of the dialogue or
of the video imagery.
Final Thoughts
Before concluding this essay I would like to add three more remarks
regarding the demographics of the judges, the "no-telecommunications" rule
and a final point about consciousness and AI.
Some controversy has surrounded the selection of the judging panel for
past Loebner contests. Part of the problem is that deciding who is
qualified to judge AI requires some assumptions about the nature of AI
itself. Our experience with the ALICE program is that the client's
experience varies widely according to his background, but because
of the exposure of the program to clients on the web, we have amassed
a remarkable corpus of what people actually try to say to a computer. Our
strategy is to develop ALICE in response to this distribution of
client inputs. As the Web clientele increasingly reflects the demographics
of the general population, we expect that the distribution of
client queries follows a popular conceptualization of what an AI
should be expected to do. It follows that the ALICE program "works better"
for the general population than for select subsets. Judges for the
Imitation Game ought to reflect these general demographics, and for that
reason it seems appropriate that a "jury of peers" include some young and
elderly people, some with little formal education, and few if
any academics.
The "no-telecommunications" rule ought to be abandoned. The emergence
of the World Wide Web makes possible the development of elaborate
client-server programs dispersed over a wide area [8]. In the case of ALICE
for example, the telerobotic "eye" is located in a lab in Pennsylvania;
the audio speech-generation is derived from a Bell Labs site in New
Jersey [6], and the natural language server may reside anywhere at all
on the worldwide Internet. Bringing these components together in one
place is costly and perhaps prohibitively expensive to some would-be
competitors. In place of the rule we ought to consider a contract
requiring prize forfeiture if any "hanky-panky" is discovered. In
practise it seems unlikely that an elaborate hoax would have much
chance of success, especially for the Grand Prize.
Finally, this essay concludes with a brief remark about consciousness
and AI. Many have argued [9] that in order to make progress AI must look
beyond simple "Eliza-like" transformations and toward something else,
usually not well defined, called "real intelligence". If anything the
reductionist approach of ALICE, who has been called "Eliza's Big Sister"
and "Super-Eliza", has shown that much more can yet be mined from this
simplistic approach. ELIZA has been called a "hoax" and perhaps
rightly so, but is the conclusion that she is not "real AI"? This author
at least is now prepared to consider the inescapable alternative
explanation: "Real consciousness" is itself a deception [5]. Turing had
a glimmer of this notion when he envisioned the computer in his
imitation game playing the role of the deceiver. The boundary between
real intelligence and deception has never been less clear.
References
- 1.
[A HREF="http://www.diemme.it/~luigi/talk.html"]
How to Pass the Turing Test by Cheating
[/A]
http://www.diemme.it/~luigi/talk.html
Jason L. Hutchens
- 2.
Lessons from a Restricted Turing Test
http://xxx.lanl.gov/ftp/cmp-lg/papers/9404/9404002.gz
Stuart M. Shieber
LANL Report-no: CRCT TR-19-92
- 3.
[A HREF="http://acm.org/~loebner/In-response.html"]
In Response [/A]
http://acm.org/~loebner/In-response.html
Hugh Loebner
- 4.
[A HREF="http://dragon.acadiau.ca/~001741c/project/turing.html"]
The Turing Test
[/A]
http://dragon.acadiau.ca/~001741c/project/turing.html
Dr. Andre Trudel (course notes)
[Note: this URL has a nice illustration of the "Turing Test" as
it is commonly misconceived.]
- 5.
The Lying Game
http://www.wired.com/wired/5.08/idees_fortes.html
Richard S. Wallace
- 6.
Bell Labs Text to Speech Synthesis
http://www.bell-labs.com/projects/tts/voices.html
John Holmgren and Michael Tanenblatt
- 7.
Home Page of The Loebner Prize--"The First Turing Test"
http://acm.org/~loebner/loebner-prize.htmlx
Hugh Loebner
- 8.
SETL for Data Processing on the Internet
http://cs.nyu.edu/~bacon/survey-pap
David Bacon
- 9. Personal communication, David Powers
|
