CAN A science be based on winning a parlour game? It is a curious-sounding game, that on the face of it involves nothing more than a man and a woman sending messages on computer screens to mislead a judge into thinking they are not the sex they appear to be.
Clearly the answer is yes, if some recent critics of artificial intelligence are to be believed. Only now some AI researchers are rebelling. They want to stop playing the game, which was bequeathed to them by the founder of their discipline as the premise for assessing machine intelligence. But will they ever be able to shake off the founderâs influence?
It was in 1950 that British mathematician and computer pioneer Alan Turing published a paper entitled âComputing Machinery and Intelligenceâ in the philosophical journal Mind with a description of his âimitation gameâ. Turing chose Mind because then, even when there were only a handful of computers on the planet, he foresaw that the machines would be dogged by the question of whether or not they were capable of âintelligence.â
Advertisement
Turing dreamt up the imitation game as a way to head off such debate. The game consists of a number of written conversations between man, woman and judge (sex unspecified). The manâs goal is to pretend to be a woman, the womanâs to resist being thought of as a man; and the judgeâs to decide which is the woman.
If a computer, said Turing, could take the place of the man, and convince the judge that it is a woman (not a machine) 70 per cent of the time, that would constitute a working definition of the computer being able to âthinkâ. Although some sources suggest that the âgenderâ issue was a red herring, the basic principle of a computer being able to convince a human of its humanity remains the same. Or as AI pioneer John McCarthy of Stanford University says, âTuring regarded it [this deception] as a sufficient condition. In arguing with philosophers who claim a machine canât be intelligent, he could say âDo you mean that if its behaviour couldnât be distinguished from that of a human, you still wouldnât regard it as intelligent? Then we have nothing left to discussâ.â
Sceptical philosophers have had a lot to discuss in the intervening forty years. AI has acquired a reputation as a science based on empty boasts and capable of only weak experimental results. Frederick Allen, writing in the American magazine The Atlantic last year, spoke for many critics when he claimed that: âToday, traditional AI is a backwater at best, and the confidence with which it was once pursued seems unimaginable. Nobody has ever designed a program that can converse at all convincingly on a single subject, and the field has splintered into disparate parts ⊠The grand vision has nearly vanished.â The LA Times columnist Michael Schrage cruelly jibed recently that thanks to âan impressive array of pompously wrong predictions, Americaâs artificial intelligentsia has a hard-earned reputation as the arrogant booboisie of bit-twiddlersâ. In some computer science departments, they joke that âIf it works, itâs no longer AIâ, adds Schrage.
As funding threatens to dry up, some AI researchers are now trying to reinvigorate the field by reassessing Turingâs strategy for silencing those troublesome philosophers. âWe must explicitly reject the Turing test [as the game is now called] in order to find a more mature description of our goals -it is time to move it from the textbooks to the history books,â say Pat Hayes of the University of Illinoisâs Beckman Institute, and Kenneth Ford of the University of West Florida.
Mechanical transvestite
Hayes and Ford used the 14th joint conference on AI, held in Montreal last year, to slam the test as âharmfulâ to AI, âdamaging its public reputation and its own intellectual coherenceâ. They allege that Turingâs experimental design is an attempt to confirm the ânull hypothesisâ â that is, trying to find nothing.
They also express concern about the emphasis on gender in the original game. A clever judge would be looking out for signs of femininity in the answers of the contestants. Women, for example, tend to be more specific about colours, distinguishing scarlet and crimson, while men tend to simply say red. âThe gender test is not a test of making an artificial human but a mechanical transvestite,â snipes Hayes.
A program playing the Turing test, says Hayes, is not so much trying to display comprehension of language, as wield it with a ridiculous level of sophistication. âIt would need more than ordinary conversational abilities, it would have to be an expert on making a good impression. It would have to be clever, to lie, to cheat, and dissemble. So we must not make an artificial intelligence but a mechanical trickster â an artificial con-artist.â
A way to judge the basis of Hayes and Fordâs dismay is to look at how todayâs computers are trying to pass the test. In 1991, a New York-based computer consultant called Hugh Loebner provided the financial backing for competitions bearing his name in which human judges would sit and type at computer terminals with unseen computer and human entrants. The aim in the first three such competitions, held in Boston, was to âfoolâ 10 human judges in the course of three hours of typed interchanges about which was human and which not.
Loebner promised $100 000 for a program capable of doing this, but the entrants were so weak that it never looked for a moment as if he would have to get out his chequebook. Instead, in past years a $2000 prize has been awarded for the program judged to be âmost humanâ. Most of these have succeeded by being programmed to stick to defined topics of conversation such as the Rolling Stones, sex, pets, or the O. J. Simpson trial â and even then they donât do very well:
Topic: Classic Star Trek
Judge: Who are the Klingons?
Program: I donât know who youâre talking about.
Judge: Youâre the trekker.
Program: Harry Mudd was one of the more comical characters.
Judge: Harry Mudd? Who was Harry Mudd?
Program: No I havenât.
Judge: Me neither.
Program: Okay, I will.
Judge: Why did you choose to converse about Star Trek?
Program: Their minds were stored in these white globes.
Judge: Whose minds?
Program: Spockâs ears are pointed and there was this strange hippy guy in âThe way to Edenâ that had these bizarre pretzel-shaped ears.
Judge: So how big were the ears?
Program: There must have been 70 or 80. Iâm not sure.
Judge: Ears?
Program: Iâm sure I would know if I could see it on TV.
Often the best results in Loebner competitions are achieved by tricks such as deliberately mistyping answers, creating the illusion of âlisteningâ by repeating snatches of a userâs input back (like the famous 1960sâ AI therapist program Eliza â see âOh look, heâs brought me a presentâ, 51¶ŻÂț, 16 September 1995), steering the conversation the computerâs way when it is in trouble by introducing controversial statements or throwing in humour to seem more human. As one team from the Centre for Machine Translation at Carnegie Mellon University in Pittsburgh, Pennsylvania, admitted in a background document on its system: âHere we unashamedly describe some of the better tricks, confident in the belief that when someday a computer program does pass the test, it will use many of themâ.
Ironically, one contestantâs efforts were ârejectedâ as not human because its answers on literature were too detailed. The female Shakespeare scholar in question was dismayed. As Hayes quips: âThe ability to produce paragraphs of well-written English is plainly now considered an inhuman ability.â
The Loebner test may be a bit of fun, but researchers who have invested a large part of their lives in search of an intelligent machine appear to fair no better. âExpert systemsâ â AI programs that try to consolidate enough expertise on a defined area of knowledge, such as areas of medicine or process control â are incapable of any interaction beyond their tightly-defined domains. Even the best ones, such as MYCIN, an expert system programmed to answer questions about eye diseases, do not know what time is, and are much less able to negotiate their way in the real world than infants.
Hayes and Ford believe that these limitations, along with the well-publicised failings of the Loebner competition have contributed to giving AI a bad name. âLast year, a couple in a bar asked me what I did for a living. With some trepidation, I replied that I worked in AI. âAI,â said the lady, âI think Iâve read about that. They tried it, didnât they, last year up in Boston?â âYes,â said her husband. âBut I thought that the machine failed the test. So why are you still working on it?'â jokes Hayes.
Why indeed? And it is not just remarks in bars that are forcing researchers to reassess their work. One of AIâs most bitter critics is John Searle, professor of philosophy at the University of California at Berkeley. Searle scored a near-mortal blow to AI in 1980 with his insistence in a paper called âMinds, Brains and Programsâ that at best all it would ever achieve is a superficial likeness of intelligent behaviour.
Probably his most famous stab at AI is the Chinese room argument. Searle contends that machine intelligence is actually like passing English symbols into a sealed chamber. A person is hiding inside the room who simply looks up the given character in a translation table, then passes out a Chinese hexagram equivalent. In other words, computers do not understand what they are passing around, even if from outside it looks as if what emanates from the Chinese room is a perfectly intelligent translation.
In this light, even if a computer successfully played the imitation game, Searle would still say that it did not display intelligence. âFormal symbol-manipulations [of the kind computers do] by themselves donât have any intentionality, they are quite meaningless,â he says. âThey have only a syntax but no semantics. Such intentionality as computers appear to have is solely in the minds of those who program them and those who use them, those who send in the input and those who interpret the output.â
But many computer scientists resist Searleâs attack. They are driven by a grand vision of building intelligent devices. Even critics of the Turing test such as Hayes and Ford are still in search of the intelligent machine. But how will they know when they have developed one? For starters, they will not be creating an alternative Turing test. âHaving such a neatly-defined aim is the sign of an immature discipline. What is the Turing test for civil engineering or physics?â
Five hundred years ago, alchemists sought to pass their own Turing test by searching for the philosopherâs stone, a magical stone or substance that could transmute base metal into gold. âBut they only became chemists after they stopped looking for that stone. As a call to arms, Turingâs paper was a magical success. But that was 1950. Itâs time to put alchemy behind us,â says Hayes.
The historical analogy is chosen with care. Hayes and Ford think AI must be freed from the obligation to copy nature before it can find its true path. âNatural systems â products of evolution in a complex world â tend to be very intricate, ad hoc devices. We need to know what weâre looking for when we examine them, and direct imitation isnât a good way to discover what it is weâre looking for,â says Hayes.
People consider a transatlantic trip on Concorde âflyingâ, but birds canât fly at Mach 3 nor can Jumbo jets catch salmon in a stream or land in trees. Thus âartificial flightâ and âartificial intelligenceâ are both misnomers, says Hayes. AI should not be about copying human intelligence (as the Turing test implies) but more about cognition in terms of computation. That means understanding that which can be deemed intelligent â whether it is exhibited by ants, humans or machines. In other words, the critics see AI transmuting into the science of computation, rather as alchemy became the science of chemistry.
âAs we develop a general science of computation, it is the aspects of thought which are ânotâ distinctively human that seem the most fundamental,â says Ford. So linguists, psychologists, cognitive scientists and some philosophers are beginning to move toward the position that being human is not a sine qua non of intelligence, and that there are core computational functions that are useful â being able to tell red from blue, for example, or provide accurate information on what happens if a patient takes two different kinds of drug â and which can be reproduced by writing programs.
There are many AI scientists who reject Searle, but who also disagree with Hayes and Ford in their condemnation of the Turing test. Danny Bobrow, developer of some pioneering AI systems and now a researcher at Xerox PARC in Palo Alto, argues that, âthe Turing test is a systems test â AI programs arenât really worth it of they canât all be put togetherâ. It is all very well building systems that can cope with aspects of intelligent behaviour, such as understanding speech or discovering mathematical theorems, but we will only have intelligence as we understand it when we can put it all together.
Bobrow uses the old Indian philosopherâs joke, in which three blind wise men each grab disparate parts of something and fail to synthesise their findings. âYou donât know if youâve got a snake, a tree trunk and a big wall or if youâve in fact got something that proves to be an elephant.â So unless we synthesise AIâs findings into a suite of programs that could pass the test, we donât know if weâve actually cracked the problem â so weâre back to Turing and his test.
Intelligent species
Others feel that Hayes and Ford are wrong to blame the test for all AIâs ills. âI donât think that even Turing regarded the imitation game as a definition,â says McCarthy. âHis other writings on machine intelligence donât even mention it. Anyway, I never regarded the imitation game as a useful goal.â
Still others want to stick with the test, confident that it is the benchmark of what an artificial â and they think âhigherâ â intelligence will achieve. One of the scientists behind the Loebner competition is Robert Epstein, of the Cambridge Center for Behavioral Studies in California, who dismisses Hayesâs and Fordâs objections as âvery primitive,â claiming, âthe major problem is taking Turing too literally. He never pretended to give the details of a practical testâ.
Epstein is an AI visionary of the old school. âI suspect that much more is coming â that the winning program will be the first member of a new intelligent species that will populate our planet virtually overnight, by means of replicating over the Internet,â he says. âWeâre going to have to adapt, just as we would if intelligent aliens settled here.â
Such optimism â or daftness â is a poor ad for what AI researchers have achieved in problem solving, natural speech understanding, vision, robotics and expert systems. Yet it may be the curse of AI that the Turing test still defines what it is they can and cannot do.
A fitting footnote to the controversies over what the Turing test is and isnât is the fact that the Loebner prize has been cancelled. The prize committee is now set to launch an alternative competition which will be designed in âthe spirit of Turingâs original proposalâ.
The controversy over what exactly that is, and what we should make of it fifty years on, is set to continue. In the meantime, Al in the sense of the science fiction authorâs dream of creating an artificial intelligence that will be a kind of mechanical sibling to us, seems as far away as ever.
