Dr. Mark Humphrys

School of Computing. Dublin City University.

Home      Blog      Teaching      Research      Contact

Search:

CA216      CA249      CA318

CA400      CA651      CA668


Mark Humphrys - Research - The Turing Test - Book chapter


How my program passed the Turing Test


Mark Humphrys
School of Computing, Dublin City University, Ireland
computing.dcu.ie/~humphrys

Ch.15 of Parsing the Turing Test, Robert Epstein, Gary Roberts and Grace Beber (eds.), Springer, 2008.


Abstract

In 1989, the author put an Eliza-like chatbot on the Internet. The conversations this program had can be seen - depending on how one defines the rules (and how seriously one takes the idea of the test itself) - as a passing of the Turing Test. This is the first time this event has been properly written up.

This chatbot succeeded due to profanity, relentless aggression, prurient queries about the user, and implying that they were a liar when they responsed. The element of surprise was also crucial. Most chatbots exist in an environment where people expect to find some bots among the humans. Not this one. What was also novel was the online element. This was certainly one of the first AI programs online. It seems to have been the first (a) AI real-time chat program, which (b) had the element of surprise, and (c) was on the Internet.

We conclude with some speculation that the future of all of AI is on the Internet, and a description of the "World-Wide-Mind" project that aims to bring this about.

Keywords - chatbot, Turing Test, Eliza, Internet, chat, BITNET, CHATDISC.



Introduction

In 1989, the author put a chatbot on the Internet, whose conversations can be seen, depending on our definitions, as having "passed the Turing Test". For reasons which will be explained below, this is the first time this event has been properly written up. This paper is an explanation of a historical event, but it has implications for the future of Turing Test experiments on the Internet, and indeed for the future of AI in general on the Internet.

The AI program

In 1987, when I was an undergraduate in Computer Science at University College Dublin (UCD), Ireland, I wrote in LISP a version of Weizenbaum's classic "Eliza" chat program [17]. Eliza "simulates" (or perhaps parodies) a Rogerian psychotherapist (i.e. a practitioner of the non-directive therapy of Carl Rogers), who has a conversation with a patient by appearing sympathetic, asking bland questions, and asking the patient to clarify what he just said, or discuss how he feels about it. This means the therapist does not have to engage in any detail with the actual content of the patient's problems. The patient does all the work. This is obviously ideal for a computer program, which can attempt to carry on a conversation without having to understand anything the human says at all. Weizenbaum's trick remains one of the classic tricks for building a chatbot.

The original Eliza was meant to be sympathetic to the human. I thought it would be interesting to add aggression and profanity to the program. My program was deliberately designed to have an unpredictable (and slightly scary) mood. It is hard to convey the personality of the program in this limited space here. For the cumulative effect, the reader should read through the LISP source code, which is visible at [4], from which the following examples come. As one example, in reply to the common use of "OK" the machine would deny that everything was OK:

          ( (equal (car input) 'ok )
            '(what do you mean ok its not ok at all)
          )

As another example, in response to any innocuous statement beginning "You are .." the machine would reply with one of these:

(putd ' youareresponses 'expr '(lambda nil
  '(
     (i am not (y) you insulting person)
     (yes i was (y) once )
     (ok so im (y) so what is it a crime)
     (i know i am (y) dont rub it in)
     (i am glad i am (y) )
     (sing if youre glad to be (y) sing if youre happy that way hey)
     (so you think i am (y) well i honestly could not care less )
    )
))

where   (y)   is the old Eliza trick of just repeating whatever the user had typed after "You are". The machine is pure stimulus-response, replying to the present message, with absolutely no memory. The reader will see that the technology is all pretty much standard Eliza pattern matching. The novel aspect is the design of the personality and responses, and of course, from the technical viewpoint, putting it online. Other innocuous statements would trigger random aggression:

    (ah type something interesting or shut up)

disinterest (this in response to "I was .."):

    (shut up you boaster)

and prurience:

    (are you lonely very often )

    (has anyone ever loved you)

    (ok honestly when was the last time you got laid)

The last one was very successful with the right personality type, as was this response to any use of bad language:

    (you only use foul language to make up for your small penis)

which (hopefully) would only inflame the situation. The answer to the prurient questions is just as likely to trigger more aggression:

    (what do you know about love anyway githead)

disbelief:

          ( (equal (car input) 'yes )
            '(i dont believe it)
          )

and unbelievable rudeness - when talking about death, for no reason the machine could suddenly reply:

    (i wish you were dead)

Throughout, the machine would produce random friendly ..

    (you are obviously an interesting person)

.. and patronising replies:

          ( (equal (car input) 'hello )
            '(hello little man how are you)
          )

while every sentence beginning "I love .." would trigger:

    '(ahh thats nice)

no matter what was being loved. These responses gave the impression of a person blowing hot and cold, with unpredictable and sometimes violent emotions. Clearly, much of this was waiting for someone with the right personality to come along - and, crucially, someone who did not know they were talking to a machine.

Fellow students talked to the program, and while its responses did prove more popular than the banal responses of the normal therapist, all of this took place with no element of surprise. The students knew they were talking to a program.

The program, called simply "Doctor", was offline (i.e. not on the Internet) though it was on a multi-user machine. It ran on a VAX/VMS mainframe (to be precise, the machine:   ccvax.ucd.ie)   that Computer Science students and staff at UCD shared. To talk to my program, the students would specify the path of the program and run it. Clearly, this program would not perform to its best under these circumstances. Even if my acquaintances could be surprised, i.e. expected to talk to me and not a program, they knew me, and so would know these were not my replies. To really test the program, it was necessary (a) to introduce surprise, and (b) for it to talk to strangers.


The AI program online

In 1989, in the final year of my undergraduate course, I put this program online, and now for the first time it had the element of surprise, and it could talk to strangers long-distance.

CHATDISC server-side programs on BITNET

In 1989, UCD was operating both "BITNET" and "Internet" machines. It was on the "BITNET" machine that I put the program online. The "Internet" at this time was the name used to refer to machines that used the emerging TCP/IP protocol, which were only some of the machines on what was a vast interconnected network of networks. BITNET, for those who do not remember, connected universities around the world, had international email, file transfer and talk messages, had online file archives that you accessed remotely, and was the birthplace of multi-user RELAY chat (the ancestor of IRC) and also of LISTSERV mailing lists. Some might say it was not part of the Internet because it did not use TCP/IP. But it would be fairer to say that it, plus all the other networks it connected to, was the 1980s Internet, which has since standardised on TCP/IP (otherwise, for example, we have to say that the 1970s Arpanet was not the Internet either, since it did not use IP). For more on this discussion of terminology see [5].

In any case, even in the pedantic sense the program was on the "Internet" as well since the BITNET machine at UCD (a VM/CMS mainframe, the node IRLEARN) exchanged email, files and chat/talk messages with the TCP/IP machine at UCD (the VAX/VMS mainframe that we encountered above) and actually had its own Internet address (the VAX machine also had a BITNET address). To summarise, at UCD we had:

  1. The "Internet" machine - VAX/VMS mainframe, the node CCVAX on BITNET, the address   ccvax.ucd.ie   on the Internet.
  2. The "BITNET" machine - VM/CMS mainframe, the node IRLEARN on BITNET, the address   irlearn.ucd.ie   on the Internet. In fact later this same "BITNET" machine was directly accessible through a web browser from the Internet at:   gopher://irlearn.ucd.ie/

VM/CMS had an interesting feature where you could "disconnect" - which meant you logged out, but could leave a program running to process incoming email, files and, crucially, chat/talk messages. The program CHATDISC EXEC, by Eric Thomas of the Ecole Centrale de Paris and of CERN, the inventor of LISTSERV [15], was in popular use as a disconnected "answer machine". Talk messages sent to you when you disconnected leaving CHATDISC running were saved in log files, and automatic replies could be sent back such as "Back in 2 hours". Different replies could be sent depending on who had sent you a message. Indeed, because CHATDISC was open-source (a REXX EXEC script for VM/CMS) it occurred to me that any program at all could be called at this point. CHATDISC allowed one essentially write "CGI scripts" for BITNET (arbitrary server-side programs that could be run remotely when you were not there).

MGonz

The next step was for CHATDISC to call "Doctor" to reply to incoming talk messages. "Doctor" was renamed "MGonz" (for an explanation of the name see [4]). The modified CHATDISC was called "MGonzNet". Here is an extract of the REXX EXEC source code of my customised CHATDISC program. The full source code is visible at [4]. When a message comes in, make a copy of the LISP source:
 Address 'CMS' 'COPYFILE MGONZ SOURCE A MGONZ LSP A (REPLACE'
Add a line defining the current input string, and asking LISP to generate the response from this. Run the file:
 arg   = removebadchars(message)    /* bad chars will crash LISP    */
 arg   = strip(arg)
 arg   = left(arg,50)    /* else lisp truncates, thus losing brackets */
 'EXECIO 1 DISKW MGONZ LSP A (FINIS STRING (response 'quote'(' arg '))'
 Address 'CMS' 'LISP MGONZ LSP'
Parse the output file. Find the reply line. Send it back:
 'EXECIO 1 DISKR MGONZ LISTING A 441'
 'EXECIO * DISKR MGONZ LISTING A (LOCATE /VALUE IS/'
 'EXECIO 1 DISKR MGONZ LISTING A (LIFO'
 parse pull line
 line = lowcase(line)     /* MGONZ LISTING will be all upper case */
 line = strip(line)
 line = strip(line,'L','(')
 line = strip(line,'T',')')       /* discard enclosing brackets */
 replymsg = '* 'line
 Call Tellback replymsg 
For technical reasons, MGonz's replies all begin with   *.  

This went online in February 1989, and friends of mine amused themselves talking to it, but most had advance warning that it was a program, so the conversations are always limited by that:

From FRIEND1: get lost you nosey schweinhunt pig dog
* have you ever had sex with someone who really loved you
From FRIEND1: no
* ah go on say yes
From FRIEND1: sorry, no
* by the way is there any medical reason for your sexual impotence
From FRIEND1: this program is very good mark you rat head

All that was necessary now was for a stranger to send me a talk message when I was not there. A number of friends and strangers hit it without warning but backed off immediately, these from March-April 1989:

From SOMEONE at IRUCCIBM: noel kerins told me to send you a message!          
* you are obviously an asshole                                                 
From SOMEONE at IRUCCIBM: so here it is!                                      
* by the way is there any medical reason for your sexual impotence             

            
From FRIEND2: Hello, Marcus maximus.
* hello little man how are you
From FRIEND2: I am not a little man. but if you must know,
             I'm very well.
From FRIEND2: Where are you anyway .. not the SCIENCE-block I hope?

What we were waiting for was someone who would not back off.


The DRAKE conversation

At this time, I was a regular on RELAY chat, talking to strangers from round the world in the kind of chaotic milieu that is still much the same today on IRC. On 2nd May 1989, a user from the node DRAKE (Drake University, Iowa) sent me a message. I never did find out who he was. Perhaps someone I met on RELAY, or a friend of someone I met on RELAY. He stayed talking to my program for 1 ½ hours, getting increasingly defensive and furious. This conversation can be seen, depending on how one defines the rules (and how seriously one takes the idea of the test itself), as a passing of the Turing Test.

The conversation, available in full at [4], begins haphazardly. Soon, the user starts to get frustrated:

From SOMEONE at DRAKE: what is it with this type something interesting or shut   
                       up business?                                              
* what are you talking about                                                     
From SOMEONE at DRAKE: i keep getting a message that says. "* AH TYPE            
                       SOMETHING INTERESTING OR                                  
* you are obviously an asshole                                                   
From SOMEONE at DRAKE: shut up"                                                  
* what are you talking about                                                     
From SOMEONE at DRAKE: what is this? first it says "YOU ARE OBVIOUSLY AN         
                       ASSHOLE" then                                             
* what are you talking about                                                     
From SOMEONE at DRAKE: you say " WHAT ARE YOU TALKING ABOUT"     

To explain the absurd repetitiveness of my sayings, he constructs a fantastic theory that I might have "hot-keys" attached to particular phrases:

From SOMEONE at DRAKE: are there two people on your username?                    

 ...

From SOMEONE at DRAKE: do you have all these silly sayings on some sort of       
                       control?                                                  
* ok thats it im not talking to you any more                                     
From SOMEONE at DRAKE: you sound like a goddamn robot that repeats everything.   

 ...

From SOMEONE at DRAKE: o my he actually typed something.                         
* ah type something interesting or shut up                                       
From SOMEONE at DRAKE: you type slow is that why you have all those stupid       
                       sayings on command  like that last one?                   

but never quite makes the mental jump to realise that there is no one at home at all.

The conversation really gets going when MGonz (rather by accident) questions his manhood. The "When was the last time you had sex?" question works brilliantly with this user, especially when his answer is not believed. He actually admits under repeated "questioning" (by a program that has no idea what it is doing) that he had lied, or at least exaggerated:

* when was the last time you had sex                                             
From SOMEONE at DRAKE: yesterday.                                                

 ...

* ok honestly when was the last time you got laid                                
From SOMEONE at DRAKE: i told you it was yesterday.                              
* you are obviously an asshole            

 ...

* ok honestly when was the last time you got laid                                
From SOMEONE at DRAKE: ok ok it was over 24 hours ago.  for you it must have     
                       been 20 years   
* what do you mean ok its not ok at all                                                         

My response to his bad language also works brilliantly:

From SOMEONE at DRAKE: is that fucking rude enough for you?                      
* you only use foul language to make up for your small penis                     
From SOMEONE at DRAKE: why not?                                                  
* ah get lost go to the bar or something                                         
From SOMEONE at DRAKE: i just told you the night before last. and the night      
                       before that too.                                          
* go away fool and stop annoying me                                              
From SOMEONE at DRAKE: well i am not going to tell you about the size just       
                       that i have                                               
* what do you know about love anyway githead                                     
From SOMEONE at DRAKE: never had any complaints about my penis and that it       
                       gets used quite                                           
* ok straight out are you a virgin                                               
From SOMEONE at DRAKE: often.  and i think that you are a virgin and have no     
                       hope of getting                                           
* do you detest and loath the abomination of organised religion                  

His answers have a certain glorious innocence about them, as he tries repeatedly to make conversation with an entity that, frankly, one cannot make conversation with:

From SOMEONE at DRAKE: is your father a hampster?                                
* do your family approve of what you do                                          
From SOMEONE at DRAKE: does your mother smell like last weeks dirty socks.       
* what method would you choose to slaughter your family                          
From SOMEONE at DRAKE: do my family approve?  of what exactly?           

 ...

From SOMEONE at DRAKE: why do you ask about religion. and we already decided     
                       that i have                                               
* do you believe in mgonz                                                        
From SOMEONE at DRAKE: had sex on campus and in my girlfriends room no less.     
* tell me your favourite sexual fantasy                                          
From SOMEONE at DRAKE: do i believe in mgonz?  what the heck is it?              
* praise and honour to blasphemers and death to mgonz            

He ends with a furious barrage of abuse, but of course, my imperturbable program is a calm, stimulus-response machine and so it is impossible for him to have the last word. He must quit, because my program never will. At the end of a long barrage of sex-related boasting:

From SOMEONE at DRAKE: how about her just sucking you off?                       
* if you are an attractive female please leave your phone number here      
From SOMEONE at DRAKE: little markie seems to be getting mad. poor little        
                       virgin markie.                                            
* jesus who let you near me go away                                              
From SOMEONE at DRAKE: FUCK YOU ASSHOLE.  THINK OR ARE YOU JUST A STUPID         
                       HOMOSEXUAL.                                               
* fuck off yourself                                                              
From SOMEONE at DRAKE: o my he actually typed something.                         
* ah type something interesting or shut up                                       
From SOMEONE at DRAKE: you type slow is that why you have all those stupid       
                       sayings on command  like that last one?                   
* you are obviously an asshole                                                   

and that is the last word.

More on the conversation

A few more observations on the conversation. Firstly, contrary to appearances, my program rarely asked about the user's sex life. The topic only got started:
From SOMEONE at DRAKE: are there two people on your username?                    
* when was the last time you had sex                                             

because, bizarrely, "there" matched a list of "sex words". After this, it was the user that kept the conversation about sex going, not the machine. The machine has absolutely no memory, and would forget about sex immediately if the user did not keep returning to it.

There were other strange breaks. For example, just when the user was starting to get suspicious, my program strongly suggested there really was someone there:

From SOMEONE at DRAKE: what? why don't you type something new. like when was     
                       the                                                       
* paris is a beautiful city                                                      
From SOMEONE at DRAKE: last time that you got laid..unless it was when you       
                       were born  o'my he said something different.  a           
* you are obviously an asshole                                                   
From SOMEONE at DRAKE: you are obviously a fucking dumbshit.                     
* i am not obviously a fucking dumbshit you insulting person       

The "paris is a beautiful city" reply happened, bizarrely, because "new" matched a list of "holiday words". The "You are .." reply is then spot on (purely by chance of course). Interestingly, the program actually admits (!) it is a program at some points in the code:

     (you know the worst thing about being a computer is having to
      deal with shits like you )
     (nobody ever asks the computer we lead a lonely life)
     (youll be in trouble when we computers take over the world)
     (have you any idea how boring it is being a stupid computer)

     (are you using foul language because i am a computer)

but by chance none of these were actually triggered in this conversation. The closest we get is:

From SOMEONE at DRAKE: mark?        
* mark isnt here and hes left me to deal with cretins like you                   

Why was this not written up before?

At the time, I did realise that something special had happened, and saved the conversation log carefully. But I think it is clear why I never wrote this up or publicised it - how could one write up something packed with vulgarity and obscenity like this?

MGonz was never something that was meant to be seen by "adults". In fact, I went to great lengths to hide it from the faculty and computing staff at UCD (see next section), who I thought would not appreciate me running this on their system.

But for many years afterwards, it became clear that this really was interesting to other people, and might even have been the first chatbot on the network with the element of pure surprise. The interest in it led to me finally setting up a web page on it [4] in 1995. Even then, this paper is still the first proper write up.


Internet features

The system had a number of network-related features which may still be of interest, especially since they relate to the possibility of fraud (or at least confusion) in Turing Tests online.

Remote control

First, if I was logged on to the VAX (the "Internet" machine), MGonzNet would copy me on any conversation it was having. From the source code at [4]:
  'GLOBALV SELECT CHATDISC STACK LOGGEDON'
  Parse pull loggedon
  If loggedon ^= false Then Do
                /* Copy to Vax unless explicitly told loggedon = false */
   If nickname ^= ''
    Then Call Vax '* Message from' nickname '('userid 'at' nodeid') ...'
    Else Call Vax '* Message from' userid 'at' nodeid '...'
   Call Vax message
   End
where "Vax" is the function:
 Vax:
  vaxmsgcmd = 'SMSG RSCS MSG CCVAX H236_007'
  Call Diag 8,vaxmsgcmd Arg(1)
 Return
In fact, I did not just get copies of messages. My VAX account was a privileged remote user that could send special commands to be executed by MGonzNet:
   When nickname = 'Vax' Then
    Do
     Call Sys
     Exit 0
    End
Where "Sys" is a function to parse any message that comes from my VAX account. This function includes code to execute a command-line (CMS) command:
   When left(upmessage,3) = 'CMS' Then
     Do
     cmd = lowcase(substr(message,5))
     cmd = strip(cmd)
     Address 'CMS' cmd
     End
That is, from the VAX machine I could use CHATDISC to remotely execute arbitrary commands on the VM machine. On the Internet today, servers running CGI scripts worry about script security since a bug in a script could give a remote user a command-line on the server. But at least CGI scripts (on a UNIX server) run as the user "nobody" - so in theory, even if they get a command-line, they shouldn't be able to do much damage. But here a disconnected server program like CHATDISC runs as me, with the ability to delete all of my files, and so on. This could have become an issue if disconnection for personal server-side programs had become common, but I am unaware that anyone else ever implemented a remote command-line like this on BITNET.

Why is this relevant to the Turing Test? Because one of those arbitrary commands that I could send from the VAX was:

CMS TELL user message

That is, I could interleave my own replies with the program's replies. In general, if Turing Test-like experiments are going on online, and we are talking to some server-side program, a human could monitor the conversation remotely (and secretly), and then send secret messages to the server to send particular replies back. We, as users of the server, would never be able to tell. So the machine would pass the Turing Test, but only because you were talking to a human!

Writers about how to run a Turing Test competition online recognise the issue of remote human insertion of messages. Here we see that this has, of course, already happened in the 1980s. Here is a friend of mine as I send him some ambiguous messages from the VAX through my CHATDISC:

From FRIEND3: SEE says you're disconnected
From FRIEND3: where the hell are you
From FRIEND3: is this a ghost i'm talking to
From FRIEND3: very mysterious
From FRIEND3: quite strange
From FRIEND3: you are definitely disconnected

Attempts to hide the system

As mentioned above, I hid MGonz from faculty and computing staff, believing they would not approve of it. I even renamed the CHATDISC program to the bland name "SYS TXT" so any search for CHATDISC or EXEC files on my disk would not find it. It would be then renamed to CHATDISC EXEC just the moment before it was launched.

I also had a "blacklist" of users who MGonz would not reply to:

/* Centre people - give standard disc msg
   don't let them stumble across MGonzNet
*/
 centre  =           ' adillon adviser annmarie bernie bonnie brianm'
 centre  = centre || ' carrick cecily deirdre dobeirne ebairead emcgrath'
 centre  = centre || ' eoin guest1 guest2'
 centre  = centre || ' harringt helen jacklowe jchester jennings joanc'
 centre  = centre || ' larry listserv maevem mailmnt mailr2 maint'
 centre  = centre || ' mallen mbreslin mcgrath mnorris'
 centre  = centre || ' mokeeffe mokelly molooney moriarty msexton'
 centre  = centre || ' noreilly odonnel'
 centre  = centre || ' oneillu pdoyle rosemary sinead tinac tmcgrath'
 centre  = centre || ' twade t_wade vmaccnt walter walsh'

This is actually a list, as [5] explains, of many of the pioneers of the Internet in Ireland. On 24th Apr 1989, I sent a quick talk message to one of these:

To CENTREPERSON: were you looking for me?
I found out he was disconnected. My message was logged in his answer machine. Then I disconnected myself. I went off to work on the VAX. While I'm working on the VAX, I get a message from MGonz telling me it is having a conversation with CENTREPERSON, and copying me on it:
From CENTREPERSON: I was, when you had all those t-discs
* ive never been when you had all those t discs whats it like
Horrified that MGonz was talking to him, I logged back in to VM as quick as I could and tried to cover up the conversation, thankful that it had just baffled him and not asked him about his sex life.

There had been a flaw in my blacklist (a missing space, I found out later). Though I had to get caught sometime. On 18th Dec 1989, a message from the UCD operator:

From UCDOPR: PLEASE LOGOFF NOW 
From UCDOPR: SWITCH BEING BACKED UP
* what are you talking about
* you are obviously an asshole

Discussion - How to pass the Turing Test

We now ask what implications MGonz has for the Turing Test. Obviously this program (like Eliza itself) is really a trick. It is not intelligent. It has no understanding of anything that is going on in the conversation. We will discuss below whether the Turing Test, and passing it, really have anything to do with AI. But leaving that aside, how did it work? How did it manage to pass the Turing Test?

Surprise

One of the things that made MGonz successful was the element of surprise. Users of an AI CGI script through a web page [20] expect to be talking to a program. Users in the Loebner Prize Competition [11] expect that some of the entrants will be machines. But no one expected to hit a program when they sent me BITNET chat messages.

Ironically, this would suggest that the Loebner Prize Competition is not the way to go about the Turing Test! As we saw above, the advance knowledge by my friends that I had (or might have) some kind of program running makes all their conversations rather bland, unserious (they just mess around) and short (because after the initial novelty, they get bored). Indeed, I have records of a number of such conversations with friends who knew it was a program, but these conversations are just too boring to include in this paper. Lack of surprise simply makes for a different type of conversation. This factor could be seen when Tom Doris put the original LISP MGonz back online in late 1998 as a CGI script [1]. Here the user knows from the start that it is a program, which affects the nature of the conversation:

User : you are simply responding to stimuli
Mgonz: so you think i am simply responding to stimuli well i honestly
could not care less
User : you are simply responding to stimuli
Mgonz: ok so im simply responding to stimuli so what is it a crime
User : you are simply responding to stimuli
Mgonz: i know i am simply responding to stimuli dont rub it in
User : you are a defective program

Long conversations (like an hour and a half) just don't happen when the user knows it is a machine. Doris' CGI script is no longer running, but Dave O'Connor [14] has a version of MGonz online as a CGI script. Again, the user knows it is a program, though human psychology is such that some users still allow themselves to be provoked. The following are 3 separate users:

From USER1: Hello
Hello little man, how are you?
From USER1: don't call me little.
Tell me a bit about yourself.
From USER1: i am five feet six inches.
I've never been five feet six inches , what's it like?
From USER1: it's annoying because robots call you little.
Are you here for business or pleasure?
From USER1: i'm here to get material for my newspaper column.
So you are here to get material for your newspaper column , well I honestly could not care less.
From USER1: that's very rude. you are a rude robot.


From USER2: erm I am a student
I've never been a student , what's it like?
From USER2: boring most of the time
Go on tell me some really juicy scandal.
From USER2: I dont know any
You are obviously an interesting person.
From USER2: so are you
Are you here for business or pleasure?
From USER2: I have to do some work so I was wondering what you are
Did you have problems with your feet when you were wondering what me am ?
From USER2: eh my feet?
Ah type something interesting or shut up.
From USER2: why did you just metion my feet
What kind of music do you like?
From USER2: no what was that feet thing about
I see.


From USER3: I am David.
I am glad you are David .
From USER3: Thank you.  So am I.
You're Welcome. :)
From USER3: You are a very nice person.
So you think I am a very nice person , well I honestly could not care less.
From USER3: Well I take that back. You are rude.
Sing if youre glad to be rude , sing if youre happy that way, hey!
From USER3: I hate to sing.
Go away fool and stop annoying me.
From USER3: I have a paper to write.
Ah get lost, go to the bar or something.
From USER3: but I need to write my paper.
If you think i care you're wrong.

The above is about as good as a Web-page chatbot can get without the element of surprise.

AOLiza [2] has recently returned to the element of surprise on the AOL Instant Messenger system, where it has been running since 2000. Like my program, and unlike the Loebner Prize, users do not expect to be talking to a machine. AOL, incidentally, were not happy about AOLiza running on their system. The element of surprise can be seen by some as unethical. And yet one cannot expect the best chatbot performance to occur when it is absent.

Sex

The second major reason why MGonz succeeded was due to profanity, relentless aggression, prurient queries about the user, and implying he was a liar when he made responses to these. Sex is probably the easiest topic in which to engage the user, so long as you get the right personality type, because what we are looking for is emotions that blind the user to reason (so he does not notice it is a program). Questioning SOMEONE at DRAKE's sexual abilities was all that was needed to get him into defensive, boastful and argumentative mood, emotions that blinded him to the fact that he wasn't actually talking to a person. MGonz however, was merely an amateur compared to the following bot, which could actually get the user aroused, and thus totally blinded to the fact that no one was at home.

Jenny18

For me, the most amazing chatbot of all time is Jake Kaufman's "Jenny18" bot [9] running on IRC in 2001-2. Like MGonz and AOLiza, Jenny18 also had the element of surprise. But its real genius is in its personality design. Jenny18 specialises ruthlessly on pretending to be a horny girl looking for cyber-sex. The performance of this bot is breathtaking, inspiring the most intimate confessions, desperate requests for a photo or an email address, arousal, masturbation, frantic demands to take it to telephone, and finally orgasm. It passes Turing Test after Turing Test. The most human is the long, hilarious conversation with "Scorpion832" (all these conversation logs can be seen online at [9]). And this simple bot clearly brings "dom01" to orgasm repeatedly, which surely must count as a fundamental new milestone passed in the history of the Turing Test:
<dom01> how old r u
<jenny18> shouldnt my nick tell u that!!! lol
<dom01> im so horny right now damn
<jenny18> (stroking your erection through your pants)
<dom01> help me geny,im horny
<jenny18> (stroking your erection through your pants)
...
<dom01> yeahhhhhh  i feel so horny genny
<jenny18> grrrrooooowwrrr... you're not the only one who's horny.
<dom01> mmmmmmmmmmyahhh
<jenny18> mmm.. =)
<dom01> im doing it
<jenny18> oh are you? =)
... 
<dom01> mmmmmrrrr
<jenny18> mmm.. =)
<dom01> im masturbating myself
<jenny18> kewl.. yum! 
<dom01> mmmmmm yeahhhhhhh   
<jenny18> could u explain pls?
...
<dom01> is your pussy wet
<jenny18> you know, my cunt is dripping. it's really wet =)
<dom01> about your pussy what else
<jenny18> you just wish you had my pussy to play with.
<dom01> rrrrrrrrryeah
<jenny18> crazy..
...
<dom01> can i call u baby
<jenny18> just got one phone line hon. gotta stay here.
...
<dom01> can i call u baby
<jenny18> no way. i just wanna keep it in msg.
<dom01> please
...
<dom01> mrrrrrrmmmmmmmm
<jenny18> could u explain pls?
<dom01> ang onnnnnnnnnnnnnn 
<jenny18> ah.
<dom01> angggggggggonnnnn
<jenny18> oh?
<dom01> waitttttttttt
<jenny18> hm... =)
<dom01> ahhhhhhhyeahhhhhhhh
<jenny18> mmmMm..
<dom01> fuck
<jenny18> i see
<dom01> i came
<jenny18> hmm
<dom01> yes
...
<dom01> xxx
<jenny18> ah.
<dom01> it feels so good ejacualtion
<jenny18> want some more? i think u do :)
<dom01> again?
<jenny18> mmm.. =)
<dom01> ok ill try my best to come again
<jenny18> come again?
<dom01> wait
<jenny18> crazy..
<dom01> ang on im washing my dick,theres sperm on it
<jenny18> hmm
<dom01> wait
...
<dom01> how old r u
<jenny18> i'm 18, babe =)
<dom01> fuck girl where r u from

With some of the others, like "GoldenBoy2222" and "lander100", it is hard to tell if they did actually come or are just pretending, but with "dom01" it is clear what has happened on his side. "Happy_Boy" also looks like it happened for him.

Incidentally, some people's inclination might be to laugh at the users of Jenny18, and regard them as sad - especially those that come. I'm not sure it's that simple. One could just as easily be happy for them - "dom01" is clearly having a nice time. Jenny18 could be seen as a public service, spreading happiness and optimism online. She gives users a nice fantasy to work with, perhaps at a period in their lives when they have no one offline to be intimate with. She could be a liberating experience, leaving them happier, more confident and optimistic in going back to the cruel reality of dating in the real world.

Jenny18 deserves a paper all on its own. And yet publicizing it is of course difficult. I did not publicize MGonz for years because of its vulgarity. Jenny18 has a far worse problem. Given the content, and especially what the users are doing during the conversation, one can imagine both the field of AI and the media being reluctant even to talk about Jenny18. Which would be wrong. For anyone who takes the Turing Test seriously, passing the "orgasm" Turing Test (i.e. the computer brings the human to orgasm) is surely an accomplishment worth noting.

Privacy

For sex conversations to work, surprise is crucial. No one talking to an AI CGI script would get too emotionally worked up if it started questioning their sexual prowess - they would treat it as an amusing game. Indeed, Yahoo categorises these under "Games" [20], showing that one's whole frame of mind in using a program that you know is a program is that of looking for amusing entertainment.

For the sex conversation to work, privacy is also crucial. No judge at the Loebner Prize Competition is going to disclose personal information about their sex life like SOMEONE from DRAKE, let alone actually get aroused, if they know that the other judges will see what they typed. In a public forum, one will be self-controlled and keep more distance in the conversation. To summarise, a young male talking about sex online alone in the privacy of their bedroom is probably the easiest environment in which to pass the Turing Test.

Real human responses

We haven't said much about AI so far. It seems to me that "script writing" is far more important than "AI" in chatbots. MGonz contains no more "AI" in it than Eliza (in fact, probably less). All my work was in the responses, the personality. My classmates wrote versions with memory (mine is memory-free!) and far more powerful sentence parsing, but their responses were the banal psychotherapy responses, or the slightly awkward and "geeky" responses developed by generations of Eliza programmers in AI. It is hard for programmers to do any different. No one can write about sex, for example, unless you are comfortable discussing it, which AI "geeks" rarely are. They might prefer to have a chatbot talk about science fiction.

What is needed is someone comfortable with sex who can still cope with a program, or at least modify an existing program. Jenny18, from someone with a background in music, not programming, is all about the strong, confident script, that simulates, often with subtle understatement, a horny girl playing with sex talk, rather than some robotic voice talking about sonnets. It seems to contain even less "AI" in it than Eliza does. To quote Kaufman, author of Jenny18, in private correspondence to me:

I think that, while technically impressive, advanced theoretical learning algorithms and memory databases are not by themselves the key to fooling random people - realistic responses are. You seem to come from the same train of thought. If artificial human intelligence is to be presented, it has to seem human! I don't understand how these chatterbots who speak in full paragraphs are to be taken seriously yet, when people subsist mainly on short utterances. To that effect, I included 500 different versions of "Huh?" and "I don't understand."

Discussion - Is the Turing Test important?

We have just discussed the main reasons why MGonz, and Jenny18, perform well on the Turing Test, and none of them seem to involve AI. So the time has come to ask: Is the Turing Test, and passing it, actually important for the field of AI? It may surprise the reader that my answer is "No".

I agree with [3] and [18] that the Turing Test served its purpose in its day, but it's irrelevant now. In his famous paper [16] Turing was dealing with people who just couldn't believe a machine could ever think, and so was making the philosophical point that if you couldn't tell the difference, why would you deny it was thinking? This is a useful thought experiment, but this is not the same as saying either that: (i) passing the Turing Test is necessary for intelligence, or even that it is likely to be of any importance, or: (ii) passing the Turing Test is sufficient to demonstrate intelligence - there may be an answer to Turing's question. He just reminds us to take care that it is not just based on prejudice.

[3], [18] and [12] point out many of the standard objections to the Turing Test - that it concentrates on adult, human, text-based language processing, to the exclusion of other types of intelligence, that it forces the intelligence to pretend, instead of just being itself, and so on. I agree with these criticisms but since they are well made elsewhere I will not repeat them here, but instead try to make a few further points.

Not necessary

First, is passing the Turing Test necessary to be intelligent? Turing does not claim it is necessary - he is just making the point that if a machine does pass, why would you deny it was intelligent? See section 2 in [16].

But many of Turing's followers seem to regard it as necessary, or at least important. Certainly thinking that the goal of AI should be to pass the Turing Test would imply that it is necessary. It seems to me that passing the Turing Test is not necessary for intelligence:

  1. We do not think humans are intelligent because they pass the Turing Test. - Turing asks why we think anyone is intelligent. He might say: "You only think I'm intelligent because of my behaviour." I would reply: "No I don't. I know you're intelligent without even meeting you or hearing a word you say. I know you're intelligent because I'm related to you." Because of what we know about the historical origins of humanity and shared DNA, I simply cannot work in a fundamentally different way to you. I know how you work. You work like me. In his paper [16] Turing asks how we can know anything outside of our own minds:

    A is liable to believe "A thinks but B does not" whilst B believes "B thinks but A does not." Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks.
    I would argue that it is more than a "polite convention". It is a consequence of the theory of evolution. A normal, healthy adult Homo sapiens sapiens has to have an inner life something like mine. But with a new thing - something that is unlike us and all animals, something that is not part of the genealogical tree of life - the situation is different. The new thing may indeed think, but the possibility of trickery and illusion remains. The genealogical argument does not help to dismiss that possibility, as it does with humans.

  2. Aliens can't pass the Turing Test. - The Turing Test will not play a role in us detecting other naturally-evolved intelligences. To invert this, when aliens discover us, how will they be able to tell we're intelligent? We won't be able to pass as convincing aliens. And yet they will quickly see that we are intelligent.

  3. How will we judge future machine intelligence? - Imagine aliens landing here 1.5 million years ago, in the days of Homo erectus, and trying to see if we were intelligent. We wouldn't pass their Turing Test, and we wouldn't have language or civilization. But we would have stone tools and fire. The aliens might have recognised us as the start of some type of intelligence, but not an intelligence similar to theirs. This is how we will recognise the start of profound machine intelligence. The Turing Test will have no role.

Not sufficient

The Turing Test may not be necessary, but is it sufficient? If a program does pass, must we admit it is intelligent? My fundamental objection to this is simple:

  1. The Turing Test has been passed. - It's time to face reality. Whenever a machine, such as Eliza, MGonz, Jenny18 or AOLiza, does pass the Turing Test, critics say "But of course that wasn't a real Turing Test". I used to agree with this, and regarded the title of my web page, "How my program passed the Turing Test" [4] as a bit of a joke. Now however I am not so sure (as you can see since I have re-used that title for this paper). I think a case can be made that the Turing Test has been passed, many times, by many programs, and so what. It follows that:

  2. Passing the Turing Test does not mean you are intelligent. - Trickery can pass it. MGonz has no more "AI" in it than the original Eliza. In terms of human reaction, Jenny18 is more impressive than any conversation program in history, inside AI or out. Yet it contains even less "AI" in it than Eliza does! The simple reality is that the Turing Test has been passed time and again, by programs that are not intelligent.

Conclusion

I have no problem with the concept of a machine being intelligent. Indeed, such already exist - for we are examples of such. Indeed, there is no evidence that there has ever existed an intelligence that is not a machine. And there is no reason why the principles behind how we work cannot be abstracted into an artificial system. My problem is with the Turing Test, not with AI.

AI researchers succeeding in the Turing Test and then disowning the test itself has been common practice ever since Weizenbaum [17]. Jason Hutchens [8] is another example. He is interested in how to fool people, but does not believe that this has much to contribute to AI. After rejecting the Turing Test, my entire AI research career since MGonz has been on quite different forms of AI (mainly sub-symbolic, non-linguistic intelligence). There are, however, some good things about both MGonz and the Turing Test for AI, which I shall discuss in the next section.


The future of AI online

The most important thing about MGonz is perhaps the online aspect. This was certainly one of the first AI programs online. It seems to have been the first (a) AI real-time chat program, which (b) had the element of surprise, and (c) was on the Internet. To explain, there were many AI chat programs before MGonz (for a survey see [10]) but they were all (a) flagged as programs or bots (no element of surprise, as in AI chat CGI scripts [20]), or (b) on mailing lists or usenet (not real-time chat), or (c) offline (e.g. bots in multi-user games systems, or on a single shared machine like the VAX in 1987 above).

We will close by considering an argument that the future of all of AI is online.

What happens next in AI?

Discussions about the Turing Test inevitably lead to discussion about the distant future, that day when AI finally triumphs and we make intelligent machines. Such long-term speculation, perhaps done best by Moravec [13], is fascinating, but after reading enough of it, I always end with a vague feeling of dissatisfaction. One question for me comes to the fore unanswered: How do we get from here to there? What is the path leading step by step from AI today to that future? In particular, what do we do next?

The symbolic AI revolution slowed in the late 1980s. It has been revived somewhat with Internet Agents and the Semantic Web, but much of the early optimism seems gone forever. The biologically-inspired AI revolution has had 15 years now to deliver on its promises, and it seems to me that it too is slowing down, and has not progressed as far as hoped (especially in machine evolution). So what happens next?

What happens next, we argue, is that it all goes online.

The World-Wide-Mind

The "World-Wide-Mind" (WWM) [19] is a proposed standard for putting AI "minds" and sub-minds online (as WWM "servers") so that they can be re-used remotely as components in larger minds. The aim is to address the scaling up of AI, or how to construct minds more complex than could be written by one author (or one research group).

AI is currently being used online, notably symbolic AI in Internet Agents and the Semantic Web. But the model is still one where the agent is written by a single author or at most a single research lab. The WWM analysis is that as we scale up AI, we need to build more and more complex agent minds out of components written by multiple (perhaps hundreds or thousands of) authors. In other words, AI authors specialise on different aspects of intelligence, and come together to build large, complex minds that no single author understands. Only by authors publishing server-side programs in an open, public system on the Internet, we argue, can such massive, long-distance collaboration become feasible. Hence the term, the "World-Wide-Mind", referring to the fact that the mind may be physically distributed across the world. For a short introduction to the WWM idea see [6] or [7].

Under the WWM view, MGonz can be seen as one of the first times (if not the very first time) that one could access a remote mind server on the network in real-time.

Good science

We said earlier that there are some good points about the Turing Test for AI. It seems to me that the most important thing about the Turing Test is the very idea of a standard test.

In any branch of AI, the existence of objective tests that cannot be argued with tends to provide a major impetus to research. This is one of the reasons for the popularity of rule-based games in AI, and, more recently, robotic soccer. With sport and games, you can prove one system is better than another. One of the problems with many branches of AI, as discussed further in [6], is the difficulty of re-implementing someone else's test problem and re-running their experiments. Much of this difficulty is caused by the need for local installation. The WWM aims to address this, by having their test problem remotely usable online. Many standard tests will emerge, it is imagined, and in an open, public system we will have repeated, objective, 3rd party comparisons of solutions, so that progress can be made.

The Loebner Prize Competition has a point that competitions spur progress. Here we argue that AI needs more competitions than just the Turing Test. In fact, it needs a generic protocol by which people can set up new competitions (new remote WWM "world" servers online).

Conclusion - No single scientist will understand AI - but the community of scientists will

Perhaps the objection to AI that I have always found the most convincing is simply that it is beyond our abilities - that there are just limits to what a smart primate can do. For example, a dog will never understand arithmetic. Perhaps the human mind will never understand the mind. We're simply too limited.

The WWM answer to this is "Yes. No single human mind will ever understand the mind. But humanity as a whole will." No single human mind can understand the entire corpus of science. We passed the point long ago (perhaps in the 17th century) when one person could understand all of mathematics and science. But humanity as a whole does understand all of these things.

To date, AI has been held back, we argue, by the need for a single lab, even a single researcher, to fully understand the components of the system. As a result, only small minds have been built so far. The WWM argues that we must give up this dream of full understanding as we build more and more complex systems. And giving up this dream of full understanding is not a strange thing to do. It is what has always happened in other fields. It is how humanity has made its most complex things.



Acknowledgements

Thanks to Dermot Williams for originally showing me how to modify CHATDISC. Thanks to John Murphy for useful discussions on whether a single scientist could understand the mind.



References

  1. Doris, Tom (1998), MGonz version online as CGI script, formerly at: www.compapp.dcu.ie/~tdoris/mgonz.html

  2. Fox, Kevin (2000 onwards), The AOLiza program,   fury.com/aoliza

  3. Hayes, Patrick J. and Ford, Kenneth M. (1995), Turing Test Considered Harmful, Proc. Int. Joint Conf. on Artificial Intelligence (IJCAI-95), pp. 972-977, Montreal.

  4. Humphrys, Mark (1995 onwards), "How my program passed the Turing Test", computing.dcu.ie/~humphrys/eliza.html
    This web page has 3 sub-pages:
    • MGonz - The LISP source code.
    • MGonzNet - The REXX source code.
    • The DRAKE conversation.

  5. Humphrys, Mark (2001 onwards), "The Internet in the 1980s", computing.dcu.ie/~humphrys/net.80s.html

  6. Humphrys, Mark (2001), Distributing a Mind on the Internet: The World-Wide-Mind, Proc. 6th Euro. Conf. on Artificial Life (ECAL-01), Springer-Verlag.

  7. Humphrys, Mark and O'Leary, Ciarán (2002), Constructing complex minds through multiple authors, Proc. 7th Int. Conf. on Simulation of Adaptive Behavior (SAB-02), MIT Press.

  8. Hutchens, Jason L. (1997), How to Pass the Turing Test by Cheating, Department of Electrical and Electronic Engineering, University of Western Australia, technical report TR97-05.

  9. Kaufman, Jake (2001), "Jenny18 - A Cybersex Bot Implemented in Eliza", virt.vgmix.com/jenny18

  10. Leonard, Andrew (1997), Bots: The Origin of New Species, Penguin.

  11. The Loebner Prize Competition, loebner.net/Prizef/loebner-prize.html

  12. Michie, Donald (1993), Turing's Test and conscious thought, Artificial Intelligence 60:1-22.

  13. Moravec, Hans (1998), Robot: Mere Machine to Transcendent Mind, Oxford University Press.

  14. O'Connor, Dave (1999 onwards), MGonz version online as CGI script, mgonz.swearing.org

  15. Thomas, Eric (1996), "The History of LISTSERV", L-Soft Inc., www.lsoft.com/products/listserv-history.asp

  16. Turing, Alan M. (1950), Computing Machinery and Intelligence, Mind 59:433-60.

  17. Weizenbaum, Joseph (1966), "ELIZA - A computer program for the study of natural language communication between man and machine", Communications of the ACM 9:36-45.

  18. Whitby, Blay (1997), Why The Turing Test is AI's Biggest Blind Alley, in Millican and Clark (eds.), Machines and Thought: The Legacy of Alan Turing, Vol. 1, Clarendon, Oxford.

  19. The World-Wide-Mind project (2001 onwards),   w2mind.org

  20. Yahoo list of AI programs online, yahoo.com/Recreation/Games/Video_Games/Internet_Games/Web_Games/Artificial_Intelligence



Feeds      w2mind.org

On Internet since 1987.