How to Build Eliza Chatterbot A Program that can Chat with Humans

How to Build Eliza Chatterbot
A Program that can Chat with Humans

Written by Amit Mathur on 10^th to 12^th Dec, 2002

What is Eliza ?

Eliza is an AI Program that simulates the behavior of a therapist. The first program of this sort was developed in 1967 in MIT. Such programs, which interact with user in simple English language and can simulate a conversation are known as Chatterbot.

A program like Eliza requires knowledge of three domains:

1.    Artificial Intelligence
2.    Expert System
3.    Natural Language Processing

Even though last two are sub-parts of the first one, they are emerging as science in themselves.

Eliza can not, of course, think on its own. It has a repository or database of facts and rules, which are searched to give the best possible response.

Eliza works by matching process. Very rarely an entire sentence is matched to give the response.

The rules are indexed by keywords. Some rules require no keyword.

How does it work ?

Ninety percent of what Eliza says is found in the associated Data File. This file acts as Knowledge Base for the complete system.

The in-built responses comprise the Static Database of the system. These are the responses for the following cases:

1. When Eliza does not understand what the user is talking about.

2. When the user repeats himself.

3. When the user does not type anything and just keeps on pressing Enter.

4. For the greeting statements.

The following strategy is used to respond to a request:

Step 1: Eliza finds out if the user has given any null input. If so, it takes the fact from the static database to respond.

Step 2: There are some in built responses that Eliza can recognize readily. It finds the presence of any such sentence after fragmenting the user’s input and remembers the associated keyword. This keyword defines the Context of the talk.

Step 3: If no in-built sentence frame work is found, then the Eliza searches for the specific keyword to define the context. If no context is found, it deliberately motivates the user to speak about a specific topic.

Step 4: A response is chosen (at this time, randomly) from the database of available responses.

Step 5: Any necessary transpositions are done. For example, consider the following conversation:

SAM> I PLAN TO GO TO JAIPUR TOMORROW WITH MY WIFE.

ELIZA> AND WHAT HAPPENS IF YOU WON'T GO TO JAIPUR WITH YOUR WIFE ?

Here, the word My has to be transposed to YOUR.

Step 6: To simulate the human conversationalists, Eliza simulates Typing and does so slowly with making spelling mistakes and correcting them.

What about Coding ?

Now let us start the real coding portion. I am using Turbo C IDE 3.0 as this is the IDE that most Indian Students use.

Note that the complete source code is in the Zip file that accompanies this file. But my main stress is on approach and not on coding. The code, which is written in just 90 minutes, is good as a working skeleton.

Before going into the detailed coding aspect, let us first see the structure of a sample Data File. Eliza recognizes certain keywords. If these keywords are found in the user input, then corresponding to that, from a predefined set of responses, one is chosen and displayed.

A keyword is separated in the data file (called Dictionary) from the responses by @KWD@ token. This token indicates that the next line that follows is actually a keyword, not a response.

@KWD@ HELLO HI, HOW ARE YOU HELLO DEAR ! @KWD@ I WILL YOU WILL DO SO. I BELIEVE IT TOO... WILL YOU BE ABLE TO DO SO ? @KWD@ YES ARE YOU SURE ? HOW CAN YOU BE SO SURE ? YOU SEEM TO BE VERY OPTIMISTIC. @KWD@ NO YOU SEEM TO BE VERY PESSIMISTIC. NEVER SAY NO... @KWD@ COMPUTER I KNOW HOW TO WORK ON COMPUTER. YOU ARE CURRENTLY USING A COMPUTER. RIGHT ?

<< Add whatever you want in above format >>

Contents of file : Eliza.dat

For example, in response to 'Hello', from the above dictionary, Eliza will give one of the following responses:

HI, HOW ARE YOU
HELLO DEAR !

Once this thing is clear, let us now define the Data Structures that we will be using. We create two classes :

progstr - This is used to store the user's input related information.
resp - This is used to store the information about the various responses.

Let me give the code first and then I will explain it.

class progstr { public: char userip[MAX_USER_INPUT]; char keyword[30]; int keyfound; int keyno; int nullip; // constructor progstr() { keyno=-1; nullip=0; keyfound=0; } }ip; class resp { int tot_resp; int last_resp; char replys[MAX_RESP_NO][MAX_RESP_LEN]; char word[MAX_KWD_LEN]; public: // constructor resp() { tot_resp=0; last_resp=-1; } int getcount() { return last_resp; } void addword(char str[MAX_KWD_LEN]) { strcpy(word,str); } char * getword() { return word; } void addresp(char str[MAX_RESP_LEN]) { strcpy(replys[++last_resp],str); } // defined later void display_resp(int num); void quit_display_resp(int num); };

Basic Data Structures Involved

The character array userip is used to store the line typed by the user. Another array keyword is used to store the keyword, if any, found in that input. If a keyword is found, we make int keyfound to 1 else, it remains 0, as it is initialized to 0 in the Constructor. keyno stores the key number of the corresponding keyword.

nullip indicates whether the user has given any Null input ie, he is just pressing enter and doing nothing else.

Now let us come to the second class, resp. The first data member, tot_resp indicates the total number of responses for a given keyword. For example, for the keyword Hello, we have 2 responses (see Eliza.Dat). So, for that, tot_resp holds a value of 2. last_resp is used for the function processing and its use will be clear later on.

The replies are actually stored in replys[MAX_RESP_NO][MAX_RESP_LEN] and the corresponding keyword is stored in the array word.

Description of Functions in Class resp :

Constructor:
This is used to initialize the total number of responses to 0. Why last_resp is initialized to -1 will be clear when you look at the function add_resp.
int getcount():
This function is used to get a count of how many responses are there for a given keyword.
void addword(char str[MAX_KWD_LEN]):
This is used to add a keyword.
char * getword():
Used to return the keyword for a particular object of class resp.
void addresp(...):
This is used to add a response corresponding to a given keyword.
void display_resp(int):
This is used to display the response to the user corresponding to a given index number for the responses. (actually it does more than that !).
void quit_display_resp(int):
Difference between this function and above function is that it is used in the end when the user is quitting. So, it does not return the prompt to the user.

Let us now create a function that reads the contents of file Eliza.Dat in an array of objects of class resp, which we name keys. Since I have already explained both - the format of .Dat file and data structures, this function should by clear with little or no effort. The code is commented wherever necessary.

void read_from_file() { ifstream fin; int index=-1; fin.open("eliza.dat"); char line[MAX_RESP_LEN]; while(fin) { fin.getline(line,MAX_RESP_LEN); char *ptr=NULL; ptr=strstr("@KWD@",line); if(strlen(line)<1) { break; } else if(ptr!=NULL) { // the next line is a keyword fin.getline(line,MAX_RESP_LEN); keys[++index].addword(line); } else { // it is a response keys[index].addresp(line); } } // end of while } // end of function

Read Contents of Eliza.Dat File

Now let us create a function for global initialization of the transposition words. This function is easy and I will not belabor it.

void initialize_global() { strcpy(wordin[0],"ARE"); strcpy(wordout[0],"AM"); strcpy(wordin[1],"AM"); strcpy(wordout[1],"ARE"); strcpy(wordin[2],"WERE"); strcpy(wordout[2],"WAS"); strcpy(wordin[3],"WAS"); strcpy(wordout[3],"WERE"); strcpy(wordin[4],"YOU"); strcpy(wordout[4],"ME"); strcpy(wordin[5]," I "); strcpy(wordout[5],"YOU"); strcpy(wordin[6],"YOUR"); strcpy(wordout[6],"MY"); strcpy(wordin[7],"MY"); strcpy(wordout[7],"YOUR"); strcpy(wordin[8],"I'VE"); strcpy(wordout[8],"YOU'VE"); strcpy(wordin[9],"YOU'VE"); strcpy(wordout[9],"I'VE"); strcpy(wordin[10],"I'M"); strcpy(wordout[10],"YOU'RE"); strcpy(wordin[11],"YOU'RE"); strcpy(wordout[11],"I'M"); strcpy(wordin[12],"ME"); strcpy(wordout[12],"YOU"); strcpy(wordin[13],"YOU"); strcpy(wordout[13],"ME"); }

Basic Transformations

Let us now write a function for displaying the responses to the user. The first if statement in the for loop is used to make a deliberate typing error to make it appear more human like ;-). One character is randomly chosen for typing error. Special cases like New Line and Backspace are separately considered. (Think why ?). Now I introduce something new. A special character - *. Char * represents all of the text found AFTER the identified keyword, and before one of the following punctuation marks.

For example, consider the user input

AMIT > CAN I GO TO INDORE TOMORROW ?
ELIZA > WHAT IF YOU DO NOT GO TO INDORE TOMORROW ?

The underlined portion is not stored in the dictionary, rather it is taken from the user input. In the file Eliza.Dat, we store this information as

CAN I
WHAT IF YOU DO NOT *

Star (*) asks the program to simply copy whatever is typed after the keyword (here CAN I ) in the user input, as it is. I hope that now the function of * as a special keyword is clear. So, let us consider a more complicated case.

AMIT > CAN I GO TO INDORE TOMORROW WITH MY FRIEND ?
ELIZA > WHAT IF YOU DO NOT GO TO INDORE TOMORROW WITH MY FRIEND?

Obviously this is not what we wanted. I am supposed to go with my friend, not one of Eliza ! So, we must perform some transformation also. When we think of transformation, the sentence gets divided in the following 3 sections:

Text Before Transposition Word. (here, GO TO INDORE TOMORROW WITH )
The Transposed keyword. (here, YOUR, in place of MY)
Text After Transposition Keyword. (here, FRIEND ? )

The following code tackles the three cases in a very lucid manner.

void resp :: display_resp(int num) { cout<<"ELIZA > "; for(int i=0;i<strlen(replys[num]);i++) { // for deliberate typing errors // (for simulating the human behavoir ;-) if(RanNum(6)==0) { char c=RanNum(100); if(c=='\n' || c=='\b' || c==13) cout<<"w"; else cout<<c; delay(RanNum(DELAY)); // correcting the deliberate typing error cout<<"\b"; } // * is used to write anything after the keyword // as it is, but with some transformations like // converting MY to YOUR. if(replys[num][i]=='*') { char * s1=ip.userip+strlen(ip.keyword); short int flag=0; for(int m=0;m<TRANSPOSE;m++) { char * s2=wordin[m]; char *ptr=NULL; ptr=strstr(s1,s2); if(ptr!=NULL) { // transposition word found in the // user input flag=1; // printing text before wordin[m] int times=ptr-s1; for(int i=0;i<times;i++) { delay(DELAY); cout<<ip.userip[strlen(ip.keyword)+i]; } // printing the wordout cout<<wordout[m]; // printing the left overs char c; c=*(ptr+strlen(wordin[m])); int t=0; while(c!='\0') { cout<<*(ptr+strlen(wordin[m])+t); t++; c=*(ptr+strlen(wordin[m])+t); } } } // end of for // if flag is still zero , this means no need for // transposing any word. if(0==flag) { char c; c=*(s1+strlen(ip.keyword)); int t=0; while(c!='\0') { cout<<*(s1+t); t++; c=*(s1+t); } } // end of if break; } else { cout<<replys[num][i]; delay(RanNum(DELAY)); } } // end of for // giving the prompt back to user cout<<"\n"<<user<<" > "; }

Function for displaying Eliza's Response

Finally we can work out a procedure for searching the keyword in the user's input. MAX_KEY indicates the number of keywords in the DAT file. Here we are simply searching whether the keyword exists in the user input (anywhere).

void find_keyword() { int len=0; int lenkey=0; int key_no=0; char teststr[50]; while((ip.keyfound==0) &&(key_no!=MAX_KEY)) { // getting the length of the keyword lenkey=strlen(keys[key_no].getword()); char *ptr=NULL; ptr=strstr(ip.userip,keys[key_no].getword()); if (ptr!=NULL) { // keyword found ! ip.keyfound=1; ip.keyno=key_no; strcpy(ip.keyword,keys[key_no].getword()); break; } key_no++; } }

Simple Search Routine.

When all these routines are made, we integrate them in the main function. For complete source code, please download the accompanying Zip file with the article.

Future Prospects:

Like other AI programs, this program also has immense possibilities of improvement.

The following are the improvements, which can be made in it.

Learning by Time or Experience can be Implemented.
All the previous talking can be stored in a array of strings, so that in case of user contradicting himself/herself, ELIZA can contradict him.
A database or a flat file, at least, can be used for the data and talk storage.
Sessions and User-Password Pairs can be established, so that, even after the completion of one session, next time, whenever the user enters his User Name and Password, ELIZA, will get all the relevant data and previous talks, related to the user, from the database itself.
A prospect for the cache Memory can be made, so as make the retrieval of data and information can be faster. I had written a Research Paper on this topic one year ago, Intelligent Information Retriever - Inception of Artificial Intelligence in Search Engines. Follow the link to download it if you are interested.
Standard Search algorithm can be used for the faster or better search of the relevant result or answer.
Various Graphical signs and/or symbols can be incorporated to show emotions, making the conversations more lively and more realistic in nature.

Important Note About Sample Code:

Please note that the current functionality and features of this program are very limited and they are just for accompanying the article. If you want to make this program more intelligent, make entries in Eliza.Dat file.

You can also increase the string manipulation power of the program, like considering multiple lines from the user, etc. I had written this code in 1 1/2 hr. just to make it more easier for the readers of my article about what is happening.

HOW SMART YOU MAKE YOUR ELIZA DEPENDS ON HOW FAR YOU EXTEND THIS PROGRAM. THERE IS PRACTICALLY NO LIMIT !
THIS CODE IS THE MINIMAL WORKING SKELETON !!

Don't forget to read README.TXT before making conclusions about the program !

Complete Source Code:

Included in the accompanying Zip File.