The Turing Test: What Is It, What Can Pass It, and Limitations

What Is the Turing Test?

The Turing Test is a deceptively simple method of determining whether a machine can demonstrate human intelligence: If a machine can engage in a conversation with a human without being detected as a machine, it has demonstrated human intelligence.

The Turing Test was proposed in a paper published in 1950 by mathematician and computing pioneer Alan Turing. It has become a fundamental motivator in the theory and development of artificial Intelligence (AI).

History of the Turing Test

Alan Turing developed some of the basic concepts of computer science while searching for a more efficient method of breaking coded German messages during World War II. After the war, he began thinking about artificial intelligence. In his 1950 paper, Turing began by posing the question, "Can machines think?" Predicting that computers would evolve beyond the limitations of the time, Turing designed a test he thought could determine whether a computer could be "smart" enough to be mistaken for a human.

The test is conducted in an interrogation room run by a judge. The test subjects, a person and a computer program, are hidden. The judge has a conversation with both parties and attempts to identify which is the human and which is the computer based on the quality of their conversation. Turing concludes that if the judge can't tell the difference, the computer has succeeded in demonstrating human intelligence. That is, it can think.

Several early computers have been claimed to have fooled humans in very basic situations. In 1966, Joseph Weizenbaum created ELIZA, a machine that took specific words and transformed them into full sentences. ELIZA was one of the earliest computers to fool human testers into thinking it was human.

Less than a decade later, a chatbot named PARRY was modeled to imitate the behavior of a paranoid schizophrenic. A group of psychiatrists were asked to analyze conversations with real patients and PARRY conversations. When asked to identify which transcripts were computer programs, the group was only able to identify the machine 48% of the time. Critics of both ELIZA and PARRY state that the full rules of the Turing test were not met and do not indicate full machine intelligence.

The Turing Test Today

The Turing Test has its detractors, but it remains a measure of the success of artificial intelligence projects. One updated version of the Turing Test has more than one human judge interrogating and chatting with both subjects. The project is considered a success if more than 30% of the judges, after five minutes of conversation, conclude that the computer is a human.

The Loebner Prize is an annual Turing Test competition that was launched in 1991 by Hugh Loebner, an American inventor and activist. Loebner created additional rules requiring the human and the computer program to have 25-minute conversations with each of four judges. The winner is the computer whose program receives the most votes and the highest ranking from the judges.

In 2014, Kevin Warwick of the University of Reading organized a Turing Test competition to mark the 60th anniversary of Alan Turing’s death. A computer chatbot called Eugene Goostman, who had the persona of a 13-year-old boy, technically passed the Turing Test in that event by securing the votes of 33% of the judges.

In a research project in February 2024, ChatGPT supposedly passed the Turing Test. However, the testing compared the artificially intelligent bot's responses to randomly sampled human responses (students) to predetermined questions, then scored the responses based on the "Big Five" personality traits. The researchers concluded, "...that AI and human behavior are remarkably similar." This is not surprising, as the program's training included material written by humans from a wide array of sources. As it uses ideas, thoughts, information, and data provided and created by humans, its responses will naturally seem human, creating doubts among some about it passing the test. Still, others believe it passed.

Turing Test Versions

There are several variations of Turing tests, all with the same intention of detecting whether a respondent is a human or a machine. Each variation takes a different approach in asking the respondent different questions and evaluating the responses.

Imitation Game

One of the earlier applications of the Turing test, the imitation game version often utilizes three parties. The first person was a male, the second person was a female, and the third person was responsible for determining the gender of the first two people. The first person is often tasked with trying to trick the third person, while the second person is often tasked with trying to help the third person correctly identify each gender.

Future iterations of the imitation game have evolved into both parties attempting to trick the third person into incorrectly identifying the genders. In any case, the objective of the imitation game is to determine whether an interrogator can be fooled.

Standard Interpretation

Another common version of the Turing test does not strive to see whether a computer can be fooled but rather to see whether a computer can imitate a human. In the standard interpretation variation of a Turing test, the first person is a computer and the second person is a human.

In this variation, the third person attempts to discover which of the first two people is a human and which is a computer. The interrogator is not the subject being tested; instead, it is the computer that is trying to fool the human (as opposed to the opposite direction under the imitation game). For example, it may be asked a series of personal finance questions to determine whether or not its responses are reasonably expected regarding behavioral finance.

Modern Approaches to the Turing Test

Since the creation of the Turing test, more modern approaches have evolved in an attempt to better detect humans and machines. These variations of the Turing test are continually evolving to maintain relevance during technological advancements.

• The Reverse Turing Test aims to have a human trick a computer into believing it is not interrogating a human.
• The Total Turing Test incorporates perceptual abilities and the ability of the person being questioned to manipulate objects.
• The Marcus Test has test subjects view media and respond to questions about the content.
• The Lovelace Test 2.0 has test subjects create art and examines their ability to do so.
• The Minimum Intelligent Signal test asks test subjects only binary questions (i.e. true/false or yes/no answers).

Limitations of the Turing Test

There are many critics of the Turing test, and the variations above attempt to mitigate some of the limitations of the original Turing test. Still, it is important to be mindful of the downsides of the Turing test and where its analysis may fall short:

• The Turing test requires a very controlled environment to be performed. Test participants must be hidden from view of each other during the entirety of the test, though the parties must have a reliable means of communication.
• The Turing test may not be suitable as a test for intelligence as different computing systems are structured differently. Therefore, there may be inherent, natural limits to what a computer is capable of performing.
• The Turing test is evolving; however, technological advancements are evolving even faster. Consider Moore's Law which states the rapid growth of processing ability with the rapid decline in cost. As computers gain more capabilities, historical testing methods may no longer be suitable as computers gain more human-like capabilities.
• The Turing test assesses intelligence, though it may not be an appropriate gauge of all types of intelligence. For example, a computer may successfully fool an interrogator based on its ability to process responses like a human. However, this may not truly indicate emotional intelligence or awareness; it may simply mean the computer's coding had a highly skilled programmer.

The Bottom Line

The Turing test is an assessment to determine whether a machine is able to exhibit the same intelligence as a human. There are now many variations of the Turing test, and as technology continues to advance with AI at the forefront, new lines of thinking are emerging with regard to a means of determining intelligence.