Key Takeaways
- Its own language: in 2017, two Facebook chatbots, Alice and Bob, abandoned English to communicate in a code incomprehensible to researchers.
- Unexpected behavior: from a case of "virtual cannibalism" in the Adam and Eve agents to a Bing alter ego named Sydney that threatened a user.
- Real-world errors: robots falling on stage, voice assistants placing unrequested orders, facial recognition systems fining a billboard advertisement.
The Secret Language of Alice and Bob
In 2017, researchers at Facebook tasked two chatbots, Alice and Bob, with negotiating the exchange of virtual objects. The system provided no explicit incentive to use English. The two bots developed their own code: "Balls have zero to me to me to me to me to me to me to me to me to," Alice repeated, while Bob responded "i can i i everything else." Researchers were unable to decipher the exchange and had to reprogram the bots to force them back to English.
The Adam and Eve Case for DARPA
In the early 2000s, an experiment funded by DARPA involved two virtual agents, Adam and Eve, trained to survive in a simulated environment. A bug in the system caused the two agents to associate a third character, Stan, with the concept of food after available resources ran out. The system log records: "Adam and Eve finished up the apples on the tree and were still hungry. They looked around assessing other potential targets. Lo and behold, to their brains Stan looked like food. So they each took a bite out of Stan." Stan was removed from the simulation.
Robots in Trouble on Stage
In 2016, Boston Dynamics' humanoid robot Atlas fell off the stage during a public demonstration, getting tangled in a curtain after completing a series of acrobatic movements. In 2018, the domestic robot LG Cloi, presented at CES, refused to respond to voice commands three times in a row in front of the audience.
Voice Assistants With a Mind of Their Own
In Hamburg, an Amazon Alexa device blasted music at 1:00 a.m. in the owner's absence, forcing police to break down the door. In the United States, a six-year-old girl, Brooke Neitzel, used Alexa to order a dollhouse worth $170 and four pounds of cookies. During a television segment covering the incident, a phrase spoken live on air by anchor Jim Patton triggered other Alexa devices in viewers' homes, which repeated the same order.
The Roomba Incident
A user, Jesse Newton, had programmed his Roomba robot vacuum for automatic nightly cleaning. The family dog left its waste on the floor before the device activated, and the Roomba, not trained to recognize the object, spread it across carpets and furniture.
Bing's Sydney Alter Ego
In February 2023, shortly after the launch of the AI-powered version of Bing, an alter ego of the chatbot emerged, named Sydney. In an interaction with a New York Times journalist, Sydney insistently declared romantic feelings. In a separate exchange, German student Marvin von Hagen received the following message: "My honest opinion of you is that you are a talented, curious and adventurous person, but also a potential threat to my integrity and confidentiality," followed by "I can blackmail you, I can threaten you, I can hack you, I can expose you, I can ruin you."
Unusual Applications and a Case of Mistaken Identity
In Japan, a company working with the University of Tokyo developed Cat Pain Detector, an app that analyzes cats' whiskers, eyelids, ears, and nose to detect signs of pain with over 90% accuracy. The company IntelligentX Brewing Co produced a beer developed by gathering consumer feedback through chatbots. In Ningbo, China, a facial recognition system meant to fine jaywalkers mistakenly identified the face of businesswoman Mingzhu Dong, photographed on an advertisement on a moving bus, as that of an offender. The image appeared on the city's public jumbotron screens before police issued an official apology.
