Facebook negotiation chatbots create secret language
Occurred: August 2017
Report incident ๐ฅ | Improve page ๐ | Access database ๐ข
Two AI-powered chatbots which were designed to learn how to negotiate developed a form of communication that appeared to be gibberish to human observers.
Facebook's Artificial Intelligence Research (FAIR) group developed two chatbots, named Alice and Bob, that were intended to be capable of negotiation and used techniques like self-play and dialogue rollouts to improve their performance.ย
The chatbots created an efficient shorthand language to communicate with each other that appeared to be gibberish but which in fact was a result of not being incentivised to stick to English-language rules. The behaviour is a well-established phenomenon where AI agents develop shorthands or "codewords" to maximise efficiency.
The episode was seen to highlight the complexities of developing AI systems able to understand and use natural language effectively, as well as the challenges in creating AI that can negotiate and reason strategically.ย
It was also seen to demonstrate the need for careful consideration of communication protocols and standards as AI systems become more advanced and potentially develop their own efficient ways of interacting.
Operator:
Developer: Meta/Facebook
Country: Global
Sector: Business/professional services
Purpose: Simulate negotiations
Technology: Machine learning
Issue: Incentivisation
Research, advocacy ๐งฎ
News, commentary, analysis ๐๏ธ
https://www.dailymail.co.uk/sciencetech/article-4747914/Facebook-shuts-chatbots-make-language.html
https://www.thesun.co.uk/tech/4141624/facebook-robots-speak-in-their-own-language/
https://nypost.com/2017/08/01/creepy-facebook-bots-talked-to-each-other-in-a-secret-language/
https://www.snopes.com/fact-check/facebook-ai-developed-own-language/
https://nypost.com/2017/08/01/creepy-facebook-bots-talked-to-each-other-in-a-secret-language/
https://www.fastcompany.com/90132632/ai-is-inventing-its-own-perfect-languages-should-we-let-it
Page info
Type: Issue
Published: August 2024