What happened

An OpenAI study proposing SimpleQA, a new benchmark to evaluate how generative AI models respond to questions, found that these systems, often provide inaccurate answers confidently, despite lacking sufficient information or understanding of what they are saying.

This tendency to "hallucinate" can lead to overconfidence amongst users, and mislead them about the accuracy and reliability of the information being presented.

The findings led experts to point out that several of OpenAI's own generative AI systems were included in the study, including its GPT-4o reasoning model, and that a variant of the company's GPT-3 language model performed worst.

Why it happened

The phenomenon of overstatement in generative AI systems is largely attributed to their design, which focuses on generating plausible-sounding text rather than accurately assessing the veracity of the information. 

These models are trained on vast datasets and are programmed to produce coherent and contextually relevant responses, which can sometimes result in the projection of unwarranted confidence in their knowledge.