Men's Journal publishes AI article riddled with errors

Occurred: February 2023

The first AI-generated article published by Men's Journal was found to be full of factual inaccuracies, raising questions about the accuracy and reliability of its technology system and editing processes.

The article 'What All Men Should Know About Low Testosterone' set forth multiple medical claims and advice, including recommending testosterone replacement therapy. 

However, medical expert Bradley Anawalt told Futurism that it contained 18 distinct errors, from basic factual mistakes and unsupported claims to sweeping mischaracterisations of medical science. The article was swiftly re-written and the disclosure removed, though no mention of its first incarnation and its mistakes is visible.

Each Men's Journal AI article has a disclosure at the top of the page stating 'This article is a curation of expert advice from Men’s Fitness, using deep-learning tools for retrieval combined with OpenAI’s large language model for various stages of the workflow. This article was reviewed and fact-checked by our editorial team.'

Days before, Men's Journal owner Arena Group had said it was partnering with AI companies Nota and Jasper 'to speed and broaden its AI-assisted efforts in content workflows, video creation, newsletters, sponsored content, and marketing campaigns.' 

Announcing the partnerships, Arena Group CEO Ross Levinsohn told the Wall Street Journal 'It’s not about ‘crank out AI content and do as much as you can.' 'Google will penalize you for that and more isn’t better; better is better.'

The incident comes weeks after CNET was revealed to have been quietly publishing AI-generated articles, some of which also turned out to be factually incorrect.

Operator: Arena Group/Men's Journal
Developer: Arena Group/Men's Journal; OpenAI; Jasper

Country: USA

Sector: Media/entertainment/sports/arts

Purpose: Automate journalism

Technology: Large language model (LLM); NLP/text analysis; Neural networks; Deep learning; Machine learning
Issue: Accuracy/reliability; Mis/disinformation

Transparency: Governance; Marketing

Page info
Type: Incident
Published: February 2023