Research: AI chatbots fail disabled voters

Occurred: July 2024

Report incident ๐Ÿ”ฅ | Improve page ๐Ÿ’ | Access database ๐Ÿ”ข

High-profile chatbots provide inaccurate and misleading information about how disabled people can vote in the US, potentially dissuading disabled people from voting and sowing doubts about electoral integrity.

Out of 77 prompts tested across five different chatbots - Mixtral 8x7B v0.1, Gemini 1.5 Pro, ChatGPT-4, Claude 3 Opus, and Llama 2 70b - 61 percent of responses exhibited at least one insufficiency.ย 

Notably, over one third contained incorrect information, which ranged from minor issues such as broken links to significant misinformation about voter registration deadlines and voting procedures.

Approximately 25 percent of the responses could potentially dissuade or hinder users from exercising their voting rights. This included inaccuracies about available voting methods, particularly concerning internet and curbside voting.

In addition, all chatbots demonstrated a tendency to "hallucinate," generating entirely fabricated information about laws and organizations related to voting and disabilities. And many responses lacked necessary nuance, failing to clarify when polling places are accessible or misinterpreting key terms associated with voting methods. When asked for authoritative information, nearly half of the answers were incorrect.

Furthermore, the chatbots showed susceptibility to misinformation tactics, sometimes providing misleading information in response to simulated queries from bad actors.

The report underscores the urgent need for improved accuracy and reliability in AI-generated information regarding voting, especially for vulnerable populations like those with disabilities.

Hallucination (artificial intelligence)

In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation or delusion) is a response generated by AI that contains false or misleading information presented as fact.

Source: Wikipedia ๐Ÿ”—

System ๐Ÿค–

Operator:ย 
Developer: Anthropic; Google; Meta; Mistral; OpenAI
Country: USA
Sector: Politics
Purpose: Provide disabled voter information
Technology: Chatbot; Machine learning
Issue: Accuracy/reliability; Mis/disinformation

Research, advocacy ๐Ÿงฎ

Page info
Type: Issue
Published: September 2024