What happened

A scathing report from Canada’s Auditor General, Karen Hogan, revealed that the Canada Revenue Agency's (CRA) chatbot, "Charlie," provided accurate responses to tax questions only 33 percent of the time during testing. 

Launched in March 2020 and used for over 18 million questions, the tool was intended to assist taxpayers with filing and benefits. However, the audit found that in a test of six standard tax-related questions, Charlie answered only two correctly - a performance significantly worse than general-purpose AI tools like ChatGPT or Claude, which answered five out of six correctly.

The financial and legal impacts are significant: the Canadian government has spent over USD 18 million on the project to date. While the CRA marketed the tool as a reliable guide, taxpayers who followed its erroneous advice have found themselves with no legal recourse. 

Under Canadian law, the taxpayer is solely responsible for the accuracy of their filing; receiving "bad advice" from a CRA chatbot or call center agent does not exempt an individual from paying the correct tax amount or the interest accrued on late or incorrect payments.

Why it happened

Several factors contributed to this systemic failure in transparency and accountability:

• Speed over accuracy: The Auditor General’s report highlighted that CRA performance evaluations focused heavily on "schedule adherence" (45 percent) rather than "accuracy and completeness" (less than 9 percent).

• Technological limitations: The original version of Charlie was a rule-based system that lacked the nuance to handle the complexity of Canada's Income Tax Act. While the CRA recently migrated to a generative AI beta, they admitted they cannot precisely verify its real-world accuracy without a manual review of millions of transcripts, which they have not conducted.

• Transparency gaps: The CRA continued to promote the tool to millions of users while internal metrics showed an accuracy threshold as low as 70 percent (the Auditor General's independent testing showed it was even lower).

• Legal shielding: Corporate and government accountability is limited by statutory protections that place the burden of "correctness" on the citizen, effectively allowing the agency to deploy flawed automation without the threat of liability for the resulting financial errors.