AI/automation ethics glossary
Accuracy & reliability
Embedded Files
Accuracy and reliability refers to the extent to which an AI/automation system behaves dependably, accurately, and consistently in the situation for which it is designed, and to the ethical consequences should it fail to do so.
What it entails
What it entails
Accuracy refers to how often an AI system produces the right answer or outcome; reliability refers to whether it does so consistently across different contexts, populations, and conditions. Together, they form a foundational expectation: that a system does what it claims to do.
This matters because AI systems are increasingly deployed in high-stakes domains (such as healthcare, criminal justice, social welfare, financial services ) where errors carry real consequences.
A diagnostic AI that misidentifies tumours, a welfare system that wrongly denies benefits, or a chatbot that confidently invents false information can cause material harm to individuals and erode public trust in institutions.
The ethical weight of accuracy failures scales directly with the stakes of the context in which a system is deployed.
Closely linked to accuracy is the problem of hallucination in generative AI: AI hallucination occurs when AI determines its response has a high degree of accuracy when it is demonstrably and undeniably wrong. This is especially dangerous when systems deliver fabricated information with apparent confidence.
Related terms
Related terms
Automation bias
Fairness
Human rights/civil liberties
Mis/disinformation
Safety
Causes
Causes
Common sources of accuracy and reliability failures include:
Poor or unrepresentative training data. Systems trained on incomplete, biased, or low-quality datasets generalise poorly to real-world conditions.
Distributional shift. Performance degrades when deployed in contexts that differ from training environments.
Overfitting to benchmarks. Systems optimised for test metrics may perform poorly outside those controlled conditions.
Hallucination in generative models. Language models that produce plausible-sounding but false outputs.
Inadequate testing and pre-deployment evaluation. Systems released without sufficient real-world stress-testing.
Opacity of proprietary systems. Vendors refusing independent audits, making external verification of accuracy claims impossible.
Overconfident design. Systems that do not communicate uncertainty to users, presenting guesses as facts.
Dilemmas
Dilemmas
Speed versus safety. Should a company deploy a "good enough" model to stay competitive, or wait until it reaches a higher threshold of reliability?
Confidence vs. honesty. AI systems designed to give fluent, confident responses may actively suppress expressions of uncertainty in order to appear helpful, prioritising user experience over accuracy. Who is responsible when this causes harm?
Human oversight and automation bias: If a system is 98% accurate, humans may stop paying attention, making them unable to catch the 2% of errors that are often the most critical.
Interpretability: If a model is highly accurate but functions as a "black box," can we ethically rely on it if we don't understand why it is making those choices?
Transparency and trade secrets. Independent verification of accuracy requires access to systems and data that vendors often withhold on commercial grounds. Should mandatory third-party auditing override proprietary interests?
Terms of use
Terms of use
You are welcome to use, copy, adapt, and redistribute this definition under a CC BY-SA 4.0 licence.
Let us know if you have any comments or suggestions about how to improve this definition, or would like to suggest and/or contribute additional terms to define.
Author: Charlie Pownall 🔗
Published: April 27, 2026
Last updated: April 27, 2026
Page updated
Google Sites
Report abuse