Safety isn’t just about a machine "breaking"; it’s about how a system behaves when things go wrong. It involves:

• Robustness. Ensuring an AI system can handle "noise" or unusual data without crashing or acting erratically.

• Control & oversight. The ability for humans to intervene or shut the system down (the "kill switch" problem).

• Predictability. Ensuring the system's actions are consistent so humans can safely work alongside it.

• Alignment. Making sure an AI system’s goals actually match the user's intent so it doesn't take dangerous shortcuts to finish a task.

Why it matters

As AI moves from our screens into the physical world, such as driving cars, performing surgery, or managing power grids, the stakes shift from "annoying software bugs" to "life-or-death risks." Society relies on a "social contract" where we trust that the technology around us won't suddenly become a threat to our physical well-being. 

Impacts & consequences

Poor safety can have serious results, from serious physical injury and loss of life to psychological trauma, financial loss, and the erosion of trust in public and other institutions. 

Causes & contributing factors

Common sources of AI safety failures include:

• Inadequate testing. Systems are released before failure scenarios are fully understood.

• Distribution shift. AI trained in one context performs poorly in a different real-world context.

• Overconfidence in AI. Users and organisations trust AI outputs without sufficient scepticism or oversight.

• Pressure to deploy quickly. Commercial and competitive pressures lead to shortcuts in safety processes.

• Complexity and opacity.  AI systems (especially large neural networks) are difficult to fully understand, making it hard to predict when they will fail.

• Misuse. Bad actors deliberately exploit AI systems to cause harm.

• Misaligned objectives. AI systems optimise for the wrong thing, causing unintended harm as a side effect.