Playing it safe with AI

This arrestingly titled new work is another addition to the burgeoning literature on AI and the law. The subject is the regulation of AI in high-risk civil applications – situations where there is a risk of serious physical harm should AI fail.

It begins with a thoughtful exploration of how AI might deal with difficult moral questions. The answer, as the author accurately states, is that AI would not ‘decide’ moral questions in any meaningful sense. Its programmers would have already done so, since AI is ultimately no more than a composition of probabilities and sensors.

The author then considers the civilian AI applications of self-driving cars, the medical system and product liability (explicitly omitting military uses). She looks at the tools for AI development, training and analysis, and how they interact with legal concepts. She covers Singapore, the EU, the US and the UK, chosen because they have some of the more advanced AI technology and hence have (or should have) the early lead in legal considerations as well. The inclusion of the EU means both civil law and common law jurisdictions are discussed.

She concludes with a proposed regulatory ‘blueprint’ based upon the ‘reasonable human counterpart’ and ‘reasonable producer’ standard. In summary, she argues that where potential harm is known to be high, the general regulator (central government) should have the power to decide if an application is too risky to be acceptable. The regulator should also have the power to require certain disclosures and design choices from AI producers. Regulators of defined sectors should have the power to set standards and expectations in their field (if not, the general regulator should do so). Where incidents occur, an inquisitorial arm of the state will have the mandate to investigate the AI producer’s records to determine if it has breached the standard of a reasonable human performing the same task and/or the standard of a reasonable producer of that speciality. A due diligence defence would be available to the latter charge, but compensation should be payable either way. An inquisitorial arm of the state would not bar the consumer from other remedies, including usual consumer protection litigation.

That approach raises questions in principle (why an inquisitorial arm of the state as opposed to private litigation?) and will doubtless raise many more in any real-life incident. But it constitutes a sound starting point for an increasingly urgent discussion. The book is recommended accordingly.

James Wilson FRHistS is an independent legal author. His most recent book is Lord Denning: Life, Law and Legacy (Wildy, Simmonds & Hill, 2023)