AI without the hype - or scaremongering

Amazingly, the software that started the current cycle of excitement about so-called artificial intelligence is still less than a year old. ChatGPT was released on 30 November 2022. And, while much that has been written about AI is undoubtedly nonsense, it cannot be denied that generative AI software is a game-changer.

Just how far it has come in the legal sector was demonstrated this week when representatives of half a dozen organisations - law firms, in-house teams and alternative legal services providers - shared real experiences of deploying the new wave of software. None will be named as the event was held under the Chatham House Rule - the (often misunderstood) convention that participants at an event are free to use the information received, but may not reveal identities. 

The theme was 'How, not hype'. Mostly this was adhered to, in a free exchange of experiences featuring a refreshingly frequent use of the words 'I don't know'. But what is absolutely clear is that generative AI is already being deployed widely across the sector with, in some cases, measurable benefits.

In the words of one speaker, GenAI's big breakthrough is that it lowers the barrier to entry, allowing organisations of all shapes and sizes to innovate. The pace of implementation is startling: one system described at the event went from initial concept to enterprise-wide deployment in just seven months. One participant observed that they had no budget for their project: nobody knew it was coming when plans for the financial year were set. 

A common lesson learned was the need to tread a path between the hype merchants who say AI is capable of everything - specifically, doing lawyers' jobs - and those who dismiss it as useless. 'There’s a massive chasm between expectations of generative AI and its capabilities,' one participant observed. Such expectations can drive lawyers into defensive mode. 

The way to get users on board is to be upfront about the technology's limitations: it is there to augment lawyers' work, not to take it over. 'When we explain this their reaction totally changes,' we heard.  

But what work is it actually doing? The event heard of use-cases including contract drafting, preparing witness statements from transcripts and translating documents. In all these cases it provides only the first run at a task; human input is essential. However the machine itself can provide the vital 'second pair of eyes' on a human job. 

And how good is it, really, at these tasks? Much depends on the quality of the questions it is asked - and users need to be trained to get this right. Expect to hear much this year about the new discipline of 'prompt engineering'. Several users talked of difficulties in handling large documents, which need to be divided up. But one organisation was confident enough to put figures on real productivity gains: In summarising documents, a 75% boost; legal research 60%; reviewing and analysing documents 50%; drafting documents 40%.

If all this sounds too good to be true, the event heard plenty about obstacles to deployment. 

One is the sheer cost of enterprise-wide systems. While participants predicted that competition between suppliers would bring prices down, the technology is up against the laws of physics. Soaring demand for GenAI is already overloading cloud-based systems - with a not inconsiderable environmental impact. 

The other much-talked-about obstacle is the use of client data for training systems. ‘Clients have very different expectations about what you can and cannot do with their data,’ a participant observed. Here it is vital to stress that no legal professional should be feeding real client material into ChatGPT - we are talking about walled-off systems. One firm revealed that is has been putting a clause on the use of data for training internal AI systems into its client terms for four years. 

Overall, it is essential to 'recognise that this is brand new and that not everything is going to be perfect at the start,’ a participant said. As with other investments there will be a ‘valley of despair’ before returns start to kick in. 

The lesson is that, even in its infancy, GenAI is here and it will find niches in law. Will it transform the whole sector? A poll on when GenAI will have an impact on the financial health of legal services providers found 90% of the admittedly self-selecting audience saying within the next five years - and half of those though it would happen within the next 24 months. One of the more bullish participants was even more specific: ‘In 10 months if you’re not using this you’re not going to be competitive.’ 

But on the tedious of topic of 'is this going to put me out of work' even the enthusiasts seemed divided. A snap poll on will there be more or fewer lawyers in 2033 found 52% predicting more, 18% fewer and 35% 'about the same'. 

Perhaps the safest bet on GenAI is that both the doom-mongers and nay-sayers will be disappointed.