We asked ChatGPT for legal advice – here are five reasons why you shouldn't
At some point in your life, you are likely to need legal advice. A survey carried out in 2023 by the Law Society, the Legal Services Board and YouGov found that two-thirds of respondents had experienced a legal issue in the past four years. The most common problems were employment, finance, welfare and benefits, and consumer issues. However, not everyone can afford to pay for legal advice. Of those survey respondents with legal problems, only 52% received professional help, 11% had assistance from other people such as family and friends, and the remainder received no help at all.
Accuracy of AI Chatbots
Many people turn to the internet for legal help. With the rise of artificial intelligence (AI) chatbots like ChatGPT, Google Bard, Microsoft Co-Pilot, and Claude, the prospect of asking them legal questions may be tempting. While these tools are powered by generative AI and can explain legal information in a conversational style, their accuracy is often questionable.
Common Mistakes Found
In a study published in the International Journal of Clinical Legal Education, we tested several AI chatbots by posing legal questions. We discovered five common mistakes:
- Providing answers based on American law without specifying
- Omitting explanations on how laws differ by region, especially in the UK
- Referring to outdated legal information
- Inaccurate advice on family and employment queries
- Missing key aspects of the law in responses
Use of English Law
When prompted, the chatbots could provide answers based on English law, but this required additional instructions. It became evident that the accuracy of the responses varied, and some were not up-to-date with the latest legal changes.
Limitations of AI Chatbots
Although AI chatbots can offer well-written answers, determining their correctness without legal knowledge is challenging. In some cases, the responses lacked sufficient detail for individuals to address their legal issues effectively.
As the technology evolves, there might be improvements in AI chatbots' ability to provide reliable legal advice. Until then, seeking professional help remains the recommended course of action.
