Using AI tools for legal analysis carries real risks for non-lawyers and for the organisations that employ them. Key concerns include confidentiality breaches, AI “hallucinations” (fabricated legal references), outdated or wrong-jurisdiction advice, unintentional waiver of legal professional privilege, and gaps in professional indemnity insurance cover. This article outlines those risks and the practical steps organisations can take to manage them.
AI is now ubiquitous. But for all of its benefits, there are of course dangers. A reasonable analogy is with the increased use of “Dr Google” in recent years. While using internet searches to self-diagnose can be of some assistance, it is at best a very initial overview and you should never rely upon it to the exclusion of a medical examination by a health professional and their in-person diagnosis.
AI legal reviews can have significant and far-reaching commercial implications. This is not a detailed analysis of all of the issues, merely an overview of some key considerations. Of course, all of these will vary depending on the AI tools you use and how you use them.
Who should be using AI tools for legal analysis, and who shouldn’t?
The old saying that “a little knowledge can be a dangerous thing” was never more apt. AI analysis is available free through numerous platforms with no access restrictions. If organisations do not place limitations on who can use what analysis and for what purposes, then there is a real danger that inexperienced personnel may utilise AI as part of their decision-making processes without full knowledge of the broader implications. Anyone who has read AI analysis will be aware that it is generally quite well-written or phrased and can be very persuasive. Without a broader knowledge base of the subject matter in question, a person may not recognise fundamental errors or omissions (or even a generally wrong approach) in the analysis. There is a significant difference in a thirty-year veteran’s understanding of an issue and their ability to genuinely question and interrogate an AI finding compared to that of a first-year cadet. This is compounded by the way we communicate in business and on projects today. Twenty years ago, communications were sent by hard copy and generally had to be signed off first by a senior person. Today, email, text, document management systems and other social media allow instant communication, often without oversight. Put simply, anyone who lacks the experience to interrogate an AI output should not be relying on it for decisions without senior review.
The takeaway is that AI use must be managed and supervised in the same way that other elements of business are. Consider developing a business AI policy for personnel to follow.
A related consideration is consistency of position. If a tool (or a saved prompt, project workspace or conversation history) has been used to develop a position for one particular project – amendments, special conditions, a negotiating stance – there is a real risk that the same position is carried across to unrelated projects where it does not fit. Where there is training and a protocol, outputs can be reviewed by the relevant technical and legal experts so that a project-specific position does not quietly become the default.
What are the risks of using free AI tools for legal work?
The distinction that matters is less free versus paid than consumer versus enterprise. A consumer product (whether free or on a personal paid subscription) is generally used under standard terms that permit the provider to retain and train on what you input. An enterprise or contracted deployment is usually governed by negotiated terms that do not. There is a big difference in the utility and risks between the two. A number of the key areas of concern are as follows.
Does uploading documents to an AI tool breach confidentiality?
Many tools (particularly consumer products) retain and reserve the right to use the data you input. This means your data is utilised and may be disclosed to third parties. Inputs may be used to train the model and, in some circumstances, could surface in outputs provided to other users. This is significant on a number of levels.
Putting documents through an AI tool can breach confidentiality obligations both under general arrangements with other parties (such as a non-disclosure agreement) and under the very documents you are reviewing. Almost every commercial contract of significance has a confidentiality clause that prohibits disclosure to third parties, and uploading that contract to a consumer AI platform may well amount to exactly that. The AI platform’s own terms typically confirm that data is processed, stored, and potentially used for training. The consequences could include the counterparty withdrawing an invitation to tender, terminating the contract, a damages claim or injunctive relief.
It can also potentially breach privacy legislation obligations – particularly where you give access to a whole document that has personal information included. Data may also be stored or processed in data centres outside Australia, which engages the cross-border disclosure rules under the Privacy Act 1988 (Cth) (Australian Privacy Principle 8) and may have further data protection implications.
Is your data secure when using AI tools?
Hand in hand with confidentiality, there are significant concerns about the data security of AI platforms, particularly consumer products. Both of these considerations are key to why there should be a clear, enforceable business AI policy and, as part of that, the tracking of AI use. If you do not know who is using AI in your business and what they are uploading, then how do you know what, if any, real or potential data and cyber security breaches have occurred? A limitation on using personal computers for work-related AI usage is also fundamental in this regard.
What are AI hallucinations, and why are they dangerous in legal analysis?
The tools you use, and how much control you have over their parameters, are key. AI tools can generate factually incorrect, made-up and misleading information. This is often presented as correct (and, as noted above, in very compelling or persuasive language). AI works by predictive methodology. It is not always correct. AI can give incorrect conclusions and even generate non-existent case citations or legislative references. Court records show that lawyers have made this mistake. It is therefore even less likely that a non-lawyer will recognise an incorrect citation or the incorrect use of a case or legal principle. Purpose-built legal tools that ground their answers in a verified database of legislation and case law, and cite the source, can significantly reduce hallucinations. General-purpose consumer tools offer few such controls.
This raises another key issue: training. Whatever tool is used, a key element of a business AI policy is training staff on the correct and appropriate use of AI tools and the information they generate. You would not let a person on site without safety inductions, and the same principles should apply to AI use.
Is the AI tool you are using up to date?
You need to ensure that the AI tools being used are up to date. Legislation, case law and even common understanding of contract processes are constantly changing and evolving. Every AI model has a training cut-off date, and a tool will not know about developments after that date unless it is connected to current sources. Even tools that can search the internet will rely on whatever they happen to find, and the user often has no way of knowing whether the answer reflects the current law. If the AI tool is out of date, then so is the advice it is generating.
Does AI apply the right laws for your jurisdiction?
Many AI tools are developed and based in overseas jurisdictions. The danger is that the advice, legal principles, legislative references and case law cited are not relevant to or applicable in Australia or even across different individual states in Australia.
It is not uncommon that legislation will be referenced generally without full particulars of the jurisdiction. This is particularly problematic where different jurisdictions often use the same or similar names for their legislation. For example, workplace health and safety, privacy and security of payment often have similar naming conventions across states and even countries.
What will AI miss if you don’t ask the right questions?
AI tools will review only the information and documents they are given and, depending on the complexity and settings of the tool, may answer only the questions asked. While not strictly a case of “rubbish in rubbish out”, what you give an AI tool and what you ask the tool to do will limit what outputs you get. For example, if you simply input a document and ask AI to analyse it for risks, then you may get very limited assistance or useful information.
AI may not tell you, for example, what is missing. In a contractual context, if there is no limit on liability or exclusion of consequential loss or no exceptions to use of intellectual property or confidential information, AI may not necessarily raise these because they are not part of what it has been tasked to analyse. If you do not know what is absent, then you cannot consider whether AI has given comprehensive advice.
Similarly, AI will have no knowledge of previous negotiations or contracts between parties, business dealings or other intangibles in relationships. Unless provided with context, AI will not take into account ancillary correspondence, tender information, telephone calls and meetings or internal policies – all of which may be relevant to the final deal struck.
These can, of course, be added into the AI request for review, but it takes time, a knowledge of the tool you are using and consideration of what answers you are after to achieve this. If you ask the wrong or an incomplete question, you may get a wrong or incomplete answer.
Lastly, and very importantly, while a lawyer can explain how he or she reached a conclusion and be questioned on it, an AI tool cannot be questioned in the same way. Many tools will now set out their reasoning, but that explanation is a generated output like any other – it is not a reliable account of how the answer was actually produced, and it cannot be cross-examined. This makes it very difficult to audit or challenge the process. AI will not come to a board or senior management meeting to explain its “thinking”.
Can using AI waive legal professional privilege?
A very real risk that is often not considered is that where you enter information into a third-party AI tool you may waive legal professional privilege or other confidentiality rights.
The decision in Mann v Carnell [1999] HCA 66 provides that privilege in professional communications is impliedly waived where disclosure and use of the communication is inconsistent with maintaining the confidentiality provided by the privilege.
Uploading a legal advice, expert report, or solicitor’s letter to a consumer AI tool is arguably exactly that, and once privilege is waived, it cannot be recovered. Whether an upload to an AI platform amounts to a waiver has not yet been tested by an Australian court, but the risk is real.
What makes this really dangerous is the potential broad application of subject matter waiver: a court could order disclosure of all related privileged advice on the same topic, not just the document that was uploaded.
This is particularly relevant with respect to legal advices and expert reports. Getting a second opinion by AI may cause significant issues in this regard. This should always be checked with your lawyers first.
Where there is training and a business AI policy, a “stop and think” hold point or checklist can be included for all users to ensure that they always ask themselves – Does this document contain any confidentiality requirements or advice such as legal advice that may be privileged? If yes to either then stop and obtain advice before proceeding further.
What are the statutory and insurance risks of using AI for legal analysis?
Persons using AI to generate advice (particularly those with legal qualifications but who are not admitted to practice) must be careful to ensure they are not holding themselves out as providing legal advice in circumstances where they are not authorised to do so. This can attract significant sanctions in each state (in Queensland, for example, under the Legal Profession Act 2007 (Qld)).
Courts are increasingly regulating, and requiring disclosure of, AI-generated content in materials presented to them – see, for example, the Supreme Court of New South Wales Practice Note SC Gen 23 (Use of Generative Artificial Intelligence) and the guidelines issued by the Supreme Court of Victoria and the Queensland Courts. All of these areas are constantly evolving as we respond to the deluge of AI-generated information and its impacts.
Finally, and possibly the most significant impact, is on insurance. If a non-lawyer in your business prepares what amounts to legal advice using AI, your business is unlikely to have professional indemnity cover for it and, unlike advice obtained from a law firm, there is no lawyer’s professional indemnity policy standing behind it. Insurance generally is another area where there are significant changes happening. It is increasingly likely that we will see more and more limitations on cover where AI has been utilised to prepare or develop systems, provide professional services or to deliver legal analysis and respond to issues.
Key takeaways: how to use AI responsibly for legal analysis
Although AI is a very useful addition to the array of tools that can be used for contract development and analysis, it has limitations and should always be viewed with a healthy dose of scepticism. It will not replace technical or legal knowledge and experience, and it cannot negotiate face to face with a business counterparty. Where it is used, there should be clear rules on when, how, where (at work, not at home on personal devices, for example) and by whom to ensure it is managed and the advice and outcomes are traceable and able to be verified.
Give careful consideration to the questions you ask and the information you give to AI, both to ensure you are not breaching privacy, confidentiality or privilege and that you are correctly targeting the outcomes or advice you want.
Check your insurance, particularly when it is renewed each year, to ensure that AI use is not affecting the cover you expected to hold.
This article may provide CPD/CLE/CIP points through your relevant industry organisation.
Carter Newell Sovereign AI may have been used to assist in the development of this article.
The material contained in this publication is in the nature of general comment only, and neither purports nor is intended to be advice on any particular matter. No reader should act on the basis of any matter contained in this publication without considering, and if necessary, taking appropriate professional advice upon their own particular circumstances.
FAQs
Find quick answers to common questions about this topic.
Can non-lawyers use AI to review contracts?
Yes, but with significant caution. AI can provide a useful starting point, but it cannot identify what is missing from a contract, account for prior negotiations or context, or guarantee that the legal principles it applies are current and relevant to Australian law. Non-lawyers should not rely on AI analysis without appropriate oversight and, where the stakes are significant, legal review.
What is an AI hallucination in a legal context?
An AI “hallucination” is where an AI tool generates information that is factually incorrect or entirely fabricated, including non-existent case citations, incorrect legislative references, or wrong legal principles, and presents it as accurate. This is a particular risk in legal analysis because a non-lawyer may not have the knowledge to recognise the error.
Does uploading a contract to an AI tool breach confidentiality?
It may. Almost every commercial contract of significance contains a confidentiality clause prohibiting disclosure to third parties. Uploading that contract to a public AI platform, which retains and may use that data, is arguably exactly that. The consequences can include damages claims, injunctive relief, or termination of the contract.
Can using AI waive legal professional privilege?
Yes, it can. Uploading a legal advice, expert report, or solicitor’s letter to a third-party AI tool may constitute an implied waiver of privilege under Mann v Carnell [1999] HCA 66. Once privilege is waived, it cannot be recovered and a court could order disclosure of all related privileged advice on the same subject matter.
Is AI-generated legal analysis covered by professional indemnity insurance?
Not necessarily. If a non-lawyer prepares what amounts to legal advice using AI, professional indemnity insurance may not apply. Insurance policies are also increasingly introducing limitations on cover where AI has been used to generate professional advice or analysis. Check your policy carefully and review it each time it is renewed.
What should a business AI policy include?
A business AI policy should address: who is authorised to use AI tools and for what purposes; which tools are approved; prohibitions on using personal devices for work-related AI tasks; data security and confidentiality obligations; training requirements; and a “stop and think” checklist for any document that may be confidential or subject to legal professional privilege.
