AI Governance Part 3: Court Case Incident Risk Mitigation Techniques Analysis of AI Hallucinations
For the previous analysis in Part 1 and Part 2, we looked at AI Incident 541. In this particular incident ChatGPT reportedly produced false court cases that were then presented as legitimate examples by legal counsel in a court of law. This writing will address potential risk mitigation techniques for those negative consequences that arise from these types of AI incidents.
The underlying cause of negative consequences in this incident is AI hallucination. We will analyze two critical secondary effects of this problem and potential risk mitigation strategies. The first being false information generated via an AI hallucination being considered valid and the second a situation where a false verdict was reached due to said false information.
False information considered real via a failed validation process
In the case of AI Incident 541 it seems that someone at the court realized that the information was indeed incorrect before the final verdict was given. In this example, we are assuming that the supporting case examples generated by AI would count as evidence in the court case since they are used to come to a specific verdict. It is important to consider two different kinds of digital evidence:
Computer records - Recording of information and storing it digitally. Computer records might include something like a timestamp of an email.
Computer generated evidence - Generating new information based on previous information. This would include generating examples of court cases via ChatGTP.
Both types of digital evidence could run into issues of reliability. We know that with the creation of “deep fakes” it's becoming increasingly difficult to tell if digital records such as photos or videos are real or generated. For the sake of this assignment, we will focus on computer generated evidence since this is how we would categorize AI hallucinations.
How reliable is data generated through a tool such as ChatGTP and how could a court of law validate it? As per the US Federal Rules of Evidence, there are specific criteria in place for evaluation evidence and there have been recent modifications to the law to better support AI generated evidence.
One mitigation strategy is to make sure that the laws are updated appropriately to cover new use cases involving AI. At the core of most current AI related evidence laws, it is the responsibility of the defendant to explain exactly how the AI generated content was generated and why it is valid. They can usually either do this themselves or provide an expert who will speak with the judge. Therefore, another risk mitigation strategy is making sure that there is a trustworthy human with enough AI expertise present to make a good judgment call.
Invalid verdict is reached
In the worst case scenario where the validation process fails, a judge would unknowingly produce an incorrect verdict. In these cases, it may be necessary to expand the law to allow for an extended appeal process in AI cases where there was not enough AI expertise present at the court to make an accurate judgment call.
In more extreme examples where AI might even be used to make the verdict, it is critical to still include humans in the decision making process. Even though as of today AI plays a limited role in most courts of law, it may be that this increases overtime. In this way, it becomes critical to explicitly include human veto power for any decisions that an AI makes concerning the human life of any individual.
There should always be an appeal process for any AI-based decision even if it takes additional time for a verdict to be reached. This allows for initial verdicts perhaps to be reached more quickly using AI, which is hopefully a more efficient way of quickly coming to verdicts, but then allows a safety fallback to a human. This will allow the system to fix any decisions made in error due to AI but allow for some degree of increased efficiency and the adoption of AI in the court system which is likely evitable. Beyond this, education on AI hallucination awareness to legal experts and judges would also be a decent risk mitigation strategy.
Additional potential problems:
Humans lose their ability to know what is an AI hallucination and what is not as a species
There is a shortage in AI experts and courts are unable to process AI based information or are coming to incorrect verdicts
Invalid verdicts are reached and justice is not implemented with no way for those affected to appeal the ruling
Conclusions:
AI does have the potential of increasing efficiency in courts of law, but retaining a human element in the decision making and validation process is absolutely critical to maintaining a functional judicial system. The greatest potential risk is simply reaching invalid verdicts due to various reasons described above. It would likely be easiest to implement additional checks and balances into the judicial system via updating laws on AI evidence validation as a risk mitigation strategy as opposed to educating judges on AI. That being said, it might be easier to require AI experts to be present when any decision involving AI is being made.
Citations:
This AI governance series was created based on the coursework that I complete for my AI Goverance course with Anna Bethke via ELVTR’s AI Governance program. I highly recommend the course and Anna Bethke for the incredible knowledge that I gained related to the future of AI governance. @AnnaBethke
