I keep meaning to write a post about my experiences with ChatGPT-4, which I refer to as the “Twenty-First Century Delphic Oracle,” or just “the Oracle.” Those who know her well call her “Pythia” or “the Pythoness,” depending, I guess, on how they feel about her, where they’re from, or what language they speak.
(By the way, I’ve also tried other AI Oracles, like Gemini, with similar experiences to what I will talk about here.)
tl;dr version: I wouldn’t trust AI for actual legal work as far as I can spit Saturn.
A “Housekeeping” Note
I’m going to try something to significantly shorten this post for most readers. Because I will include some discussion of specific issues where you might not care about the details, I’ll use WordPress’s “accordion” blocks to include those. So you can easily skip over them if you want, or click on them to expand each section.
Without the accordions, this post is 1,416 words; with all the accordions, it hits 3,774. The WordPress prediction algorithm (driven by AI???) says it’s almost a 20-minute read if you read all the accordions. If you skip them, it’s about a 7-minute read.
There’s some meat hidden in those accordions, though. If you have the time, you’ll want to check them out.
The Ancient Oracle at Delphi
The Oracle at Delphi, one of ancient Greece’s most significant religious sites, was considered the voice of Apollo, the god of prophecy. Located on Mount Parnassus, the sanctuary featured Pythia, the high priestess who served as the oracle. Delphi’s origins date back to the 8th century BCE, and it remained influential for over a thousand years.
According to myth, Apollo killed the serpent Python, who originally guarded the site. He then established his oracle, with the priestess named the Pythia after Python. The Pythia entered trances, often induced by inhaling fumes from a chasm, to deliver cryptic messages. Priests interpreted these messages and relayed them to seekers.
Consultation involved elaborate rituals. Individuals and city-states sought the oracle’s guidance on personal dilemmas and state matters. The oracle’s prophecies were famously ambiguous, requiring careful interpretation. For example, when King Croesus of Lydia asked if he should attack Persia, the oracle replied he would destroy a great empire. This prophecy proved true when his own empire fell.
The reliability of the Oracle at Delphi is debated. Some ancient sources revered her insights, claiming her pronouncements were divinely inspired and accurate. Others pointed to the ambiguity and potential for misinterpretation, suggesting the oracle’s statements could fit any outcome. Accounts also indicate that political and financial influences sometimes swayed the oracle’s pronouncements, leading to skepticism about her reliability.
Despite criticisms, the Oracle at Delphi remained central in Greek culture. She embodied the intersection of religion, politics, and social life in the ancient world. The site, now an important archaeological location, continues to attract interest for its historical and cultural significance.
Twenty-First Century Delphic Oracle
By the time you reach the end of this post, you’ll understand why I refer to ChatGPT-4 and other artificial intelligence “chatbots,” as twenty-first century Delphic oracles.
Artificial intelligence programs like ChatGPT are just one type of artificial intelligence, known as Large Language Models, or LLMs.
LLMs (and other AIs) don’t just spring into being ex nihilo nihil fit. They actually require specific kinds of “coddling” or training with specific kinds of data. You just don’t find an AI sitting on a bus bench, or hanging out at the park, and say, “Hey! Get a job! Go out and read the Internet! Learn something!”
Computer scientists “train” these AI programs on huge amounts of fondled data. In the end they create something that appears almost human, and seems to have reasoning, creativity, and emotional understanding. But the fact is, it doesn’t. AI’s dependency on human-designed algorithms and curated data underscores the point. AI is not an independent form of intelligence. Human intelligence creates (coding/programming), trains (with massaged/tagged data sets), and directs (through “prompts”) the program.
And while so far I’m talking about AI generally, Scott Greenfield — you almost had to know that was coming — points out one of the weaknesses of AI relating to a specific domain of knowledge: in this case, the law, and the idea that AI will fundamentally transform out legal systems, and, thus, our societies.
[T]he point is that the people pushing this notion, and creating the algorithms necessary to enable AI to play judge, jury and executioner, aren’t lawyers but computer nerds. They wouldn’t know a legally correct answer if it bit them in the butt. At least they have a grudging grasp that there might be issues.
— Scott Greenfield, “All Rise for Judge AI” (May 3, 2024)
Scott references numerous drawbacks to the fact that “computer nerds,” and not lawyers drive the car.
Twenty-First Century Delphic Fumes
A part of Scott’s analysis implies — and I completely agree — that the issue really boils down to this:
[A]s we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.
— C-SPAN Transript, “Defense Department Briefing,” quoting Donald Rumsfeld. (February 12, 2002)
These two types of unknowns impact the utility of our Twenty-First Century Delphic Oracle. While the Ancient Oracle inhaled “Delphic vapors” or “Pythian gases” before entering a trance and making predictions, our new oracles imbibe massive amounts of processed and spoon-fed text (and, for image-generating AIs, images). The cursor throbs as it enters its trance-like state in response to a user prompt: a question, sometimes “contextualized.” Then, the LLM spits out an answer. Maybe it’s useful. Or it might be ambiguous, requiring further prompting.
Maybe it’s a hallucination.
The Unknowns
When I first started writing this section, I was going to create separate lists for the known unknowns and the unknown unknowns. There’s a problem with this, though: the known unknowns often shade into unknown unknowns. Maybe that’s true of “unknowns” generally. I don’t know. But it’s particularly true when dealing with AI systems, like LLMs.
Ultimately, no one — not even the fumigated Twenty-First Century Delphic Oracle itself — knows what’s going on under the hood.
Large language models (LLMs) like the ones that power ChatGPT and Bard are different from revolutionary technologies of the past in at least one striking way: No one — not even the people who built the models — knows exactly how they work.
— Mark Sullivan, “The frightening truth about AI chatbots: Nobody knows exactly how they work” (May 17, 2023)
Therefore, I’m going to just talk about the various unknowns. I’ll list the known unknowns. If I can, I’ll drop hints about the unknown unknowns they may conceal. (I know. If they’re unknown unknowns, how can I do that? Well, let’s see what happens.)
The Bottom Line
Despite its drawbacks and the caveats that come from what I’ve written (especially in the accordions) — the fumes it’s breathed, if you will, that impact its predictions and pontifications — I have found my Twenty-First Century Oracle useful.
I don’t use it to write briefs, or contracts, or to collect data from clients. I’ll talk more about this in a second article, already in progress. Because, ironically, I started writing this article as a response to Mark Draughn’s “How to Use AI in Your Legal Practice.” It began life as a comment to that article on his Windypundit blog.
When it passed a few paragraphs, I erased the comment, and decided to write my own article. I remembered Scott’s article, “All Rise for Judge AI,” and thought I’d incorporate both articles in my own.
As it turns out, including the accordion blocks, I’m now surpassing 3,700 words. I suspect nobody has time for that. I hope there’s enough interest that people read the article. (Leave me a comment, please? Love it? Hate it? It’s okay? Or anything else you want to say.)
And I haven’t even started talking (much) about my favorite AI issue: Hallucinations.
For that reason, I’ve decided that I’m going to do another article — probably much shorter, but separate nonetheless — specifically addressing Mark Draughn’s ideas, and talking about the ways in which I have used various AI programs in my practice.
Stay tuned!
The post Twenty-First Century Delphic Oracle appeared first on Fresno Criminal Lawyer.