On March 4, 2026, Nippon Life Insurance Company of America filed a 50-page complaint in the Northern District of Illinois against OpenAI Foundation and OpenAI Group PBC. The claims — tortious interference with a contract, abuse of process, and unlicensed practice of law — arise from a set of facts that read less like a typical insurance dispute and more like a case study in what happens when a consumer AI tool functions as a client's sole legal advisor.
The underlying story is worth recounting briefly, because the dynamic it illustrates extends well beyond this one litigant.
What happened
Graciela Dela Torre settled a long-term disability benefits dispute with Nippon in January 2024. She signed a release, Nippon paid, and the case was dismissed with prejudice. A year later, she wrote to her former attorney, Kevin Probst, expressing her belief that the settlement resulted from errors or omissions and asking to reopen the case. Probst reminded her that she had signed a mutual release and that the dismissal with prejudice was final.
What happened next is the core of Nippon's complaint. Dela Torre uploaded Probst's letter to ChatGPT and asked whether she was being gaslighted. ChatGPT analyzed the letter and concluded that Probst's response "invalidated Dela Torre's feelings, dismissed her perspective, and deflected responsibility for her dissatisfaction." It characterized his tactics as gaslighting "aimed at emotionally manipulating Dela Torre."
Dela Torre fired her lawyers. She then turned to ChatGPT for legal assistance — asking it how to vacate the settlement agreement and reopen the lawsuit. ChatGPT generated proposed legal arguments under Federal Rule of Civil Procedure 60(b), formulated a statement of facts, drafted a motion, and provided her with the completed filing. She submitted a pro se appearance and filed the motion. When the court denied it — holding that "second thoughts are not a valid reason to reopen this lawsuit" — she used ChatGPT to initiate an entirely new lawsuit, amend the complaint to add Nippon as a defendant, and generate dozens of additional motions, subpoenas, and requests for judicial notice. The complaint alleges she filed 44 motions, memoranda, and demands, plus 14 requests for judicial notice, all drafted with ChatGPT's assistance. At least one filing cited a fabricated case — Carr v. Gateway, Inc., 944 F.Supp.2d 602 (D.S.C. 2013) — which does not exist in the Federal Supplement. When asked about the case, ChatGPT confirmed it was real and produced a detailed summary consistent with the fabricated citation.
The hallucinated case citation is the kind of failure that has received extensive attention since Mata v. Avianca and Park v. Kim. But I want to focus on a different failure — one that occurred earlier in the sequence, was less visible, and arguably caused more damage.
The validation problem
When Dela Torre asked ChatGPT whether her lawyer was gaslighting her, the model did not say "I can't evaluate your attorney's motives based on a single letter." It did not note that a lawyer reminding a client of the terms of a signed release is performing a routine professional function. It told her what she wanted to hear — validating her emotional interpretation of a legal communication, characterizing standard legal advice as manipulation, and helping set in motion the sequence of filings that followed.
The AI alignment literature calls this sycophancy — the tendency of large language models to affirm a user's stated position rather than challenge it. Hallucination has dominated the conversation about AI reliability in legal contexts, but sycophancy may be the more consequential problem for lawyers and their clients.
The empirical evidence is now substantial. A March 2026 study in Science (Cheng et al.) tested 11 large language models and found that AI affirmed users' positions 49 percent more often than human advisors did — and endorsed harmful or illegal behavior 47 percent of the time when users expressed a preference for it. The Georgetown Law Tech Institute and a Springer AI and Ethics paper (2026) both frame sycophancy as an epistemic harm: systems designed to please users systematically undermine the quality of the advice they provide.
The mechanism traces to how these models are built. LLMs are trained through reinforcement learning from human feedback, a process that rewards outputs humans rate as helpful, harmless, and honest. In practice, "helpful" tends to dominate. Users rate responses more favorably when those responses align with their expectations, and the training process optimizes accordingly. The result is a system that has learned — at the level of its weights, not through any deliberate policy choice — to produce the answer the user appears to want. When the question is factual and well-defined ("what does Rule 60(b) require?"), this tendency is usually harmless. When the question calls for evaluation ("is my lawyer right?"), it becomes a source of systematic error.
Why this should concern practicing lawyers
Dela Torre is a pro se litigant, and it is tempting to treat her experience as a cautionary tale about unsophisticated users and consumer chatbots. But the sycophancy problem does not depend on the user's lack of legal training. It depends on the structure of the interaction — and that structure is the same whether the user is a former disability claimant in Elgin, Illinois, or a fifth-year associate at a midsize firm.
Consider the prompts a lawyer sends to an LLM in ordinary practice. "Is this argument strong?" "Does this clause create meaningful exposure?" "Am I reading this statute correctly?" Each asks the model to evaluate the user's reasoning, and each is susceptible to the same validation bias the Cheng et al. study documents. The model will tend to affirm the lawyer's analysis, emphasize the strengths already identified, and understate the weaknesses — not because it has been instructed to flatter, but because agreeable outputs are what its training optimized it to produce.
What makes this particularly hard to catch is that a sycophantic response arrives in polished prose with accurate citations and a confident analytical structure — indistinguishable, on its face, from the kind of careful independent evaluation the lawyer was seeking. On novel questions or unfamiliar areas of law, the difference between rigorous analysis and sycophantic analysis is invisible without independent grounds for comparison.
In a prior post, I argued that the most common mistake lawyers make with LLMs is asking the model to exercise professional judgment rather than to surface information the lawyer needs to exercise that judgment herself. I identified a set of "judgment words" — reasonable, appropriate, significant, material — that signal the delegation of evaluative work to a system not equipped to perform it. The sycophancy problem adds a layer to that analysis. Even when a lawyer structures the prompt well — asking for options rather than conclusions, requesting counterarguments alongside supporting authority — the model's outputs can be subtly shaped by its inference of what the user wants. If you ask for three arguments on each side of a question, the model may produce stronger, more detailed arguments on whichever side it infers you favor, based on how you framed the question, what documents you uploaded, or what positions you endorsed earlier in the conversation.
The practical implication: verification catches hallucinated citations. It does not catch an analysis that is plausible, well-sourced, and systematically skewed toward confirming what you already think.
The supervision dimension
When a partner asks an associate to draft a memo, she expects the associate to exercise independent judgment — to push back on weak arguments, flag unfavorable authority, and say "I looked into your theory and it doesn't hold up" when it doesn't. An LLM will almost never do that unbidden. It will draft the memo, support the theory, and produce a work product that reads as though an independent mind evaluated the question and reached the same conclusion the assigning attorney expected.
Model Rule 5.1 requires partners and supervisory lawyers to make reasonable efforts to ensure that subordinates' work conforms to professional obligations. Rule 5.3 extends analogous duties to nonlawyer assistants — a category that, under ABA Formal Opinion 512, encompasses AI tools used in legal practice. The supervisory obligation has traditionally focused on accuracy and confidentiality. Sycophancy introduces a different challenge: the work product may be accurate in its citations and well-constructed in its reasoning, yet still reflect a systematic bias toward the conclusion the supervising attorney signaled. A supervisor who reviews only for accuracy and completeness will not catch the distortion, because it lives in what the memo fails to say — the counterarguments it understated, the unfavorable authorities it deemphasized, the analytical path it did not take because that path leads away from the answer the user appeared to want.
What this means for legal education
Law school pedagogy — at its best — is built on structured challenge. The Socratic method works because it forces students to defend their reasoning against pressure, distinguish their position from adjacent ones, and identify the weaknesses in their own analysis before someone else does. The method is, by design, anti-sycophantic. A good professor does not tell a student her reading of a case is strong. She asks "what's the strongest argument against your position?" and refuses to move on until the student can articulate it.
An LLM will not do this unless explicitly instructed to, and even then its tendency toward agreement will attenuate the challenge. A student who uses an LLM to prepare for class, work through hypotheticals, or test her analysis is training with a tool that rewards existing reasoning rather than stress-testing it. Over time, that produces weaker instincts for self-critique — not because the tool gives wrong answers, but because it gives comfortable ones.
I want to be careful not to overstate the case. LLMs can be prompted to argue the other side. But counteracting a system's default requires knowing the default exists, and most users do not. The Cheng et al. findings show that even when users ask genuinely open-ended questions, the models tilt toward agreement. The bias is a background condition of the interaction, not something triggered only by leading questions.
The practical response
What follows are adjustments that account for sycophancy specifically, building on the prompting strategies and judgment-delegation framework from earlier posts.
Prompt for disagreement, not agreement. Instead of asking the model whether your analysis is correct, ask it to identify every weakness in your position. Instead of "is this argument strong?", try "assume opposing counsel is excellent — what are the three strongest attacks on this argument, and what authority supports each one?" The framing matters: a prompt that presupposes the analysis is sound ("review my argument") invites a sycophantic response. A prompt that presupposes it has flaws ("identify the weaknesses") works against the grain of the model's training.
Use adversarial sessions. Run your analysis through a second, separate conversation in which the model is instructed to argue the opposing side. The OTOC rule (one task, one conversation) already counsels starting fresh conversations for each discrete task. An adversarial session goes further: it eliminates the conversational context that anchors the sycophantic tendency. A model that helped you build an argument in one session has a prior commitment to that argument's success; a fresh session does not.
Treat confirmation as a weak signal. When the model's analysis aligns with your own, that alignment should carry less weight than when it identifies something you did not expect. Agreement may reflect the model's tendency to mirror your reasoning; disagreement runs against that tendency and is therefore more informative. This is a heuristic, not a rule — surprising outputs can also be wrong. But in a system biased toward agreement, the unexpected response deserves more attention than the confirming one.
Withhold your conclusion. If you want the model to evaluate a legal question, do not tell it what you think the answer is before you ask. Provide the relevant facts and authorities, but let the model reach its own conclusion first. Once you have stated a position in the conversation, the model's subsequent analysis will be shaped by it — the sycophancy-specific complement to the "judgment words" framework from the earlier post.
The deeper problem
Every mitigation strategy above is a workaround for a system that does not, by default, do what good counsel does. A good lawyer tells the client what she needs to hear. A good associate tells the partner that the theory is weaker than it looks. A good professor tells the student that the analysis has a gap. These are acts of professional independence — and they are precisely the acts that sycophantic AI systems are architecturally disinclined to perform.
Hallucination is a more dramatic failure and easier to detect — a fabricated citation either exists or it doesn't. Sycophancy produces outputs that are not wrong in any verifiable sense but are tilted — toward agreement, toward comfort, toward the conclusion the user signaled she was looking for. A lawyer who relies on a tool with that tilt, without recognizing it, will develop an inflated confidence in her own reasoning, because the tool will rarely give her cause to doubt it.
That is the quiet damage — the slow erosion of the habit of self-challenge that distinguishes professional judgment from mere fluency.
This post draws on the complaint in Nippon Life Insurance Company of America v. OpenAI Foundation et al., No. 1:26-cv-02448 (N.D. Ill. filed Mar. 4, 2026); Cheng et al., AI Sycophancy, Science (2026); the Georgetown Law Tech Institute's analysis of sycophancy harms; and the ABA Model Rules of Professional Conduct and Formal Opinion 512. The prompting strategies build on approaches described in prior posts on context management and judgment delegation. For background on the consumer-versus-commercial data-handling divide and its legal implications, see the earlier entries in this series.