Most lawyers who use AI regularly have reached a plateau. They open the same tool each time, accept whatever model it defaults to, ask for a draft or a summary, and judge the result by how well it reads. That is a reasonable place to start, and it produces useful work. But it leaves the user with no way to tell a good answer from a fluent one, and no way to know whether this month’s model is better or worse than last month’s at the tasks they care about. The habits below are the ones I would recommend to a lawyer who uses AI regularly and wants to get more out of it without taking on more risk. None of them requires technical training, and none depends on a particular tool or vendor.
1. Know the models well enough to choose one
Not every tool lets you choose the model; some make that choice for you. But if you can choose, learn enough about the options to make the choice deliberately. Some models are fast and do well at reformatting, extraction, and short drafting; others spend more time reasoning before they answer and do better at tracing how an indemnification provision interacts with a limitation-of-liability clause. Chat applications often default to a faster or cheaper model, so a user who never changes the setting may be judging the vendor by its least capable option. Choosing well does not require knowing the technical specifications, but it does require knowing, from experience, which model you trust for which kind of task, and the quickest way to learn that is to run the same task on two models and compare the results.
2. Use it adversarially at least as often as you use it to brainstorm
Beginners use AI to generate ideas, and the models are good at it, but a model asked to help with your work tends to agree with it, the failure mode I have written about as sycophancy. The defense is to ask for opposition at least as often as you ask for help. Ask the model to argue against your recommendation, to find the weakest step in your reasoning, to list the questions a skeptical reader would ask first, or to review your draft as the person on the other side of it would read it, whether that is a counterparty, a regulator, or a client’s board. Run the request in a new conversation without saying the draft is yours, since a model that knows whose work it is reviewing will soften the critique. And ask it to rank what it finds and to say plainly if nothing is serious, because a model told to find problems will find some whether or not they exist.
3. Keep a personal benchmark
Vendor announcements report gains on benchmarks that measure tasks unlike yours, and the only way to know whether a new release is better at your work is to test it on tasks from your own practice whose answers you already know. Mine is a set of difficult legal research tasks that I give to each new version of the models I use. I use hard questions because nearly every current model handles an easy research question well enough, and releases differ most on the difficult ones. Run the same tasks with the same prompts each time, keep the outputs so you can compare one release with the last, and build the set from material you are permitted to put into the tools you are testing.
4. Develop a feel for how much of the context window you are using
Everything in a conversation occupies the model’s context window: the documents you attach, the model’s answers, and every follow-up. So does material you never see, such as the instructions the tool’s developer gives the model before your first message and any saved memories or user profile the tool adds to the conversation. That hidden material takes up space and also shapes the answers. As the window fills, the model attends to its contents less reliably, particularly material in the middle, and some tools drop or compress older turns without saying so. A few tools show how much of the window is in use, but most do not, so an intermediate user keeps a rough sense of the load, knowing for instance that a long agreement and an extended discussion of it can fill much of the window. When a model forgets an instruction you gave earlier or misses a provision you know is in the document, suspect a crowded context, and start a new conversation with only what the task needs.
5. Know the knowledge and source limits of your tool
Every tool answers from some body of material, and each kind has its own limits. A tool connected to a database of cases, statutes, or firm documents can answer only from what that database holds, so learn how often it is updated and how far back its coverage goes. A tool that searches the web reaches only the sites that allow it, and much of the legal and financial press blocks AI crawlers. A tool with neither answers from training data that stops at a cutoff date, so know roughly when that date is and what it rules out. A question about a rule amended or a case decided since then will get an answer about the law as it stood before, stated with the same confidence as any other answer. The same model can fall into different categories in different products, and sometimes in different requests within one product, depending, for example, on whether a file is attached.
6. Ask for options, not conclusions
For legal analysis especially, ask for the competing readings and the strongest case for each rather than for the answer. The choice among them belongs to the lawyer, but there is a second reason. A model that states a conclusion early writes everything after it to fit, so the model presents a wrong conclusion with supporting authority and counterarguments already distinguished, which makes the error harder to catch than a bare wrong answer. Push back in the same conversation and the model will often dig in with further justification, or give way for no better reason than that you objected. If a conversation has already reached a conclusion you doubt, start a new one and ask for the options.
7. Know when to refine, when to start over, and when to finish by hand
When an output disappoints, you can refine it, start over, or finish it yourself, and beginners tend to use one of these responses for everything, either re-prompting indefinitely or fixing by hand what a clearer request would have fixed. Refine in the same conversation when the output is close and you can say exactly what is wrong with it. Start over when the model has built on a wrong premise or when your corrections have begun to pile up, because each correction stays in the context alongside the error it corrects. Finish by hand when the remaining changes would take longer to describe than to make, or when they are judgment calls you should be making anyway.
8. Ask what it needs before it starts
Before a substantial task, describe what you want and ask the model what it would need to know to do it well, or ask it to restate the task in its own words before it begins. Its questions show you the gaps in your request, such as the audience you did not name or the term you used in two senses, before the model fills them with guesses. The extra exchange is short, and it often shows that the request you were about to send was not quite the one you meant.
9. Look for new uses
Most users settle on a few tasks the tool handles well and stop there. But the models change with each release, and a task that failed on an earlier model may work now, so make a habit of trying the model on work you assumed it could not do, such as building a chronology from a set of documents, comparing two versions of an agreement, or turning a regulation into a compliance checklist. When a task is tedious, ask whether the model could do part of it; better yet, describe your workflow to the model and ask where it could help. Some attempts will fail, and those failures are good candidates for your benchmark.
10. Keep doing some of it yourself
The ability to judge a model’s output comes from having done the task yourself. A lawyer who hands every first draft to a model gradually loses that ability, and a junior lawyer who starts with the model may never build it, the risks the medical literature calls deskilling and never-skilling. One common improper delegation looks harmless: “make it shorter.” Shortening a draft means deciding which points are important and which can go, and lawyers develop that judgment by cutting their own drafts. A model decides by its own sense of importance, which may drop the qualification that controls the outcome and keep a paragraph that reads well and says little. Do the cutting yourself, or ask the model to identify candidates for cutting and explain why each is expendable, and make the decisions on your own.
Making them routine
Most of these habits add a step the default workflow does not require, such as choosing the model, opening a new conversation, or asking for the case against your own draft. Each step is slow the first few times, but it becomes quick once it is routine, and a user who keeps up the habits learns from their own testing which outputs to trust.
This post builds on earlier posts on sycophancy, context-window degradation, what AI legal research can’t read, the delegation framework, and the ten-minute conversation.