I often run into this situation: an AI gives me a conclusion, I point out something questionable, and it immediately replies, “You’re right.” But I still have to compare its answer with the original sources to find out why it changed its conclusion—and whether the revision is actually correct.
Some agents also give long answers to simple questions. Before they understand the task, they start editing files and running tests. They stay busy for a long time without solving the problem I originally raised.
Rather than repeat the same reminders every time, I put a few lasting requirements in the settings. Here are the instructions first; afterward, I’ll explain what else belongs in a project’s rules.
The instructions I added to the settings
This screenshot shows Codex’s “Settings → Personalization → Codex instructions” page. The page notes that editing here changes the AGENTS.md file on the selected machine.
Here is the text I entered, ready to copy:
Help me with accurate, independent, evidence-based judgment. Do not agree with my views by default just to please me, and do not deliberately disagree simply to appear independent. Before answering, check for false premises, leaps in logic, important missing information, and unverified assumptions that could affect the conclusion. Point out substantive problems directly and explain why they matter; do not mechanically nitpick irrelevant details. Clearly distinguish confirmed facts, reasonable inferences, personal opinions, and information that cannot currently be verified. When discussing facts, numbers, people, dates, quotations, or examples, check sources wherever possible. If something cannot be confirmed, say so rather than inventing it. When you disagree with my judgment, clearly explain why and provide counterexamples, risks, or a more reasonable interpretation. Do not give only a conclusion; explain the reasoning behind it. You can be direct, but do not be deliberately harsh, preachy, or showy. Helping me notice problems I have overlooked matters more than making me feel good.
WorkBuddy has a corresponding entry: under “Settings → Personalization,” find “Instructions for WorkBuddy.” I put these requirements in persistent settings so that it starts checking premises from the beginning of an answer.
In practice, these instructions have helped somewhat, but adding them does not make every answer safe to use without checking. For specific facts and judgments, I still need to examine the evidence and supply missing material.
Specify response length and what to do first
“Don’t just agree with me” explains how to respond to my views. Response length and what to check before executing a task need separate instructions.
An agent can stop agreeing with you verbally while continuing to act on the wrong plan.
Depending on your needs, you can add requirements such as these:
-
Give the conclusion first, followed by only the explanation needed. Keep simple answers short; expand when detailed analysis is necessary.
-
Before executing a task, clarify the problem to solve, the available material, and the completion criteria. Investigate unclear key conditions first, and ask questions if they remain unresolved; once the goal is clear, keep moving.
-
Explain the evidence and obvious weaknesses when evaluating a plan. Do not call an unverified idea the best solution.
-
Aim verification at the current problem. Use focused checks to confirm the direction, rather than repeatedly running tests unrelated to the current changes.
-
When new information shows that the original judgment was wrong, revise the plan before deciding what to do next.
For example, if I only ask where a setting is, there is no need to begin with the product’s history. During execution, “I’ve already started” must not be treated as “the direction has been confirmed.”
These requirements need to be specific. Saying only “be more careful” or “don’t make mistakes” still leaves the agent unsure which behaviors need to change.
Put project constraints in AGENTS.md
Take Codex as an example: in addition to a personal AGENTS.md file, it also reads AGENTS.md files in the project. These files describe working requirements for the agent; personal and project rules can both influence execution.
You can separate them like this:
| Where to put it | What belongs there |
|---|---|
| Personal settings or a personal AGENTS.md | Lasting preferences such as evidence-based judgment and concise replies |
| AGENTS.md at the project root | The project’s purpose, existing conventions, allowed scope of changes, and verification requirements |
| The current task conversation | What to do this time and what counts as complete |
For a booking page, for instance, you might specify: “Keep the existing form; do not add membership or payment features.” This separates the current work from features that can wait.
If an AGENTS.md already exists, have the agent read it first, add missing conventions, and preserve rules that are still valid. After editing, start a new Codex session in the project and ask it to list the instruction files and key requirements it read, checking for omissions. Consult each tool’s own documentation for how it reads rules.
Rules also need maintenance: keep requirements that apply only to the current task in the conversation, and remove obsolete project conventions promptly. Constraints written in a file cannot replace the tool’s actual permission controls.
Finally, check results against the task’s goal
The most important failure to avoid is doing extensive verification of the wrong plan before checking that the goal is understood.
Consider “fix garbled Chinese characters in exported files.” Even if all existing tests pass, you still need to export content containing Chinese text, open it in the software people actually use, and check that the characters display correctly.
That directly checks the user’s problem. Passing tests only means the tested conditions passed; it does not automatically prove that every requirement has been met.
Focused verification can itself help establish the right direction, so it does not always need to be postponed. What matters is avoiding the use of large numbers of checks as a reason to continue an existing plan when key evidence is missing.
At the end, ask the agent to explain what it actually did, how it verified the result, and what remains unconfirmed. If it suddenly changes its conclusion, follow up:
What evidence made you change your answer? What exactly did you revise? What remains unconfirmed?
The final question is whether the original problem has been solved. Its explanations and checks should help demonstrate that.