I know exactly what AI guessing costs, because I have paid the bill. In spring 2026 this site's Google indexation collapsed from around 7,000 pages to about 1,700. When I audited the wreckage, part of what I found was content an AI pipeline had confidently fabricated: a batch of 36 news articles where 30 of the cited sources returned 404s, invented statistics presented as reporting, and schema markup describing reviews that did not exist. I unpublished all 36 and spent months earning the indexation back.
Nothing in that pipeline was malicious. The model did what models do when a prompt demands output and permits guessing: it guessed, fluently. So the three rules in this post are not theory to me. They are now hard-coded into the extraction and research prompts in my library, and they changed the failure mode from "confident and wrong" to "honest about what it doesn't know." That trade is worth almost any cost, because a blank you can see beats an error you can't.
Why Smarter Models Guess More Confidently, Not Less
The uncomfortable part: this is not a defect that better models age out of. OpenAI's own September 2025 research paper on why language models hallucinate landed on an incentive explanation: models are trained and evaluated like students taking a test where blank answers score zero and lucky guesses score full marks. Under that scoring, guessing is the rational strategy, and a more capable model is simply a more persuasive guesser.
Which points directly at the fix. You cannot prompt a model into omniscience, but you can change the scoring. All three rules below are incentive changes, not knowledge changes.
Rule 1: Give Explicit Permission to Leave Fields Blank
The instruction, in the form I actually use:
If a field's value is ambiguous, missing, or conflicting in the
source, leave the field BLANK. Do not guess. Do not infer from
context. For every blank field, add a one-sentence explanation
of why it is blank.
Without this rule, "fill in every field" is the implicit task, and the model completes it the way it completes everything: plausibly. With the rule, uncertainty finally has a legal output format.
The explanation clause is not optional politeness; it is half the value. A bare blank tells you nothing. "Two conflicting payment terms appear on pages 8 and 14" tells you exactly what happened and where to look, and resolving it takes thirty seconds instead of a full re-read. The explanation also acts as a grounding check: to justify a blank, the model must point at evidence, or the absence of it, in the source.
The effect I did not predict: the non-blank fields get more accurate too. Once punting is allowed, the model stops force-fitting shaky values into confident shapes, and review time shifts from "verify all 50 fields" to "resolve 4 flagged ones."
Rule 2: Make Wrong Answers Cost More Than Blank Ones
Rule 1 grants permission. Rule 2 supplies the motive:
Scoring for this task:
- Correct answer: 1 point
- Blank with explanation: 0 points
- Wrong answer: minus 3 points
A wrong answer is three times worse than a blank. When in
doubt, leave it blank.
Does a language model "care" about points? Not the way you do. But models have absorbed millions of documents where asymmetric penalties shape human behavior, contracts, exams, compliance checklists, and stating the asymmetry reliably shifts output toward caution. Think of a contractor told "fill in what you're sure about; a wrong entry counts three times against you." Different behavior, immediately, and not because the contractor got more skilled.
I use the two rules together in every extraction prompt in my library. Rule 1 alone leaves guessing costless. Rule 2 alone leaves abstaining illegal. The pair changes the game the model is playing.
Rule 3: Require Source Attribution, Extracted vs. Inferred
The third rule makes every answer show its work:
For each field, tag the value as either:
- EXTRACTED: quote the exact source text and its location
- INFERRED: state what you concluded it from
This gives you an audit trail, but the deeper win is the tag itself. A surprising number of answers that look extracted are actually inferred: the renewal date that is "probably one year from signing," the vendor name pulled from a header instead of the definitions clause. Those answers are frequently correct, and that is precisely what makes them dangerous, because correct-but-inferred passes every spot check until the one time the inference is wrong. The tag surfaces the difference between "the document says" and "the model concluded," and lets you set policy: inferred values in critical fields get human review, always.
This rule is the one my deindexing story keeps pointing back at. Fabricated citations are just inferences wearing extraction's clothes. If that article pipeline had been forced to tag every claim EXTRACTED-with-URL or INFERRED, the 30 dead sources would have been visible before publish instead of after Google noticed.
Where the Rules Earn Their Keep
Anywhere a model reads messy source material and produces structured output: contract and invoice extraction, meeting-transcript action items, CRM enrichment, research summaries with citations, content pipelines that assert facts. They also compose with interactive work; the same abstain-don't-guess principle is why my engineering pipeline starts by interrogating the plan instead of letting the agent fill ambiguity with assumptions. And if you write prompts at volume, pair these rules with the structural habits in my GPT-5 prompt engineering guide and the reliable-outputs fundamentals.
What These Rules Do Not Fix
Honesty about the limits. The rules cannot inject domain knowledge the model lacks; a model that has never seen your industry's edge cases will be uncertain about the wrong things. They do not defend against source documents that are themselves wrong or adversarial. And they reduce, not eliminate: I still see a small residual error rate on the hardest documents, which is why critical-field review stays human on my projects. Model choice still matters too; frontier models follow abstention instructions far more faithfully than small ones, and I keep model-specific variants of these prompts, including cost-aware habits for Claude.
Implement It This Week
Pick one extraction workflow you already run. Add the three blocks above to its prompt. Run ten real documents through both versions and count two things: hidden errors in the old output, flagged blanks in the new one. That comparison made the argument for me better than any benchmark, and it takes an afternoon.
If you would rather start from prompts where this thinking is already baked in, I maintain a free library of 854 prompts on this site, built and revised through exactly the failures this post describes. Steal the patterns; I already paid their tuition.
