What this prompt does
This prompt makes the model decide between direct answering and chain-of-thought (CoT) for a specific task on a specific model, and specify it tightly enough to ship to production. The reflex to slap CoT on everything quietly triples latency and cost on tasks that never needed it, so this weighs complexity, latency, cost, and consistency before recommending an approach instead of defaulting to reasoning everywhere.
The [task] variable is what the model must do, [model] is the target model whose cost and latency profile the analysis uses, and [latency_budget] is the hard ceiling the recommendation must respect. The output gives a recommendation (direct, CoT, or a hidden/structured reasoning variant), production-ready prompts for both approaches with reasoning separated so callers can strip it, the expected token/cost and latency delta on [model], an eval plan to settle it with data, and a guardrail for when to fall back to the other approach in production.
When to use it
- A task uses CoT by default and you suspect it doesn't need the latency and cost.
- Deciding whether reasoning helps for extraction-style work that direct answering often wins.
- Working under a strict per-request
[latency_budget]that CoT might blow. - Wanting both prompt variants plus an eval to choose between them objectively.
- Estimating the token and cost difference between approaches before committing.
- Needing a production guardrail for when to fall back to the other approach.
Example output
You get the recommendation, both prompts, and the eval plan in clearly labeled sections. The recommendation weighs complexity, [latency_budget], cost, and consistency. You get a production-ready direct-answer prompt and a CoT prompt for the same [task] with reasoning separated from the final answer so callers can strip it, the expected token/cost and latency delta on [model], an eval comparison plan (a small labeled set, metrics, and a pass bar), and a guardrail for falling back in production. It's a decision plus shippable artifacts, not just an opinion.
Pro tips
- Run the cheap eval first — for extraction-style tasks direct answering usually wins outright, and the eval saves you from guessing.
- Set
[latency_budget]as a real number; CoT's extra tokens directly threaten a tight budget, and the recommendation hinges on it. - Keep CoT reasoning separated from the final answer so your caller can strip it cleanly before using the result.
- Describe
[task]precisely so the complexity judgment, which drives the whole recommendation, is grounded in what you actually need. - Name
[model]accurately, since the token/cost and latency delta is model-specific and a different model can flip the decision. - Wire the fallback guardrail into production so a task can switch approaches if quality or latency drifts.
