Chain of Thought Prompting

The Problem with Direct Answers

Large Language Models (LLMs) are powerful pattern-completion machines. But they have a well-documented weakness: when asked to solve complex reasoning problems directly, they often arrive at wrong answers with total confidence.

Consider this prompt: "A store has 50 shirts. They sell 30% on Monday and then sell half of what remains on Tuesday. How many shirts are left?"

Without guidance, an LLM might shortcut the reasoning and give a wrong answer. The model jumps to an answer without working through the intermediate steps.

Chain of Thought (CoT) prompting is the technique that fixes this problem.

---

What is Chain of Thought Prompting?

Chain of Thought prompting instructs the model to show its reasoning step by step before arriving at a final answer. Instead of jumping to a conclusion, the model "thinks out loud," breaking the problem into logical steps.

This technique, introduced by Google researchers in 2022, dramatically improves LLM performance on tasks that require:

• Multi-step arithmetic
• Logical reasoning
• Sequential decision-making
• Comparing options with trade-offs
• Drawing conclusions from evidence

---

How to Trigger Chain of Thought Reasoning

There are two main approaches:

Approach 1: Zero-Shot Chain of Thought

Simply add the phrase "Let's think step by step" or "Think through this carefully before answering" to your prompt. This is remarkably effective and requires no examples.

Example: "A store has 50 shirts. They sell 30% on Monday and then sell half of what remains on Tuesday. How many shirts are left? Let's think step by step."

Model Output: "Step 1: Start with 50 shirts. Step 2: They sell 30% on Monday. 30% of 50 = 15 shirts sold. Remaining: 50 - 15 = 35 shirts. Step 3: They sell half of what remains on Tuesday. Half of 35 = 17.5, round to 17 (or 18 depending on rounding). Remaining: 35 - 17 = 18 shirts. Answer: 18 shirts remain."

By asking the model to think step by step, you get the correct answer because the model is forced to work through each intermediate step.

Approach 2: Few-Shot Chain of Thought

Provide examples where you show the reasoning steps, not just the final answers. This is more powerful but requires more work upfront.

Example:

"Q: A bakery bakes 120 loaves on Monday. They sell 75% during the day. How many are left? A: Step 1: Start with 120 loaves. Step 2: They sell 75%, which is 0.75 x 120 = 90 loaves. Step 3: Remaining = 120 - 90 = 30 loaves. Answer: 30 loaves.

Q: A store has 50 shirts. They sell 30% on Monday and then sell half of what remains on Tuesday. How many shirts are left? A:"

The model follows the pattern from your example and applies the same step-by-step reasoning.

---

Why Chain of Thought Works

LLMs generate text one token at a time. When the model generates intermediate reasoning steps, those steps become part of the context that influences subsequent tokens. In other words:

• The model is not "doing math" in its head
• Writing out step 1 makes step 2 easier to get right
• Each step builds on the previous one in the context window
• Errors are more visible and the model can self-correct

This is also why asking for explanations alongside answers generally improves answer quality -- the explanation process guides the model toward the correct conclusion.

---

Chain of Thought for Business Reasoning

CoT prompting is not limited to arithmetic. It is extremely valuable for business logic and decision-making tasks.

Example: Contract Risk Assessment

Prompt: "Review the following contract clause and assess its risk level (Low, Medium, High). Think through your reasoning step by step before giving your final assessment.

Clause: 'Either party may terminate this agreement with 48 hours notice for any reason.'"

Model Output: "Step 1: Identify what this clause does. It gives both parties the right to end the contract with only 48 hours notice, for any reason.

Step 2: Assess the implications for the business. A 48-hour termination window is extremely short. If a client terminates with 48 hours notice, the business has almost no time to find replacement revenue or wind down operations.

Step 3: Consider industry norms. Most service contracts use 30-90 day termination clauses. 48 hours is significantly below standard.

Step 4: Assess the risk. The very short notice period creates significant business continuity risk.

Assessment: HIGH risk. Recommend negotiating for at least a 30-day termination notice period."

This output is far more useful than a direct "High risk" answer because the reasoning is transparent, auditable, and can be reviewed by a human.

---

Self-Consistency: Taking Chain of Thought Further

One powerful extension of CoT is self-consistency. Instead of generating one chain of thought, you generate several (by running the prompt multiple times or with slight variations), then select the most common final answer.

This works because:

• Different reasoning paths sometimes lead to different intermediate conclusions
• The correct answer is more likely to appear across multiple reasoning chains
• Outlier answers are likely errors

Self-consistency is particularly valuable in high-stakes applications where getting the right answer matters more than API cost.

---

When to Use Chain of Thought Prompting

Use CoT when the task involves:

Do not use CoT for:

• Simple fact retrieval ("What is the capital of France?")
• Straightforward classification where reasoning is not needed
• Cases where you only want a concise answer, not a detailed explanation

---

Practical Tips for Chain of Thought Prompting

1. Specify where the answer goes: Tell the model to put the final answer at the end, after the reasoning. "Work through the problem step by step, then state your final answer on the last line."

2. Use delimiters for structured output: Ask the model to wrap its reasoning and answer in tags. "Put your reasoning in <reasoning> tags and your final answer in <answer> tags."

3. Combine with few-shot examples: For complex tasks, show a full example with reasoning before the actual question.

4. Ask the model to check its work: "After reaching your answer, verify it is correct by checking each step."

---

Practice Exercise

Write a chain of thought prompt for one of these real-world tasks:

1. Determining whether a loan applicant qualifies based on their income and debt levels
2. Deciding which software tool best fits a company's budget and requirements
3. Assessing whether a marketing campaign met its goals based on metrics

Test your prompt and observe whether the step-by-step output leads to more reliable answers than asking directly.

Key Takeaways

• Chain of Thought prompting instructs the model to show its reasoning step by step before giving a final answer.
• Simply adding "Let's think step by step" to a prompt (zero-shot CoT) significantly improves performance on reasoning tasks.
• CoT works because intermediate reasoning steps become context that guides subsequent outputs, reducing errors.
• Few-shot CoT provides examples with full reasoning chains, making it even more powerful for complex tasks.
• Use CoT for multi-step calculations, logical deductions, risk assessments, and any task where showing the reasoning is as valuable as the answer itself.