Reducing GenAI Hallucinations with Chain of Thought and Tree of Thought

Have you ever wondered how some AI models seem to “think” through complex problems with an eerie, almost human-like flow? Like when they solve intricate math puzzles, break down logical reasoning tasks, or even generate surprisingly coherent essays (hopefully like this one). The secret sauce often lies in how we guide their thinking, a process known as prompt engineering. In a previous post, we talked about the Interviewer Method, this article talks about two other methods: Chain of Thought (CoT) and Tree of Thought (ToT) reasoning.

These are transformative methods that shape how models process information, reach conclusions, and even, “reason.” In this post, we’re diving deep into these concepts, unpacking their mechanics, exploring where they shine (or stumble), and pondering what they mean for the future of AI. We will go through some examples to help illustrate these techniques in practical terms.

What Is Chain of Thought Reasoning?

Imagine trying to solve a math problem without jotting down your intermediate steps. You’re mentally juggling numbers, hoping not to drop any along the way. I can recall countless times my Math teacher would ask me for my workings, how did I come to that answer.

Now, think about how easier it gets when you write things out, step-by-step. That’s Chain of Thought reasoning in a nutshell, or CoT for short.

CoT is all about breaking down complex tasks into manageable, sequential steps. Instead of asking an AI model to jump straight to the answer, we prompt it to articulate the reasoning process. In this scenario you are the teacher, ChatGPT is the pupil, and you ask it for its workings. This helps it think with reasoning, reducing hallucinations.

A Simple Example

Let’s use the following Math problem. Type this in a new GenAI prompt:

If a train leaves Chicago at 60 mph and another leaves New York at 80 mph, how long until they meet if the distance between them is 800 miles?

We are not using CoT, so we might expect the AI to return something like “5 hours,” but you’re left wondering, how did it get there?

With CoT:
We would prompt something like this: Use chain-of-thought reasoning to solve this problem: Two trains start from cities 800 miles apart and head toward each other. One goes 60 mph, the other 80 mph. How long until they meet?

The response could look like this:

 We have: 
• Two trains 
• Starting 800 miles apart 
• Traveling toward each other 
• Train A is going 60 mph 
• Train B is going 80 mph

We need to find how long until they meet.
⸻
🚂 Step 2: Combine their speeds
Since they are moving toward each other, we can combine their speeds:
\text{Relative speed} = 60 \text{ mph} + 80 \text{ mph} = 140 \text{ mph}
⸻
🕰 Step 3: Use the formula
The formula for time when distance and speed are known is:
\text{Time} = \frac{\text{Distance}}{\text{Speed}}
\text{Time} = \frac{800 \text{ miles}}{140 \text{ mph}} = \frac{80}{14} = \frac{40}{7}
⸻
🧮 Step 4: Convert to hours and minutes
\frac{40}{7} \approx 5.714 \text{ hours}
Now convert 0.714 hours to minutes: 0.714 \times 60 \approx 42.86 \text{ minutes}
⸻
✅ Final Answer:
They will meet in approximately 5 hours and 43 minutes. ```
Now, not only do you get an answer, but you also see the reasoning that led to it. This transparency makes CoT incredibly useful in applications, instills greater confidence where interpretability matters, in domains such as finance, healthcare, or legal domains.

Why It Works

CoT leverages the model’s strength in pattern recognition while compensating for its inherent weakness in abstract reasoning. By structuring prompts to mimic human thought processes, we nudge the model to “think” more methodically.

Interestingly, a 2022 study by Google researchers found that even large language models like GPT-3 showed significant performance gains in math and logic tasks when CoT prompting was used. It’s like flipping a switch from “guess mode” to “reason mode.”

What is Tree of Thought Reasoning?

Cot is great for linear problems, like the train Math problem illustrated earlier, but what happens when the path to answering isn’t as straight? This is where Tree of Thought (ToT) comes in, a method that branches out, quite literally, to explore multiple reasoning paths simultaneously, like a tree.

How It Works

ToT can be thought as the AI version of brainstorming, instead of following a single thread of logic, the model generates several possibilities, evaluates them, and then decides which path to pursue further. It’s like playing chess: you don’t just consider one move—you anticipate several moves ahead, weighing different scenarios.

A Real-World Scenario

Let’s use a road-trip scenario here to illustrate ToT:

Let’s use Tree of Thought method. You are a road trip planner for a trip from Seattle to San Francisco. Consider:

- **Option A:** Take the scenic route. It’s longer but more enjoyable.  
- **Option B:** Stick to the highway. Faster but boring.  
- **Option C:** Mix both, balancing efficiency and fun.

We can expect a response similar to:

# 🌲 Tree of Thought: Road Trip from Seattle to San Francisco

You're heading out on an epic road trip from Seattle to San Francisco. Let's explore your options using the **Tree of Thought** method — branching out ideas, evaluating each path, and pruning to find the most delightful (or efficient) journey.

---

## 🧠 Root Thought: What's the goal of this road trip?
- Is it to **relax and sightsee**?
- To **reach San Francisco quickly**?
- Or to **create unforgettable moments without totally killing your schedule**?

Let’s branch into three options.

---

## 🌿 Option A: The Scenic Route  
*Longer, but filled with majestic views and unique stops*

### 🌲 Thought Branches:
- **Highway 101 down the coast**: Ocean views, beach towns, redwood forests.
- **Stops in**: 
  - Olympic National Park 🌄  
  - Cannon Beach, OR 🏖️  
  - Redwood National Park 🌳  
  - Mendocino & Point Reyes 🐋

### ✅ Pros:
- Instagram-worthy.
- Serotonin-boosting nature exposure.
- Local eats and cozy coastal inns.

### ❌ Cons:
- Adds ~6–8 hours to total trip time.
- Potentially tiring if you're not into winding roads and small towns.

---

## 🌿 Option B: The Interstate Speedrun  
*Zoom down I-5 like you’ve got a date with destiny (or a hotel check-in)*

### 🛣️ Thought Branches:
- **Main route**: I-5 south all the way.
- **Possible stops**:
  - Portland for food + coffee ☕  
  - Ashland for Shakespeare 🎭 (if that’s your jam)  
  - Sacramento for a final leg stretch 🧘

### ✅ Pros:
- Fastest route (~13–14 hours).
- Easier to plan and navigate.
- More gas stations and rest stops.

### ❌ Cons:
- Boring scenery after a while.
- Misses coastal magic.
- Feels more like commuting than adventuring.

---

## 🌿 Option C: The Balanced Adventure  
*Best of both asphalt worlds*

### 🌀 Thought Branches:
- **Start on I-5** to Portland for a foodie fix.  
- **Cut west to the coast** via OR-18 to Highway 101.  
- Cruise through **Redwoods**, then reconnect to I-5 near Santa Rosa.

### ✅ Pros:
- You get scenic chunks without adding too much time.
- Variety keeps things fresh.
- Easier to adapt depending on weather or energy levels.

### ❌ Cons:
- Some planning required to time the switchovers.
- Could get stuck in traffic or detours if not planned well.

---

## 🌳 Pruning and Choosing

| Option | Duration | Scenic Value | Fun Factor | Energy Drain | Planning Required |
|--------|----------|--------------|-------------|---------------|--------------------|
| A      | 🌅🌅🌅🌅     | 🌲🌲🌲🌲🌲     | 🎉🎉🎉🎉       | 🔋🔋🔋🔋        | 📋📋📋📋📋           |
| B      | ⏱️⏱️       | 🌲            | 🎉           | 🔋            | 📋                |
| C      | ⏱️⏱️⏱️     | 🌲🌲🌲        | 🎉🎉🎉🎉       | 🔋🔋🔋         | 📋📋📋             |

---

## 🌟 Recommendation: Option C – The Balanced Adventure

**Why?**  
It keeps the journey dynamic and full of surprises without burning you out or blowing the timeline. Think of it as the "road trip tapas menu" — a little of everything.

You’ll come back with both memories *and* momentum.

---

> “It’s not just about the destination, it’s about how many quirky roadside attractions you can hit without needing a chiropractor after.”

With ToT, the AI doesn’t just pick one option blindly. It maps out the pros and cons of each, compares them, and then recommends the best choice based on your preferences (like time constraints or scenic value).

Why It’s Powerful

ToT shines in decision-making tasks, creative problem-solving, and strategic planning. It allows AI to:

• Consider alternative hypotheses
• Handle ambiguous or open-ended problems
• Adapt to dynamic environments

In fact, recent advancements in ToT have been pivotal in developing AI systems for gaming, robotics, and even scientific discovery, where the ability to explore multiple possibilities is crucial.

Chain vs. Tree: When to Use What

Knowing when to use CoT or ToT isn’t just academic—it’s practical. If you’re building an AI for legal contract analysis, CoT might be your go-to. But if you’re designing an AI to strategize in a video game, ToT could be the game-changer.

Final Thoughts

Whether you’re a TPM, a developer, or just an intrigued reader, understanding CoT and ToT offers insights not just into how AI works, but how reasoning itself unfolds. They remind us that thinking, whether by humans or machines, is rarely a straight line. Sometimes it’s a chain. Sometimes it’s a tree. And sometimes, it’s a beautiful, tangled mess of both.