Hill Climbing & Simulated Annealing (Final Lesson)

In the previous lesson, we studied Uniform Cost Search, which guarantees the least-cost solution in weighted graphs.

In this final lesson, we complete the Algorithms module by learning optimization-based algorithms — Hill Climbing and Simulated Annealing.

These algorithms are not about finding a path in a graph. They are about finding the best possible solution among many possibilities.

What Is Optimization in Algorithms?

Optimization problems aim to find a solution that minimizes or maximizes an objective function.

Examples include:

• Minimizing delivery cost
• Maximizing profit
• Reducing energy usage
• Improving model accuracy

Hill Climbing and Simulated Annealing are widely used when the search space is very large.

Hill Climbing Algorithm

Hill Climbing is a greedy optimization algorithm.

It starts with an initial solution and repeatedly moves to a neighboring solution that improves the objective value.

The algorithm stops when no better neighbor exists.

Real-World Analogy

Imagine standing on a foggy hill.

You can only see nearby ground. You keep walking uphill until you cannot go higher.

That highest point may be a local peak — not necessarily the highest mountain.

Hill Climbing – Pseudocode

current = initial_solution

while True:
    neighbor = best_neighbor(current)
    if neighbor is better than current:
        current = neighbor
    else:
        break

return current

Problem With Hill Climbing

Hill Climbing often gets stuck in:

• Local maxima
• Plateaus
• Ridges

This is why a more advanced technique is needed.

Simulated Annealing

Simulated Annealing improves Hill Climbing by sometimes accepting worse solutions.

This allows the algorithm to escape local optima.

The probability of accepting worse solutions decreases over time.

Real-World Inspiration

Simulated Annealing is inspired by metallurgy.

When metal cools slowly, its atoms settle into a strong, stable structure.

Fast cooling creates weak structures.

Simulated Annealing – Pseudocode

current = initial_solution
temperature = high_value

while temperature > minimum:
    neighbor = random_neighbor(current)
    cost_diff = cost(neighbor) - cost(current)

    if cost_diff < 0:
        current = neighbor
    else:
        accept_prob = exp(-cost_diff / temperature)
        if random() < accept_prob:
            current = neighbor

    temperature = cool_down(temperature)

return current

Capstone Project – Route Optimization System

Project Goal:
Find the most cost-efficient delivery route between multiple locations.

This problem has a massive search space and cannot be solved efficiently using brute force.

Project Approach

We represent a route as an ordered list of locations.

The total cost is calculated as:

def route_cost(route, distance_matrix):
    cost = 0
    for i in range(len(route) - 1):
        cost += distance_matrix[route[i]][route[i+1]]
    return cost

Simulated Annealing is used to explore different route permutations and minimize the total cost.

Why Simulated Annealing Works Here

Delivery routing has many local optima.

Simulated Annealing avoids getting stuck by occasionally accepting worse routes early.

As the temperature cools, the algorithm becomes more selective.

Project Outcome

The algorithm gradually converges towards an efficient delivery route.

This approach is used in:

• Logistics companies
• Ride-sharing platforms
• Supply chain optimization

Additional Project Ideas

You can further strengthen your algorithm skills by working on these projects:

• Exam Timetable Optimization – minimize student conflicts
• Server Load Balancing – distribute traffic efficiently
• Warehouse Picking Optimization – shortest picking paths
• Energy Consumption Optimization – reduce power usage

Final Words

Algorithms are not just interview topics — they shape how real systems think and decide.

You now understand searching, sorting, optimization, graphs, dynamic programming, and modern algorithms.

Keep practicing, keep building, and keep pushing your problem-solving skills forward.