LeNet Architecture

In the previous lesson, we understood how classic CNN architectures are structured using convolution, pooling, and dense layers.

Now we study the first successful Convolutional Neural Network in history — LeNet.

LeNet proved that neural networks could automatically learn features directly from images.

What Is LeNet?

LeNet is a CNN architecture introduced by Yann LeCun in the 1990s.

It was originally designed to recognize handwritten digits, such as postal codes and bank checks.

Although simple by today’s standards, LeNet laid the foundation for all modern CNN architectures.

Why LeNet Is Important

Before LeNet, feature extraction was done manually.

LeNet showed that:

Neural networks can automatically learn features from raw pixel data.

This idea changed the future of computer vision completely.

High-Level Structure of LeNet

LeNet follows a clean and logical flow:

Input → Convolution → Pooling → Convolution → Pooling → Fully Connected → Output

Each layer gradually transforms raw pixel information into meaningful patterns.

Layer-by-Layer Breakdown

LeNet starts with a grayscale image as input.

The first convolution layer detects simple edges and shapes.

Pooling reduces spatial size and keeps important features.

The second convolution layer detects more complex patterns, such as curves and combinations of edges.

Finally, fully connected layers perform classification.

Why LeNet Uses Small Filters

LeNet uses small convolution filters (usually 5×5).

Small filters:

Capture local patterns Reduce number of parameters Improve learning stability

This design principle is still used in modern CNNs.

LeNet vs Modern CNNs

LeNet is shallow and simple.

Modern CNNs are deeper, use more filters, and include advanced techniques like batch normalization and residual connections.

However, the core idea remains the same.

LeNet in Code (Simplified)

Below is a simplified implementation of LeNet using Keras.

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, AveragePooling2D, Flatten, Dense

model = Sequential([
    Conv2D(6, (5,5), activation="tanh", input_shape=(32,32,1)),
    AveragePooling2D(),

    Conv2D(16, (5,5), activation="tanh"),
    AveragePooling2D(),

    Flatten(),
    Dense(120, activation="tanh"),
    Dense(84, activation="tanh"),
    Dense(10, activation="softmax")
])

This structure closely follows the original LeNet design.

Why Average Pooling?

Unlike modern CNNs, LeNet uses average pooling instead of max pooling.

At that time, average pooling provided smoother learning and better numerical stability.

Later research showed that max pooling often works better, which is why it is commonly used today.

Real-World Understanding

Think of LeNet as a beginner reader.

First, it learns strokes.

Then, it learns numbers.

Finally, it understands digits as symbols.

This step-by-step learning is exactly how CNNs work.

Mini Practice

Think about this:

Why do you think LeNet places convolution layers before dense layers?

Exercises

Quick Quiz

In the next lesson, we will study AlexNet, the architecture that restarted the deep learning revolution and introduced GPUs to CNN training.