Computer Vision Cheat Sheet
CNN Β· YOLO Β· object detection Β· segmentation Β· augmentation Β· OCR Β· ResNet
Sheet 2 of 4
Specialized ML
Intermediate
Printable
CNN Architecture
Convolutional Neural Networks extract spatial features by sliding learnable filters across input images through stacked layers.
Convolution Operation
(I * K)(i,j) = Ξ£mΞ£n I(i+m, j+n)Β·K(m,n)
I = input feature map, K = kernel/filter. Output size = β(W β K + 2P) / Sβ + 1. W=input, K=kernel, P=padding, S=stride.
Output Feature Map Size
Hout = β(Hin β K + 2P) / Sβ + 1
Same padding (P = K/2) keeps spatial dims. Valid padding (P=0) shrinks output.
| Layer Type | Purpose | Key Param |
|---|---|---|
| Conv2D | Feature extraction | filters, kernel, stride |
| BatchNorm | Stabilise training | momentum, Ξ΅ |
| ReLU | Non-linearity | β |
| MaxPool | Spatial downsampling | pool_size, stride |
| Dropout | Regularisation | rate (0.2β0.5) |
| Flatten | 3D β 1D vector | β |
| Dense | Classification head | units, activation |
PyTorch β Simple CNN Block
import torch.nn as nn class ConvBlock(nn.Module): def __init__(self, in_c, out_c): super().__init__() self.block = nn.Sequential( nn.Conv2d(in_c, out_c, kernel_size=3, padding=1), nn.BatchNorm2d(out_c), nn.ReLU(inplace=True), nn.MaxPool2d(2, 2) ) def forward(self, x): return self.block(x)
Receptive field: Each neuron "sees" a growing patch of the original image. Deeper layers capture larger, more abstract features β edges β textures β objects.
Pretrained Models
| Model | Params | Top-1 (ImageNet) | Best For |
|---|---|---|---|
| VGG16 | 138M | 71.5% | Baseline, feature extraction |
| ResNet-50 | 25M | 76.1% | General backbone |
| EfficientNet-B0 | 5.3M | 77.1% | Mobile / lightweight |
| ViT-B/16 | 86M | 81.8% | Large datasets, patches |
| ConvNeXt-T | 28M | 82.1% | Modern CNN baseline |
| DINOv2 | 307M | 86.2% | Self-supervised features |
Fine-tuning ResNet-50 (PyTorch)
from torchvision import models model = models.resnet50(weights='IMAGENET1K_V2') # Freeze backbone for p in model.parameters(): p.requires_grad = False # Replace classification head model.fc = nn.Linear(2048, num_classes)
Residual connections in ResNet solve the vanishing gradient problem: output = F(x) + x. Allows training very deep networks (50β152+ layers).
Object Detection
| Model | Type | Speed | Accuracy |
|---|---|---|---|
| R-CNN | 2-stage | Slow | High |
| Fast R-CNN | 2-stage | Medium | High |
| Faster R-CNN | 2-stage | ~5 fps | Very High |
| SSD | 1-stage | ~46 fps | Medium |
| YOLOv8 | 1-stage | ~160 fps | High |
| DETR | Transformer | ~28 fps | Very High |
IoU β Intersection over Union
IoU = Area(A β© B) / Area(A βͺ B)
Measures bounding box overlap. IoU β₯ 0.5 = True Positive (PASCAL VOC). IoU β₯ 0.75 = stricter COCO threshold.
NMS (Non-Maximum Suppression): Removes duplicate detections. Keeps the highest-confidence box; suppresses overlapping boxes with IoU above threshold (typically 0.4β0.5).
YOLO β You Only Look Once
YOLO divides the image into an SΓS grid. Each cell predicts B bounding boxes + confidence scores + C class probabilities β all in a single forward pass.
YOLO Confidence Score
Conf = P(Object) Γ IoUpredtruth
High confidence = high probability an object exists AND the box is accurate. Zero if no object in cell.
YOLO Loss Function
L = Ξ»coordLbox + Lobj + Ξ»noobjLnoobj + Lcls
Ξ»coord=5 upweights localisation. Ξ»noobj=0.5 downweights empty cells. CIoU loss in YOLOv8.
| Version | Key Improvement |
|---|---|
| YOLOv1 | Single pass, 7Γ7 grid, 45fps |
| YOLOv3 | Multi-scale anchors, Darknet-53 |
| YOLOv5 | PyTorch, CSP backbone, auto-anchor |
| YOLOv8 | Anchor-free, C2f module, best default |
YOLOv8 β Inference & Training
from ultralytics import YOLO # Load pretrained model model = YOLO('yolov8n.pt') # nano # Inference on image/video results = model.predict( 'image.jpg', conf=0.25, # confidence threshold iou=0.45, # NMS IoU threshold device='cuda' ) # Fine-tune on custom dataset model.train( data='dataset.yaml', epochs=100, imgsz=640, batch=16, lr0=0.01 ) # Access results for r in results: print(r.boxes.xyxy) # bbox coords print(r.boxes.conf) # confidence print(r.boxes.cls) # class IDs
Model sizes: YOLOv8n (nano, 3.2M params) β YOLOv8s β YOLOv8m β YOLOv8l β YOLOv8x (68.2M params). Start with nano, scale up if mAP is insufficient.
Image Segmentation
| Type | Output | Model |
|---|---|---|
| Semantic | Class label per pixel | FCN, DeepLab, SegFormer |
| Instance | Separate mask per object | Mask R-CNN, YOLO-seg |
| Panoptic | Semantic + Instance | Panoptic FPN, Mask2Former |
| SAM | Prompt-based any mask | Segment Anything (Meta) |
U-Net Architecture (PyTorch)
# Encoder-Decoder with skip connections # Encoder: downsample (Conv + Pool) # Bottleneck: deepest feature map # Decoder: upsample + concat skip enc1 = ConvBlock(1, 64) # 572β572 pool1 = nn.MaxPool2d(2) # 572β286 # ... deeper layers ... up = nn.ConvTranspose2d(512, 256, kernel_size=2, stride=2) x = torch.cat([up, skip], dim=1)
Dice Loss (Segmentation)
Dice = 1 β (2|Pβ©G| + Ξ΅) / (|P| + |G| + Ξ΅)
P = predicted mask, G = ground truth. Ξ΅ prevents division by zero. Better than CE for class-imbalanced masks.
Data Augmentation
| Technique | What it does | Use case |
|---|---|---|
| HorizontalFlip | Mirror image left-right | General (not text) |
| RandomCrop | Random crop + resize | Classification |
| ColorJitter | Brightness/contrast/sat | Lighting variation |
| RandomRotation | Rotate Β±degrees | Orientation-agnostic |
| GaussianBlur | Blur with Ο range | Focus robustness |
| Normalize | Mean/std per channel | Always apply last |
| Mixup | Blend 2 images + labels | Classification boost |
| CutMix | Paste patches + labels | ImageNet SOTA |
Albumentations Pipeline
import albumentations as A train_tfm = A.Compose([ A.RandomResizedCrop(224, 224), A.HorizontalFlip(p=0.5), A.ColorJitter( brightness=0.2, contrast=0.2), A.Normalize( mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]), ])
Evaluation Metrics
Mean Average Precision (mAP)
mAP = (1/|C|) Ξ£c APc
AP = area under precision-recall curve per class. mAP@0.5 = IoU threshold 0.5. mAP@[.5:.95] = COCO standard (avg over 10 thresholds).
| Metric | Task | Formula |
|---|---|---|
| IoU | Detection | Intersection / Union |
| mAP@0.5 | Detection | AP avg over classes @IoU=0.5 |
| mAP@[.5:.95] | COCO | Avg mAP over 10 IoU thresholds |
| Dice / F1 | Segmentation | 2TP / (2TP + FP + FN) |
| Pixel Acc. | Segmentation | correct pixels / total pixels |
| mIoU | Semantic seg | Mean IoU across all classes |
Class imbalance: Pixel accuracy is misleading when background dominates. Always report mIoU or Dice for segmentation tasks.
OCR Pipeline
- Preprocessing β grayscale, denoise, deskew, binarize (Otsu threshold)
- Text Detection β locate text regions (CRAFT, DBNet, EAST)
- Text Recognition β read characters per region (CRNN + CTC loss)
- Post-processing β spell check, NLP cleanup, structured output
EasyOCR β Quick Start
import easyocr reader = easyocr.Reader(['en']) results = reader.readtext('image.jpg') for (bbox, text, conf) in results: print(f"{text} ({conf:.2f})")
PaddleOCR / Tesseract
# PaddleOCR (fast, accurate) from paddleocr import PaddleOCR ocr = PaddleOCR(lang='en') res = ocr.ocr('img.jpg') # Tesseract (classic, offline) import pytesseract from PIL import Image text = pytesseract.image_to_string( Image.open('img.png'))
CTC Loss (Sequence Recognition)
LCTC = βlog P(y* | x)
Connectionist Temporal Classification β aligns variable-length sequences without needing character-level labels. Used in CRNN for scene text recognition.
Image Preprocessing
Standard ImageNet Preprocessing
from torchvision import transforms preprocess = transforms.Compose([ transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), transforms.Normalize( mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225] ) ])
| Operation | Why |
|---|---|
| Resize to 224Γ224 | Standard input for ImageNet models |
| ToTensor | PIL [0,255] β float32 [0,1] |
| Normalize | Zero-mean unit-variance per channel |
| BGRβRGB | OpenCV reads BGR; most models expect RGB |
| Channel first | PyTorch: [C,H,W]; TF: [H,W,C] |
Don't forget: Apply the same mean/std normalization at inference as during training. Mismatched normalization is a common silent bug that drops accuracy significantly.
Vision Transformers (ViT)
ViT splits the image into fixed-size patches, flattens them into tokens, and applies standard transformer self-attention.
Patch Embedding
xp = Flatten(patch) Β· E + Epos
Image HΓWΓC split into N = HW/PΒ² patches of size PΓP. Each patch projected to embedding dim D. Learnable positional embeddings added.
| Model | Patch | Layers | Params |
|---|---|---|---|
| ViT-B/16 | 16Γ16 | 12 | 86M |
| ViT-L/16 | 16Γ16 | 24 | 307M |
| Swin-T | 4Γ4 | Hierarchical | 28M |
| DeiT-S | 16Γ16 | 12 | 22M |
CNN vs ViT: CNNs excel with small datasets (inductive bias: locality, translation invariance). ViTs excel with large datasets β no spatial bias, global attention from layer 1.
CV Libraries & Tools
| Library | Use Case | Key Feature |
|---|---|---|
OpenCV | Image I/O, preprocessing | C++ speed, 2500+ algos |
Pillow | Simple image ops | Easy Python interface |
torchvision | PyTorch CV utils | Pretrained models, transforms |
timm | Model zoo | 700+ pretrained CV models |
albumentations | Augmentation | Fastest, bbox-aware transforms |
ultralytics | YOLO detection | YOLOv8, simple API |
detectron2 | Research detection | Mask R-CNN, Faster R-CNN |
supervision | Annotation utils | Works with any detector |
OpenCV β Common Operations
import cv2 import numpy as np # Read / write img = cv2.imread('img.jpg') # BGR img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # Resize img = cv2.resize(img, (224, 224)) # Grayscale + threshold gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY) _, thresh = cv2.threshold( gray, 127, 255, cv2.THRESH_BINARY) # Edge detection edges = cv2.Canny(gray, 100, 200) # Draw bounding box cv2.rectangle(img, (x1,y1), (x2,y2), (0,255,0), 2)
timm library:
import timm; model = timm.create_model('efficientnet_b0', pretrained=True, num_classes=10) β fastest way to get any pretrained CV model.Computer Vision Mastery Checklist
CNN Foundations
- Calculate output size of a Conv2D layer given kernel, stride, padding
- Explain max pooling and its effect on spatial dimensions
- Describe residual connections and why they help training
- Fine-tune a pretrained ResNet for a custom classification task
- Choose the right pretrained backbone for speed vs accuracy
- Apply ImageNet normalization correctly at train and inference time
Detection & Segmentation
- Explain IoU and how it's used as a detection threshold
- Describe Non-Maximum Suppression and when to adjust the threshold
- Distinguish semantic vs instance vs panoptic segmentation
- Train a YOLOv8 model on a custom dataset with dataset.yaml
- Interpret mAP@0.5 and mAP@[.5:.95] for detection results
- Apply Dice loss for class-imbalanced segmentation problems
Augmentation, OCR & ViT
- Build an augmentation pipeline with Albumentations
- Apply Mixup or CutMix to improve classification accuracy
- Set up an OCR pipeline using EasyOCR or PaddleOCR
- Explain CTC loss and why it's needed for sequence recognition
- Compare CNN vs ViT for small vs large dataset scenarios
- Use timm to load any pretrained vision model in 2 lines