AI Lesson 86 – Object Tracking | Dataplexa

AI Course

I. AI Foundations
1. Introduction to AI 2. History of AI 3. AI vs ML vs DL 4. Search Algorithms 5. Intelligent Agents 6. Informed & Uninformed Search 7. Constraint Satisfaction 8. Adversarial Search 9. Logic in AI 10. Planning in AI 11. Knowledge Representation 12. Probabilistic AI 13. Bayesian Networks 14. Markov Models 15. AI Ethics 16. AI Applications 17. Intro to PyTorch 18. Intro to TensorFlow 19. AI Workflows 20. Data for AI
II. Machine Learning
21. Introduction to ML 22. ML Workflow 23. Supervised Learning 24. Unsupervised Learning 25. Reinforcement Learning 26. Linear Regression 27. Logistic Regression 28. SVM 29. Decision Trees 30. Random Forest 31. Gradient Boosting 32. XGBoost 33. K-Means 34. Hierarchical Clustering 35. Dimensionality Reduction 36. Feature Engineering 37. Feature Selection 38. Model Evaluation 39. Overfitting vs Underfitting 40. Cross Validation
III. Deep Learning
41. Intro to Deep Learning 42. Neural Networks 43. Activation Functions 44. Backpropagation 45. Loss & Optimizers 46. Regularization 47. CNN Basics 48. CNN Architectures 49. RNN, LSTM, GRU 50. Seq2Seq 51. Transformers 52. Attention 53. Positional Encoding 54. Training DL 55. Hyperparameter Tuning 56. Transfer Learning 57. Autoencoders 58. GAN Basics 59. GAN Applications 60. Model Serving
IV. Natural Language Processing
61. Intro to NLP 62. Text Processing 63. Tokenization 64. Stopwords & Lemmatization 65. BoW & TF-IDF 66. Word Embeddings 67. Sentence Embeddings 68. NLP Classification 69. Sequence Modeling 70. RNN in NLP 71. Transformers in NLP 72. BERT Basics 73. BERT Fine-tuning 74. RoBERTa & DistilBERT 75. Sentiment Analysis 76. NER 77. Machine Translation 78. Text Generation 79. NLP Evaluation 80. NLP Use Cases
V. Computer Vision
81. Intro to CV 82. Image Processing 83. Filters & Edges 84. Convolutions 85. Object Detection 86. Object Tracking 87. Image Segmentation 88. Face Recognition 89. Feature Detection 90. OCR 91. Image Augmentation 92. Generative Vision 93. Vision Transformers 94. Multimodal Models 95. CV Use Cases
VI. LLM Engineering
96. Intro to LLMs 97. Tokenization 98. LLM Architecture 99. Pretraining 100. Fine-tuning 101. RLHF 102. Prompt Engineering 103. Embedding Models 104. Vector Databases 105. LLM Agents 106. Guardrails & Safety 107. AI in Production 108. End-to-End Project

Lesson 86: Object Tracking

Object tracking is the process of following a detected object as it moves across video frames. While object detection tells us what is present and where it is in a single frame, object tracking focuses on continuity over time.

In most real-world systems, detection and tracking work together. First, objects are detected, then they are tracked frame by frame.

Real-World Connection

Object tracking is used in video surveillance to follow people, in sports analytics to track players, in self-driving cars to monitor nearby vehicles, and in retail analytics to study customer movement.

Any system that analyzes video instead of static images relies heavily on tracking.

What Is Object Tracking?

Object tracking assigns a consistent identity to an object across multiple frames in a video. Even if the object moves, changes direction, or partially disappears, the tracker tries to keep following it.

  • Works on video streams
  • Tracks object position over time
  • Maintains object identity

Detection vs Tracking

Understanding the difference is important:

  • Object Detection: Finds objects in a single frame
  • Object Tracking: Follows objects across frames

Tracking is usually faster than detection because it does not re-detect objects in every frame.

How Object Tracking Works

Most tracking systems follow this workflow:

  • Detect objects in the first frame
  • Initialize a tracker for each object
  • Update object positions in subsequent frames
  • Re-detect if tracking fails

Popular Object Tracking Algorithms

  • CSRT: Accurate, slower
  • KCF: Faster, less accurate
  • MOSSE: Very fast, basic
  • SORT / Deep SORT: Tracking with identity preservation

Simple Object Tracking Example

Below is a basic example of tracking an object using OpenCV’s CSRT tracker. 


import cv2

cap = cv2.VideoCapture("video.mp4")

ret, frame = cap.read()
bbox = cv2.selectROI("Select Object", frame, False)

tracker = cv2.TrackerCSRT_create()
tracker.init(frame, bbox)

while True:
    ret, frame = cap.read()
    if not ret:
        break

    success, box = tracker.update(frame)

    if success:
        x, y, w, h = map(int, box)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    cv2.imshow("Object Tracking", frame)
    if cv2.waitKey(30) & 0xFF == 27:
        break

cap.release()
cv2.destroyAllWindows()

What This Code Is Doing

The program allows you to select an object in the first frame. Once selected, the tracker follows that object throughout the video.

Instead of re-detecting the object, the tracker predicts its new position in each frame.

Understanding the Output

The output video shows a bounding box that moves along with the selected object. As long as the object remains visible, the tracker maintains its position.

If the object moves too fast or becomes hidden, tracking may fail.

Challenges in Object Tracking

  • Occlusion (object temporarily hidden)
  • Lighting changes
  • Fast motion
  • Multiple similar objects

Why Object Tracking Matters in AI

Tracking allows machines to understand motion and behavior over time. It is essential for video analytics, robotics, and autonomous navigation.

Modern AI systems often combine detection, tracking, and prediction for better accuracy.

Practice Questions

Practice 1: Object tracking is mainly used on images or video?



Practice 2: What follows an object across multiple frames?



Practice 3: Name a tracking algorithm known for accuracy.



Quick Quiz

Quiz 1: What follows an object over time?





Quiz 2: What does tracking try to preserve?





Quiz 3: Which library provides CSRT tracker?





Coming up next: Image Segmentation — dividing images into meaningful regions.

← Previous Course Index Next →