AI Lesson 58 – GAN Basics (Generative Adversarial Nets) | Dataplexa

AI Course

I. AI Foundations
1. Introduction to AI 2. History of AI 3. AI vs ML vs DL 4. Search Algorithms 5. Intelligent Agents 6. Informed & Uninformed Search 7. Constraint Satisfaction 8. Adversarial Search 9. Logic in AI 10. Planning in AI 11. Knowledge Representation 12. Probabilistic AI 13. Bayesian Networks 14. Markov Models 15. AI Ethics 16. AI Applications 17. Intro to PyTorch 18. Intro to TensorFlow 19. AI Workflows 20. Data for AI
II. Machine Learning
21. Introduction to ML 22. ML Workflow 23. Supervised Learning 24. Unsupervised Learning 25. Reinforcement Learning 26. Linear Regression 27. Logistic Regression 28. SVM 29. Decision Trees 30. Random Forest 31. Gradient Boosting 32. XGBoost 33. K-Means 34. Hierarchical Clustering 35. Dimensionality Reduction 36. Feature Engineering 37. Feature Selection 38. Model Evaluation 39. Overfitting vs Underfitting 40. Cross Validation
III. Deep Learning
41. Intro to Deep Learning 42. Neural Networks 43. Activation Functions 44. Backpropagation 45. Loss & Optimizers 46. Regularization 47. CNN Basics 48. CNN Architectures 49. RNN, LSTM, GRU 50. Seq2Seq 51. Transformers 52. Attention 53. Positional Encoding 54. Training DL 55. Hyperparameter Tuning 56. Transfer Learning 57. Autoencoders 58. GAN Basics 59. GAN Applications 60. Model Serving
IV. Natural Language Processing
61. Intro to NLP 62. Text Processing 63. Tokenization 64. Stopwords & Lemmatization 65. BoW & TF-IDF 66. Word Embeddings 67. Sentence Embeddings 68. NLP Classification 69. Sequence Modeling 70. RNN in NLP 71. Transformers in NLP 72. BERT Basics 73. BERT Fine-tuning 74. RoBERTa & DistilBERT 75. Sentiment Analysis 76. NER 77. Machine Translation 78. Text Generation 79. NLP Evaluation 80. NLP Use Cases
V. Computer Vision
81. Intro to CV 82. Image Processing 83. Filters & Edges 84. Convolutions 85. Object Detection 86. Object Tracking 87. Image Segmentation 88. Face Recognition 89. Feature Detection 90. OCR 91. Image Augmentation 92. Generative Vision 93. Vision Transformers 94. Multimodal Models 95. CV Use Cases
VI. LLM Engineering
96. Intro to LLMs 97. Tokenization 98. LLM Architecture 99. Pretraining 100. Fine-tuning 101. RLHF 102. Prompt Engineering 103. Embedding Models 104. Vector Databases 105. LLM Agents 106. Guardrails & Safety 107. AI in Production 108. End-to-End Project

Overfitting and Underfitting

When training a machine learning model, the goal is not just to perform well on training data, but to perform well on new, unseen data. Two common problems prevent this from happening: overfitting and underfitting.

Understanding these concepts is critical because even a complex, high-accuracy model can completely fail in real-world use if it is not trained correctly.

Real-World Connection

Imagine preparing for an exam. If you memorize only past question papers word by word, you may score well on similar questions but fail when new questions appear. This is overfitting. If you barely study and only know basic definitions, you will perform poorly everywhere. This is underfitting.

What Is Underfitting?

Underfitting happens when a model is too simple to capture the patterns in the data. It fails to learn meaningful relationships, leading to poor performance on both training and test data.

  • Model is too simple
  • High bias
  • Poor training and test accuracy

What Is Overfitting?

Overfitting occurs when a model learns the training data too well, including noise and random fluctuations. While it performs very well on training data, it performs poorly on unseen data.

  • Model is too complex
  • High variance
  • Very high training accuracy, low test accuracy

The Bias–Variance Tradeoff

Bias represents error from overly simple assumptions. Variance represents error from sensitivity to small changes in training data. A good model balances both bias and variance.

Visual Intuition

A straight line trying to fit curved data underfits. A curve passing through every data point overfits. The best model captures the overall trend without memorizing noise.

Underfitting Example (Python)


from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error
import numpy as np

X = np.array([[1], [2], [3], [4], [5]])
y = np.array([1, 4, 9, 16, 25])

model = LinearRegression()
model.fit(X, y)

predictions = model.predict(X)
print("MSE:", mean_squared_error(y, predictions))
MSE: 22.0

Understanding the Output

The linear model fails to capture the quadratic relationship, resulting in high error. This is a clear example of underfitting.

Overfitting Example (Conceptual)

A very deep decision tree may perfectly classify training data but fail on new data because it memorizes specific examples instead of learning general patterns.

How to Detect Overfitting and Underfitting

  • Compare training and validation performance
  • Plot learning curves
  • Use cross-validation

How to Fix Underfitting

  • Use a more complex model
  • Add more relevant features
  • Reduce regularization

How to Fix Overfitting

  • Simplify the model
  • Collect more training data
  • Apply regularization
  • Use feature selection

Practice Questions

Practice 1: What problem occurs when a model is too simple?



Practice 2: What problem occurs when a model memorizes training data?



Practice 3: What balance explains underfitting and overfitting?



Quick Quiz

Quiz 1: Overfitting is associated with which issue?





Quiz 2: A model performing poorly on both train and test data indicates?





Quiz 3: Which technique helps reduce overfitting?





Coming up next: Cross-Validation — testing models reliably using multiple data splits.

← Previous Course Index Next →