AI Lesson 29 – Decision Trees | Dataplexa

AI Course

I. AI Foundations
1. Introduction to AI 2. History of AI 3. AI vs ML vs DL 4. Search Algorithms 5. Intelligent Agents 6. Informed & Uninformed Search 7. Constraint Satisfaction 8. Adversarial Search 9. Logic in AI 10. Planning in AI 11. Knowledge Representation 12. Probabilistic AI 13. Bayesian Networks 14. Markov Models 15. AI Ethics 16. AI Applications 17. Intro to PyTorch 18. Intro to TensorFlow 19. AI Workflows 20. Data for AI
II. Machine Learning
21. Introduction to ML 22. ML Workflow 23. Supervised Learning 24. Unsupervised Learning 25. Reinforcement Learning 26. Linear Regression 27. Logistic Regression 28. SVM 29. Decision Trees 30. Random Forest 31. Gradient Boosting 32. XGBoost 33. K-Means 34. Hierarchical Clustering 35. Dimensionality Reduction 36. Feature Engineering 37. Feature Selection 38. Model Evaluation 39. Overfitting vs Underfitting 40. Cross Validation
III. Deep Learning
41. Intro to Deep Learning 42. Neural Networks 43. Activation Functions 44. Backpropagation 45. Loss & Optimizers 46. Regularization 47. CNN Basics 48. CNN Architectures 49. RNN, LSTM, GRU 50. Seq2Seq 51. Transformers 52. Attention 53. Positional Encoding 54. Training DL 55. Hyperparameter Tuning 56. Transfer Learning 57. Autoencoders 58. GAN Basics 59. GAN Applications 60. Model Serving
IV. Natural Language Processing
61. Intro to NLP 62. Text Processing 63. Tokenization 64. Stopwords & Lemmatization 65. BoW & TF-IDF 66. Word Embeddings 67. Sentence Embeddings 68. NLP Classification 69. Sequence Modeling 70. RNN in NLP 71. Transformers in NLP 72. BERT Basics 73. BERT Fine-tuning 74. RoBERTa & DistilBERT 75. Sentiment Analysis 76. NER 77. Machine Translation 78. Text Generation 79. NLP Evaluation 80. NLP Use Cases
V. Computer Vision
81. Intro to CV 82. Image Processing 83. Filters & Edges 84. Convolutions 85. Object Detection 86. Object Tracking 87. Image Segmentation 88. Face Recognition 89. Feature Detection 90. OCR 91. Image Augmentation 92. Generative Vision 93. Vision Transformers 94. Multimodal Models 95. CV Use Cases
VI. LLM Engineering
96. Intro to LLMs 97. Tokenization 98. LLM Architecture 99. Pretraining 100. Fine-tuning 101. RLHF 102. Prompt Engineering 103. Embedding Models 104. Vector Databases 105. LLM Agents 106. Guardrails & Safety 107. AI in Production 108. End-to-End Project

Decision Trees

Decision Trees are supervised machine learning models that make decisions in a way that closely resembles human thinking. Instead of complex mathematical formulas, they use simple rules and conditions to arrive at an answer.

Because of this structure, decision trees are one of the easiest machine learning models to understand and explain.

Why Decision Trees Are Important

Many real-world decisions are made step by step. Decision trees follow the same approach, making them highly interpretable and practical.

Examples include:

  • Loan approval systems
  • Medical diagnosis
  • Customer churn prediction
  • Fraud detection

What Is a Decision Tree?

A decision tree is a flowchart-like structure where data is split at each step based on a condition.

  • Root Node: Starting point of the tree
  • Decision Nodes: Nodes where conditions are applied
  • Leaf Nodes: Final predictions

Each path from root to leaf represents a decision rule.

Real-World Connection

Consider how a bank evaluates a loan application:

  • Is the credit score above a threshold?
  • Is the income stable?
  • Does the applicant have existing loans?

Each answer leads to another question until a final decision is reached. This is exactly how a decision tree operates.

How Decision Trees Split Data

Decision trees split data using metrics that measure how well a split separates classes.

  • Gini Impurity
  • Entropy
  • Information Gain

The goal is to create splits that make each group as pure as possible.

Simple Decision Tree Example

The following example demonstrates how to train a decision tree classifier using Python. 


from sklearn.tree import DecisionTreeClassifier

# Sample data
X = [[25, 50000], [30, 60000], [45, 80000], [35, 65000], [50, 90000]]
y = [0, 0, 1, 0, 1]  # 0 = Reject, 1 = Approve

# Create model
model = DecisionTreeClassifier()

# Train model
model.fit(X, y)

# Predict
result = model.predict([[40, 70000]])
print(result)
[1]

The model predicts approval for the new applicant based on learned decision rules.

How the Tree Makes This Decision

The model checks one condition at a time. For example, it may first check income, then age, and finally reach a leaf node with a decision.

Unlike SVM or logistic regression, decision trees do not rely on probabilities or distance-based logic.

Advantages of Decision Trees

  • Easy to understand and explain
  • Handles both numerical and categorical data
  • No need for feature scaling

Limitations of Decision Trees

  • Prone to overfitting
  • Unstable with small data changes
  • Lower accuracy compared to ensemble methods

Practice Questions

Practice 1: Decision trees make predictions using?



Practice 2: Where does a decision tree give the final output?



Practice 3: Name one metric used to split data.



Quick Quiz

Quiz 1: What is the starting point of a decision tree?





Quiz 2: A major drawback of decision trees is?





Quiz 3: One major strength of decision trees is?





Coming up next: Random Forest — combining multiple decision trees to build stronger models.

← Previous Course Index Next →