Introduction to Exploratory Data Analysis

Before you build any model or write a single line of analysis code, you need to actually look at your data — that's exactly what Exploratory Data Analysis is, and this lesson walks you through what it means, why it matters, and how to do it in Python.

What EDA Actually Is

Exploratory Data Analysis — EDA for short — is the process of getting familiar with a dataset before doing anything formal with it. Think of it like reading a restaurant menu before ordering. You're not committing to anything yet. You're scanning, asking questions, and getting a feel for what's there.

The term was coined by statistician John Tukey in 1977. His core idea was simple: don't jump straight to hypothesis testing or modeling. First, explore. Look at the shapes, the patterns, the weird values, the things that don't make sense. The data will tell you things you didn't expect.

In practice, EDA means:

• Looking at the first and last few rows of data
• Checking column names, data types, and dimensions
• Finding missing values and duplicates
• Computing basic summary statistics (mean, median, min, max)
• Visualising distributions and relationships between variables

None of this is fancy. But skipping it is one of the most common mistakes beginners make — and even experienced analysts get burned by it.

A Real-World Analogy

EDA is that orientation phase. It saves you from building a beautiful chart on dirty data, or drawing conclusions from a column that turned out to be 40% empty.

The EDA Process at a Glance

Here is a simple visual of the typical EDA workflow — the steps flow from raw data all the way to insights:

This course follows exactly this flow — we start with structure and loading, move through cleaning, statistics, and visualisation, and end with real-world case studies.

Your First EDA in Python

The scenario: You have just joined a small electronics retailer as a data analyst. Your manager hands you a sales dataset and says "tell me what's in here." There is no dashboard yet, no report template — just raw data. This is exactly where EDA begins.

We will build the dataset inline using pandas — no file needed — and run our first three inspection commands.

import pandas as pd    # pandas is the core data analysis library in Python
import numpy as np     # numpy handles numerical operations; pandas uses it internally

# Build a small electronics sales dataset directly in Python
# Each key is a column name, each list is that column's values
sales_df = pd.DataFrame({
    'order_id':    [1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008],   # unique order ID
    'customer':    ['Alice', 'Bob', 'Carol', 'Dave', 'Eve', 'Frank', 'Grace', 'Hank'],
    'product':     ['Laptop', 'Mouse', 'Keyboard', 'Laptop', 'Monitor', 'Mouse', 'Keyboard', 'Laptop'],
    'quantity':    [1, 3, 2, 1, 1, 5, 2, 1],                           # units sold per order
    'unit_price':  [999.99, 29.99, 79.99, 999.99, 349.99, 29.99, 79.99, 999.99],
    'revenue':     [999.99, 89.97, 159.98, 999.99, 349.99, 149.95, 159.98, 999.99],
    'region':      ['North', 'South', 'East', 'West', 'North', 'South', 'East', None]  # None = missing
})

# Step 1 – How big is this dataset? Returns a tuple (rows, columns)
print("Shape:", sales_df.shape)

# Step 2 – What does the data look like? Shows first 5 rows by default
print(sales_df.head())

# Step 3 – What data type is stored in each column?
print(sales_df.dtypes)

Shape: (8, 7)

   order_id customer   product  quantity  unit_price   revenue region
0      1001    Alice    Laptop         1      999.99    999.99  North
1      1002      Bob     Mouse         3       29.99     89.97  South
2      1003    Carol  Keyboard         2       79.99    159.98   East
3      1004     Dave    Laptop         1      999.99    999.99   West
4      1005      Eve   Monitor         1      349.99    349.99  North

order_id        int64
customer       object
product        object
quantity        int64
unit_price    float64
revenue       float64
region         object
dtype: object

The describe() Function — Your Instant Summary

The scenario: Your manager asks — "Give me a quick statistical snapshot of the sales numbers." Instead of calculating mean, min, and max one by one for each column, pandas gives you a single command that handles everything at once. This is the fastest way to understand the numerical range and spread of your data.

import pandas as pd

# Rebuild the sales dataset (numeric columns only for this example)
sales_df = pd.DataFrame({
    'order_id':    [1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008],
    'quantity':    [1, 3, 2, 1, 1, 5, 2, 1],
    'unit_price':  [999.99, 29.99, 79.99, 999.99, 349.99, 29.99, 79.99, 999.99],
    'revenue':     [999.99, 89.97, 159.98, 999.99, 349.99, 149.95, 159.98, 999.99]
})

# describe() computes 8 statistics for every numeric column automatically
# .round(2) is chained on top to keep decimal places clean and readable
print(sales_df.describe().round(2))

       order_id  quantity  unit_price    revenue
count      8.00      8.00        8.00       8.00
mean    1004.50      2.00      321.24     488.74
std        2.45      1.41      413.58     402.43
min     1001.00      1.00       29.99      89.97
25%     1002.75      1.00       79.99     159.98
50%     1004.50      1.50      214.99     254.99
75%     1006.25      2.25      999.99     999.99
max     1008.00      5.00      999.99     999.99

Spotting Missing Values Immediately

The scenario: Before any analysis, you need to know if your data has gaps. A column that looks complete might be 30% empty. If you skip this check and later group or average that column, your results will be silently wrong — no error message, just bad numbers. Here we find missing values and report them as a percentage, which is far more actionable than a raw count.

import pandas as pd

# Rebuild the full sales dataset — note the None in the last row's region column
sales_df = pd.DataFrame({
    'order_id':   [1001, 1002, 1003, 1004, 1005, 1006, 1007, 1008],
    'customer':   ['Alice', 'Bob', 'Carol', 'Dave', 'Eve', 'Frank', 'Grace', 'Hank'],
    'product':    ['Laptop', 'Mouse', 'Keyboard', 'Laptop', 'Monitor', 'Mouse', 'Keyboard', 'Laptop'],
    'quantity':   [1, 3, 2, 1, 1, 5, 2, 1],
    'unit_price': [999.99, 29.99, 79.99, 999.99, 349.99, 29.99, 79.99, 999.99],
    'revenue':    [999.99, 89.97, 159.98, 999.99, 349.99, 149.95, 159.98, 999.99],
    'region':     ['North', 'South', 'East', 'West', 'North', 'South', 'East', None]
})

# isnull() scans every cell — returns True if value is missing, False if not
# .sum() then counts the True values per column = total missing per column
missing_count = sales_df.isnull().sum()

# Divide by total rows (len) and multiply by 100 to get a percentage per column
missing_pct = (sales_df.isnull().sum() / len(sales_df) * 100).round(1)

# Combine count and percentage into one clean summary DataFrame
missing_summary = pd.DataFrame({
    'missing_count': missing_count,
    'missing_pct':   missing_pct
})

# Filter to show only columns that actually have missing data
print(missing_summary[missing_summary['missing_count'] > 0])

        missing_count  missing_pct
region              1         12.5

A Quick Visual Check — Revenue by Product

Numbers tell part of the story. A chart can reveal the same insight in seconds. Here is a mockup of what a revenue-by-product bar chart looks like from our dataset — we will write the actual matplotlib and seaborn code to produce this in later lessons.

This is the kind of insight EDA surfaces in the first ten minutes. Laptops dominate revenue, but mice are ordered more frequently. A business strategy built on order volume looks completely different from one built on revenue — and EDA is what makes that distinction visible before you build anything on top of it.

Practice Questions

Quiz