Univariate Analysis

import pandas as pd      # pandas: Python's core data table library — all statistical methods and DataFrame ops
import numpy as np       # numpy: numerical library — np.nan for missing values, np.percentile for IQR

# Patient dataset — 12 rows, realistic clinical fields
df = pd.DataFrame({
    'patient_id':    range(1001, 1013),
    'age':           [45, 62, 38, 71, 55, 48, 67, 29, 83, 52, 44, 61],
    'bmi':           [24.1, 31.8, 22.4, 28.6, np.nan, 35.2, 27.9, 21.3, 33.1, 26.5, np.nan, 29.4],
    'systolic_bp':   [118, 145, 122, 158, 134, 142, 128, 115, 172, 136, 119, 141],
    'num_admissions':[1, 4, 1, 6, 2, 3, 5, 1, 8, 2, 1, 3],
    'los_days':      [3, 7, 2, 12, 4, 6, 9, 2, 15, 5, 3, 6]   # length of stay in days
})

def numeric_profile(dataframe):
    """Produces a full univariate statistical profile for every numeric column."""
    results = []
    for col in dataframe.select_dtypes(include='number').columns:   # only numeric columns
        s = dataframe[col].dropna()                                  # drop NaN for clean stats — note separately

        q1  = s.quantile(0.25)                                       # first quartile
        q3  = s.quantile(0.75)                                       # third quartile
        iqr = q3 - q1                                                # interquartile range — the middle 50% spread

        # IQR outlier rule: anything outside [Q1 - 1.5×IQR, Q3 + 1.5×IQR] is a candidate outlier
        lower_fence = q1 - 1.5 * iqr
        upper_fence = q3 + 1.5 * iqr
        n_outliers  = ((s < lower_fence) | (s > upper_fence)).sum()  # count rows outside the fences

        results.append({
            'column':       col,
            'count':        s.count(),                               # non-null count
            'missing_pct':  round(dataframe[col].isnull().mean() * 100, 1),  # % missing from original column
            'mean':         round(s.mean(), 2),
            'median':       round(s.median(), 2),
            'std':          round(s.std(), 2),
            'iqr':          round(iqr, 2),
            'min':          s.min(),
            'max':          s.max(),
            'skewness':     round(s.skew(), 3),                      # .skew() — Fisher-Pearson coefficient
            'kurtosis':     round(s.kurt(), 3),                      # .kurt() — excess kurtosis (normal = 0)
            'n_outliers':   int(n_outliers)
        })

    return pd.DataFrame(results).set_index('column')

profile = numeric_profile(df)
print(profile.to_string())

What just happened?

pandas is providing every statistical method in the profile. .quantile(0.25) and .quantile(0.75) return Q1 and Q3 — the 25th and 75th percentiles. The IQR (Q3 − Q1) is the width of the middle 50% of values. The fences at Q1 − 1.5×IQR and Q3 + 1.5×IQR define the boundary beyond which a value is flagged as a potential outlier — the same rule used by box plots.

numpy is present as the numerical backbone — all pandas operations ultimately use numpy arrays under the hood.

Reading the profile: bmi has 16.7% missing — flag that immediately. num_admissions (skew 0.830) and los_days (skew 1.023) are both moderately-to-highly right-skewed — expect a few patients with very long stays or many admissions dragging the mean up. Both have one flagged outlier. patient_id has kurtosis −1.2, which is expected — it's a uniform sequence of IDs, not a natural distribution at all.

Bimodal is the one that surprises people most. If you see two peaks, you might have two different populations mixed together — e.g. a dataset with both adults and children, or weekday vs weekend behaviour.

import pandas as pd      # pandas: data library — .value_counts(), .nunique(), and categorical analysis
import numpy as np       # numpy: numerical library — standard import for EDA scripts

# Extend the patient dataset with categorical columns
df['admission_type'] = ['Emergency','Elective','Emergency','Emergency','Urgent',
                         'Elective','Emergency','Elective','Emergency','Urgent',
                         'Emergency','Elective']
df['primary_diagnosis'] = ['Cardiac','Respiratory','Orthopaedic','Cardiac','Diabetes',
                            'Respiratory','Cardiac','Orthopaedic','Cardiac','Diabetes',
                            'Respiratory','Cardiac']
df['ward'] = ['Ward A','Ward B','Ward C','Ward A','Ward B','Ward C',
              'Ward A','Ward B','Ward A','Ward C','Ward B','Ward A']

cat_cols = ['admission_type', 'primary_diagnosis', 'ward']  # columns to profile

for col in cat_cols:
    n_unique  = df[col].nunique()                            # .nunique() — count of distinct values
    top_value = df[col].value_counts().index[0]              # most frequent category
    top_pct   = df[col].value_counts(normalize=True).iloc[0] * 100  # normalize=True gives proportions

    # Rare category threshold: any category with fewer than 2 occurrences
    rare = df[col].value_counts()
    rare_cats = rare[rare < 2].index.tolist()               # list of category names with count < 2

    print(f"--- {col} ---")
    print(f"  Unique values (cardinality): {n_unique}")
    print(f"  Most frequent: '{top_value}' ({top_pct:.1f}% of rows)")
    print(f"  Rare categories (<2 rows):   {rare_cats if rare_cats else 'None'}")
    print(f"  Missing: {df[col].isnull().sum()}")
    print()
    print(df[col].value_counts().to_string())               # full frequency table
    print()

What just happened?

pandas provides the two key categorical analysis methods here. .nunique() counts the number of distinct values in a column — this is cardinality. A column with 500 unique values in 600 rows is probably a free-text field or an ID, not a useful categorical feature. .value_counts(normalize=True) returns proportions (0.0–1.0) instead of raw counts, making it easy to see percentage dominance.

The rare category check matters for modelling: if "Urgent" only appears once in 1,000 rows, one-hot encoding will create a column that's 999 zeros and 1 one. That column carries almost no predictive signal and wastes memory. In practice you'd either merge rare categories into an "Other" bucket or drop them.

Emergency admissions dominate at 58.3%. Cardiac is the most common diagnosis at 41.7%. Both are class imbalance signals — if you're building a classifier predicting any of these labels, the model will be biased toward the majority class without rebalancing.

import pandas as pd      # pandas: data library — Series operations and value binning via pd.cut()
import numpy as np       # numpy: numerical library — np.histogram() for bin counting

# Use a larger, more realistic age distribution to demonstrate bimodality clearly
ages = pd.Series([28, 31, 34, 29, 33, 27, 35, 30, 32, 36,   # younger cohort: 27–40
                  58, 63, 71, 65, 68, 72, 59, 66, 74, 61,   # older cohort: 55–75
                  44, 52, 48])                               # middle few patients

print(f"Mean:     {ages.mean():.1f}")
print(f"Median:   {ages.median():.1f}")
print(f"Std dev:  {ages.std():.1f}")
print(f"Skewness: {ages.skew():.3f}")
print()

# Bin the ages to reveal the shape — pd.cut() divides values into discrete intervals
# bins defines the bucket edges; labels gives each bucket a readable name
bins   = [20, 30, 40, 50, 60, 70, 80]
labels = ['20s', '30s', '40s', '50s', '60s', '70s']
age_bins = pd.cut(ages, bins=bins, labels=labels, right=False)  # right=False: intervals are [left, right)

# Count how many patients fall in each decade bucket
freq = age_bins.value_counts().sort_index()   # .sort_index() restores chronological order after value_counts sorts by count
print("Age distribution by decade:")
print(freq)
print()

# Simple text histogram to visualise shape without matplotlib
print("Text histogram:")
for label, count in freq.items():
    bar = '█' * count                         # one block character per patient
    print(f"  {label}: {bar} ({count})")

What just happened?

pandas provides pd.cut() — it bins continuous values into discrete intervals you define with the bins parameter. The right=False argument makes intervals left-inclusive: age 30 goes into the '30s' bucket, not the '20s' bucket. .value_counts().sort_index() then counts entries per bucket and restores the natural order (by decade rather than by frequency).

numpy is imported as standard — used here for the numerical infrastructure under the pandas operations.

The mean (49.9) and skewness (0.121) suggest a nearly normal, symmetric distribution. But the text histogram completely disproves this: there's a spike of 10 in the 30s, a valley in the 40s–50s, and a second peak in the 60s. This is textbook bimodality — two distinct patient populations merged into one dataset. Mean and skewness are both blind to this. This is exactly why you bin and visualise, not just compute summary statistics.

import pandas as pd      # pandas: data library — all column analysis methods used in the report function
import numpy as np       # numpy: numerical library — underpins all pandas statistical computation

def univariate_report(dataframe, numeric_cols, cat_cols):
    """Prints a plain-English univariate summary for each column."""

    print("=" * 55)
    print("UNIVARIATE ANALYSIS REPORT")
    print("=" * 55)

    for col in numeric_cols:
        s        = dataframe[col].dropna()
        skew     = s.skew()
        miss_pct = dataframe[col].isnull().mean() * 100
        q1, q3   = s.quantile(0.25), s.quantile(0.75)
        iqr      = q3 - q1
        n_out    = ((s < q1 - 1.5*iqr) | (s > q3 + 1.5*iqr)).sum()

        # Build a plain-English skew flag
        if abs(skew) < 0.5:   skew_flag = "symmetric"
        elif abs(skew) < 1.0: skew_flag = "moderately skewed"
        else:                  skew_flag = "HIGHLY SKEWED — consider log transform"

        print(f"\n[NUMERIC] {col}")
        print(f"  Range: {s.min():.1f} – {s.max():.1f}  |  Mean: {s.mean():.1f}  |  Median: {s.median():.1f}")
        print(f"  Shape: {skew_flag} (skew={skew:.2f})")
        print(f"  Missing: {miss_pct:.1f}%  |  Outliers (IQR rule): {n_out}")

    for col in cat_cols:
        n_unique  = dataframe[col].nunique()
        top       = dataframe[col].value_counts()
        top_label = top.index[0]
        top_pct   = top.iloc[0] / len(dataframe) * 100
        rare      = top[top < 2].index.tolist()

        print(f"\n[CATEGORICAL] {col}")
        print(f"  Cardinality: {n_unique} unique values")
        print(f"  Dominant: '{top_label}' ({top_pct:.0f}% of rows)")
        print(f"  Rare categories (<2 rows): {rare if rare else 'None'}")
        print(f"  Missing: {dataframe[col].isnull().sum()}")

    print("\n" + "=" * 55)

# Run the report on the patient dataset (df from earlier blocks)
univariate_report(
    df,
    numeric_cols=['age', 'bmi', 'systolic_bp', 'num_admissions', 'los_days'],
    cat_cols=['admission_type', 'primary_diagnosis', 'ward']
)

What just happened?

pandas is the engine throughout this report function. Every method — .dropna(), .skew(), .quantile(), .isnull().mean(), .nunique(), .value_counts() — is a pandas Series method. The function loops through your specified column lists and builds human-readable output rather than a raw table. This is exactly the kind of artefact you'd paste into a Jupyter notebook markdown cell or a project Confluence page.

Three action items jump out of this report: BMI is missing 16.7% — needs imputation strategy. Length of stay is highly skewed at 1.02 — log transform before modelling. Emergency admissions dominate at 58% — class imbalance must be addressed if predicting admission type. A report that generates action items, not just statistics — that's what univariate analysis is for.

Teacher's Note

The single most common mistake in univariate analysis is stopping at .describe(). It gives you mean, std, min, max and quartiles — but no skewness, no kurtosis, no outlier count, no missing rate, no shape flag. .describe() is a starting point, not the finish line.

The second mistake is treating numeric and categorical columns the same way. Running .describe() on a categorical column gives you count, unique, top, and freq — a completely different set of numbers that answer completely different questions. Know which type of column you're looking at before you run a single method.

Up Next · Lesson 17

Bivariate Analysis

Move beyond single columns — explore how two variables interact, whether they correlate, and what relationships your model will need to learn.