Pandas Lesson 8 – Data Cleaning | Dataplexa

Data Cleaning in Pandas

Data cleaning is the process of preparing raw data for analysis by fixing errors, inconsistencies, and unwanted values.

Even after handling missing values, datasets often contain duplicated rows, incorrect formats, inconsistent text, or unnecessary columns.

Why Data Cleaning Is Important

Clean data ensures:

In real projects, data cleaning can take more time than actual analysis.

We continue using the same dataset. Make sure it is loaded correctly.

import pandas as pd

df = pd.read_csv("dataplexa_pandas_sales.csv")

Duplicate rows can distort totals and averages.

To check for duplicates:

df.duplicated()

To remove duplicate rows:

df = df.drop_duplicates()

Inconsistent column names make code harder to read.

A common practice is to convert all column names to lowercase and remove spaces.

df.columns = df.columns.str.lower().str.replace(" ", "_")

This makes column access predictable and clean.

Text columns often contain extra spaces, mixed letter cases, or inconsistent values.

Example: Cleaning a region column.

df["region"] = df["region"].str.strip().str.title()

This removes extra spaces and standardizes text formatting.

Sometimes numeric data is stored as text. This prevents calculations.

Check data types:

df.dtypes

Convert a column to numeric:

df["sales"] = pd.to_numeric(df["sales"], errors="coerce")

Invalid values automatically become NaN.

Datasets often include columns that are not useful.

Remove unwanted columns:

df = df.drop(columns=["unnecessary_column"])

Keeping only relevant columns improves performance and clarity.

Some values may be logically incorrect, such as negative sales.

Filter out invalid values:

df = df[df["sales"] >= 0]

This ensures data consistency.

After cleaning, always recheck the dataset.

df.info()
df.describe()

These summaries confirm the dataset is ready for analysis.

Using the dataset:

Now that the dataset is clean, the next lesson focuses on sorting and ranking data to organize information meaningfully.