Advanced Filtering in Pandas

Filtering is one of the most important skills in data analysis. In real projects, you rarely filter data using a single condition. Instead, you combine multiple conditions, ranges, and logical rules.

In this lesson, you will learn how to perform advanced filtering using Pandas.

Why Advanced Filtering Is Important

Advanced filtering helps you:

• Find specific patterns in large datasets
• Apply business rules to data
• Prepare clean datasets for analysis and modeling

All examples in this lesson use the same dataset you have been working with so far.

Filtering with Multiple Conditions

To apply multiple conditions, use:

• & for AND
• | for OR

Example: Sales greater than 5000 AND region is North.

filtered = sales[
    (sales["sales_amount"] > 5000) &
    (sales["region"] == "North")
]

filtered

Filtering with OR Conditions

Example: Sales from North OR South regions.

filtered = sales[
    (sales["region"] == "North") |
    (sales["region"] == "South")
]

filtered

Filtering Using isin()

When filtering against multiple values, isin() is cleaner and easier.

Example: Filter only Laptop and Phone products.

filtered = sales[
    sales["product"].isin(["Laptop", "Phone"])
]

filtered

Filtering Using Between()

Use between() for numeric ranges.

Example: Sales amount between 3000 and 8000.

filtered = sales[
    sales["sales_amount"].between(3000, 8000)
]

filtered

Filtering with String Conditions

You can filter text columns using string methods.

Example: Customers whose name starts with "A".

filtered = sales[
    sales["customer_name"].str.startswith("A")
]

filtered

Filtering with NOT Conditions

Use the ~ operator to negate conditions.

Example: Exclude Online sales.

filtered = sales[
    ~(sales["sales_channel"] == "Online")
]

filtered

Filtering Using Query()

The query() method allows SQL-like filtering. It is often more readable for complex logic.

Example: Sales greater than 6000 in South region.

filtered = sales.query(
    "sales_amount > 6000 and region == 'South'"
)

filtered

Combining Multiple Advanced Rules

Real-world filtering often combines everything together.

Example:

• Region is North or South
• Product is Laptop
• Sales amount above 5000

filtered = sales[
    (sales["region"].isin(["North", "South"])) &
    (sales["product"] == "Laptop") &
    (sales["sales_amount"] > 5000)
]

filtered

Common Mistakes to Avoid

• Using and instead of &
• Forgetting parentheses around conditions
• Filtering strings without handling missing values

Practice Exercise

Using the dataset:

• Filter rows where sales are above 7000
• Select only Offline sales
• Include only two specific regions
• Exclude one product category

What’s Next?

In the next lesson, you will learn Outlier Detection and how to identify unusual values in datasets.