Regular Expressions in Python | Python Course | Dataplexa

Regular Expressions in Python

Every application that handles text — web forms, log files, search engines, data pipelines — eventually needs to find patterns rather than exact strings. Regular expressions (regex) are a compact, powerful language for describing text patterns. Python's built-in re module puts the full power of regex at your fingertips without installing anything extra.

This lesson builds from basic matching all the way to groups, backreferences, flags, non-capturing groups, and a real-world data extraction example — everything you need to handle text confidently in production code.

Importing re and the Core Functions

Everything in this lesson comes from the standard library — just import re.

re.search(pattern, string) — scans the entire string, returns the first match object or None
re.match(pattern, string) — checks for a match only at the beginning of the string
re.findall(pattern, string) — returns a list of all non-overlapping matches
re.finditer(pattern, string) — returns an iterator of match objects (includes positions)
re.sub(pattern, replacement, string) — replaces all matches with a new string
re.split(pattern, string) — splits the string at every match
re.compile(pattern) — compiles a pattern into a reusable regex object
re.fullmatch(pattern, string) — matches only if the entire string matches the pattern

Your First Pattern — search and match

Always use raw strings r"..." for regex patterns. Without the r, Python interprets backslashes as escape sequences — \d becomes a problem, \n becomes a newline. Raw strings prevent this.

import re

text = "Order #4821 was shipped on 2024-03-15"

# search() — finds pattern anywhere in the string
result = re.search(r"\d+", text)   # \d+ = one or more digits

if result:
    print("Found     :", result.group())         # matched text
    print("Position  :", result.start(), "to", result.end())

# match() — only checks from the START
m = re.match(r"\d+", text)
print("match()   :", m)    # None — string starts with "Order", not a digit

# fullmatch() — the ENTIRE string must match
code = "AB123"
print(re.fullmatch(r"[A-Z]{2}\d{3}", code))    # match object — full match
print(re.fullmatch(r"[A-Z]{2}\d{3}", "AB123X")) # None — extra character

Found : 4821 Position : 7 to 11 match() : None <re.Match object; span=(0, 5), match='AB123'> None

Always check if result: before calling .group() — the function returns None if nothing matches.
re.fullmatch() is ideal for validating that an entire input — a postal code, product code, or plate number — matches a required format exactly.

Pattern Syntax Reference

Pattern	Meaning	Example Match
`\d`	Any digit 0–9	`5`, `9`
`\D`	Any non-digit	`a`, `!`
`\w`	Word char (letter, digit, underscore)	`a`, `3`, `_`
`\W`	Non-word character	`!`,
`\s`	Any whitespace	space, `\t`, `\n`
`\S`	Non-whitespace	`a`, `5`
`.`	Any character except newline	`a`, `!`, `5`
`^`	Start of string	`^Hello` → "Hello world"
`$`	End of string	`end$` → "the end"
`*`	0 or more	`ab*` → `a`, `ab`, `abb`
`+`	1 or more	`\d+` → `1`, `42`
`?`	0 or 1 (optional)	`colou?r` → `color`, `colour`
`{n}`	Exactly n times	`\d{4}` → `2024`
`{n,m}`	Between n and m times	`\d{2,4}` → `12`, `1234`
`[abc]`	Any one char from set	`[aeiou]` → any vowel
`[^abc]`	Any char NOT in set	`[^0-9]` → non-digit
`a\|b`	Either a or b	`cat\|dog`
`(abc)`	Capturing group	captures `abc`
`(?:abc)`	Non-capturing group	groups but does not capture
`(?P<name>)`	Named capturing group	access by name

Finding All Matches — re.findall

re.findall() is the most commonly used function. It returns every match in a plain list — no match objects, no loops needed. Returns an empty list if nothing matches — safe without an if check.

import re

log = "Errors on 2024-01-05, 2024-02-18, and 2024-03-22"

# Extract all dates
dates = re.findall(r"\d{4}-\d{2}-\d{2}", log)
print("Dates:", dates)

receipt = "Total: $12.99  Tax: $1.04  Tip: $2.50  Subtotal: $15.45"

# Extract all dollar amounts
amounts = re.findall(r"\$\d+\.\d{2}", receipt)
print("Amounts:", amounts)

# Extract all hashtags from a tweet
tweet = "Loving #Python and #DataScience! #AI is the future #100DaysOfCode"
tags = re.findall(r"#\w+", tweet)
print("Hashtags:", tags)

# Extract all words (sequences of letters only)
sentence = "Hello, world! Python is great."
words = re.findall(r"[a-zA-Z]+", sentence)
print("Words:", words)

Dates: ['2024-01-05', '2024-02-18', '2024-03-22'] Amounts: ['$12.99', '$1.04', '$2.50', '$15.45'] Hashtags: ['#Python', '#DataScience', '#AI', '#100DaysOfCode'] Words: ['Hello', 'world', 'Python', 'is', 'great']

When the pattern contains a capturing group (), findall returns the contents of the group, not the full match.
Use re.finditer() when you also need position information — it returns match objects with .start(), .end(), and .span().

Capturing Groups — Extract Parts of a Match

Wrapping part of a pattern in parentheses creates a capturing group. Groups let you extract specific portions of a match. Named groups make the code even clearer.

import re

# Extract year, month, day separately
date_str = "Invoice date: 2024-07-19"

m = re.search(r"(\d{4})-(\d{2})-(\d{2})", date_str)
if m:
    print("Full match:", m.group(0))   # entire match
    print("Year      :", m.group(1))   # first group
    print("Month     :", m.group(2))   # second group
    print("Day       :", m.group(3))   # third group
    print("All groups:", m.groups())   # tuple of all groups

# Named groups — even clearer
m2 = re.search(r"(?P\d{4})-(?P\d{2})-(?P\d{2})", date_str)
if m2:
    print("Named year :", m2.group("year"))
    print("Named month:", m2.group("month"))

# findall with groups — returns list of tuples when multiple groups
log = "2024-01-05 ERROR 2024-02-18 WARNING 2024-03-22 INFO"
entries = re.findall(r"(\d{4}-\d{2}-\d{2}) (\w+)", log)
print("Log entries:", entries)

Full match: 2024-07-19 Year : 2024 Month : 07 Day : 19 All groups: ('2024', '07', '19') Named year : 2024 Named month: 07 Log entries: [('2024-01-05', 'ERROR'), ('2024-02-18', 'WARNING'), ('2024-03-22', 'INFO')]

group(0) or group() returns the entire match. group(1), group(2) return individual groups left to right.
Named groups (?P<name>...) let you reference captures by name — much cleaner in long patterns.
When findall() has multiple groups, it returns a list of tuples — one tuple per match.

Non-Capturing Groups

Sometimes you need to group part of a pattern (for alternation or repetition) without capturing it. Use (?:...) for a non-capturing group — it groups but does not appear in .groups() or affect findall()'s output.

import re

# Capturing group — "www." captured unnecessarily
url1 = "Visit https://www.dataplexa.com for more"
m1 = re.search(r"https?://(www\.)?(\w+\.\w+)", url1)
if m1:
    print("With capture :", m1.groups())      # ('www.', 'dataplexa.com')

# Non-capturing group — "www." grouped but not captured
m2 = re.search(r"https?://(?:www\.)?(\w+\.\w+)", url1)
if m2:
    print("Non-capture  :", m2.groups())      # ('dataplexa.com',) — cleaner

# Alternation with non-capturing group
text = "I have a cat and a dog and a fish"
animals = re.findall(r"(?:cat|dog|fish)", text)
print("Animals:", animals)

With capture : ('www.', 'dataplexa.com') Non-capture : ('dataplexa.com',) Animals: ['cat', 'dog', 'fish']

Substitution — re.sub

re.sub() finds all matches and replaces them. It is the regex equivalent of str.replace() but with full pattern power. Use backreferences \1, \2 to insert captured group content into the replacement.

import re

# Mask all digits (e.g. hide card numbers)
text = "Card: 4532-1568-2345-6789  PIN: 4821"
masked = re.sub(r"\d", "*", text)
print(masked)

# Normalise spacing — collapse multiple spaces to one
messy = "Name:    Alice     Age:   30     City:  London"
clean = re.sub(r"\s+", " ", messy).strip()
print(clean)

# Backreference — wrap all numbers in brackets
tagged = re.sub(r"(\d+)", r"[\1]", "Order 4821 has 3 items costing $42")
print(tagged)

# Reformat date from MM/DD/YYYY to YYYY-MM-DD
date = "Invoice date: 07/19/2024"
reformatted = re.sub(r"(\d{2})/(\d{2})/(\d{4})", r"\3-\1-\2", date)
print(reformatted)

# Limit replacements — only replace first 2 occurrences
text2 = "apple apple apple apple"
print(re.sub(r"apple", "mango", text2, count=2))

Card: ****-****-****-**** PIN: **** Name: Alice Age: 30 City: London Order [4821] has [3] items costing $[42] Invoice date: 2024-07-19 mango mango apple apple

re.sub() always returns a new string — the original is not modified.
The date reformatting pattern — swapping group order with backreferences — is one of the most practical everyday uses of regex.
Pass count=n to limit how many substitutions are made.

Splitting — re.split

re.split() divides a string at every match of the pattern — more powerful than str.split() because the delimiter can be a complex pattern.

import re

# Split on any punctuation followed by optional whitespace
sentence = "First.Second! Third? Fourth; Fifth"
parts = re.split(r"[.!?;]\s*", sentence)
print(parts)

# Split on one or more whitespace characters
data = "alice   bob\tcarol\ndave    eve"
names = re.split(r"\s+", data)
print(names)

# Split on comma or semicolon (inconsistent CSV-like data)
csv_line = "alice,25;London,engineer;active"
fields = re.split(r"[,;]", csv_line)
print(fields)

# maxsplit — limit the number of splits
text = "a:b:c:d:e"
print(re.split(r":", text, maxsplit=2))

['First', 'Second', 'Third', 'Fourth', 'Fifth'] ['alice', 'bob', 'carol', 'dave', 'eve'] ['alice', '25', 'London', 'engineer', 'active'] ['a', 'b', 'c:d:e']

Compiling Patterns — re.compile

If you use the same pattern many times — in a loop, across many strings, or in a frequently-called function — compile it once with re.compile(). The compiled object is faster on repeated use and supports all the same methods.

import re

# Compile once — reuse many times
EMAIL = re.compile(r"[\w\.-]+@[\w\.-]+\.\w{2,}")
PHONE = re.compile(r"\+?\d[\d\s\-\(\)]{8,}\d")

contacts = [
    "Alice: alice@dataplexa.com, +44 7911 123456",
    "Bob: not-an-email, 07700 900123",
    "Carol: carol@example.org, (555) 867-5309",
    "No contact info here"
]

for line in contacts:
    email = EMAIL.search(line)
    phone = PHONE.search(line)
    print(f"  Email: {email.group() if email else 'none':<28} Phone: {phone.group() if phone else 'none'}")

Email: alice@dataplexa.com Phone: +44 7911 123456 Email: none Phone: 07700 900123 Email: carol@example.org Phone: (555) 867-5309 Email: none Phone: none

Compiled patterns are typically stored as module-level constants in uppercase — EMAIL, PHONE.
The compiled object supports all the same methods: .search(), .findall(), .sub(), .split().

Flags — Modifying Match Behaviour

import re

text = "Python is great. PYTHON is fast. python is fun."

# re.IGNORECASE — case-insensitive matching
matches = re.findall(r"python", text, re.IGNORECASE)
print("Case insensitive:", matches)

# re.MULTILINE — ^ and $ match at each LINE boundary
multi = "first line\nsecond line\nthird line"
starts = re.findall(r"^\w+", multi, re.MULTILINE)
print("Line starts:", starts)

# re.DOTALL — make . match newlines too
html = "\nsome content\n"
tag = re.search(r"(.*?)", html, re.DOTALL)
print("DOTALL match:", tag.group(1).strip() if tag else "no match")

# re.VERBOSE — add comments and whitespace to complex patterns
DATE_PATTERN = re.compile(r"""
    (\d{4})   # year
    -
    (\d{2})   # month
    -
    (\d{2})   # day
""", re.VERBOSE)

m = DATE_PATTERN.search("Date: 2024-07-19")
if m:
    print("Verbose match:", m.groups())

# Combine flags with |
hits = re.findall(r"^python", text, re.IGNORECASE | re.MULTILINE)
print("Combined flags:", hits)

Case insensitive: ['Python', 'PYTHON', 'python'] Line starts: ['first', 'second', 'third'] DOTALL match: some content Verbose match: ('2024', '07', '19') Combined flags: ['Python']

re.VERBOSE (or re.X) lets you write complex patterns across multiple lines with comments — whitespace inside the pattern is ignored.
Combine any flags with |: re.IGNORECASE | re.MULTILINE.

Real-World Example — Log File Parser

This parses a server log file to extract structured data from each line — the most common real-world regex task in backend development and DevOps.

import re
from collections import Counter

# Sample server log lines
log_lines = [
    "2024-03-15 08:01:22 INFO  Server started on port 8080",
    "2024-03-15 08:05:31 INFO  User alice logged in from 192.168.1.10",
    "2024-03-15 08:12:44 ERROR Database connection timeout after 30s",
    "2024-03-15 08:15:02 WARN  Disk usage at 85% on /dev/sda1",
    "2024-03-15 08:22:18 ERROR Failed to send email: SMTP refused",
    "2024-03-15 08:30:05 INFO  User bob logged in from 10.0.0.5",
]

# Pattern: timestamp, level, message
LOG_PATTERN = re.compile(
    r"(?P\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\s+"
    r"(?P\w+)\s+"
    r"(?P.+)"
)

entries  = []
for line in log_lines:
    m = LOG_PATTERN.match(line)
    if m:
        entries.append(m.groupdict())

# Count by level
level_counts = Counter(e["level"] for e in entries)
print("Log level summary:")
for level, count in sorted(level_counts.items()):
    print(f"  {level:<8}: {count}")

# Extract all IP addresses
ip_pattern = re.compile(r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b")
all_ips = []
for e in entries:
    all_ips.extend(ip_pattern.findall(e["message"]))
print("IPs found:", all_ips)

# Show only error and warning messages
print("Issues:")
for e in entries:
    if e["level"] in ("ERROR", "WARN"):
        print(f"  [{e['level']}] {e['timestamp']} — {e['message']}")

Log level summary: ERROR : 2 INFO : 3 WARN : 1 IPs found: ['192.168.1.10', '10.0.0.5'] Issues: [ERROR] 2024-03-15 08:12:44 — Database connection timeout after 30s [WARN] 2024-03-15 08:15:02 — Disk usage at 85% on /dev/sda1 [ERROR] 2024-03-15 08:22:18 — Failed to send email: SMTP refused

groupdict() returns all named groups as a dictionary — much cleaner than accessing by index when there are many groups.
The IP address pattern \b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b uses \b (word boundary) to avoid matching partial numbers.
This parse-then-analyse pattern — regex for extraction, Python for logic — is the standard approach in log monitoring tools.

Quick Reference Table

Function	Returns	Best Used For
`re.search()`	First match object or None	Checking if pattern exists anywhere
`re.match()`	Match at start or None	Validating format from the beginning
`re.fullmatch()`	Match if entire string matches	Strict format validation
`re.findall()`	List of all matches	Extracting all occurrences
`re.finditer()`	Iterator of match objects	Matches with positions
`re.sub()`	New string with replacements	Cleaning, masking, reformatting
`re.split()`	List of substrings	Splitting on complex delimiters
`re.compile()`	Compiled pattern object	Reusing patterns efficiently

Practice

Which function scans the entire string and returns only the first match?

What regex pattern matches one or more digits?

What type does re.findall() always return?

What flag makes a regex pattern case-insensitive?

What string prefix prevents backslash issues in regex patterns?

What syntax creates a non-capturing group?

Quick Quiz

What is the difference between re.match() and re.search()?

match() checks only the start, search() scans the whole string
search() checks only the start, match() scans the whole string
they are identical

What does the pattern \d{4}-\d{2}-\d{2} match?

A date in YYYY-MM-DD format
Any sequence of digits
A time in HH:MM:SS format

What does re.sub(r"\s+", " ", text) do?

Replaces all sequences of whitespace with a single space
Removes all spaces from the string
Splits the string on whitespace

In a regex, what does ? mean after a character?

The preceding character is optional (0 or 1 times)
Match any single character
Match one or more of the preceding

Why is re.compile() recommended when using the same pattern repeatedly?

It compiles the pattern once for faster repeated use
It automatically fixes errors in the pattern
It makes the pattern case-insensitive by default

Which match object method returns all named groups as a dictionary?

groups()
groupdict()
namedgroups()

NEXT UP

Iterators in Python

Discover how Python's for loops actually work under the hood, and learn to build your own iterable objects using the iterator protocol.

← Previous Course Index Next →