
Python Course
Multithreading in Python
Most programs run one instruction at a time — sequentially. But many real-world tasks spend most of their time waiting: for a network response, a file load, a database query. Multithreading lets your program do other work during those waits by running multiple threads concurrently within the same process.
This lesson covers Python's threading module, thread synchronisation with locks, the Global Interpreter Lock, the ThreadPoolExecutor, and daemon threads.
What is a Thread?
A thread is a unit of execution within a process. All threads share the same memory — the same variables, objects, and files. This makes communication easy but introduces the risk of two threads modifying the same data simultaneously, causing unpredictable results.
- A Python program always starts with one thread — the main thread.
- Threads are lightweight — creating one is fast and uses little memory.
- Threads are best for I/O-bound tasks — tasks that spend time waiting for external resources.
- For CPU-bound tasks, use multiprocessing instead (next lesson).
Creating and Starting Threads
threading.Thread creates a thread. Pass it a target function and optional args, then call .start().
import threading
import time
def download(url, duration):
print(f"[{threading.current_thread().name}] Starting: {url}")
time.sleep(duration) # simulate network wait
print(f"[{threading.current_thread().name}] Done: {url}")
# Sequential — total time = sum of all durations
start = time.perf_counter()
download("page_a.html", 2)
download("page_b.html", 1)
download("page_c.html", 3)
print(f"Sequential: {time.perf_counter() - start:.2f}s
")
# Concurrent — total time ≈ longest single duration
start = time.perf_counter()
threads = [
threading.Thread(target=download, args=("page_a.html", 2), name="T1"),
threading.Thread(target=download, args=("page_b.html", 1), name="T2"),
threading.Thread(target=download, args=("page_c.html", 3), name="T3"),
]
for t in threads: t.start()
for t in threads: t.join() # wait for all to finish
print(f"Concurrent: {time.perf_counter() - start:.2f}s")t.start()begins the thread — the main thread continues immediately without waiting.t.join()blocks the calling thread until the target thread finishes — always join before using results.- Thread output order is non-deterministic — the OS schedules threads however it likes.
The Global Interpreter Lock (GIL)
Python has a Global Interpreter Lock — a mutex that allows only one thread to execute Python bytecode at a time, even on multi-core hardware.
- I/O-bound tasks: threads work well. When a thread waits for I/O, it releases the GIL, letting other threads run — giving real speedups.
- CPU-bound tasks: threads do not help and can be slower. Only one thread runs Python code at a time, so threads compete for the GIL rather than running truly in parallel. Use multiprocessing for CPU-bound tasks.
import threading, time
def cpu_task(n):
"""CPU-bound — holds the GIL while computing."""
return sum(i * i for i in range(n))
def io_task(seconds):
"""I/O-bound — releases the GIL while sleeping."""
time.sleep(seconds)
# I/O-bound: real speedup from threading
start = time.perf_counter()
threads = [threading.Thread(target=io_task, args=(1,)) for _ in range(5)]
for t in threads: t.start()
for t in threads: t.join()
print(f"5 I/O tasks with threads: {time.perf_counter() - start:.2f}s") # ~1.0s
# CPU-bound: no speedup — GIL prevents true parallelism
start = time.perf_counter()
threads = [threading.Thread(target=cpu_task, args=(2_000_000,)) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print(f"4 CPU tasks with threads: {time.perf_counter() - start:.2f}s") # no improvementThread Synchronisation — Lock
Because threads share memory, two threads can read and modify the same variable simultaneously — producing incorrect results. A Lock ensures only one thread accesses a critical section at a time.
import threading
counter = 0
def increment_unsafe():
global counter
for _ in range(100_000):
counter += 1 # read-modify-write — NOT atomic, unsafe under threads
def increment_safe(lock):
global counter
for _ in range(100_000):
with lock: # only one thread here at a time
counter += 1
# Unsafe — race condition → wrong result
counter = 0
threads = [threading.Thread(target=increment_unsafe) for _ in range(5)]
for t in threads: t.start()
for t in threads: t.join()
print("Unsafe:", counter) # typically much less than 500,000
# Safe — Lock prevents race condition
counter = 0
lock = threading.Lock()
threads = [threading.Thread(target=increment_safe, args=(lock,)) for _ in range(5)]
for t in threads: t.start()
for t in threads: t.join()
print("Safe: ", counter) # always exactly 500,000- Use
with lock:— the context manager form guarantees the lock is released even on exception. - A thread that tries to acquire an already-held lock blocks until it is released.
- Keep locked sections short — holding a lock too long slows other threads unnecessarily.
threading.RLock()is a re-entrant lock — a thread can acquire it multiple times without deadlocking itself.
ThreadPoolExecutor — The Modern Approach
concurrent.futures.ThreadPoolExecutor manages thread creation, reuse, and result collection for you — the clean high-level API for running functions in a thread pool.
from concurrent.futures import ThreadPoolExecutor, as_completed
import time
def fetch(url):
time.sleep(1) # simulate a 1-second API call
return f"Response from {url}"
urls = [
"https://api.example.com/users",
"https://api.example.com/orders",
"https://api.example.com/products",
"https://api.example.com/reports",
]
start = time.perf_counter()
with ThreadPoolExecutor(max_workers=4) as executor:
# submit() returns Future objects immediately
futures = {executor.submit(fetch, url): url for url in urls}
# as_completed yields futures as they finish (not submission order)
for future in as_completed(futures):
url = futures[future]
result = future.result()
print(f"{url.split('/')[-1]}: {result[:30]}")
print(f"All done in {time.perf_counter() - start:.2f}s")
# executor.map() alternative — simpler, results in submission order
print("
Using map:")
with ThreadPoolExecutor(max_workers=4) as executor:
for result in executor.map(fetch, urls):
print(result[:40])executor.submit(fn, *args)schedules a function and returns aFutureimmediately.as_completed(futures)yields futures in finish order — not submission order.executor.map(fn, iterable)— simpler alternative when you want results in submission order.- The
withblock automatically waits for all tasks to complete before exiting.
Daemon Threads
A daemon thread runs in the background and is killed automatically when the main thread exits. Use daemon threads for background tasks that should not prevent shutdown.
import threading, time
def background_monitor():
while True:
print("[Monitor] Checking system health...")
time.sleep(2)
# daemon=True — thread dies automatically when main thread exits
monitor = threading.Thread(target=background_monitor, daemon=True, name="HealthMonitor")
monitor.start()
print(f"Main thread working... (monitor: {monitor.name}, daemon={monitor.daemon})")
time.sleep(3)
print("Main thread done — daemon thread is killed automatically")- Set
daemon=Truebefore calling.start(). - Non-daemon threads keep the program alive until they finish — the process will not exit while any non-daemon thread is running.
- Use daemon threads for log writers, health monitors, and background sync tasks.
Quick Reference Table
| Tool | Purpose | Key Usage |
|---|---|---|
threading.Thread | Create and run a thread | Thread(target=fn, args=(...)) |
t.start() | Begin thread execution | Call after creating the thread |
t.join() | Wait for thread to finish | Call before using thread results |
threading.Lock | Prevent race conditions | with lock: |
threading.RLock | Re-entrant lock | When the same thread may acquire the lock twice |
ThreadPoolExecutor | High-level thread pool | executor.submit(fn, *args) |
| Daemon thread | Background task killed on exit | Thread(..., daemon=True) |
Practice
What method starts a thread's execution after it is created?
What does t.join() do?
What is the GIL and which type of task does it prevent from running in true parallel?
What is a race condition and how does a Lock prevent it?
What happens to a daemon thread when the main thread exits?
Which function from concurrent.futures yields futures in the order they finish?
Quick Quiz
For which type of task does Python multithreading provide the most benefit?
Why does the GIL make multithreading ineffective for CPU-bound tasks?
What is the preferred way to use a Lock?
What does as_completed(futures) yield?
What prevents a program from exiting while a non-daemon thread is still running?
Which lock type allows the same thread to acquire it multiple times without deadlocking itself?