Python Course
Working with APIs
An API — Application Programming Interface — is how software systems talk to each other over the internet. When your Python script fetches weather data, submits a payment, posts to Slack, or reads from a spreadsheet, it is talking to an API. REST APIs are the dominant standard, and Python's requests library makes consuming them straightforward and readable.
This lesson covers the full picture: HTTP methods, request and response anatomy, authentication, error handling, pagination, and building a clean API client class you can reuse across projects.
HTTP Methods and What They Mean
REST APIs communicate using standard HTTP methods. Each method has a specific meaning that tells the server what operation to perform.
- GET — retrieve data. Safe and idempotent — calling it multiple times returns the same result without side effects.
- POST — create a new resource. Sending the same request twice creates two resources.
- PUT — replace an existing resource entirely.
- PATCH — partially update an existing resource.
- DELETE — remove a resource.
Making GET Requests
requests.get() sends a GET request and returns a Response object. The response contains the status code, headers, and body — usually JSON.
Real-world use: fetching a user's profile from a GitHub API, retrieving product listings from an e-commerce API, or pulling exchange rates from a financial data provider.
# Basic GET request — fetching data from a public API
import requests
# Fetch a post from JSONPlaceholder (a free fake REST API for testing)
url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)
# Status code — 200 means OK
print("Status:", response.status_code)
print("Content-Type:", response.headers["Content-Type"])
# Parse the JSON body
data = response.json()
print("Title:", data["title"])
print("User ID:", data["userId"])
# Query parameters — appended to the URL as ?key=value
response = requests.get(
"https://jsonplaceholder.typicode.com/posts",
params={"userId": 1, "_limit": 3} # fetch first 3 posts by user 1
)
posts = response.json()
print(f"\nFetched {len(posts)} posts:")
for post in posts:
print(f" [{post['id']}] {post['title'][:40]}...")Content-Type: application/json; charset=utf-8
Title: sunt aut facere repellat provident occaecati
User ID: 1
Fetched 3 posts:
[1] sunt aut facere repellat provident occ...
[2] qui est esse...
[3] ea molestias quasi exercitationem repel...
response.status_code— 200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 404 Not Found, 500 Server Errorresponse.json()— parses the response body as JSON and returns a Python dict or listparams=—requestsautomatically encodes the dict as a query string (?userId=1&_limit=3)response.text— raw response as a string;response.content— raw bytes (for files, images)
Making POST, PUT, PATCH, and DELETE Requests
Write operations send data to the server. Use json= to send a JSON body — requests sets the Content-Type: application/json header automatically.
# POST, PUT, PATCH, DELETE — write operations
import requests
BASE = "https://jsonplaceholder.typicode.com"
# POST — create a new resource
new_post = {"title": "My New Post", "body": "Hello world", "userId": 1}
r = requests.post(f"{BASE}/posts", json=new_post)
print("POST status:", r.status_code) # 201 Created
print("Created:", r.json())
# PUT — replace an existing resource entirely
updated = {"title": "Updated Title", "body": "New body", "userId": 1}
r = requests.put(f"{BASE}/posts/1", json=updated)
print("\nPUT status:", r.status_code) # 200 OK
# PATCH — partially update a resource
r = requests.patch(f"{BASE}/posts/1", json={"title": "Patched Title"})
print("PATCH title:", r.json()["title"])
# DELETE — remove a resource
r = requests.delete(f"{BASE}/posts/1")
print("DELETE status:", r.status_code) # 200 OKCreated: {'title': 'My New Post', 'body': 'Hello world', 'userId': 1, 'id': 101}
PUT status: 200
PATCH title: Patched Title
DELETE status: 200
- Use
json=to send JSON — do not manually serialize and set headers - Use
data=for form-encoded data (application/x-www-form-urlencoded) - Use
files=for multipart file uploads - 201 Created is the correct status for a successful POST; 204 No Content for a successful DELETE with no body
Error Handling
Network requests can fail in many ways — the server is down, the URL is wrong, the token expired, the rate limit was hit. Always handle errors explicitly.
Real-world use: a production data pipeline retries failed API calls with exponential backoff, logs 4xx errors for investigation, and raises alerts on 5xx server errors.
# Robust error handling for API requests
import requests
from requests.exceptions import RequestException, Timeout, ConnectionError
def safe_get(url, params=None, timeout=5):
"""Make a GET request with comprehensive error handling."""
try:
response = requests.get(url, params=params, timeout=timeout)
response.raise_for_status() # raises HTTPError for 4xx and 5xx
return response.json()
except Timeout:
print(f"Request timed out after {timeout}s: {url}")
except ConnectionError:
print(f"Could not connect to: {url}")
except requests.exceptions.HTTPError as e:
code = e.response.status_code
if code == 401:
print("Unauthorised — check your API key")
elif code == 404:
print(f"Resource not found: {url}")
elif code == 429:
print("Rate limit hit — slow down requests")
else:
print(f"HTTP error {code}: {e}")
except RequestException as e:
print(f"Unexpected request error: {e}")
return None
# Test with a valid URL
data = safe_get("https://jsonplaceholder.typicode.com/users/1")
if data:
print(f"User: {data['name']} — {data['email']}")
# Test with a 404
safe_get("https://jsonplaceholder.typicode.com/posts/99999")Resource not found: https://jsonplaceholder.typicode.com/posts/99999
response.raise_for_status()— raisesHTTPErrorfor any 4xx or 5xx status code- Always set a
timeout— without it, a slow server can hang your script indefinitely - Catch specific exceptions before the general
RequestException— more specific handling first - Return
None(or a default) on failure so callers can check rather than crash
Authentication
Most real APIs require authentication. The three most common patterns are API keys, Bearer tokens, and Basic Auth.
# Authentication patterns
import requests
# 1. API Key in query string
requests.get(
"https://api.example.com/data",
params={"api_key": "your_key_here"}
)
# 2. API Key in header (most common)
headers = {"X-API-Key": "your_key_here"}
requests.get("https://api.example.com/data", headers=headers)
# 3. Bearer token (OAuth2 / JWT)
token = "eyJhbGciOiJIUzI1NiIs..."
headers = {"Authorization": f"Bearer {token}"}
requests.get("https://api.example.com/protected", headers=headers)
# 4. Basic Auth — username and password
requests.get(
"https://api.example.com/resource",
auth=("username", "password")
)
# Best practice — load secrets from environment variables, never hardcode
import os
api_key = os.environ.get("API_KEY")
headers = {"X-API-Key": api_key}
print("Using API key from environment:", bool(api_key))- Never hardcode API keys or tokens in source code — use environment variables or a secrets manager
- Bearer tokens expire — handle 401 responses by refreshing the token and retrying
- Store credentials in a
.envfile and load them with thepython-dotenvpackage in development
Sessions — Reusing Connections and Headers
A requests.Session persists certain settings — headers, auth, cookies — across multiple requests. It also reuses the underlying TCP connection, making repeated calls to the same server faster.
# requests.Session — reuse connection and headers
import requests
BASE = "https://jsonplaceholder.typicode.com"
with requests.Session() as session:
# Set headers once — applied to every request made through this session
session.headers.update({
"Authorization": "Bearer fake-token-123",
"Accept": "application/json"
})
# All requests reuse the same connection and headers
user = session.get(f"{BASE}/users/1").json()
posts = session.get(f"{BASE}/posts", params={"userId": 1}).json()
todos = session.get(f"{BASE}/todos", params={"userId": 1}).json()
print(f"User: {user['name']}")
print(f"Posts: {len(posts)}")
print(f"Todos: {len(todos)}")Posts: 10
Todos: 20
- Use a session whenever you make more than one request to the same server — faster and cleaner
- Sessions automatically handle cookies — useful for APIs that use cookie-based auth
- Always use
with requests.Session() as session:to ensure the session is closed properly
Handling Pagination
Most APIs do not return all results at once — they return a page at a time. You must iterate through pages until there are no more results.
# Pagination — fetch all pages of results
import requests
def fetch_all_posts():
"""Fetch all posts using page-based pagination."""
all_posts = []
page = 1
while True:
response = requests.get(
"https://jsonplaceholder.typicode.com/posts",
params={"_page": page, "_limit": 10}
)
data = response.json()
if not data: # empty list means no more pages
break
all_posts.extend(data)
print(f"Page {page}: fetched {len(data)} posts (total so far: {len(all_posts)})")
page += 1
return all_posts
posts = fetch_all_posts()
print(f"\nTotal posts fetched: {len(posts)}")Page 2: fetched 10 posts (total so far: 20)
Page 3: fetched 10 posts (total so far: 30)
...
Page 10: fetched 10 posts (total so far: 100)
Page 11: fetched 0 posts (total so far: 100)
Total posts fetched: 100
- Common pagination styles: page number (
?page=2), offset (?offset=20&limit=10), cursor-based (?cursor=abc123) - Some APIs include a
nextURL in the response body or headers — use that instead of constructing your own - Always add a safety limit to the while loop in production to prevent infinite loops if the API behaves unexpectedly
Building a Reusable API Client
For any project that talks to an API repeatedly, wrap the logic in a class. This keeps authentication, base URL, error handling, and retry logic in one place.
# Reusable API client class
import requests
from requests.exceptions import RequestException
class APIClient:
def __init__(self, base_url, api_key=None):
self.base_url = base_url.rstrip("/")
self.session = requests.Session()
if api_key:
self.session.headers.update({"X-API-Key": api_key})
self.session.headers.update({"Accept": "application/json"})
def get(self, endpoint, **kwargs):
return self._request("GET", endpoint, **kwargs)
def post(self, endpoint, **kwargs):
return self._request("POST", endpoint, **kwargs)
def _request(self, method, endpoint, **kwargs):
url = f"{self.base_url}/{endpoint.lstrip('/')}"
try:
r = self.session.request(method, url, timeout=10, **kwargs)
r.raise_for_status()
return r.json()
except RequestException as e:
print(f"[{method}] {url} failed: {e}")
return None
def close(self):
self.session.close()
# Usage
client = APIClient("https://jsonplaceholder.typicode.com")
user = client.get("/users/1")
posts = client.get("/posts", params={"userId": 1, "_limit": 3})
if user:
print(f"User: {user['name']}")
if posts:
print(f"Posts returned: {len(posts)}")
client.close()Posts returned: 3
Summary Table
| Concept | Method / Tool | Purpose |
|---|---|---|
| GET request | requests.get(url, params={}) |
Retrieve data |
| POST request | requests.post(url, json={}) |
Create a resource |
| Error handling | response.raise_for_status() |
Raise on 4xx / 5xx automatically |
| Auth — Bearer | headers={"Authorization": "Bearer token"} |
OAuth2 / JWT authentication |
| Session | requests.Session() |
Reuse connection and headers |
| Pagination | Loop with params={"_page": page} |
Fetch all pages of results |
Practice Questions
Practice 1. Which HTTP method is used to create a new resource on a server?
Practice 2. What does response.raise_for_status() do?
Practice 3. Why should you always set a timeout in requests.get()?
Practice 4. What is the advantage of using requests.Session() for multiple requests to the same server?
Practice 5. Where should API keys and tokens be stored instead of being hardcoded in source code?
Quiz
Quiz 1. What is the difference between PUT and PATCH?
Quiz 2. Which argument in requests.get() automatically URL-encodes a dictionary as query parameters?
Quiz 3. Which HTTP status code indicates a resource was successfully created?
Quiz 4. In the Bearer token authentication pattern, where is the token placed in the request?
Quiz 5. What condition should a pagination loop check to know there are no more pages?
Next up — Databases: connecting to SQLite and PostgreSQL, running queries, and using SQLAlchemy ORM.