Python Lesson 40 – APIs | Dataplexa

Working with APIs

An API — Application Programming Interface — is how software systems talk to each other over the internet. When your Python script fetches weather data, submits a payment, posts to Slack, or reads from a spreadsheet, it is talking to an API. REST APIs are the dominant standard, and Python's requests library makes consuming them straightforward and readable.

This lesson covers the full picture: HTTP methods, request and response anatomy, authentication, error handling, pagination, and building a clean API client class you can reuse across projects.

HTTP Methods and What They Mean

REST APIs communicate using standard HTTP methods. Each method has a specific meaning that tells the server what operation to perform.

  • GET — retrieve data. Safe and idempotent — calling it multiple times returns the same result without side effects.
  • POST — create a new resource. Sending the same request twice creates two resources.
  • PUT — replace an existing resource entirely.
  • PATCH — partially update an existing resource.
  • DELETE — remove a resource.

Making GET Requests

requests.get() sends a GET request and returns a Response object. The response contains the status code, headers, and body — usually JSON.

Real-world use: fetching a user's profile from a GitHub API, retrieving product listings from an e-commerce API, or pulling exchange rates from a financial data provider.

# Basic GET request — fetching data from a public API

import requests

# Fetch a post from JSONPlaceholder (a free fake REST API for testing)
url = "https://jsonplaceholder.typicode.com/posts/1"
response = requests.get(url)

# Status code — 200 means OK
print("Status:", response.status_code)
print("Content-Type:", response.headers["Content-Type"])

# Parse the JSON body
data = response.json()
print("Title:", data["title"])
print("User ID:", data["userId"])

# Query parameters — appended to the URL as ?key=value
response = requests.get(
    "https://jsonplaceholder.typicode.com/posts",
    params={"userId": 1, "_limit": 3}   # fetch first 3 posts by user 1
)
posts = response.json()
print(f"\nFetched {len(posts)} posts:")
for post in posts:
    print(f"  [{post['id']}] {post['title'][:40]}...")
Status: 200
Content-Type: application/json; charset=utf-8
Title: sunt aut facere repellat provident occaecati
User ID: 1

Fetched 3 posts:
[1] sunt aut facere repellat provident occ...
[2] qui est esse...
[3] ea molestias quasi exercitationem repel...
  • response.status_code — 200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 404 Not Found, 500 Server Error
  • response.json() — parses the response body as JSON and returns a Python dict or list
  • params=requests automatically encodes the dict as a query string (?userId=1&_limit=3)
  • response.text — raw response as a string; response.content — raw bytes (for files, images)

Making POST, PUT, PATCH, and DELETE Requests

Write operations send data to the server. Use json= to send a JSON body — requests sets the Content-Type: application/json header automatically.

# POST, PUT, PATCH, DELETE — write operations

import requests

BASE = "https://jsonplaceholder.typicode.com"

# POST — create a new resource
new_post = {"title": "My New Post", "body": "Hello world", "userId": 1}
r = requests.post(f"{BASE}/posts", json=new_post)
print("POST status:", r.status_code)    # 201 Created
print("Created:", r.json())

# PUT — replace an existing resource entirely
updated = {"title": "Updated Title", "body": "New body", "userId": 1}
r = requests.put(f"{BASE}/posts/1", json=updated)
print("\nPUT status:", r.status_code)   # 200 OK

# PATCH — partially update a resource
r = requests.patch(f"{BASE}/posts/1", json={"title": "Patched Title"})
print("PATCH title:", r.json()["title"])

# DELETE — remove a resource
r = requests.delete(f"{BASE}/posts/1")
print("DELETE status:", r.status_code)  # 200 OK
POST status: 201
Created: {'title': 'My New Post', 'body': 'Hello world', 'userId': 1, 'id': 101}

PUT status: 200
PATCH title: Patched Title
DELETE status: 200
  • Use json= to send JSON — do not manually serialize and set headers
  • Use data= for form-encoded data (application/x-www-form-urlencoded)
  • Use files= for multipart file uploads
  • 201 Created is the correct status for a successful POST; 204 No Content for a successful DELETE with no body

Error Handling

Network requests can fail in many ways — the server is down, the URL is wrong, the token expired, the rate limit was hit. Always handle errors explicitly.

Real-world use: a production data pipeline retries failed API calls with exponential backoff, logs 4xx errors for investigation, and raises alerts on 5xx server errors.

# Robust error handling for API requests

import requests
from requests.exceptions import RequestException, Timeout, ConnectionError

def safe_get(url, params=None, timeout=5):
    """Make a GET request with comprehensive error handling."""
    try:
        response = requests.get(url, params=params, timeout=timeout)
        response.raise_for_status()   # raises HTTPError for 4xx and 5xx
        return response.json()

    except Timeout:
        print(f"Request timed out after {timeout}s: {url}")
    except ConnectionError:
        print(f"Could not connect to: {url}")
    except requests.exceptions.HTTPError as e:
        code = e.response.status_code
        if code == 401:
            print("Unauthorised — check your API key")
        elif code == 404:
            print(f"Resource not found: {url}")
        elif code == 429:
            print("Rate limit hit — slow down requests")
        else:
            print(f"HTTP error {code}: {e}")
    except RequestException as e:
        print(f"Unexpected request error: {e}")
    return None

# Test with a valid URL
data = safe_get("https://jsonplaceholder.typicode.com/users/1")
if data:
    print(f"User: {data['name']} — {data['email']}")

# Test with a 404
safe_get("https://jsonplaceholder.typicode.com/posts/99999")
User: Leanne Graham — Sincere@april.biz
Resource not found: https://jsonplaceholder.typicode.com/posts/99999
  • response.raise_for_status() — raises HTTPError for any 4xx or 5xx status code
  • Always set a timeout — without it, a slow server can hang your script indefinitely
  • Catch specific exceptions before the general RequestException — more specific handling first
  • Return None (or a default) on failure so callers can check rather than crash

Authentication

Most real APIs require authentication. The three most common patterns are API keys, Bearer tokens, and Basic Auth.

# Authentication patterns

import requests

# 1. API Key in query string
requests.get(
    "https://api.example.com/data",
    params={"api_key": "your_key_here"}
)

# 2. API Key in header (most common)
headers = {"X-API-Key": "your_key_here"}
requests.get("https://api.example.com/data", headers=headers)

# 3. Bearer token (OAuth2 / JWT)
token = "eyJhbGciOiJIUzI1NiIs..."
headers = {"Authorization": f"Bearer {token}"}
requests.get("https://api.example.com/protected", headers=headers)

# 4. Basic Auth — username and password
requests.get(
    "https://api.example.com/resource",
    auth=("username", "password")
)

# Best practice — load secrets from environment variables, never hardcode
import os
api_key = os.environ.get("API_KEY")
headers = {"X-API-Key": api_key}
print("Using API key from environment:", bool(api_key))
Using API key from environment: False
  • Never hardcode API keys or tokens in source code — use environment variables or a secrets manager
  • Bearer tokens expire — handle 401 responses by refreshing the token and retrying
  • Store credentials in a .env file and load them with the python-dotenv package in development

Sessions — Reusing Connections and Headers

A requests.Session persists certain settings — headers, auth, cookies — across multiple requests. It also reuses the underlying TCP connection, making repeated calls to the same server faster.

# requests.Session — reuse connection and headers

import requests

BASE = "https://jsonplaceholder.typicode.com"

with requests.Session() as session:
    # Set headers once — applied to every request made through this session
    session.headers.update({
        "Authorization": "Bearer fake-token-123",
        "Accept": "application/json"
    })

    # All requests reuse the same connection and headers
    user    = session.get(f"{BASE}/users/1").json()
    posts   = session.get(f"{BASE}/posts", params={"userId": 1}).json()
    todos   = session.get(f"{BASE}/todos", params={"userId": 1}).json()

print(f"User: {user['name']}")
print(f"Posts: {len(posts)}")
print(f"Todos: {len(todos)}")
User: Leanne Graham
Posts: 10
Todos: 20
  • Use a session whenever you make more than one request to the same server — faster and cleaner
  • Sessions automatically handle cookies — useful for APIs that use cookie-based auth
  • Always use with requests.Session() as session: to ensure the session is closed properly

Handling Pagination

Most APIs do not return all results at once — they return a page at a time. You must iterate through pages until there are no more results.

# Pagination — fetch all pages of results

import requests

def fetch_all_posts():
    """Fetch all posts using page-based pagination."""
    all_posts = []
    page      = 1

    while True:
        response = requests.get(
            "https://jsonplaceholder.typicode.com/posts",
            params={"_page": page, "_limit": 10}
        )
        data = response.json()

        if not data:          # empty list means no more pages
            break

        all_posts.extend(data)
        print(f"Page {page}: fetched {len(data)} posts (total so far: {len(all_posts)})")
        page += 1

    return all_posts

posts = fetch_all_posts()
print(f"\nTotal posts fetched: {len(posts)}")
Page 1: fetched 10 posts (total so far: 10)
Page 2: fetched 10 posts (total so far: 20)
Page 3: fetched 10 posts (total so far: 30)
...
Page 10: fetched 10 posts (total so far: 100)
Page 11: fetched 0 posts (total so far: 100)

Total posts fetched: 100
  • Common pagination styles: page number (?page=2), offset (?offset=20&limit=10), cursor-based (?cursor=abc123)
  • Some APIs include a next URL in the response body or headers — use that instead of constructing your own
  • Always add a safety limit to the while loop in production to prevent infinite loops if the API behaves unexpectedly

Building a Reusable API Client

For any project that talks to an API repeatedly, wrap the logic in a class. This keeps authentication, base URL, error handling, and retry logic in one place.

# Reusable API client class

import requests
from requests.exceptions import RequestException

class APIClient:
    def __init__(self, base_url, api_key=None):
        self.base_url = base_url.rstrip("/")
        self.session  = requests.Session()
        if api_key:
            self.session.headers.update({"X-API-Key": api_key})
        self.session.headers.update({"Accept": "application/json"})

    def get(self, endpoint, **kwargs):
        return self._request("GET", endpoint, **kwargs)

    def post(self, endpoint, **kwargs):
        return self._request("POST", endpoint, **kwargs)

    def _request(self, method, endpoint, **kwargs):
        url = f"{self.base_url}/{endpoint.lstrip('/')}"
        try:
            r = self.session.request(method, url, timeout=10, **kwargs)
            r.raise_for_status()
            return r.json()
        except RequestException as e:
            print(f"[{method}] {url} failed: {e}")
            return None

    def close(self):
        self.session.close()

# Usage
client = APIClient("https://jsonplaceholder.typicode.com")
user   = client.get("/users/1")
posts  = client.get("/posts", params={"userId": 1, "_limit": 3})

if user:
    print(f"User: {user['name']}")
if posts:
    print(f"Posts returned: {len(posts)}")
client.close()
User: Leanne Graham
Posts returned: 3

Summary Table

Concept Method / Tool Purpose
GET request requests.get(url, params={}) Retrieve data
POST request requests.post(url, json={}) Create a resource
Error handling response.raise_for_status() Raise on 4xx / 5xx automatically
Auth — Bearer headers={"Authorization": "Bearer token"} OAuth2 / JWT authentication
Session requests.Session() Reuse connection and headers
Pagination Loop with params={"_page": page} Fetch all pages of results

Practice Questions

Practice 1. Which HTTP method is used to create a new resource on a server?



Practice 2. What does response.raise_for_status() do?



Practice 3. Why should you always set a timeout in requests.get()?



Practice 4. What is the advantage of using requests.Session() for multiple requests to the same server?



Practice 5. Where should API keys and tokens be stored instead of being hardcoded in source code?



Quiz

Quiz 1. What is the difference between PUT and PATCH?






Quiz 2. Which argument in requests.get() automatically URL-encodes a dictionary as query parameters?






Quiz 3. Which HTTP status code indicates a resource was successfully created?






Quiz 4. In the Bearer token authentication pattern, where is the token placed in the request?






Quiz 5. What condition should a pagination loop check to know there are no more pages?






Next up — Databases: connecting to SQLite and PostgreSQL, running queries, and using SQLAlchemy ORM.