Mango DBLesson 31 – Transactions | Dataplexa

Transactions

A MongoDB document write is atomic by default — a single document is either written completely or not at all. But what happens when one logical operation must write to multiple documents or multiple collections simultaneously? Without a transaction, a failure halfway through leaves the database in an inconsistent state — an order inserted but stock never decremented, or a payment recorded but the order never created. Multi-document ACID transactions solve this by grouping multiple operations into a single all-or-nothing unit. Every operation in the transaction either commits together or rolls back together, leaving the database in a consistent state regardless of what goes wrong. This lesson covers when transactions are necessary, how they work internally, how to implement them safely in PyMongo, and how to handle errors and retries correctly.

1. ACID Properties — What Transactions Guarantee

ACID is the set of four properties that define a reliable transaction. Understanding what each property means helps you reason about when a transaction is necessary and what guarantees your application can rely on.

# ACID properties — what each guarantees in MongoDB transactions

acid_properties = {
    "Atomicity": {
        "guarantee": "All operations in the transaction succeed, or none do",
        "failure_scenario": "Server crashes after inserting the order but before "
                            "decrementing stock",
        "without_transaction": "Order exists, stock is wrong — inconsistent state",
        "with_transaction":    "Both operations roll back — database unchanged",
    },
    "Consistency": {
        "guarantee": "The database moves from one valid state to another valid state",
        "failure_scenario": "Order total does not match the sum of its line items",
        "without_transaction": "Inconsistent totals possible if writes interleave",
        "with_transaction":    "Validation rules enforced across the entire transaction",
    },
    "Isolation": {
        "guarantee": "Concurrent transactions do not see each other's partial writes",
        "failure_scenario": "Two users simultaneously buy the last unit of a product",
        "without_transaction": "Both see stock = 1, both succeed — stock goes to -1",
        "with_transaction":    "One transaction wins, the other sees the updated stock "
                               "and fails cleanly",
    },
    "Durability": {
        "guarantee": "Committed transactions survive crashes and restarts",
        "failure_scenario": "Server crashes immediately after commit acknowledgement",
        "without_transaction": "Write may be lost if journal not flushed",
        "with_transaction":    "Write concern w='majority' + j=True ensures durability",
    },
}

print("ACID properties in MongoDB multi-document transactions:\n")
for prop, details in acid_properties.items():
    print(f"  {prop}")
    print(f"  Guarantees:          {details['guarantee']}")
    print(f"  Without transaction: {details['without_transaction']}")
    print(f"  With transaction:    {details['with_transaction']}")
    print()

ACID properties in MongoDB multi-document transactions:

Atomicity
Guarantees: All operations in the transaction succeed, or none do
Without transaction: Order exists, stock is wrong — inconsistent state
With transaction: Both operations roll back — database unchanged

Consistency
Guarantees: The database moves from one valid state to another
Without transaction: Inconsistent totals possible if writes interleave
With transaction: Validation rules enforced across the entire transaction

Isolation
Guarantees: Concurrent transactions do not see each other's partial writes
Without transaction: Both see stock = 1, both succeed — stock goes to -1
With transaction: One transaction wins, the other fails cleanly

Durability
Guarantees: Committed transactions survive crashes and restarts
Without transaction: Write may be lost if journal not flushed
With transaction: w='majority' + j=True ensures durability

Single-document writes in MongoDB are already atomic for free — you only need a multi-document transaction when two or more documents must change together as one unit
MongoDB multi-document transactions are available on replica sets and sharded clusters — they are not available on standalone single-node servers
Transactions carry overhead compared to single-document writes — use them only when the consistency requirement genuinely demands it, not as a default approach for all writes

2. When to Use Transactions

Not every multi-document write needs a transaction. Many patterns — like embedding related data or using the two-phase commit pattern — can achieve consistency without one. Understanding exactly when a transaction is the right tool prevents over-using them and incurring unnecessary performance costs.

# When to use transactions — decision guide

use_transaction = [
    {
        "scenario": "Place an order",
        "operations": [
            "Insert new order document",
            "Decrement product stock for each item",
            "Insert order items"
        ],
        "needs_transaction": True,
        "reason": "All three must succeed together — partial write leaves "
                  "inventory inconsistent"
    },
    {
        "scenario": "Transfer funds between two user wallets",
        "operations": [
            "Decrement balance on sender's account",
            "Increment balance on receiver's account"
        ],
        "needs_transaction": True,
        "reason": "Money cannot disappear or duplicate — debit and credit "
                  "must be atomic"
    },
    {
        "scenario": "Update a user's profile name",
        "operations": ["Update name field on the user document"],
        "needs_transaction": False,
        "reason": "Single document write — already atomic without a transaction"
    },
    {
        "scenario": "Insert a new product review",
        "operations": [
            "Insert review document",
            "Update product's avg_rating (computed pattern)"
        ],
        "needs_transaction": True,
        "reason": "Review and updated rating must be consistent — "
                  "stale rating is misleading"
    },
    {
        "scenario": "Log an event to an audit collection",
        "operations": ["Insert event document"],
        "needs_transaction": False,
        "reason": "Single document insert — atomic by default, "
                  "no cross-collection consistency needed"
    },
]

print("Transaction decision guide:\n")
for s in use_transaction:
    badge = "✓ USE TRANSACTION" if s["needs_transaction"] else "✗ NOT NEEDED"
    print(f"  [{badge}] {s['scenario']}")
    print(f"  Reason: {s['reason']}")
    print()

Transaction decision guide:

[✓ USE TRANSACTION] Place an order
Reason: All three must succeed together — partial write leaves inventory inconsistent

[✓ USE TRANSACTION] Transfer funds between two user wallets
Reason: Money cannot disappear or duplicate — debit and credit must be atomic

[✗ NOT NEEDED] Update a user's profile name
Reason: Single document write — already atomic without a transaction

[✓ USE TRANSACTION] Insert a new product review
Reason: Review and updated rating must be consistent — stale rating is misleading

[✗ NOT NEEDED] Log an event to an audit collection
Reason: Single document insert — atomic by default

If your operation touches only one document, a transaction adds overhead with no benefit — single-document atomicity is free in MongoDB
Financial operations — transfers, refunds, payments — are the classic transaction use case: money must never appear from or disappear into nowhere
Consider embedding related data as an alternative — an order with embedded items can be inserted atomically as one document, completely avoiding the need for a transaction

3. Basic Transaction — Session and commit_transaction

Every MongoDB transaction runs inside a session. A session is a logical channel between the client and the server that tracks the transaction state. You start a session, start a transaction on it, pass the session to every operation that should be part of the transaction, then either commit or abort.

# Basic transaction — place an order atomically

from pymongo import MongoClient
from datetime import datetime, timezone

client = MongoClient("mongodb://localhost:27017/")
db     = client["dataplexa"]

def place_order(user_id: str, items: list) -> dict:
    """
    Place an order atomically:
    1. Insert the order document
    2. Insert each order item
    3. Decrement stock for each product
    All three steps succeed together or all roll back.
    """

    # Start a client session — the transaction runs inside this session
    with client.start_session() as session:

        # Start the transaction on the session
        with session.start_transaction():

            # Step 1: calculate total
            total = sum(item["qty"] * item["price"] for item in items)

            # Step 2: insert the order — note session= passed to every operation
            order_id = f"o{db.orders.count_documents({}) + 1:03d}_txn"
            db.orders.insert_one(
                {
                    "_id":     order_id,
                    "user_id": user_id,
                    "status":  "processing",
                    "total":   round(total, 2),
                    "date":    datetime.now(timezone.utc).strftime("%Y-%m-%d"),
                    "items":   items
                },
                session=session   # ← critical: binds operation to transaction
            )

            # Step 3: decrement stock for each product
            for item in items:
                result = db.products.update_one(
                    {
                        "_id":   item["product_id"],
                        "stock": {"$gte": item["qty"]}  # ensure enough stock
                    },
                    {"$inc": {"stock": -item["qty"]}},
                    session=session   # ← critical
                )
                if result.modified_count == 0:
                    # Not enough stock — abort the entire transaction
                    session.abort_transaction()
                    return {"success": False,
                            "error": f"Insufficient stock for {item['product_id']}"}

            # All steps succeeded — commit atomically
            # (session.start_transaction() as context manager commits on exit)

    return {"success": True, "order_id": order_id, "total": round(total, 2)}


# Test — place a valid order
result = place_order(
    user_id="u003",
    items=[
        {"product_id": "p003", "qty": 2, "price": 4.99},
        {"product_id": "p006", "qty": 1, "price": 3.49},
    ]
)
print("Order placement result:")
print(f"  success:  {result['success']}")
if result["success"]:
    print(f"  order_id: {result['order_id']}")
    print(f"  total:    ${result['total']:.2f}")

# Verify stock was decremented
p3 = db.products.find_one({"_id": "p003"}, {"name": 1, "stock": 1})
p6 = db.products.find_one({"_id": "p006"}, {"name": 1, "stock": 1})
print(f"\nStock after transaction:")
print(f"  {p3['name']:25} stock: {p3['stock']}")
print(f"  {p6['name']:25} stock: {p6['stock']}")

Order placement result:
success: True
order_id: o008_txn
total: $13.47

Stock after transaction:
Notebook A5 stock: 3
Ballpoint Pens 10-pack stock: 119

Pass session=session to every read and write operation inside the transaction — any operation that does not receive the session runs outside the transaction and will not be rolled back if it aborts
Using session.start_transaction() as a context manager (with block) automatically commits on clean exit and aborts on any unhandled exception
The stock check {"stock": {"$gte": item["qty"]}} inside the update filter is a safe atomic test-and-decrement — if it matches, the document is found and updated; if not, modified_count is 0 and you know stock was insufficient

4. Error Handling and Retry Logic

Transactions can fail for two reasons: application logic errors (like insufficient stock) and transient errors from the database itself (like write conflicts when two transactions try to modify the same document simultaneously). Transient errors must be retried — they are expected under concurrent load and do not indicate a bug.

# Transaction error handling — retrying transient errors correctly

from pymongo import MongoClient
from pymongo.errors import (
    ConnectionFailure,
    OperationFailure,
    PyMongoError
)
import time

client = MongoClient("mongodb://localhost:27017/")
db     = client["dataplexa"]

def run_transaction_with_retry(txn_func, *args, max_retries=3, **kwargs):
    """
    Wrapper that retries a transaction function on transient errors.
    Transient errors: write conflicts, network blips, primary failover.
    Non-transient errors: validation failures, logic errors — do not retry.
    """
    for attempt in range(1, max_retries + 1):
        with client.start_session() as session:
            try:
                with session.start_transaction():
                    result = txn_func(session, *args, **kwargs)
                    # Context manager commits here on clean exit
                    return result

            except OperationFailure as e:
                # Error code 112 = WriteConflict — two txns modified same doc
                # Error code 251 = TransactionAborted — retry is safe
                transient_codes = {112, 251, 267}
                if e.code in transient_codes and attempt < max_retries:
                    wait = 0.1 * (2 ** attempt)  # exponential backoff
                    print(f"  Transient error (code {e.code}) — "
                          f"retrying in {wait:.1f}s (attempt {attempt}/{max_retries})")
                    time.sleep(wait)
                    continue
                raise   # non-transient or max retries exceeded — re-raise

            except (ConnectionFailure, PyMongoError) as e:
                if attempt < max_retries:
                    print(f"  Connection error — retrying (attempt {attempt})")
                    continue
                raise


def transfer_stock_txn(session, from_product: str,
                        to_product: str, qty: int) -> dict:
    """
    Move qty units from one product's stock to another atomically.
    (Demonstrates a two-document update in one transaction.)
    """
    # Decrement source
    src = db.products.find_one_and_update(
        {"_id": from_product, "stock": {"$gte": qty}},
        {"$inc": {"stock": -qty}},
        session=session,
        return_document=True
    )
    if not src:
        raise OperationFailure(
            "Insufficient stock on source product", code=999
        )

    # Increment destination
    db.products.update_one(
        {"_id": to_product},
        {"$inc": {"stock": qty}},
        session=session
    )

    return {
        "moved":    qty,
        "from":     from_product,
        "to":       to_product,
    }

# Run with retry wrapper
result = run_transaction_with_retry(
    transfer_stock_txn,
    from_product="p005",   # USB-C Hub (stock: 50)
    to_product="p003",     # Notebook A5 (stock: 3 after previous demo)
    qty=5
)
print("Stock transfer result:", result)

# Verify
hub      = db.products.find_one({"_id": "p005"}, {"name": 1, "stock": 1})
notebook = db.products.find_one({"_id": "p003"}, {"name": 1, "stock": 1})
print(f"\n  {hub['name']:25} stock: {hub['stock']}")
print(f"  {notebook['name']:25} stock: {notebook['stock']}")

Stock transfer result: {'moved': 5, 'from': 'p005', 'to': 'p003'}

USB-C Hub stock: 45
Notebook A5 stock: 8

Error code 112 (WriteConflict) is the most common transient error — it means two concurrent transactions tried to write the same document and one lost. Always retry it
Use exponential backoff between retries — 0.1 × 2^attempt seconds — to avoid a thundering herd of retries all competing again at the same instant
Never retry non-transient errors like insufficient stock or validation failures — these will fail the same way on every attempt and retrying wastes resources

5. Transaction Limits and Best Practices

MongoDB transactions have hard limits designed to prevent runaway operations from degrading the entire cluster. Understanding these limits helps you write transactions that stay well within bounds and perform reliably in production.

# Transaction limits and best practices

limits_and_practices = {
    "Time limit": {
        "limit":    "60 seconds maximum runtime (default)",
        "why":      "Long transactions hold locks, blocking other operations",
        "practice": "Keep transactions short — seconds, not minutes. "
                    "Never call external APIs or run slow loops inside a transaction"
    },
    "Document size": {
        "limit":    "All documents touched must fit within available memory",
        "why":      "Transactions use WiredTiger's in-memory snapshot",
        "practice": "Avoid transactions that touch thousands of large documents"
    },
    "Oplog size": {
        "limit":    "16 MB total operations log size per transaction",
        "why":      "All transaction operations are recorded in the oplog",
        "practice": "Keep the number of writes per transaction small — "
                    "ideally under 1000 documents"
    },
    "Read your own writes": {
        "limit":    "Within a transaction, reads always see the transaction's own writes",
        "why":      "Snapshot isolation — each transaction sees a consistent snapshot",
        "practice": "You can read a document you just inserted in the same transaction "
                    "without any special configuration"
    },
    "No DDL inside transactions": {
        "limit":    "Cannot create or drop collections/indexes inside a transaction",
        "why":      "DDL operations are metadata changes that cannot be rolled back",
        "practice": "Create all collections and indexes before starting any transaction"
    },
}

print("Transaction limits and best practices:\n")
for name, details in limits_and_practices.items():
    print(f"  {name}")
    print(f"  Limit:    {details['limit']}")
    print(f"  Practice: {details['practice']}")
    print()

# Demonstrate the context manager pattern — the safest way to write transactions
print("Recommended transaction pattern (context manager):")
code_pattern = '''
with client.start_session() as session:
    with session.start_transaction():
        # All operations here use session=session
        # Clean exit → automatic commit
        # Exception → automatic abort + re-raise
        db.orders.insert_one({...}, session=session)
        db.products.update_one({...}, {...}, session=session)
'''
print(code_pattern)

Transaction limits and best practices:

Time limit
Limit: 60 seconds maximum runtime (default)
Practice: Keep transactions short — seconds, not minutes. Never call external APIs inside a transaction

Document size
Limit: All documents touched must fit within available memory
Practice: Avoid transactions that touch thousands of large documents

Oplog size
Limit: 16 MB total operations log size per transaction
Practice: Keep the number of writes per transaction small — ideally under 1000 documents

Read your own writes
Limit: Within a transaction reads always see the transaction's own writes
Practice: You can read a doc you just inserted in the same transaction without special config

No DDL inside transactions
Limit: Cannot create or drop collections or indexes inside a transaction
Practice: Create all collections and indexes before starting any transaction

Recommended transaction pattern (context manager):

with client.start_session() as session:
with session.start_transaction():
db.orders.insert_one({...}, session=session)
db.products.update_one({...}, {...}, session=session)

The double context manager pattern — with start_session() as session + with session.start_transaction() — is the recommended approach: sessions and transactions are always cleaned up correctly even if an exception is raised
Never perform I/O — HTTP calls, file reads, user input — inside a transaction. Every millisecond the transaction is open holds a snapshot and potentially blocks concurrent writers
If a transaction consistently approaches the 60-second limit or 16 MB oplog limit, it is a signal to redesign the operation — batch writes into smaller transactions or reconsider the data model

Summary Table

Concept	What It Means	Key Rule
Atomicity	All or nothing — no partial writes	Every operation in the transaction commits or none do
Session	Logical channel tracking transaction state	Pass `session=session` to every operation
Commit	Makes all transaction writes permanent	Context manager commits automatically on clean exit
Abort / Rollback	Undoes all writes since transaction started	Context manager aborts automatically on exception
WriteConflict (code 112)	Two transactions modified the same document	Transient — always retry with exponential backoff
Time limit	60-second maximum transaction duration	Never call external APIs inside a transaction
Oplog limit	16 MB total writes per transaction	Keep writes per transaction under 1000 documents
When NOT to use	Single-document writes are already atomic	Only use transactions when 2+ documents must change together

Practice Questions

Practice 1. What is the minimum requirement for using multi-document transactions in MongoDB?

Practice 2. What happens if you forget to pass session=session to one of the operations inside a transaction?

Practice 3. What is a WriteConflict error and how should your application handle it?

Practice 4. Why should you never make HTTP requests or call external APIs inside a MongoDB transaction?

Practice 5. Describe what the double context manager pattern does — with start_session() and with session.start_transaction().

Quiz

Quiz 1. Which ACID property ensures that a failure halfway through a transaction leaves the database unchanged?

Atomicity — all commit or all roll back, no partial writes
Consistency — the database moves between valid states
Isolation — concurrent transactions don't see each other
Durability — committed data survives crashes

Quiz 2. What is the maximum default duration a MongoDB transaction can run before being automatically aborted?

60 seconds — automatically aborted if exceeded
5 seconds
10 minutes
There is no time limit

Quiz 3. Which of the following operations does NOT require a multi-document transaction?

Transferring money between two user accounts
Inserting an order and decrementing product stock
Updating a user's email address
Inserting a review and updating the product's avg_rating

Quiz 4. What error code indicates a WriteConflict — two concurrent transactions modifying the same document?

112 — WriteConflict, always retry with backoff
11000 — DuplicateKeyError
2 — BadValue error
500 — InternalError

Quiz 5. What is the maximum total oplog size allowed for a single MongoDB transaction?

16 MB — transaction aborted if exceeded
1 GB — oplog is stored on disk
4 MB — matching the BSON document size
There is no oplog size limit

Next up — Performance Optimization: Query profiling, the explain plan, slow query detection, and the strategies that keep MongoDB fast as your data grows.

← Previous Course Index Next →