Web APIs
Final Project
Build the complete APIForge Platform API v1 from scratch — all 19 endpoints across four phases, using every pattern covered in this course.
This is the lesson where everything comes together. Every concept from the previous 36 lessons — HTTP methods, status codes, REST design, authentication, rate limiting, pagination, caching, webhooks, error handling, performance — gets applied in one real, working API. Not a toy. Not a tutorial with pre-filled gaps. A complete system built from a blank folder.
The plan is already written. Lesson 36 gave you the scope, the four phases, the API contract covering 19 endpoints, the database schema, 40 acceptance criteria, and the folder structure. This lesson is the build. Each phase produces something testable on its own. You do not move to the next phase until the current one passes its definition of done.
The APIForge Engineering team builds this over ten days. You can follow at your own pace — the code is complete and runnable at the end of each phase.
Phase 1 — Foundation
Phase 1 builds the skeleton every other phase depends on: the Express server, database connection, environment validation, request logger, and global error handler. When Phase 1 is done, GET /health returns 200 and the server refuses to boot if any required environment variable is missing.
The global error handler is the most important piece in this phase. Every unhandled error in any route lands here. It logs the full error internally, strips the stack trace from the response, and always sends back the structured error object format agreed in the API contract. No phase can safely start without this in place.
The entry point wires all middleware and routes together, validates environment variables before any connection is attempted, and registers the global error handler last — Express calls error handlers only when four parameters are present.
// WHAT: APIForge server.js — full Phase 1 foundation
// File: src/server.js
import express from 'express';
import compression from 'compression';
import morgan from 'morgan';
import { pool } from './db/pool.js';
import { generateRequestId } from './utils/generateId.js';
// ── Boot-time environment validation ──────────────────────────────────────
const REQUIRED_ENV = ['DATABASE_URL', 'JWT_SECRET', 'REDIS_URL'];
REQUIRED_ENV.forEach(key => {
if (!process.env[key]) {
console.error(`FATAL: Missing required environment variable: ${key}`);
process.exit(1);
}
});
const app = express();
// ── Core middleware ────────────────────────────────────────────────────────
app.use(compression({ level: 6, threshold: 1024 }));
app.use(express.json());
// Attach a unique request ID to every request
app.use((req, res, next) => {
req.requestId = generateRequestId();
res.setHeader('X-Request-ID', req.requestId);
next();
});
// Structured request logging
app.use(morgan(':method :url :status :response-time ms - :res[content-length]'));
// ── Health check (no auth required) ───────────────────────────────────────
app.get('/health', async (req, res) => {
try {
await pool.query('SELECT 1'); // confirm database is reachable
res.json({
status: 'ok',
version: process.env.npm_package_version || '1.0.0',
timestamp: new Date().toISOString(),
});
} catch {
res.status(503).json({ status: 'degraded', detail: 'Database unreachable' });
}
});
// ── Routes (registered in later phases) ───────────────────────────────────
// import authRoutes from './routes/auth.routes.js';
// import projectRoutes from './routes/projects.routes.js';
// app.use('/auth', authRoutes);
// app.use('/api/v1', projectRoutes);
// ── 404 handler ───────────────────────────────────────────────────────────
app.use((req, res) => {
res.status(404).json({
error: {
type: 'not_found',
message: `No route found for ${req.method} ${req.path}`,
code: 'route_not_found',
param: null,
request_id: req.requestId,
}
});
});
// ── Global error handler (4 params — Express requires all four) ───────────
app.use((err, req, res, next) => {
console.error(`[${req.requestId}] Unhandled error:`, err);
const statusCode = err.statusCode || 500;
res.status(statusCode).json({
error: {
type: statusCode >= 500 ? 'api_error' : 'invalid_request_error',
message: statusCode >= 500 ? 'An unexpected error occurred.' : err.message,
code: err.code || 'internal_error',
param: err.param || null,
request_id: req.requestId,
}
});
});
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`APIForge API running on port ${PORT}`));The error handler uses the presence of four parameters to identify itself to Express. If you write it with only three, Express treats it as a regular middleware and never calls it for errors. The next parameter must be present even if unused — this is a quirk of Express that trips up many developers.
The 500 path deliberately returns a generic message — "An unexpected error occurred" — instead of the real error message. Stack traces and internal error details in production responses are a security risk: they reveal file paths, library versions, and sometimes database schema details to anyone who can trigger an error. The full error is logged internally with the request ID so your team can look it up.
Try this: Temporarily throw an error inside the health check handler and confirm the response body contains no stack trace. Then check your server logs and confirm the full error is logged there with the matching request ID.Phase 2 — Authentication
Phase 2 builds register, login, token refresh, and the auth middleware that every Phase 3 route depends on. This phase introduces one addition beyond Lesson 32's auth project: a refresh token. The access token expires in 2 hours — a short window that limits damage from a stolen token. The refresh token expires in 7 days and can be used once to get a new access token without requiring the user to log in again.
The two tokens are separate JWTs signed with the same secret but carrying different payloads. The access token carries the user ID and role. The refresh token carries only the user ID and a type: "refresh" claim — the middleware checks this field and rejects refresh tokens presented as access tokens.
The auth controller handles register, login, and refresh. The middleware is a standalone function imported by every protected route. Both live in separate files — the controller in src/controllers, the middleware in src/middleware.
// WHAT: APIForge Phase 2 — auth controller + middleware
// Files: src/controllers/auth.controller.js + src/middleware/authenticate.js
import bcrypt from 'bcryptjs';
import jwt from 'jsonwebtoken';
import { pool } from '../db/pool.js';
import { generateId, buildError } from '../utils/index.js';
// ── Register ───────────────────────────────────────────────────────────────
export async function register(req, res, next) {
try {
const { email, password } = req.body;
if (!email || !password)
return next(buildError(400, 'email and password are required', 'parameter_missing'));
const existing = await pool.query('SELECT id FROM users WHERE email = $1', [email]);
if (existing.rows.length)
return next(buildError(409, 'An account with that email already exists', 'email_taken'));
const passwordHash = await bcrypt.hash(password, 10);
const id = generateId('usr');
const now = Math.floor(Date.now() / 1000);
await pool.query(
'INSERT INTO users (id, email, password_hash, created_at, updated_at) VALUES ($1,$2,$3,$4,$4)',
[id, email, passwordHash, now]
);
res.status(201).json({ id, email, createdAt: now });
} catch (err) { next(err); }
}
// ── Login ──────────────────────────────────────────────────────────────────
export async function login(req, res, next) {
try {
const { email, password } = req.body;
const result = await pool.query('SELECT * FROM users WHERE email = $1', [email]);
const user = result.rows[0];
// Same error for wrong email or wrong password — no enumeration
if (!user || !(await bcrypt.compare(password, user.password_hash)))
return next(buildError(401, 'Invalid credentials', 'invalid_credentials'));
const payload = { userId: user.id, email: user.email, role: user.role };
const token = jwt.sign(payload, process.env.JWT_SECRET, { expiresIn: '2h' });
const refreshToken = jwt.sign(
{ userId: user.id, type: 'refresh' },
process.env.JWT_SECRET,
{ expiresIn: '7d' }
);
res.json({ token, refreshToken, expiresIn: '2h' });
} catch (err) { next(err); }
}
// ── Refresh ────────────────────────────────────────────────────────────────
export async function refresh(req, res, next) {
try {
const authHeader = req.headers['authorization'];
const token = authHeader && authHeader.split(' ')[1];
if (!token) return next(buildError(401, 'Refresh token required', 'token_missing'));
const decoded = jwt.verify(token, process.env.JWT_SECRET);
if (decoded.type !== 'refresh')
return next(buildError(401, 'Invalid token type', 'invalid_token_type'));
const result = await pool.query('SELECT * FROM users WHERE id = $1', [decoded.userId]);
if (!result.rows.length) return next(buildError(401, 'User not found', 'user_not_found'));
const user = result.rows[0];
const newToken = jwt.sign(
{ userId: user.id, email: user.email, role: user.role },
process.env.JWT_SECRET,
{ expiresIn: '2h' }
);
res.json({ token: newToken, expiresIn: '2h' });
} catch (err) {
if (err.name === 'TokenExpiredError')
return next(buildError(401, 'Refresh token expired — please log in again', 'token_expired'));
next(err);
}
}
// ── Auth middleware ────────────────────────────────────────────────────────
// File: src/middleware/authenticate.js
export function authenticate(req, res, next) {
const token = req.headers['authorization']?.split(' ')[1];
if (!token) return next(buildError(401, 'No token provided', 'token_missing'));
try {
const decoded = jwt.verify(token, process.env.JWT_SECRET);
if (decoded.type === 'refresh')
return next(buildError(401, 'Cannot use refresh token for API access', 'invalid_token_type'));
req.user = decoded;
next();
} catch (err) {
const msg = err.name === 'TokenExpiredError' ? 'Token has expired' : 'Invalid token';
const code = err.name === 'TokenExpiredError' ? 'token_expired' : 'invalid_token';
next(buildError(401, msg, code));
}
}Every error in the auth controller goes through next(buildError(...)) instead of res.status().json() directly. This routes all errors through the global error handler from Phase 1, ensuring every error response — regardless of which controller produced it — follows the same structured format. Consistency is not automatic. You build it into the architecture.
The type: "refresh" claim in the refresh token payload is a guard against token confusion attacks — a client accidentally presenting a long-lived refresh token to a protected endpoint instead of a short-lived access token. The middleware explicitly rejects any token that carries the refresh type, no matter how valid its signature is.
Phase 3 — Core Resources
Phase 3 is the bulk of the build: Projects CRUD, Members management, cursor pagination, search, and file attachment uploads. Every route in this phase sits behind the authenticate middleware from Phase 2. Every list endpoint uses cursor pagination. Every error goes through next(buildError(...)).
The projects service is the largest file in the project — it handles the N+1 prevention pattern from Lesson 30, the cursor pagination SQL from Lesson 34, and the search index sync from Lesson 35. Keeping this logic in the service layer means the controller stays thin and readable.
The projects service encapsulates all database interaction for the projects resource. The controller calls service methods and maps the results to HTTP responses — it never writes SQL directly.
// WHAT: APIForge Phase 3 — projects service with cursor pagination
// File: src/services/projects.service.js
// All database interaction for projects lives here — controllers stay thin
import { pool } from '../db/pool.js';
import { index } from '../search/client.js'; // Meilisearch index
import { generateId } from '../utils/generateId.js';
// ── List projects (cursor pagination) ─────────────────────────────────────
export async function listProjects({ userId, limit = 20, startingAfter, status }) {
const params = [userId, limit + 1]; // fetch one extra to determine has_more
let query = `
SELECT p.*, u.email AS owner_email
FROM projects p
JOIN users u ON u.id = p.owner_id
WHERE p.owner_id = $1
`;
if (status) {
query += ` AND p.status = $${params.push(status)}`;
}
if (startingAfter) {
// Cursor: fetch rows created before the cursor project's created_at
const cursor = await pool.query(
'SELECT created_at FROM projects WHERE id = $1', [startingAfter]
);
if (cursor.rows.length) {
query += ` AND p.created_at < $${params.push(cursor.rows[0].created_at)}`;
}
}
query += ` ORDER BY p.created_at DESC LIMIT $2`;
const result = await pool.query(query, params);
const hasMore = result.rows.length > limit;
const data = hasMore ? result.rows.slice(0, limit) : result.rows;
return { object: 'list', data, has_more: hasMore, url: '/api/v1/projects' };
}
// ── Create project ─────────────────────────────────────────────────────────
export async function createProject({ name, description, status = 'active', userId }) {
const id = generateId('proj');
const now = Math.floor(Date.now() / 1000);
const result = await pool.query(`
INSERT INTO projects (id, name, description, status, owner_id, created_at, updated_at)
VALUES ($1, $2, $3, $4, $5, $6, $6)
RETURNING *
`, [id, name, description, status, userId, now]);
const project = result.rows[0];
// Sync to search index — non-blocking, failure does not break the response
index.addDocuments([{
id: project.id,
name: project.name,
description: project.description || '',
status: project.status,
}]).catch(err => console.error('Search index sync failed:', err));
return project;
}
// ── Get single project with members (JOIN — no N+1) ────────────────────────
export async function getProject(id, userId) {
const result = await pool.query(`
SELECT
p.*,
u.email AS owner_email,
COALESCE(
json_agg(
json_build_object('id', m.id, 'email', mu.email, 'role', m.role)
) FILTER (WHERE m.id IS NOT NULL), '[]'
) AS members
FROM projects p
JOIN users u ON u.id = p.owner_id
LEFT JOIN project_members m ON m.project_id = p.id
LEFT JOIN users mu ON mu.id = m.user_id
WHERE p.id = $1
GROUP BY p.id, u.email
`, [id]);
return result.rows[0] || null;
}
// ── Update project (PATCH — partial update) ────────────────────────────────
export async function updateProject(id, fields) {
const allowed = ['name', 'description', 'status'];
const updates = Object.entries(fields)
.filter(([k]) => allowed.includes(k) && fields[k] !== undefined);
if (!updates.length) return null;
const now = Math.floor(Date.now() / 1000);
const setClauses = updates.map(([k], i) => `${k} = $${i + 2}`).join(', ');
const values = [id, ...updates.map(([, v]) => v), now];
const result = await pool.query(
`UPDATE projects SET ${setClauses}, updated_at = $${values.length}
WHERE id = $1 RETURNING *`,
values
);
return result.rows[0] || null;
}
// ── Delete project ─────────────────────────────────────────────────────────
export async function deleteProject(id) {
const result = await pool.query(
'DELETE FROM projects WHERE id = $1 RETURNING id', [id]
);
// CASCADE in schema handles project_members and attachments automatically
if (result.rows.length) index.deleteDocument(id).catch(() => {});
return result.rows.length > 0;
}The PATCH update uses a dynamic SET clause builder — it only updates the fields present in the request body. A client sending only { "status": "archived" } runs one SQL statement that touches only the status column. This is correct PATCH semantics — partial update, not replacement. A PUT would require the full object and overwrite everything.
The search index sync is wrapped in a .catch() and is non-blocking. If Meilisearch is down, the project is still created and a 201 is returned. The search index being temporarily stale is a minor degradation. The create operation failing because of a search service outage would be a major incident.
Phase 4 — Production Features
Phase 4 adds the features that separate a working API from a production-ready one: idempotency on project creation, Redis rate limiting with headers on every response, outbound webhooks that fire on key events, and API versioning. Each one is a standalone middleware or service that slots into the existing architecture without touching any Phase 3 code.
The webhook service is the most interesting piece of Phase 4. When a project is created, the projects service emits an event. The webhook service listens for that event, looks up all registered webhook endpoints for the user, and fires an HMAC-signed HTTP POST to each one using a background job — the same async pattern from Lesson 35's notification use case.
Rate limiting and idempotency are registered as middleware on the projects router. The webhook service is called from the projects service after a successful create or delete — it fires asynchronously so the API response is never delayed by webhook delivery.
// WHAT: APIForge Phase 4 — rate limiter + idempotency + webhook service
// Files: src/middleware/rateLimiter.js + src/services/webhook.service.js
import Redis from 'ioredis';
import crypto from 'crypto';
import { Queue } from 'bullmq';
import { pool } from '../db/pool.js';
const redis = new Redis(process.env.REDIS_URL);
const webhookQueue = new Queue('webhooks', { connection: redis });
// ── Rate limiter middleware ────────────────────────────────────────────────
export async function rateLimiter(req, res, next) {
const userId = req.user.userId;
const window = Math.floor(Date.now() / 1000 / 60); // current minute
const key = `rl:${userId}:${window}`;
const limit = 100;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, 60);
const remaining = Math.max(0, limit - count);
const resetAt = (window + 1) * 60;
res.setHeader('X-RateLimit-Limit', limit);
res.setHeader('X-RateLimit-Remaining', remaining);
res.setHeader('X-RateLimit-Reset', resetAt);
if (count > limit) {
return next(Object.assign(
buildError(429, `Rate limit exceeded. Retry after ${resetAt - Math.floor(Date.now()/1000)}s`, 'rate_limit_exceeded'),
{ headers: { 'Retry-After': String(resetAt - Math.floor(Date.now()/1000)) } }
));
}
next();
}
// ── Idempotency middleware ─────────────────────────────────────────────────
export async function idempotency(req, res, next) {
if (!['POST', 'PATCH'].includes(req.method)) return next();
const key = req.headers['idempotency-key'];
if (!key) return next();
const cacheKey = `idem:${req.user.userId}:${key}`;
const cached = await redis.get(cacheKey);
if (cached) {
const { statusCode, body } = JSON.parse(cached);
res.setHeader('Idempotency-Replayed', 'true');
return res.status(statusCode).json(body);
}
const originalJson = res.json.bind(res);
res.json = async (body) => {
if (res.statusCode < 500) {
await redis.setEx(cacheKey, 86400, JSON.stringify({ statusCode: res.statusCode, body }));
}
return originalJson(body);
};
next();
}
// ── Webhook service ────────────────────────────────────────────────────────
export async function fireWebhooks(userId, eventType, data) {
const result = await pool.query(
'SELECT * FROM webhooks WHERE user_id = $1 AND $2 = ANY(events)',
[userId, eventType]
);
for (const webhook of result.rows) {
await webhookQueue.add('deliver', {
url: webhook.url,
secret: webhook.secret,
eventType,
payload: { type: eventType, data, created: Math.floor(Date.now() / 1000) },
}, { attempts: 3, backoff: { type: 'exponential', delay: 2000 } });
}
}
// ── Webhook delivery worker ────────────────────────────────────────────────
// File: src/workers/webhook.worker.js
import { Worker } from 'bullmq';
new Worker('webhooks', async (job) => {
const { url, secret, payload } = job.data;
const body = JSON.stringify(payload);
const timestamp = Math.floor(Date.now() / 1000);
const sig = crypto.createHmac('sha256', secret)
.update(`${timestamp}.${body}`)
.digest('hex');
const response = await fetch(url, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'APIForge-Signature': `t=${timestamp},v1=${sig}`,
},
body,
});
if (!response.ok) throw new Error(`Delivery failed: ${response.status}`);
}, { connection: redis });The rate limiter, idempotency check, and auth middleware are all applied on the router level — not registered globally on the app. This means a future public endpoint (like a documentation route or a status page) can exist on the same server without being rate-limited or requiring a token. Middleware scope matters as much as middleware logic.
The webhook worker signs each delivery with a fresh timestamp on every attempt. If the first delivery attempt fails and the job retries 3 minutes later, the new timestamp is used — not the original one. This matters because the recipient verifies the timestamp to prevent replay attacks. A retried webhook with a 3-minute-old timestamp would fail the recipient's tolerance check if you signed it once at queue time.
Try this: Register a webhook endpoint using a free service like webhook.site, create a project through the API, and watch the signed webhook arrive at your endpoint. Verify the HMAC signature manually using the secret you registered.Before and After: Four Phases Complete
Ten days of planned, phase-gated development produced a complete production-ready API. The before column is not hypothetical — it is what the same project looks like when the phases are skipped and the build is done as one unplanned sprint.
Final Result — What Was Built
| Component | What Was Built | Pattern From |
|---|---|---|
| Foundation | Express server, env validation, request ID, global error handler, health check, gzip compression | Lessons 5, 16, 30 |
| Authentication | Register, login, refresh token, auth middleware, token type guard | Lessons 17, 21, 32 |
| Projects | Full CRUD, cursor pagination, search index sync, N+1 prevention via JOIN | Lessons 13, 14, 30, 34 |
| Members | Invite by email, role assignment, duplicate prevention, cursor pagination | Lessons 13, 14 |
| Search | Meilisearch full-text, highlight tags, filter by status, offset pagination | Lessons 14, 35 |
| File Attachments | Pre-signed S3 URLs, pending/ready status, file type and size validation | Lesson 35 |
| Webhooks | Endpoint registration, HMAC signing, BullMQ delivery, exponential backoff retry | Lessons 34, 35 |
| Production Layer | Redis rate limiting, idempotency keys, API versioning header, structured errors throughout | Lessons 23, 30, 34 |
Quiz
1. The APIForge global error handler is registered last in server.js and handles all errors passed via next(err). What is the specific requirement that makes Express treat this middleware as an error handler rather than a regular middleware?
2. In the APIForge projects service, the search index sync after createProject is wrapped in a .catch() and not awaited. What failure scenario does this design decision handle correctly?
3. The APIForge webhook worker generates a new HMAC signature on every delivery attempt, including retries. Why is signing at delivery time instead of at queue time the correct approach?