Linux Administration Lesson 28 – | Dataplexa
Section III — Networking, Security & Storage

SSH and Secure Access

In this lesson

Key-based authentication SSH config and hardening Port forwarding SSH agent and jump hosts sshd hardening

SSH (Secure Shell) is the universal protocol for encrypted remote access to Linux systems. It provides a secure channel over an insecure network — encrypting all traffic, authenticating both client and server, and offering tunnelling capabilities that extend far beyond a simple terminal session. For most administrators, SSH is the primary interface to every server they manage, making its proper configuration and security a foundational skill.

How SSH Authentication Works

SSH supports multiple authentication methods. Key-based authentication is the standard for server administration — it replaces a password with a cryptographic key pair: a private key that never leaves the client machine, and a public key that is stored on the server. The server challenges the client to prove it holds the private key, without the private key ever crossing the network.

Client (~/.ssh/) id_ed25519 PRIVATE KEY — stays here id_ed25519.pub PUBLIC KEY — copy to server config per-host settings challenge signed response Key Exchange Server sends challenge Client signs with privkey Server verifies with pubkey Server (~/.ssh/) authorized_keys contains public key(s) host keys (/etc/ssh/) server identity sshd_config daemon settings Private key never crosses the network · Public key is safe to distribute widely

Fig 1 — SSH key-based authentication: the private key signs a challenge; the public key verifies it

# Generate an Ed25519 key pair (preferred — smaller and faster than RSA)
ssh-keygen -t ed25519 -C "alice@workstation"

# Generate RSA 4096 if Ed25519 is not supported on the target
ssh-keygen -t rsa -b 4096 -C "alice@workstation"

# Keys are stored in:
# ~/.ssh/id_ed25519       — private key (chmod 600, never share)
# ~/.ssh/id_ed25519.pub   — public key (safe to distribute)

# Copy the public key to a remote server
ssh-copy-id -i ~/.ssh/id_ed25519.pub alice@192.168.1.10

# Manual alternative (when ssh-copy-id is not available)
cat ~/.ssh/id_ed25519.pub | ssh alice@192.168.1.10 \
  "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

# Verify key login works BEFORE disabling password auth
ssh -i ~/.ssh/id_ed25519 alice@192.168.1.10

What just happened? ssh-keygen created two files: the private key (protected by a passphrase — strongly recommended) and the public key. ssh-copy-id appended the public key to ~/.ssh/authorized_keys on the server and set correct permissions automatically. The passphrase encrypts the private key at rest — even if someone steals the key file, they cannot use it without the passphrase.

The SSH Client Config File

The SSH client configuration file at ~/.ssh/config lets you define per-host settings — aliases, ports, key files, usernames, and jump host chains — so that a complex ssh invocation becomes a simple short command. It is one of the most productivity-improving files an administrator can maintain.

# ~/.ssh/config — client-side SSH configuration
# Permissions must be: chmod 600 ~/.ssh/config

# ── Global defaults for all hosts ────────────────────────────────
Host *
    ServerAliveInterval 60        # send keepalive every 60 seconds
    ServerAliveCountMax 3         # disconnect after 3 missed keepalives
    AddKeysToAgent yes            # automatically add keys to ssh-agent
    IdentityFile ~/.ssh/id_ed25519

# ── Production web server ─────────────────────────────────────────
Host web-prod
    HostName 203.0.113.10
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_deploy

# ── Bastion / jump host ───────────────────────────────────────────
Host bastion
    HostName bastion.example.com
    User alice
    IdentityFile ~/.ssh/id_ed25519

# ── Internal server reached via the bastion ───────────────────────
Host db-internal
    HostName 10.0.1.50
    User dbadmin
    ProxyJump bastion          # tunnel through bastion automatically

# ── GitHub ────────────────────────────────────────────────────────
Host github.com
    IdentityFile ~/.ssh/id_ed25519_github
    AddKeysToAgent yes
# With the config above, these long commands become short aliases:
# Instead of: ssh -i ~/.ssh/id_ed25519_deploy -p 2222 deploy@203.0.113.10
ssh web-prod

# Instead of: ssh -J alice@bastion.example.com dbadmin@10.0.1.50
ssh db-internal

# Test the config — show which settings would be used for a host
ssh -G web-prod

# Check config file permissions (must be 600 or SSH ignores it)
ls -la ~/.ssh/config

What just happened? ssh -G web-prod showed the merged configuration that would be applied — all per-host settings combined with matching global defaults. This is the correct way to debug an ~/.ssh/config entry without actually opening a connection. The file permissions confirm 600 — SSH will refuse to read the config file if it is world-readable.

SSH Agent and Key Management

A passphrase on a private key is essential for security but inconvenient when you connect to many servers. ssh-agent solves this by holding decrypted keys in memory for the duration of a session — you enter the passphrase once, and the agent handles all subsequent authentication automatically. Agent forwarding extends this across jump hosts.

Analogy: The SSH agent is like a hotel key card holder at the front desk. You hand your master key (private key) to the desk once in the morning (enter passphrase), and for the rest of the day the desk attendant opens any door you need without you having to produce the key again. When you check out (end the session), the desk returns the key.

# Start the SSH agent (usually started automatically on login in modern distros)
eval "$(ssh-agent -s)"

# Add a key to the agent — prompts for passphrase once
ssh-add ~/.ssh/id_ed25519

# Add a key with a time limit (auto-removed after 4 hours)
ssh-add -t 14400 ~/.ssh/id_ed25519

# List keys currently loaded in the agent
ssh-add -l

# Remove a specific key from the agent
ssh-add -d ~/.ssh/id_ed25519

# Remove all keys from the agent
ssh-add -D

# ── Agent Forwarding ──────────────────────────────────────────────
# Connect to a server AND forward your agent (so you can ssh further
# from that server without copying keys there)
ssh -A alice@bastion.example.com

# Or set ForwardAgent in ~/.ssh/config for specific hosts
# Host bastion
#     ForwardAgent yes

# IMPORTANT: Only enable ForwardAgent for trusted hosts — a compromised
# intermediate server could use your forwarded agent to authenticate elsewhere

What just happened? After ssh-add, the agent holds the decrypted key in memory. ssh-add -l confirmed the key is loaded by showing its fingerprint. Any subsequent ssh connection will be authenticated automatically without prompting for the passphrase again — until the agent is killed or the key is removed.

Port Forwarding and Tunnelling

SSH's tunnelling capabilities allow traffic for any TCP service to be carried securely through an SSH connection. This is used to access services on remote networks that are not directly reachable, to encrypt traffic for services that lack built-in encryption, and to create secure access paths through firewalls.

Local Forwarding -L

Forward a local port to a remote service. Access a remote database as if it were local.

ssh -L 5433:db.internal:5432 bastion # localhost:5433 → db.internal:5432

Remote Forwarding -R

Expose a local service on the remote server. Let a remote server reach a service only on your machine.

ssh -R 8080:localhost:3000 server # server:8080 → local:3000

Dynamic Forwarding -D

SOCKS5 proxy through SSH. Route all application traffic through the SSH server.

ssh -D 1080 alice@server # configure app to use SOCKS5 localhost:1080
# ── Local forwarding — access a remote database locally ──────────
# Forward local port 5433 to port 5432 on db.internal via bastion
ssh -L 5433:db.internal:5432 -N -f alice@bastion.example.com
# -N = no command, just tunnel; -f = run in background
# Now connect to the database as: psql -h localhost -p 5433

# ── Remote forwarding — expose local dev server to a remote host ──
# Make your local port 3000 accessible as port 8080 on the remote server
ssh -R 8080:localhost:3000 -N alice@server.example.com

# ── Dynamic forwarding — SOCKS5 proxy ────────────────────────────
ssh -D 1080 -N -f alice@server.example.com
# Configure browser/app to use SOCKS5 proxy at 127.0.0.1:1080

# ── Jump hosts / ProxyJump ────────────────────────────────────────
# Reach an internal server through a bastion in one command
ssh -J alice@bastion.example.com dbadmin@10.0.1.50

# Multi-hop jump chain
ssh -J user@hop1,user@hop2 user@final-destination

# ── Persistent tunnel with autossh ───────────────────────────────
sudo apt install autossh -y
autossh -M 0 -f -N -L 5433:db.internal:5432 alice@bastion.example.com

What just happened? The local port forward created an encrypted tunnel — psql connected to localhost:5433 as if the database were local, but all traffic was secretly routed through the SSH connection to bastion and then to db.internal:5432. The database server never needed to be exposed to the internet — it is only reachable from the bastion's internal network.

Hardening the SSH Daemon

The SSH daemon configuration in /etc/ssh/sshd_config exposes dozens of tunable security settings. A freshly installed OpenSSH server has reasonable defaults, but production hardening requires explicit configuration of authentication methods, access controls, and session limits.

sshd_config — Production Hardening Settings
Setting Purpose
PermitRootLogin no Prevent direct root login over SSH. All admin work uses a named account + sudo.
PasswordAuthentication no Disable password logins — SSH keys only. Eliminates brute-force attacks entirely.
PubkeyAuthentication yes Explicitly enable public key authentication (default yes, but make it explicit).
AllowUsers alice bob Whitelist specific users. All other accounts — including service accounts — cannot log in via SSH even if they have a valid key.
MaxAuthTries 3 Disconnect after 3 failed authentication attempts. Slows down automated credential stuffing.
LoginGraceTime 30 Disconnect unauthenticated connections after 30 seconds. Limits the window for slow brute-force attacks.
X11Forwarding no Disable X11 forwarding on servers — an unnecessary attack surface on headless systems.
ClientAliveInterval 300
ClientAliveCountMax 2
Send a keepalive every 5 minutes. Disconnect after 2 missed responses (~10 min idle). Cleans up abandoned sessions.
# Back up before editing
sudo cp /etc/ssh/sshd_config /etc/ssh/sshd_config.$(date +%Y%m%d)

# Apply the hardened configuration
sudo tee /etc/ssh/sshd_config.d/hardening.conf <<'EOF'
# Drop-in hardening config — overrides defaults without touching the main file
PermitRootLogin no
PasswordAuthentication no
PubkeyAuthentication yes
AllowUsers alice bob deploy
MaxAuthTries 3
LoginGraceTime 30
X11Forwarding no
ClientAliveInterval 300
ClientAliveCountMax 2
AllowAgentForwarding no
AllowTcpForwarding no
EOF

# ALWAYS validate before reloading — a syntax error locks you out
sudo sshd -t && echo "Config OK"

# Reload the daemon (keeps existing sessions alive)
sudo systemctl reload sshd

# Verify the settings took effect
sudo sshd -T | grep -E "permitrootlogin|passwordauthentication|allowusers"

What just happened? The hardening config was placed in /etc/ssh/sshd_config.d/ as a drop-in file rather than editing the main sshd_config directly — this survives package upgrades cleanly. sshd -T (test mode with full config dump) confirmed the settings are active. Note that AllowTcpForwarding no and AllowAgentForwarding no are included — on servers that don't need tunnelling, these reduce the attack surface.

SSH Security Monitoring and Auditing

Every SSH login attempt — successful or failed — is recorded in the authentication log. Regular review of these logs detects brute-force attacks, unauthorised access attempts, and successful logins from unexpected locations. The patterns in SSH logs are some of the most actionable security intelligence available on a Linux server.

Monitor for brute-force attempts

Count failed login attempts per source IP. Any IP with hundreds of failures in a short window is running an automated attack.

grep "Failed password" /var/log/auth.log | grep -oP '(\d{1,3}\.){3}\d{1,3}' | sort | uniq -c | sort -rn | head -10

Review successful logins

Any successful login from an unexpected IP, at an unexpected time, or for an unexpected user warrants investigation.

grep "Accepted" /var/log/auth.log | tail -20

Audit authorized_keys on all accounts

An attacker who gains access may add their own public key. Regular audits ensure only expected keys are present.

sudo find /home /root -name "authorized_keys" -exec echo "=== {} ===" \; -exec cat {} \;

Deploy fail2ban to auto-block attackers

fail2ban monitors auth.log and automatically adds iptables rules to block IPs that exceed the configured failure threshold.

sudo apt install fail2ban -y sudo systemctl enable --now fail2ban sudo fail2ban-client status sshd
# Show all current SSH sessions
who
w
last | head -20        # login history
lastb | head -10       # failed login history (requires root)

# Show failed SSH attempts from the journal
journalctl -u sshd --since "24 hours ago" | grep "Failed"

# Count unique IPs attempting to login in the last hour
journalctl -u sshd --since "1 hour ago" | \
  grep "Failed password" | \
  grep -oP '(\d{1,3}\.){3}\d{1,3}' | sort -u | wc -l

# Check fail2ban status and banned IPs
sudo fail2ban-client status sshd
sudo fail2ban-client get sshd banned

What just happened? fail2ban has blocked 7 IPs currently and has banned 142 unique attackers in total since it was deployed, based on 4,281 failed authentication attempts. The top banned IP (185.220.101.42) is the same brute-force source identified in Lesson 21's log analysis — fail2ban is automatically blocking it before it can accumulate more failures. With PasswordAuthentication no already set, these attempts will all fail anyway — but fail2ban reduces log noise and connection overhead.

Agent Forwarding Carries Risk — Only Enable It on Trusted Hosts

When you enable ForwardAgent yes or use ssh -A, the remote server gains the ability to use your agent socket to authenticate as you to any other server your key has access to — for as long as you are connected. If that remote server is compromised or run by a malicious party, the attacker can use your agent to reach your other servers. Limit agent forwarding to bastion hosts you control and trust. For multi-hop scenarios, ProxyJump (-J) is safer because your agent socket is only ever on your own machine.

Lesson Checklist

I generate Ed25519 key pairs with passphrases, use ssh-copy-id to install public keys, and verify key login before disabling passwords
I maintain a ~/.ssh/config with host aliases, ProxyJump chains, and global keepalive settings to simplify daily workflows
I can create local (-L), remote (-R), and dynamic (-D) port forwards, and I use ProxyJump instead of agent forwarding for multi-hop access
I use drop-in files in /etc/ssh/sshd_config.d/ for hardening, always validate with sshd -t, and confirm settings with sshd -T
I regularly audit authorized_keys files across all accounts and deploy fail2ban to automatically block brute-force sources

Teacher's Note

The ProxyJump directive in ~/.ssh/config deserves special attention — it is the cleanest solution to the common "I need to reach servers behind a bastion" problem, and most administrators discover it years too late. Unlike agent forwarding, ProxyJump creates the tunnel entirely on your local machine, so your private key and agent never touch the intermediate host. If you currently use agent forwarding to reach internal servers, switching to ProxyJump is a meaningful security improvement with zero loss in convenience.

Practice Questions

1. You need to access a PostgreSQL database running at db.internal:5432 which is only reachable from bastion.example.com. Write the SSH command to create a persistent background tunnel and the ~/.ssh/config block that would make this available with a short alias. Then write the psql command to use the tunnel.

2. Explain the security difference between ForwardAgent yes and ProxyJump when accessing an internal server through a bastion host. In which scenario is each approach appropriate?

3. After applying SSH hardening settings in /etc/ssh/sshd_config.d/hardening.conf, how do you verify that the settings are actually active without rebooting, and what is the risk of skipping validation before reloading sshd?

Lesson Quiz

1. Why is a passphrase on your SSH private key important even though the private key never leaves your machine?

2. What does ssh -L 8080:internal-app.local:80 -N -f user@bastion do?

3. AllowUsers alice bob is set in sshd_config. A user named deploy has a valid SSH key in their authorized_keys file. Can deploy log in via SSH?

Up Next

Lesson 29 — Firewall Management

Controlling inbound and outbound traffic with ufw, firewalld, and iptables