Ansible Course
Managing Secrets Securely
In this lesson
Managing secrets securely means choosing the right tool for each secret, at each scale, in each environment — and building the habits that prevent credentials from leaking through the gaps between tools. Ansible Vault from Lesson 28 is the right foundation for most teams. But production infrastructure at scale introduces additional requirements: secrets that rotate automatically, audit logs showing who accessed what, dynamic credentials that expire after use, and integration with cloud-native secrets services. This lesson covers the full secrets management landscape — from the simplest vault-password-file setup to HashiCorp Vault and AWS Secrets Manager integration — so you can choose the right approach for your team's security requirements and operational maturity.
Choosing Your Secrets Strategy
Not every team needs HashiCorp Vault. The right strategy depends on team size, compliance requirements, and operational maturity. Over-engineering secrets management wastes time; under-engineering it causes incidents. These four tiers map to progressively more complex and capable approaches.
Ansible Vault + password file
All secrets in
vars/secrets.yml, encrypted with ansible-vault encrypt.
Password in .vault_pass on each engineer's machine. Shared via
a team password manager. Zero external dependencies.
Ansible Vault + CI environment secrets
Same encrypted files in Git. Vault password stored as a CI/CD secret (GitHub Actions secret, GitLab CI variable). Pipeline injects it at runtime. No password file on disk.
AWS / GCP / Azure Secrets Manager
Secrets stored
in cloud-native services. Ansible retrieves them at runtime using the
aws_secret or azure_keyvault_secret lookup
plugins. Automatic rotation, fine-grained IAM, and full audit logging.
HashiCorp Vault
Centralised secrets engine with dynamic credentials, lease expiry, policy-based access control, and a complete audit trail. Works across cloud and on-premise. The industry standard for enterprise secret management.
Tier 2 — Vault in CI/CD Pipelines
The most common production setup for teams that use Ansible Vault is injecting the vault password through the CI/CD pipeline's own secrets mechanism. The encrypted files are in Git; the key is not. The pipeline assembles them at runtime without any engineer having to manage a password file manually.
Step 1
Store the vault password as a CI/CD secret
In GitHub
Actions: Settings → Secrets → Actions → New repository secret.
Name it VAULT_PASSWORD. The value is the same password in your
local .vault_pass. It is stored encrypted by GitHub and never
visible in logs.
Step 2
Write the workflow file
# .github/workflows/deploy.yml
name: Deploy to production
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install Ansible and dependencies
run: |
pip install ansible
ansible-galaxy install -r requirements.yml
- name: Write vault password file
run: echo "${{ secrets.VAULT_PASSWORD }}" > .vault_pass
# The file exists only for the duration of this job
- name: Run deployment playbook
run: |
ansible-playbook deploy.yml \
-i inventory/production/ \
--vault-password-file .vault_pass
- name: Remove vault password file
if: always() # runs even if the playbook fails
run: rm -f .vault_pass
Step 3
Alternative — use environment variable instead of a file
- name: Run deployment playbook (env var method)
env:
ANSIBLE_VAULT_PASSWORD: ${{ secrets.VAULT_PASSWORD }}
run: |
# Write password from env var to a temp file Ansible can read
echo "$ANSIBLE_VAULT_PASSWORD" > /tmp/.vault_pass
ansible-playbook deploy.yml \
-i inventory/production/ \
--vault-password-file /tmp/.vault_pass
rm /tmp/.vault_pass
The Hotel Key Card Analogy
Managing secrets in a CI/CD pipeline is like hotel key cards. The hotel (CI/CD system) holds the master key (vault password) in a secure cabinet. When a guest (pipeline job) needs access to a room (encrypted secrets), the hotel issues a temporary key card valid only for that stay (job execution). The card is returned or destroyed when the guest checks out (job completes). No guest ever possesses the master key directly — they only ever hold a temporary, scoped credential issued by the system that manages it.
Tier 3 — AWS Secrets Manager Integration
Cloud-native secrets managers offer
capabilities beyond what Ansible Vault provides: automatic rotation, fine-grained
IAM access controls, versioned secret history, and a full audit log in CloudTrail.
The amazon.aws collection provides a lookup plugin that retrieves
secrets at playbook runtime — no local copy of the secret ever exists.
# Install the AWS collection
ansible-galaxy collection install amazon.aws
---
- name: Deploy application with AWS Secrets Manager
hosts: appservers
become: true
vars:
# Fetch secrets at runtime from AWS Secrets Manager
# The IAM role on the control node must have secretsmanager:GetSecretValue
db_password: "{{ lookup('amazon.aws.aws_secret',
'prod/myapp/db_password',
region='us-east-1') }}"
api_key: "{{ lookup('amazon.aws.aws_secret',
'prod/myapp/api_key',
region='us-east-1') }}"
tasks:
- name: Deploy application configuration
ansible.builtin.template:
src: app.conf.j2
dest: /etc/app/config.conf
# In app.conf.j2:
# db_password = {{ db_password }}
# api_key = {{ api_key }}
no_log: true # prevent secret values from appearing in output
How the lookup works
The lookup plugin runs on the
control node at the moment the variable is evaluated. It calls the AWS Secrets
Manager API using the control node's IAM credentials, retrieves the current secret
value, and returns it as a string. The secret never touches the filesystem — it
exists only in memory for the duration of the playbook run. When AWS rotates the
secret, the next playbook run automatically picks up the new value.
Tier 4 — HashiCorp Vault Integration
HashiCorp Vault is the industry standard
for enterprise secrets management. It provides dynamic credentials (database
passwords that are created per-request and expire automatically), policy-based
access control, and a complete audit log. Ansible integrates with it via the
hashi_vault lookup plugin in the
community.hashi_vault collection.
# Install the HashiCorp Vault collection
ansible-galaxy collection install community.hashi_vault
# Set the Vault address and token in environment (or ansible.cfg)
export VAULT_ADDR=https://vault.example.com
export VAULT_TOKEN=s.abc123xyz
---
- name: Deploy with HashiCorp Vault secrets
hosts: databases
become: true
vars:
# KV secrets engine — retrieve a static secret
db_password: "{{ lookup('community.hashi_vault.hashi_vault',
'secret=secret/data/myapp/db_password:value
url=https://vault.example.com
auth_method=token') }}"
# Dynamic database credentials — created per-request, expire automatically
# Requires a Vault database secrets engine configured for PostgreSQL
pg_credentials: "{{ lookup('community.hashi_vault.hashi_vault',
'secret=database/creds/myapp-role
url=https://vault.example.com
auth_method=approle
role_id={{ vault_role_id }}
secret_id={{ vault_secret_id }}') }}"
tasks:
- name: Create database user with dynamic credentials
community.postgresql.postgresql_user:
name: "{{ pg_credentials.username }}"
password: "{{ pg_credentials.password }}"
state: present
become_user: postgres
no_log: true
# These credentials expire in 1 hour — Vault creates and revokes them
HashiCorp Vault auth methods for Ansible
token
A Vault token
directly. Simple for development. Tokens expire — not ideal for long-running
CI pipelines without renewal logic.
approle
A role ID and
secret ID pair — designed for machine-to-machine authentication. The standard
method for CI/CD pipelines and control nodes accessing Vault.
aws_iam
Authenticate using
the control node's AWS IAM role. Zero credentials to manage — the EC2 instance
identity itself becomes the Vault credential.
ldap
Authenticate using
Active Directory or LDAP credentials. Suitable when engineers run playbooks
interactively and already have AD credentials.
Secrets Hygiene — Universal Rules
These rules apply regardless of which secrets strategy you choose. They are the baseline that every Ansible project handling real credentials must meet.
Never log secrets — always use no_log: true
Any task
whose arguments contain a credential must set no_log: true.
Ansible output goes to terminals, CI logs, monitoring systems, and log
aggregators — all of which persist the output. A single uncovered password
task can expose credentials in a dozen places.
Rotate credentials when team membership changes
When an engineer leaves, rotate every secret they had access to — vault passwords, API keys, and service account credentials. This is a process, not a technical control, but it prevents the most common post-departure exposure scenario.
Scan Git history before making a repo public
Tools like
git-secrets, truffleHog, and
gitleaks scan every commit in a repository's history for
credential patterns. Run one before changing a private repo to public —
secrets committed and then deleted still exist in git history.
Name all vault variables with a vault_ prefix
vault_db_password not db_password. This makes
encrypted variables visually distinct in any file — critical for code reviews
where a reviewer needs to quickly identify which values are protected.
Separate secrets files by sensitivity level
Maintain
separate vault files for different sensitivity tiers:
vars/secrets.yml for application secrets and
vars/infra_secrets.yml for infrastructure credentials. This
allows Vault IDs to assign different passwords to different files and
limits exposure when one password is compromised.
Document what each secret is for in a README
Every
secret in vars/secrets.yml should have a corresponding entry
in a non-committed SECRETS.md (or in the encrypted file's
comments) explaining what the secret is used for and how to rotate it.
When a rotation is required at 2am, this document is invaluable.
Vault Password Scripts
Instead of a static password file, you
can point vault_password_file at an executable script. Ansible runs
the script and uses whatever it prints to stdout as the vault password. This is how
you connect Ansible Vault to any external secrets manager — the script handles the
retrieval, Ansible handles the decryption.
#!/usr/bin/env python3
# scripts/get_vault_password.py
# Makes this file executable: chmod +x scripts/get_vault_password.py
import boto3
import json
import sys
def get_vault_password():
"""Retrieve vault password from AWS Secrets Manager."""
client = boto3.client('secretsmanager', region_name='us-east-1')
try:
response = client.get_secret_value(SecretId='ansible/vault-password')
# Secret stored as plain string
print(response['SecretString'])
except Exception as e:
print(f"Error retrieving vault password: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == '__main__':
get_vault_password()
# ansible.cfg — point to the script instead of a static file
[defaults]
vault_password_file = ./scripts/get_vault_password.py
# The script must be executable
chmod +x scripts/get_vault_password.py
# Test the script directly before using it with Ansible
./scripts/get_vault_password.py # should print the vault password to stdout
# Now ansible-playbook automatically calls the script at runtime
ansible-playbook site.yml -i inventory/production/
Deleted Secrets Still Exist in Git History
If a plaintext secret is ever
committed to a Git repository — even for a moment, even to a private repo — it
must be treated as compromised. Simply deleting the file and committing again does
not remove it from the repository's history. Anyone who clones the repo can access
every commit, including the one that contained the secret. The correct response is
to rotate the exposed credential immediately and use
git filter-branch or BFG Repo Cleaner to purge the
secret from history — then force-push and require all team members to re-clone.
This process takes hours and causes operational disruption. Prevention via
pre-commit hooks and CI scanning is far less painful.
Key Takeaways
vault_password_file at an
executable script that retrieves the password from wherever you store it.
no_log: true on tasks that handle
credentials — regardless of which secrets manager you use, task
output exposes the decrypted value unless suppressed.
Teacher's Note
If your project uses GitHub
Actions, set up the Tier 2 pattern from this lesson today — create a
VAULT_PASSWORD repository secret and update your workflow file to
write it to .vault_pass before running Ansible. The five-minute
setup eliminates the need for any engineer to manage a local password file for
CI deployments.
Practice Questions
1. Which task attribute must be set on every task that handles a decrypted credential to prevent the value from appearing in Ansible's output or CI logs?
2. Which ansible.cfg
setting can point to an executable script that retrieves the vault password from
an external secrets manager at runtime?
3. A plaintext API key was accidentally committed to a Git repository and then deleted in the next commit. What must be done with the API key before anything else?
Quiz
1. A GitHub Actions workflow needs to run an Ansible playbook that uses Ansible Vault-encrypted variables. What is the correct approach for providing the vault password?
2. A security team requires that database passwords used by Ansible never persist longer than one hour and are unique per deployment run. Which secrets approach satisfies this?
3. An engineer committed a plaintext database password to a private GitHub repo by mistake, noticed immediately, and deleted the file in the next commit. Is the secret safe?
Up Next · Lesson 30
Application Deployment
Learn to automate full application deployment cycles — rolling releases, blue-green deployments, symlink-based release management, health checks, and automated rollback when a deployment fails.