Kubernetes Lesson 21 – Environment Variables | Dataplexa

Core Kubernetes Concepts · Lesson 21

Environment Variables

You've seen env vars used in ConfigMaps and Secrets — but that's only two of five different sources Kubernetes can pull them from. This lesson closes the loop on every injection method, including some genuinely useful ones that most engineers never discover until they need them.

The Five Sources of Environment Variables

Every env var in a container ultimately comes from one of five places. Some you've already used. A couple will be new. Understanding all five means you'll never have to ask "how do I get that value into my container?" again.

#	Source	YAML field	Best used for
1	Static literal value	env[].value	Fixed values that never change per-environment
2	ConfigMap reference	env[].valueFrom.configMapKeyRef	Non-sensitive config: hostnames, feature flags, log levels
3	Secret reference	env[].valueFrom.secretKeyRef	Sensitive values: passwords, tokens, API keys
4	Pod metadata (Downward API)	env[].valueFrom.fieldRef	Pod name, namespace, node name, Pod IP — self-discovery
5	Resource field (Downward API)	env[].valueFrom.resourceFieldRef	CPU/memory limits — so the app can self-tune thread pools

Sources 1–3: Static, ConfigMap, and Secret

The scenario: You're deploying a new order management API. It needs a static app name baked in, a handful of non-sensitive settings from a ConfigMap, and database credentials from a Secret. You also want to bulk-inject all ConfigMap keys in one block to avoid listing each one individually. Here's the complete env block covering all three familiar sources.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-api
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-api
  template:
    metadata:
      labels:
        app: order-api
    spec:
      containers:
        - name: order-api
          image: company/order-api:4.2.1
          ports:
            - containerPort: 8080
          env:
            # --- SOURCE 1: Static literal value ---
            - name: APP_NAME              # Env var name inside the container
              value: "order-api"          # Hardcoded string — use for values that never change

            - name: REGION
              value: "eu-west-1"          # Deployment region — static per cluster, not per-environment

            # --- SOURCE 2: Single key from a ConfigMap ---
            - name: LOG_LEVEL             # The env var name the container sees
              valueFrom:
                configMapKeyRef:
                  name: order-api-config  # ConfigMap name (must be in same namespace)
                  key: LOG_LEVEL          # Key within the ConfigMap to pull
                  optional: false         # Fail Pod startup if ConfigMap or key is missing

            - name: DB_HOST
              valueFrom:
                configMapKeyRef:
                  name: order-api-config
                  key: DB_HOST

            # --- SOURCE 3: Single key from a Secret ---
            - name: DB_PASSWORD           # Env var the app reads for the database password
              valueFrom:
                secretKeyRef:
                  name: order-db-secret   # Secret name (must be in same namespace)
                  key: POSTGRES_PASSWORD  # Key within the Secret — decoded automatically
                  optional: false

          envFrom:
            # Bulk-inject ALL keys from a ConfigMap as env vars in one block
            - configMapRef:
                name: order-api-config    # Every key in this ConfigMap becomes an env var
            # Bulk-inject ALL keys from a Secret as env vars in one block
            - secretRef:
                name: order-api-secret    # Every key in this Secret becomes an env var
                optional: true            # Don't fail if this Secret doesn't exist yet

$ kubectl apply -f order-api-deployment.yaml
deployment.apps/order-api created

$ kubectl exec -it order-api-7f9b4d-2xkpj -n production -- env | sort
APP_NAME=order-api
DB_HOST=postgres.production.svc.cluster.local
DB_PASSWORD=s3cur3P@ssw0rd
DB_PORT=5432
FEATURE_EXPRESS_CHECKOUT=true
LOG_LEVEL=info
MAX_CONNECTIONS=50
REGION=eu-west-1
TIMEOUT_SECONDS=30

What just happened?

env vs envFrom — precedence matters — When you use both env and envFrom, individual env entries take precedence over envFrom entries with the same key name. This means you can use envFrom to bulk-import a ConfigMap and then use env to override specific values for this particular container — without changing the ConfigMap.

optional: true on secretRef — Setting optional: true means the Pod will start even if the referenced Secret doesn't exist. The env vars from that Secret simply won't be set. Useful during initial cluster bootstrap when Secrets may be created after the Deployment — but be careful: the app may fail at runtime when it tries to use the missing value.

kubectl exec -- env | sort — Piping to sort alphabetises the output, making it much easier to scan for specific variables in a container with dozens of env vars. A small trick that saves significant time during debugging.

Source 4: The Downward API — Pod Self-Awareness

The Downward API is one of the most underused features in Kubernetes. It lets a Pod inject facts about itself — its own name, namespace, node, IP address, labels, and annotations — as environment variables or files, without the application needing to call the Kubernetes API. The Pod can know where it is and who it is.

This sounds niche until you've seen it in action. Log aggregation becomes trivial when every log line automatically includes the Pod name and namespace. Distributed tracing headers can include the node the request originated on. Health check endpoints can report their own deployment metadata. All without any application code touching the Kubernetes API.

The scenario: Your platform team is rolling out centralised logging with a structured JSON format. Every log line needs to include the Pod name, namespace, and node to make distributed tracing across dozens of replicas tractable. The development team doesn't want to call the Kubernetes API in their code — they want it pre-injected as env vars they can reference in their logger config.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: order-api
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: order-api
  template:
    metadata:
      labels:
        app: order-api
        version: "4.2.1"
    spec:
      containers:
        - name: order-api
          image: company/order-api:4.2.1
          env:
            # --- SOURCE 4: Pod metadata via Downward API (fieldRef) ---
            - name: POD_NAME              # The running Pod's name — unique per replica
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name  # Path in the Pod object to read from

            - name: POD_NAMESPACE         # Which namespace this Pod is running in
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace

            - name: POD_IP                # The Pod's cluster IP address
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP

            - name: NODE_NAME             # Which node this Pod was scheduled onto
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName

            - name: POD_SERVICE_ACCOUNT   # Which ServiceAccount this Pod is running as
              valueFrom:
                fieldRef:
                  fieldPath: spec.serviceAccountName

            # --- SOURCE 5: Container resource limits via Downward API (resourceFieldRef) ---
            - name: CPU_LIMIT             # The CPU limit set on THIS container
              valueFrom:
                resourceFieldRef:
                  containerName: order-api   # Must specify which container (for multi-container pods)
                  resource: limits.cpu       # Which resource field to read
                  divisor: "1m"              # Divisor: unit to express the value in (1m = millicores)

            - name: MEMORY_LIMIT          # The memory limit set on THIS container
              valueFrom:
                resourceFieldRef:
                  containerName: order-api
                  resource: limits.memory
                  divisor: "1Mi"             # Express in mebibytes

$ kubectl apply -f order-api-downward.yaml
deployment.apps/order-api configured

$ kubectl exec -it order-api-7f9b4d-2xkpj -n production -- env | grep -E "POD_|NODE_|CPU_|MEMORY_"
CPU_LIMIT=300
MEMORY_LIMIT=256
NODE_NAME=node-eu-west-1a
POD_IP=10.244.2.14
POD_NAME=order-api-7f9b4d-2xkpj
POD_NAMESPACE=production
POD_SERVICE_ACCOUNT=default

$ kubectl exec -it order-api-7f9b4d-r2skl -n production -- env | grep POD_NAME
POD_NAME=order-api-7f9b4d-r2skl

What just happened?

Each Pod gets unique values — Notice that POD_NAME on the first Pod is order-api-7f9b4d-2xkpj and on the second Pod it's order-api-7f9b4d-r2skl. The Deployment YAML is identical for both — but the Downward API dynamically resolves to each Pod's actual runtime values at start time. This is the magic: one manifest, three unique identities.

fieldRef paths — The most useful fieldPath values are: metadata.name, metadata.namespace, metadata.uid, status.podIP, status.hostIP, spec.nodeName, spec.serviceAccountName. You can also expose labels and annotations, but those require the volume mount approach since they can contain multiple values and can change at runtime.

resourceFieldRef divisor — The divisor field controls the unit. For CPU: 1m gives millicores (so 300m CPU limit → CPU_LIMIT=300), 1 gives whole cores. For memory: 1Mi gives mebibytes, 1 gives bytes. A Java application can read MEMORY_LIMIT and use it to auto-set the JVM heap size to 75% of the container limit — no manual tuning, no OOM kills.

Variable Substitution Inside env

Kubernetes supports a limited but useful form of variable substitution inside the env block. A variable defined earlier in the same env list can be referenced in a later value using $(VAR_NAME) syntax.

The scenario: Your application reads its database connection as a single DATABASE_URL string in the format postgresql://user:password@host:port/db. The individual parts (user, password, host, port) come from different sources — ConfigMap and Secret. You need to compose them into one URL without changing application code.

containers:
  - name: order-api
    image: company/order-api:4.2.1
    env:
      - name: DB_USER                   # Step 1: inject individual parts from their sources
        valueFrom:
          secretKeyRef:
            name: order-db-secret
            key: POSTGRES_USER

      - name: DB_PASSWORD
        valueFrom:
          secretKeyRef:
            name: order-db-secret
            key: POSTGRES_PASSWORD

      - name: DB_HOST
        valueFrom:
          configMapKeyRef:
            name: order-api-config
            key: DB_HOST

      - name: DB_PORT
        valueFrom:
          configMapKeyRef:
            name: order-api-config
            key: DB_PORT

      - name: DB_NAME
        valueFrom:
          configMapKeyRef:
            name: order-api-config
            key: DB_NAME

      - name: DATABASE_URL              # Step 2: compose them into a connection string
        value: "postgresql://$(DB_USER):$(DB_PASSWORD)@$(DB_HOST):$(DB_PORT)/$(DB_NAME)"
        # $(VAR_NAME) references earlier env vars defined in this same list
        # CRITICAL: only works for vars defined ABOVE in the same env[] list — not from envFrom
        # Result: postgresql://order_user:s3cur3P@ssw0rd@postgres.production.svc.cluster.local:5432/orders

$ kubectl exec -it order-api-7f9b4d-2xkpj -n production -- env | grep DATABASE_URL
DATABASE_URL=postgresql://order_user:s3cur3P@ssw0rd@postgres.production.svc.cluster.local:5432/orders

$ kubectl exec -it order-api-7f9b4d-2xkpj -n production -- env | grep DB_
DB_HOST=postgres.production.svc.cluster.local
DB_NAME=orders
DB_PASSWORD=s3cur3P@ssw0rd
DB_PORT=5432
DB_USER=order_user
DATABASE_URL=postgresql://order_user:s3cur3P@ssw0rd@postgres.production.svc.cluster.local:5432/orders

What just happened?

$(VAR_NAME) substitution — Kubernetes resolves the $(VAR_NAME) references before the container starts, assembling the composed string from the individual values. The app receives the final DATABASE_URL string and never needs to know about the individual pieces.

Order matters — critically — Substitution only works for variables defined earlier in the same env list. If DATABASE_URL appears before DB_USER in the YAML, the substitution won't resolve — the literal string $(DB_USER) is injected as-is. Always define component variables above the composed ones.

Only works in env[], not envFrom — Variable substitution does not work for variables coming from envFrom blocks. If DB_USER was bulk-imported via envFrom rather than defined in env[], the $(DB_USER) reference in DATABASE_URL would produce a literal $(DB_USER) in the composed value.

The Complete env Reference Map

All five sources in one diagram — how they flow from their origin into the running container:

Environment Variable Sources → Container

Static Value

ConfigMap

Secret

Pod Metadata
(Downward API)

Resource Fields
(Downward API)

→

env[] in container spec

value: "literal"
configMapKeyRef
secretKeyRef
fieldRef
resourceFieldRef

→

📦

Running Container

process.env / os.environ / System.getenv()

Precedence: Individual env[] entries override envFrom entries with the same key name. $(VAR) substitution only works within env[] and only for variables defined earlier in the same list.

Debugging Environment Variable Problems

The scenario: A developer reports that their service is failing to connect to the database. They swear the ConfigMap and Secret are correct. You need to verify exactly what env var values are actually reaching the container — not what the manifests say they should be.

kubectl exec -it order-api-7f9b4d-2xkpj -n production -- env | sort
# Dump and sort all env vars in the running container
# Compare against what the ConfigMap and Secret contain
# Discrepancies here reveal the actual problem

kubectl exec -it order-api-7f9b4d-2xkpj -n production -- printenv DATABASE_URL
# printenv: print the value of a single specific env var
# Faster than piping env | grep when you know exactly what you're looking for

kubectl get configmap order-api-config -n production -o jsonpath='{.data}'
# Dump the full ConfigMap data as JSON to compare against what the Pod received
# If these match, the issue is in the Pod spec mapping — wrong key name, wrong ConfigMap name

kubectl get secret order-db-secret -n production -o jsonpath='{.data}' | \
  python3 -c "import sys,json,base64; d=json.load(sys.stdin); print({k: base64.b64decode(v).decode() for k,v in d.items()})"
# Decode all Secret values to compare against what the container is actually receiving
# Only run this on a secure terminal — outputs plaintext secrets to stdout

kubectl describe pod order-api-7f9b4d-2xkpj -n production | grep -A3 "Environment:"
# describe pod: shows the resolved env var sources in the pod spec
# Useful for verifying which ConfigMap/Secret is wired to which env var name
# Shows "optional" status and whether the reference resolved

$ kubectl exec -it order-api-7f9b4d-2xkpj -n production -- printenv DATABASE_URL
postgresql://order_user:s3cur3P@ssw0rd@postgres.production.svc.cluster.local:5432/orders

$ kubectl describe pod order-api-7f9b4d-2xkpj -n production | grep -A40 "Environment:"
    Environment:
      APP_NAME:    order-api
      REGION:      eu-west-1
      LOG_LEVEL:   <set to the key 'LOG_LEVEL' of config map 'order-api-config'>    Optional: false
      DB_HOST:     <set to the key 'DB_HOST' of config map 'order-api-config'>      Optional: false
      DB_PASSWORD: <set to the key 'POSTGRES_PASSWORD' in secret 'order-db-secret'> Optional: false
      POD_NAME:     (v1:metadata.name)
      POD_NAMESPACE: (v1:metadata.namespace)
      NODE_NAME:    (v1:spec.nodeName)
    Environment Variables from:
      ConfigMap order-api-config  Optional: false
      Secret    order-api-secret  Optional: true

What just happened?

kubectl describe pod — Environment section — The Environment section in kubectl describe pod is invaluable for debugging. It shows each env var, its source (literal, ConfigMap key, Secret key, or Downward API), and whether it's optional. Critically, it does NOT show the actual values — it shows the source mapping. This tells you what Kubernetes is trying to inject — then kubectl exec -- env tells you what actually reached the container.

Two-step debugging workflow — Step 1: kubectl describe pod to verify the wiring (which ConfigMap/Secret key maps to which env var). Step 2: kubectl exec -- printenv VAR_NAME to verify the actual resolved value. If step 1 looks correct but step 2 shows the wrong value, the ConfigMap or Secret itself has the wrong data — check it with kubectl get configmap -o jsonpath.

Teacher's Note: The Downward API is more useful than it looks

I've seen engineers spend days writing code to call the Kubernetes API from inside their application just to get the Pod name for log correlation — when fieldRef: metadata.name would have done it in three lines of YAML. The Downward API is not glamorous but it's genuinely useful and it's free — no API calls, no RBAC permissions needed, no network latency.

The resourceFieldRef for memory limits is one of my favourite tricks. A JVM application that reads MEMORY_LIMIT_MI and sets -Xmx to 75% of that value will never get OOM-killed by Kubernetes — because the JVM heap is always sized relative to the actual container limit. Change the limit in the manifest and the JVM self-adjusts on the next restart. No manual -Xmx tuning required.

One last caution: don't let your env block grow to 40+ entries. When a container has that many env vars, it becomes unreadable and unmaintainable. If you're past 15–20 env vars, consider switching to a mounted ConfigMap file and having the app read structured config from disk instead.

Practice Questions

1. Which valueFrom field in an env entry lets you inject the Pod's own name, namespace, or IP address using the Downward API?

2. What syntax does Kubernetes use to reference a previously defined env var's value inside another env var's value field for variable substitution?

3. A container spec has both an env entry and an envFrom block that both define a key called LOG_LEVEL. Which one wins?

Quiz

Up Next · Lesson 22

Resource Requests and Limits

How Kubernetes schedules Pods onto nodes, what happens when a container exceeds its memory limit, and why getting resource sizing wrong is the most common cause of production incidents.

← Previous Course Index Next →

Kubernetes Course

Environment Variables

The Five Sources of Environment Variables

Sources 1–3: Static, ConfigMap, and Secret

Source 4: The Downward API — Pod Self-Awareness

Variable Substitution Inside env

The Complete env Reference Map

Debugging Environment Variable Problems

Practice Questions

Quiz