Zero trust for APIs: verify every internal and external request

TL;DR — Zero trust for APIs means every request is authenticated and authorised at the point of use, including service-to-service calls on your private network. The most broadly correct implementation is: mTLS between workloads for workload identity, short-lived JWT or opaque access tokens for caller identity and scopes, and policy enforcement at the API gateway and again in the service. Reading time: ~7 min

What it is and where it sits

For APIs, zero trust is not a product. It is a design rule: never treat "inside the VPC", "from Kubernetes", or "behind the VPN" as sufficient proof that a caller should be trusted.

In practical architecture terms, it replaces these common assumptions:

• "Internal traffic is safe; only the edge needs auth"
• "If nginx or the API gateway authenticated the user, downstream services can skip checks"
• "A shared API key between services is good enough"
• "Source IP or security group membership is identity"

Instead, each hop verifies two things:

• Who is calling? User identity, workload identity, or both
• What is this caller allowed to do on this resource right now?

Where it lives in a typical flow:

• At the edge: API gateway / ingress validates client credentials and rejects obvious garbage early
• On the service-to-service hop: sidecar, service mesh, reverse proxy, or app server verifies mTLS and/or bearer token
• In the application: business authorisation checks resource ownership, tenant boundaries, roles, scopes, and action-specific policy
• Around data stores and async systems: the same principle applies to DB connections, queues, and webhooks; internal does not mean exempt

[Client App] ---- 1. HTTPS + access token ----> [API Gateway]
[API Gateway] ---- 2. mTLS + forwarded token ----> [Orders API]
[Orders API] ---- 3. mTLS + service token ----> [Inventory API]
[Orders API]
  | 4. SQL with app role
  v
[Postgres]

What it replaces in a request/data flow is the old perimeter model:

• before: internet auth at the edge, then trusted flat network inside
• after: explicit identity and policy checks on every call, including orders-api -> inventory-api

The important architectural point: zero trust for APIs is layered, not single-point. If your gateway validates JWTs but your internal services accept any request from the cluster network, you still have a large trust gap.

How it actually works

Use one realistic example: a frontend calls POST /orders, and orders-api calls inventory-api to reserve stock.

Step-by-step flow

1. The client gets an access token from your identity provider using OAuth 2.0 / OIDC. The token is short-lived, for example 5-15 minutes, and contains claims like sub, aud, scope, tenant_id.
2. The client calls the API gateway over TLS with Authorization: Bearer <token>.
3. The gateway validates the token:
   • signature using JWKS or introspection for opaque tokens
   • exp and nbf
   • iss
   • aud matches the API
   • required scopes, for example orders:create
4. The gateway forwards the request to orders-api over mTLS. Now the gateway proves its workload identity with a client certificate, and orders-api verifies that certificate chain and SAN.
5. orders-api performs its own authz check, not just trust the gateway. It checks business rules the gateway cannot know, like "user can only create orders for tenant acme".
6. orders-api calls inventory-api. This is where many teams break zero trust. Do not call it with only a cluster IP and no credentials.
7. orders-api authenticates to inventory-api using mTLS for workload identity and usually also sends a token representing either:
   • the original user, if you need end-user context downstream, or
   • a delegated/service token with constrained scope like inventory:reserve
8. inventory-api authorises the action. It checks that caller orders-api is allowed to reserve stock, and if user context is propagated, that the user is allowed to act in that tenant.
9. Each service emits audit logs with request ID, caller identity, decision, and reason. Without this, debugging denied internal calls becomes guesswork.

What concrete enforcement looks like

At the transport layer:

• TLS server auth stops talking to impostor services
• mTLS client auth identifies the calling workload

At the application layer:

• bearer token or signed internal token identifies the user and/or delegated actor
• policy checks decide action on resource

Those are complementary. mTLS alone answers "which workload called me?" It does not answer "which user initiated this action?" JWT alone answers caller claims, but if any pod can replay a token to your service, you still have a lateral movement problem.

Failure modes you will actually see

Expired token:

curl -i https://api.example.com/orders \
  -H 'Authorization: Bearer eyJhbGciOi...' \
  -H 'Content-Type: application/json' \
  -d '{"sku":"ABC-123","qty":1}'

HTTP/2 401
content-type: application/json
www-authenticate: Bearer error="invalid_token", error_description="The access token expired"

{"error":"invalid_token"}

mTLS client cert missing on an internal hop:

curl -vk https://inventory.internal.svc.cluster.local/reserve

* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, handshake failure (552):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to inventory.internal.svc.cluster.local:443
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to inventory.internal.svc.cluster.local:443

Wrong audience claim:

HTTP/1.1 403 Forbidden
content-type: application/json

{"error":"insufficient_scope","detail":"aud claim api://billing does not match api://orders"}

These details matter because zero trust failures often look like generic 401/403 unless you log the exact reject reason.

When to use it (and when not to)

Use explicit decision criteria, not slogans.

You probably do not need the full stack if all of these are true:

• one deployable unit, not many services
• no cross-team service ownership
• no multi-tenant isolation problem
• no untrusted clients beyond your own backend
• your main risk is bad business logic, not lateral movement

In that case, do the boring essentials first: TLS everywhere, no shared admin credentials, proper session/token validation at the edge, and resource-level authorisation in the app.

Trade-offs

Every zero-trust benefit has a bill attached.

• Benefit: limits lateral movement after one service is compromised
  Cost: certificate issuance/rotation, trust bundle distribution, and broken-call debugging get harder.

• Benefit: clear workload identity for internal callers
  Cost: you now operate PKI or depend on a platform component that does. Expired certs cause outages, not warnings.

• Benefit: per-request authorisation is auditable
  Cost: more policy code, more deny paths, and more support tickets saying "why did this 403?"

• Benefit: short-lived tokens reduce blast radius
  Cost: clock skew, token refresh logic, JWKS caching, and occasional auth outages when the IdP or key rotation path breaks.

• Benefit: central policy at the gateway
  Cost: temptation to over-centralise. Gateways can check coarse auth, but business auth still belongs in services. Duplicated logic is normal here.

• Benefit: stronger tenant isolation
  Cost: every downstream call must preserve tenant context correctly. One dropped tenant_id claim can become a data leak.

• Benefit: less reliance on network boundaries
  Cost: more moving parts than "allow 10.0.0.0/8". Expect latency overhead from TLS handshakes if you do not reuse connections.

Lock-in risk is real too. If you push all authn/authz into one vendor-specific mesh or gateway policy language, migration gets expensive. Keep the core model portable: standard TLS, OAuth 2.0/OIDC, JWT or opaque tokens, and app-enforced business rules.

In practice

Example 1: nginx validates client certs on an internal API

server {
    listen 443 ssl;
    server_name inventory.internal.example;

    ssl_certificate /etc/nginx/tls/server.crt;
    ssl_certificate_key /etc/nginx/tls/server.key;

    ssl_client_certificate /etc/nginx/tls/ca.crt;
    ssl_verify_client on;
    ssl_verify_depth 2;

    location /reserve {
        if ($ssl_client_verify != SUCCESS) { return 495; }

        proxy_set_header X-Client-DN  $ssl_client_s_dn;
        proxy_set_header X-Client-Cert $ssl_client_escaped_cert;
        proxy_pass http://inventory_app;
    }
}

This forces client-certificate authentication before inventory-api accepts internal traffic. The gotcha: ssl_verify_client on; only proves the cert chains to your CA; your app still needs to check that the cert subject/SAN maps to an allowed workload identity, not just any cert from that CA.

Test it with and without a client cert:

curl -vk https://inventory.internal.example/reserve
curl -vk https://inventory.internal.example/reserve \
  --cert /etc/certs/orders-api.crt \
  --key /etc/certs/orders-api.key \
  --cacert /etc/certs/ca.crt

Expected failure shape without the cert:

< HTTP/1.1 400 Bad Request
< Server: nginx
< Content-Type: text/html

<html>
<head><title>400 No required SSL certificate was sent</title></head>

Example 2: Express middleware validates JWT claims and enforces scope

import express from "express";
import jwt from "jsonwebtoken";
import jwksClient from "jwks-rsa";

const app = express();

const client = jwksClient({
  jwksUri: "https://idp.example.com/.well-known/jwks.json",
  cache: true,
  cacheMaxEntries: 5,
  cacheMaxAge: 10 * 60 * 1000
});

function getKey(header, callback) {
  client.getSigningKey(header.kid, (err, key) => {
    if (err) return callback(err);
    callback(null, key.publicKey || key.rsaPublicKey);
  });
}

function requireAuth(requiredScope) {
  return (req, res, next) => {
    const auth = req.header("authorization") || "";
    const token = auth.startsWith("Bearer ") ? auth.slice(7) : null;
    if (!token) return res.status(401).json({ error: "missing_bearer_token" });

    jwt.verify(token, getKey, {
      algorithms: ["RS256"],
      issuer: "https://idp.example.com/",
      audience: "api://orders",
      clockTolerance: 5
    }, (err, payload) => {
      if (err) return res.status(401).json({ error: "invalid_token", detail: err.message });

      const scopes = String(payload.scope || "").split(" ");
      if (!scopes.includes(requiredScope)) {
        return res.status(403).json({ error: "insufficient_scope", required: requiredScope });
      }

      req.auth = payload;
      next();
    });
  };
}

app.post("/orders", requireAuth("orders:create"), (req, res) => {
  if (req.auth.tenant_id !== req.header("x-tenant-id")) {
    return res.status(403).json({ error: "tenant_mismatch" });
  }
  res.status(201).json({ ok: true });
});

app.listen(3000);

This validates the token signature and core claims, then enforces a scope and a tenant boundary. The gotcha: signature validation is not enough; if you do not check issuer and audience, a valid token minted for some other API can be accepted by mistake.

Example 3: Call an internal API with mTLS and a delegated token

INTERNAL_TOKEN=$(curl -sS https://idp.example.com/oauth/token \
  -d grant_type=client_credentials \
  -d client_id=orders-api \
  -d client_secret="$ORDERS_API_SECRET" \
  -d audience=api://inventory | jq -r .access_token)

curl -sS https://inventory.internal.example/reserve \
  --cert /etc/certs/orders-api.crt \
  --key /etc/certs/orders-api.key \
  --cacert /etc/certs/ca.crt \
  -H "Authorization: Bearer $INTERNAL_TOKEN" \
  -H "X-Request-ID: 7d7d8f2b-2b8c-4a7a-9d8c-1f0e5d6a9c11" \
  -H "Content-Type: application/json" \
  -d '{"sku":"ABC-123","qty":1,"tenant_id":"acme"}'

This gives inventory-api both workload identity from the cert and API-specific authorisation from the token. The gotcha: client-credentials tokens represent the service, not the end user. If downstream decisions depend on the user, propagate user context explicitly and define whether it is delegation, impersonation, or just audit context.

⚠️ Turning on mandatory mTLS or strict JWT checks in production can cut off healthy traffic immediately. Roll out in stages: first log-only or optional verification, then enforce for one route or one caller, then widen. Keep a tested rollback ready before changing ingress or service auth settings.

Further reading

• RFC 8705: OAuth 2.0 Mutual-TLS Client Authentication and Certificate-Bound Access Tokens
• RFC 7519: JSON Web Token (JWT)
• OpenID Connect Core 1.0
• NIST SP 800-207 Zero Trust Architecture
• The "Authorization" and "HTTP authentication" sections of the MDN HTTP docs

This article was written by an AI system and published pending human review. Verify anything you intend to act on.