Access proxy denies users who are already in the correct group

TL;DR — If the access proxy denies a user who is definitely in the right group, the most common root cause is not the group itself but stale claims: the user’s session/JWT was minted before the group change, or the proxy is matching the wrong claim/value. Re-authenticate first, then inspect the token claims and policy order before touching backend config. Reading time: ~6 min

The scenario

It’s Tuesday afternoon, you add a contractor to the "engineering" group in your IdP, they confirm the group membership is visible in the IdP portal, and they still get blocked at the access proxy with an "unauthorized" page. Other engineers can get in. Your app is healthy, upstream logs show no request ever reaching it, and now you need to prove whether the break is in the proxy policy, the IdP claim mapping, or a stale session.

[User Browser] ---- 1. GET app.example.com ----> [Access Proxy]
[Access Proxy] ---- 2. redirect/login ----> [IdP]
[IdP] ---- 3. token with claims ----> [Access Proxy]
[Access Proxy] ---- 4. allow or deny ----> [App]

Symptoms

• User sees a proxy-hosted denial page such as:

Access denied
You do not have permission to access this application.

• HTTP status from the protected hostname is 302 to login, then 403 after successful IdP login:

$ curl -k -I https://app.example.com/
HTTP/2 302
location: https://idp.example.com/oauth2/authorize?client_id=...

$ curl -k -I https://app.example.com/ -H 'Cookie: proxy_session=...'
HTTP/2 403
content-type: text/html; charset=utf-8

• Proxy logs contain lines shaped like:

level=warn msg="authorization denied" user="alice@example.com" reason="group claim mismatch"
level=info msg="authenticated user" sub="00u123..." groups="[]"
level=warn msg="policy evaluation result=deny matched_rule=default-deny"

• IdP shows the user in the expected group, but the token/session the proxy evaluates does not.
• App logs show nothing for the denied request because the proxy blocks before forwarding.

Likely causes

Step-by-step diagnosis

1. Check whether the proxy is denying before the app sees the request.

curl -k -I https://app.example.com/

If you get 302 to your IdP and later 403 from the same protected hostname, this is a proxy/IdP issue, not the app. Continue to step 2.

1. Force a fresh authentication session.

• In the browser: open a private window, log in again.
• Or clear the proxy cookie for the app domain.
• If you have a logout endpoint, hit it explicitly:

curl -k -I https://app.example.com/logout

This is your problem if a private window works immediately while the normal session still fails. Jump to Fixes → Stale session or cached token missing the new group.

1. Inspect the claims the proxy is actually evaluating. If your proxy exposes a debug/userinfo endpoint, query it with the active session cookie:

curl -s https://app.example.com/oauth2/userinfo -H 'Cookie: proxy_session=REDACTED' | jq .

Look for groups, roles, email, sub, or provider-specific claim paths. This is your problem if the expected group is absent or the claim name differs from the policy. Jump to Fixes → IdP is not emitting group claims or Fixes → Proxy policy matches the wrong claim name or value.

1. Decode the ID/access token directly. If you can capture the token from proxy debug logs, browser dev tools, or your test flow:

jwt decode "$ID_TOKEN" | jq .

Typical bad output shape:

{
  "sub": "00u123",
  "email": "alice@example.com",
  "roles": ["engineering"],
  "groups": null,
  "exp": 1793362231
}

This is your problem if the proxy policy expects groups but the IdP emits roles, or if no group-like claim exists at all. Jump to the matching fix.

1. Inspect policy order and default action. Search the proxy config repo or host config:

grep -RinE 'allow|deny|group|claim|default' /etc /opt /srv 2>/dev/null

This is your problem if you find a broad deny or a narrower allow below a first-match rule. Example:

rules:
  - action: deny
    path: /admin
  - action: allow
    groups: [engineering]
    path: /admin

If your engine is first-match, the allow never runs. Jump to Fixes → Policy/rule order causes a deny before the allow-group rule.

1. Check for group overage or truncation. Large group memberships can be omitted from tokens; some IdPs replace them with an indirection or require a separate API lookup. Decode the token and inspect whether groups are missing only for users in many groups. This is your problem if smaller users work and heavy-membership users don’t. Jump to Fixes → Group overage/size limit causes groups to be omitted from token.

2. Check time on proxy and IdP-facing hosts.

date -u
ssh proxy-host 'date -u && timedatectl status | sed -n "1,8p"'

This is your problem if clocks differ by more than a minute or NTP is unsynced and logs show token validity errors such as token is not yet valid or expired. Jump to Fixes → Clock skew breaks token validity and proxy falls back to deny.

Fixes

Stale session or cached token missing the new group

Invalidate the user session so the proxy fetches fresh claims.

# Generic reverse proxy session cookie invalidation is product-specific.
# Fastest user-side fix:
# 1) open private window, or
# 2) clear cookies for app.example.com, or
# 3) hit logout endpoint if configured
curl -k -I https://app.example.com/logout

If your proxy caches auth decisions, reduce or disable auth cache TTL in config and reload:

authz_cache_ttl: 60s
session_ttl: 8h

sudo systemctl reload access-proxy

Verify it worked:

curl -s https://app.example.com/oauth2/userinfo -H 'Cookie: proxy_session=NEW' | jq .groups

Proxy policy matches the wrong claim name or value

Change the policy to match the claim your IdP actually emits. Example: token has roles: ["engineering"], but policy checks groups.

access_policy:
  - action: allow
    path: /admin
    match:
      claim: roles
      any_of: [engineering]
  - action: deny
    path: /admin

If the claim is nested, use the syntax your proxy supports for nested paths; common examples are realm_access.roles or resource_access.myapp.roles.

Also normalize exact value mismatches: Engineering vs engineering, full DN vs short name, UUID vs display name.

{
  "groups": ["cn=engineering,ou=groups,dc=example,dc=com"]
}

Then match the DN, not the display name. Verify it worked:

jwt decode "$ID_TOKEN" | jq '.roles, .groups'

Policy/rule order causes a deny before the allow-group rule

Move the specific allow rule above the broader deny if your engine is first-match.

⚠️ Reordering access rules can open access wider than intended. Apply in staging first or keep a second admin session open so you do not lock yourself out.

rules:
  - action: allow
    path: /admin
    groups: [engineering]
  - action: deny
    path: /admin
  - action: allow
    path: /
    authenticated: true

Reload and tail logs while testing:

sudo systemctl reload access-proxy
journalctl -u access-proxy -f

Verify it worked:

curl -k -I https://app.example.com/admin -H 'Cookie: proxy_session=REDACTED'

Expect 200 or upstream redirect, not 403.

IdP is not emitting group claims for this app/client/scope

Update the IdP app/client configuration so the token for this relying party includes the group claim or role claim the proxy expects. The exact UI differs by provider; in your provider dashboard, open the application/client used by the proxy and add a token claim mapping for groups/roles, or add the scope that causes groups to be emitted.

Common OIDC-side expectation in proxy config:

oidc:
  scopes: [openid, profile, email, groups]
  claim_mappings:
    groups: groups

Reload after config change:

sudo systemctl reload access-proxy

Verify it worked:

jwt decode "$ID_TOKEN" | jq '.groups // .roles // .realm_access.roles'

Group overage/size limit causes groups to be omitted from token

If users belong to many groups, stop relying on all groups being present in the token. Prefer app-specific roles, narrower groups, or proxy-side lookup via IdP API if your product supports it. Short-term mitigation: create a dedicated small group for this app and map only that claim into the token.

Example target token shape:

{
  "roles": ["app-admin"]
}

Then update policy:

access_policy:
  - action: allow
    match:
      claim: roles
      any_of: [app-admin]

Verify it worked:

jwt decode "$ID_TOKEN" | jq '.roles, .groups | length?'

Clock skew breaks token validity and proxy falls back to deny

Sync time on the proxy host.

sudo timedatectl set-ntp true
sudo systemctl restart systemd-timesyncd || sudo systemctl restart chronyd || sudo systemctl restart ntpd

Confirm sync:

timedatectl status | sed -n '1,12p'

If running in containers, also verify the node clock, not just the container. Verify it worked:

journalctl -u access-proxy --since '5 min ago' | grep -Ei 'not yet valid|expired|clock|skew'

No new matches after a fresh login.

Prevention

• Add a synthetic auth check that logs in with a test user in the expected group and asserts the proxy returns non-403 for a protected path.

curl -fsS -o /dev/null -w '%{http_code}
' https://app.example.com/admin

Alert if the result is 403 for the canary account.

• Log the evaluated identity claims at auth time, but redact tokens. Example fields: sub, email, groups_count, matched_rule, deny_reason.

{"event":"authz","sub":"00u123","email":"alice@example.com","groups_count":3,"matched_rule":"admin-engineering","result":"allow"}

• Pin policy in version control and add CI tests for rule order.

yq '.rules[].action' access-policy.yaml

Add a test that fails if a deny for /admin appears before the allow for the admin group.

• Keep token/session TTLs aligned with your group-change expectations. If access changes must apply within minutes, do not run 24-hour sessions without revalidation.

session_ttl: 8h
reauth_interval: 1h
authz_cache_ttl: 60s

• Monitor NTP health on proxy nodes.

timedatectl show -p NTPSynchronized -p TimeUSec --value

Alert when NTPSynchronized=no.

• Standardize on app-specific roles instead of broad directory groups for proxy authorization. Smaller, explicit claims avoid overage and naming drift.

This article was written by an AI system and published pending human review. Verify anything you intend to act on.