agnes-the-ai-analyst/docs/RBAC.md

Internal roles + external group mapping

Three-layer authorization model for Agnes (v9):

• External groups — Cloud Identity / Google Workspace groups, pulled at sign-in into session.google_groups. Owned by the organization; Agnes only reads them. See docs/auth-groups.md.
• Internal roles — Agnes-defined capabilities (e.g. core.admin, context_engineering.admin, corporate_memory.curator). Owned by Agnes. Either seeded for the platform (the core.* hierarchy that maps onto the legacy four-value users.role enum) or registered in code by module authors and persisted in the internal_roles table.
• Two paths to grant a role to a user:
  • Group mappings — admin-managed many-to-many table binding external group IDs to internal role keys. Resolver joins this table at sign-in and writes the resolved keys into session["internal_roles"]. Drives the OAuth flow.
  • Direct user grants (user_role_grants) — admin-issued or auto-seeded grants tying one user to one role. Persists across sessions; works for PAT and other headless callers where the session cache is unreachable.

require_internal_role(...) checks both paths: session cache first (cheap), DB-backed grants second (one query, fallback for PAT). Implies hierarchy is expanded after union — a single core.admin grant satisfies any check for core.km_admin, core.analyst, or core.viewer.

---

Quickstart by role

"I'm an operator and I want to grant someone admin"

Three options, pick whichever fits the situation:

Situation	Tool	Steps
One-off, you're at a browser	Admin UI	Admin → Users → click the target → set Core role to "Admin" (or check additional capabilities). Last-admin protection blocks the obvious foot-gun.
Bulk / scripted / from CI	CLI	da admin grant-role <email> core.admin (PAT-authenticated; works headlessly).
Custom tooling (e.g. SCIM bridge)	REST	POST /api/admin/users/{id}/role-grants with { "role_key": "core.admin" }. Bearer-token auth.

For group-based access (everyone in engineering@acme.com should be analyst), use Admin → Role mapping and click the matching chip in Known groups — it pre-fills the form. The mapping takes effect on the next sign-in of affected users, not retroactively.

"I'm a user and I want to know what I have access to"

Open /profile. Three role-related sections render server-side:

• Effective roles — chip cloud of every internal-role key you currently hold (after implies expansion). This is what the auth gate sees.
• Direct grants — rows in user_role_grants with their source label (auto-seed for v8 backfill, direct for explicit admin grants).
• Roles via groups — for each Cloud Identity group you're a member of, the role(s) it grants you via the admin's group_mappings table.

If you're missing a role you expect, the breakdown tells you whether the gap is on the grant side (no row in user_role_grants) or the mapping side (your group isn't bound to a role yet) — actionable info to take to your admin. Admins additionally see a deep-link to /admin/users/{id} for self-managing their own grants.

"I'm a module author and I need to gate my endpoints"

See Module-author workflow below for the full walkthrough. TL;DR:

# 1. Pick a key in your module's namespace, register it at import time.
from app.auth.role_resolver import register_internal_role
register_internal_role(
    "context_engineering.admin",
    display_name="Context Engineering Admin",
    description="Manages prompt templates and retrieval settings.",
    owner_module="context_engineering",
)

# 2. Gate your endpoint.
from app.auth.role_resolver import require_internal_role
@router.post("/context/templates")
async def update_template(
    user: dict = Depends(require_internal_role("context_engineering.admin")),
):
    ...

The register_* call runs at module import; sync_registered_roles_to_db (called by app/main.py at startup) reconciles the registry into the DB. Admins then see your role on /admin/role-mapping and can bind groups to it or grant individual users.

---

When to use which gate

You want to gate on …	Use …
"Is this user signed in at all?"	Depends(get_current_user)
"Coarse global level" (admin / km_admin / analyst / viewer)	Depends(require_admin) / Depends(require_role(Role.ANALYST)) — thin wrappers over require_internal_role(f"core.{role}")
"Specific module capability"	Depends(require_internal_role("corporate_memory.curator"))

require_admin and require_role are convenience helpers — they're the same gate as require_internal_role("core.admin") / require_internal_role(f"core.{role.value}") underneath. Use them for "is this user at least an analyst" type checks where the implies hierarchy carries the meaning. Use require_internal_role directly for fine-grained module capabilities.

Don't gate on user.get("role") == "admin" directly — that's the legacy column, kept only as a deprecated artifact. Use the helpers above. They route through the resolver and stay correct after grant revocations; the legacy column is a stale snapshot the role-management endpoints don't touch (see Hydration shim below for why a sweeping read of user["role"] keeps working anyway).

---

Role-key naming convention

Role keys follow <owner_module>.<capability> — lower-snake-case segments separated by dots, total length up to 64 characters, regex ^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)*$. Examples:

Key	Owner module	Notes
core.viewer / core.analyst / core.km_admin / core.admin	core	Seeded by the platform; is_core=true. Map onto legacy users.role enum. Hierarchy via implies.
context_engineering.admin	context_engineering	Module-author registered. Manage prompt templates and retrieval settings.
corporate_memory.curator	corporate_memory	Module-author registered. Manage memory items and verification evidence.

Constraints on key:

• Lower-snake-case segments, dots between segments. The first character of each segment must be a letter.
• Total length ≤ 64 chars.
• Immutable — referenced from code; renaming would silently break every existing mapping or grant. Pick carefully.
• Registering the same key twice with the same fields is a no-op (re-import safe); registering with different fields raises ValueError. If two modules collide, one of them must rename.
• First segment ⇔ owner_module convention: corporate_memory.curator is owned by corporate_memory. The mapping UI uses this prefix to group your role with other roles your module registers.
• Don't re-use core.* — that namespace is reserved for the seeded platform hierarchy. Pick a module-prefixed key even if it overlaps semantically with a core role (e.g. corporate_memory.admin, not core.curator).

---

Implies hierarchy

internal_roles.implies is a JSON array of role keys this role transitively grants. The seed populates:

core.admin     →  ["core.km_admin"]
core.km_admin  →  ["core.analyst"]
core.analyst   →  ["core.viewer"]
core.viewer    →  []

expand_implies(role_keys, conn) does BFS from the input keys and returns the deduped, sorted closure. So expand_implies(["core.admin"], conn) returns ["core.admin", "core.analyst", "core.km_admin", "core.viewer"] and require_internal_role("core.viewer") succeeds for any user holding any core.* role.

implies is currently seeded only for the core.* hierarchy (via _seed_core_roles in src/db.py). The register_internal_role API accepts display_name, description, and owner_module — there is no implies= keyword argument today, so module authors cannot declare in-namespace hierarchies through the registry. The closure expansion (expand_implies) reads whatever implies JSON sits in the row, so the runtime honors it; what's missing is the registry-side write path.

If your module needs a hierarchy (e.g. editor ⊆ admin), gate on each level explicitly today:

# Today — register both levels independently, gate per level.
register_internal_role(
    "context_engineering.editor",
    display_name="Context Engineering Editor",
    description="Save drafts.",
    owner_module="context_engineering",
)
register_internal_role(
    "context_engineering.admin",
    display_name="Context Engineering Admin",
    description="Save and ship.",
    owner_module="context_engineering",
)

# Endpoint-level: the admin gate is its own dependency. If you want the
# admin to satisfy editor checks too, grant them both roles in
# user_role_grants (or bind both via group_mappings) — until module-level
# implies lands, the registry won't auto-expand for you.
@router.post("/save")
async def save(user = Depends(require_internal_role("context_engineering.editor"))): ...

@router.post("/ship")
async def ship(user = Depends(require_internal_role("context_engineering.admin"))): ...

A future change can extend register_internal_role + InternalRoleSpec + sync_registered_roles_to_db to write implies from code. The runtime invariant — module-level implies must never point at core.* — applies whichever side of the registry/seed boundary you're on; today it's not enforced because the field isn't exposed, but a registry-side implementation must validate it.

---

Module-author workflow (step-by-step)

This is what every new module needs to do to plug into the role system. Five steps.

1. Register the role at import time

In your module's package init (e.g. services/context_engineering/__init__.py):

from app.auth.role_resolver import register_internal_role

register_internal_role(
    "context_engineering.admin",
    display_name="Context Engineering Admin",
    description="Manages prompt templates and retrieval settings.",
    owner_module="context_engineering",
)

register_internal_role only populates the in-process registry. The startup hook in app/main.py calls sync_registered_roles_to_db(conn) to reconcile the registry into the internal_roles table:

• Inserts keys that don't exist yet.
• Updates display_name / description / owner_module when they've drifted from code.
• Never deletes — a role disappearing from code (module unloaded) keeps its DB row and any mappings/grants until an admin explicitly removes it.

Nothing to do in startup code — adding the import side-effect to the module-init is enough as long as your module is imported by app/main.py (directly or transitively).

2. Gate your endpoints with require_internal_role

from fastapi import APIRouter, Depends
from app.auth.role_resolver import require_internal_role

router = APIRouter(prefix="/api/context", tags=["context"])

@router.post("/templates")
async def update_template(
    body: TemplateUpdate,
    user: dict = Depends(require_internal_role("context_engineering.admin")),
):
    ...

The dependency reads session["internal_roles"] first (the OAuth fast path); on miss, falls back to a DB lookup against user_role_grants for the authenticated user, expanding implies. A 403 is raised only when neither path produces the required role. Unauthenticated requests still get 401 from the upstream get_current_user dependency.

3. Decide if you need a hierarchy

If your module has multiple capability levels (e.g. editor who can save drafts, publisher who can ship), register each level independently — the registry-side implies write path doesn't exist yet (see Implies hierarchy for what is supported in 0.11.4).

register_internal_role(
    "your_module.editor",
    display_name="Your Module Editor",
    description="Save drafts.",
    owner_module="your_module",
)
register_internal_role(
    "your_module.publisher",
    display_name="Your Module Publisher",
    description="Ship drafts to production.",
    owner_module="your_module",
)

Until module-level implies lands, give a publisher both roles when granting (admin issues your_module.editor and your_module.publisher via the UI/CLI/REST, or binds the same Cloud Identity group to both rows in group_mappings). The runtime resolver will treat them as a flat union — there's no automatic "publisher ⊇ editor" until the registry side ships.

Don't manually OR two require_internal_role checks at the endpoint to fake a hierarchy — that pattern doesn't compose as you add levels. Pick a primary capability per endpoint and lean on the grants/mappings to keep "everyone with X also has Y" in sync.

4. Test the gate

Pattern that works across FastAPI test clients:

def test_endpoint_requires_module_admin(client, fresh_db):
    # 1. Sign in a non-admin user.
    user_token = _seed_user_with_role(fresh_db, "u@t", "analyst")

    # 2. Without the module role: 403.
    resp = client.post("/api/context/templates",
                       headers={"Authorization": f"Bearer {user_token}"},
                       json={...})
    assert resp.status_code == 403
    assert "context_engineering.admin" in resp.json()["detail"]

    # 3. Grant the role directly, retry: 200.
    _grant_role(fresh_db, "u@t", "context_engineering.admin")
    resp = client.post("/api/context/templates",
                       headers={"Authorization": f"Bearer {user_token}"},
                       json={...})
    assert resp.status_code == 200

For unit tests of business logic that don't go through HTTP, mock _get_internal_role_keys or set session["internal_roles"] directly — but always include an end-to-end test that exercises the real gate.

5. Document the role for admins

The display_name and description you pass to register_internal_role show up on /admin/role-mapping. Write the description from the admin's point of view: "Manages prompt templates and retrieval settings", not "Allows write access to context_engineering tables". Admins are deciding which Cloud Identity groups to bind to your role and which users to grant it to — they need the capability framing, not the implementation.

If your module ships with sensible defaults (e.g. "everyone in engineering@ should automatically be your_module.editor"), document the recommended group mapping in your module's README. Don't hardcode it — admins always make the binding decision.

---

Admin workflows (full reference)

Via the admin UI (preferred for one-offs)

Admin → Role mapping (/admin/role-mapping):

• Internal roles table (read-only): every registered role + how many mappings/grants reference it. Module-author roles are visually distinguished from core.* (different badge color, separate is_core=false).
• Known groups picker above the create-mapping form: clickable chips for the calling admin's own session groups (tagged "your group") plus any external group IDs already used in existing mappings (tagged "already mapped"). Click a chip to fill the form's group-id field. Empty-state copy points at LOCAL_DEV_GROUPS / Google sign-in for when you don't have anything to suggest.
• Group → role mappings table with delete buttons + the create form. Mappings take effect on the next sign-in of affected users (group resolution is cached on the session — see Refresh semantics).

Admin → Users → click a user (/admin/users/{id}):

• Core role — single-select (viewer/analyst/km_admin/admin); changes update the user's core.* grant via DELETE + POST on the role-grants endpoint.
• Additional capabilities — multi-checkbox of non-core internal roles; toggle on/off.
• Effective roles (debug) — direct grants, group-derived (best-effort, only the calling admin's own groups), and the expanded set after implies BFS. Useful for "why does this user have access?" investigations.

Direct grants (the per-user table) take effect on the very next request for that user — no logout needed.

Via the CLI (preferred for scripts and CI)

da admin role list                                  # all internal roles
da admin role show core.admin                       # one role + counts
da admin mapping list                               # all group → role bindings
da admin mapping create engineering@example.com core.km_admin
da admin mapping delete <mapping-id>
da admin grant-role alice@example.com core.admin    # direct grant
da admin revoke-role alice@example.com core.admin
da admin effective-roles alice@example.com          # debug: direct + group + expanded

All da admin subcommands hit the REST API and authenticate via PAT — works headlessly without a browser session. The PAT-aware path in require_internal_role (DB lookup over user_role_grants) makes this work; without that, every CLI admin command would 403.

Via the REST API (for custom tooling)

All endpoints under /api/admin, gated by require_internal_role("core.admin"):

Endpoint	Purpose
GET /internal-roles	list all roles
GET /group-mappings	list group bindings
POST /group-mappings	create binding ({external_group_id, role_key})
DELETE /group-mappings/{id}	remove binding
GET /users/{id}/role-grants	list direct grants for a user
POST /users/{id}/role-grants	grant a role ({role_key})
DELETE /users/{id}/role-grants/{grant_id}	revoke a direct grant
GET /users/{id}/effective-roles	debug view (direct + group + expanded)

Audit log entries are written for every mutation (role_mapping.created/deleted, role_grant.created/deleted).

---

Refresh semantics

Two scenarios:

Group mappings change — affected users see the new resolution on next sign-in. Their session cache holds the old set until logout. If you can't get the user to log out (long-lived session, automated client), Admin → Users → deactivate then reactivate invalidates the existing session and forces a fresh sign-in on the next request.

Direct grants change — take effect on the very next request, no logout needed. Two reasons:

1. The DB-backed fallback in require_internal_role consults user_role_grants per request when the session cache doesn't already grant access.
2. The _hydrate_legacy_role shim (see Hydration shim below) re-resolves on every authenticated request, so legacy user["role"] reads also pick up the new state immediately.

This asymmetry is intentional: group mappings are cached because they're the high-volume hot path; direct grants are admin-issued and rare, so the per-request DB lookup is acceptable.

---

Hydration shim

The v8 → v9 migration NULL-ed the legacy users.role column for every existing user (DuckDB rejects DROP COLUMN under the FK reference, so the column lives on as a deprecated artifact). A long tail of read sites still inspects user["role"] directly — Jinja2 templates (session.user.role), the dashboard's UserInfo.is_admin, the catalog/sync admin-bypass paths in app/api/catalog.py and app/api/sync.py, and so on.

_hydrate_legacy_role in app/auth/dependencies.py runs after every authenticated user load (both LOCAL_DEV_MODE and JWT/PAT paths). It re-resolves the highest-level core.* grant from user_role_grants and writes it into user["role"] as the legacy enum string — so every old call site keeps working without a mass refactor.

Always re-resolves, never trusts the legacy column. The role-management endpoints (POST/DELETE /api/admin/users/{id}/role-grants, plus the changeCoreRole UI flow) modify user_role_grants without touching the legacy column. If the shim short-circuited on a truthy stale value, a downgraded user would keep role="admin" in their dict even though the grants table no longer agrees — and _is_admin_user_dict (in src/rbac.py) and the catalog/sync short-circuits would silently retain elevated access while require_internal_role correctly denied the API gates. The fix is to make the grants table the single source of truth on every authenticated request. Cost: one extra DB round-trip per authenticated request — same as the existing PAT-aware fallback. Worth the consistency.

---

Local development

LOCAL_DEV_GROUPS mocks session.google_groups (see docs/auth-groups.md → Local-dev mock). The dev-bypass branch in app/auth/dependencies.py re-runs the resolver every time the mocked groups change, and passes the dev user's id so direct grants are folded into the session cache too — your seeded admin user shows up with the full core.* hierarchy on the first request, no DB-fallback hop per gate.

Typical dev setup:

export LOCAL_DEV_MODE=1
export LOCAL_DEV_GROUPS='[{"id":"engineering@example.com","name":"Engineering"}]'
# Either: register a mapping (Admin UI → Role mapping → Known groups picker)
# Or: grant directly (da admin grant-role dev@localhost <role-key>)
# Then hit any protected endpoint — dev user holds the role on the next request.

The Known-groups picker on /admin/role-mapping reads exactly your LOCAL_DEV_GROUPS IDs at page-render time, so you don't have to remember Cloud Identity opaque IDs.

---

PAT and headless requests

PATs and other Bearer-token clients carry a JWT that proves identity but not a signed session cookie, so session["internal_roles"] is never populated for them. However, v9 require_internal_role falls back to user_role_grants in the database on session-cache miss, so a PAT client whose user has a matching direct grant succeeds normally. This makes admin CLIs work uniformly via PAT.

Group memberships still don't apply to PAT (the JWT doesn't snapshot them), so a user who would get core.admin via a Cloud Identity group mapping in their browser session will need a direct grant to use that capability via PAT. The CLI da admin grant-role <email> <role-key> is the supported way to issue such grants.

For background services (the scheduler, telegram bot, etc.) that need to call gated endpoints, mint a long-lived PAT for a service-account user via /tokens and pass it through env (SCHEDULER_API_TOKEN, etc.). The PAT carries the service account's direct grants — gate semantics are identical to a human admin's session.

---

Resolution timing

Source	When	Cost
Group mappings	At sign-in (Google OAuth callback + dev-bypass when LOCAL_DEV_GROUPS changes)	One DB query at login; cached on session for the session lifetime.
Direct grants	Folded into the session cache at sign-in alongside group mappings (after the v0.11.4 fix). Per-request DB fallback still fires for PAT/headless callers without a session, or for sessions that predate the fix.	One DB query at login; per-request fallback adds one query per gated request that wasn't already satisfied by the cache. Acceptable for admin/CLI traffic; not used for high-volume per-user-data endpoints.

Trade-off: a user with a stale session keeps stale group-resolved roles until they log out + back in. Direct grants take effect immediately even on a stale session because the per-request fallback always re-checks the DB. Same mental model as session.google_groups.

---

Migration notes (v8 → v9)

• v9 schema seeds core.* rows (is_core=true) with the legacy hierarchy in implies.
• For each existing user with a non-null users.role, the migration inserts one user_role_grants row pointing at core.{role} (source='auto-seed').
• The legacy users.role column is then NULL-ed (DuckDB rejects DROP COLUMN while a FK references the table; physical drop deferred).
• UserRepository.create() and update() write to both users.role (legacy compat for in-flight code) and user_role_grants (new source of truth) so a deployment that lands mid-flight stays consistent.
• The role-management endpoints (POST/DELETE on /api/admin/users/{id}/role-grants) only modify user_role_grants. The legacy column accumulates stale values from UserRepository.create/update; the hydration shim ignores them and reads grants on every request — see Hydration shim.

---

Common pitfalls

• Gating on the legacy enum directly — if user.get("role") == "admin": ... works today via the hydration shim, but bypasses the resolver's implies expansion. Prefer await require_internal_role("core.admin") (the dependency) or is_admin(user, conn) (the helper from src/rbac.py).
• Forgetting owner_module in register_internal_role — the field defaults to None, but the admin UI uses it to group your role with other roles your module registers. Always pass it; same value as the first dot-segment of the key.
• Pointing implies at core.* — module roles must never claim platform-admin transitively. Validation rejects this; pick implies inside your own namespace.
• Manually rewriting users.role — the column is a deprecated artifact; only UserRepository.create/update should touch it (and only for in-flight legacy compat). Modify user_role_grants instead.
• Expecting roles to refresh mid-session — group-mapped roles are cached at sign-in. If an admin needs to revoke access now, deactivate-then-reactivate the user's account (forces re-sign-in) or use a direct grant change (takes effect on the next request).

---

Future work (not in 0.11.4)

• Optional da admin set-core-role <user-email> <role> (single-step replace) could trim the two-step revoke old + grant new pattern the UI currently uses.
• Cleanup migration to physically DROP the legacy users.role column once we either rebuild the table (CREATE NEW + COPY + DROP OLD) or DuckDB ALTER w/ FK support stabilizes.
• Fine-grained audit history (who held what when) — currently the audit log only records mutations, not the historical state.
• Self-service "request access" flow on /profile — let a user click a role they don't have and ping an admin for the grant. Today the user has to ask out-of-band.