agnes-the-ai-analyst/docs/local-development.md
Petr Simecek 1c18cdf15f
release(0.11.2): LOCAL_DEV_GROUPS dev mock + Makefile defaults + docs/local-development.md (#70)
* feat(auth): mock session.google_groups in LOCAL_DEV_MODE via LOCAL_DEV_GROUPS

LOCAL_DEV_MODE auto-logged-in the dev user but left session.google_groups
empty, so group-aware UI/code paths can't be exercised on localhost without
a real Google OAuth round-trip. New LOCAL_DEV_GROUPS env var (JSON array
matching the production {id, name} shape) populates the session on every
dev-bypass request — same structure the OAuth callback writes, so mock and
prod stay in lockstep. Compare-then-write avoids spurious Set-Cookie noise
on PAT/CLI requests; malformed input falls back to [] with a WARNING so
the dev mock never breaks the dev flow.

* refactor(auth): fail-fast LOCAL_DEV_GROUPS at startup + cache + no-mutate

Three small follow-ups on the same dev-mock vector before merge:

- Validate LOCAL_DEV_GROUPS at app startup and report the parsed group IDs
  in the LOCAL_DEV_MODE banner. A malformed value now warns loudly at boot
  instead of silently logging on the first authenticated request, where
  it's easy to miss.
- Cache the parsed result single-slot, keyed by the raw env-string. Avoids
  re-parsing JSON on every authenticated request without test-isolation
  surprises — when the env value changes, the key changes and the cache
  transparently rebuilds.
- Stop mutating the parsed-input dicts (item.setdefault → spread-merge)
  so the cached list stays a fresh value on every rebuild.
- Replace the try/except guard around request.session with hasattr —
  SessionMiddleware is always registered, the silent except was paranoid.

Tests grow by a direct session-cookie inspection (decoupled from the
profile template) and three startup-banner log assertions.

* fix(auth): drop fragile session-decoder test + actually skip empty-target write

Two follow-ups on the LOCAL_DEV_GROUPS feature before merge:

- Drop test_session_holds_mocked_groups_directly. It manually decoded the
  signed session cookie via TimestampSigner + base64, hardcoding both the
  Starlette session-cookie format and the 14-day max_age. Starlette has
  changed its session encoding before (URLSafeTimedSerializer pre-0.20)
  and would do so again silently — the test would fail with a cryptic
  BadSignature, not a clear "mock is broken" signal. The remaining
  test_dev_user_sees_mocked_groups_on_profile already covers the same
  observable signal (mocked groups in /profile body) without coupling to
  Starlette internals.

- Actually skip the session write when target_groups is empty. The previous
  comment claimed compare-then-write avoided spurious Set-Cookie noise on
  PAT/CLI requests, but on those requests session.get("google_groups") is
  None and target is [], so None != [] always evaluates True and the write
  fired anyway, marking the session dirty and re-issuing Set-Cookie on
  every request. Adding `target_groups and ...` to the guard makes the
  comment honest: empty mock now genuinely no-ops, stable browser sessions
  still skip via value-equality, and the only remaining write is the one
  that actually changes state.

33 auth tests still pass locally.

* fix(auth): match production's always-write semantics for stale dev groups

Devin code-review finding on PR #70: my earlier `target_groups and ...`
short-circuit silently diverged from the production OAuth callback. In
app/auth/providers/google.py:189-194 the callback always writes
session.google_groups on each login — including [] on failure or empty
token — so the session always reflects authoritative current state. The
mock should match.

Failure mode the previous guard left open: a developer sets
LOCAL_DEV_GROUPS=[{...}] for a session, the groups land in the signed
cookie, then the developer unsets the env var and reloads. target → [],
session.get → [{...}], `if target_groups and ...` is False, no write,
stale groups stay in the browser session indefinitely. Mock now lies
about state until logout.

Fix splits the guard:
- target_groups truthy + value-changed → write the new mock (existing path)
- target_groups falsy + non-empty stored → write [] to clear stale state
- otherwise no-op (target [] + stored None/[]: no transition to record)

PAT/CLI requests with no prior session still take the no-op path
(target=[], session.get → None which is falsy), so the original goal of
suppressing spurious Set-Cookie noise on token traffic is preserved.

Tests already cover the populated and unset paths; the new clear-stale
branch is correct by construction (production has the same shape) and
the rare manual reset workflow.

* release(0.11.2): default mocked groups in make local-dev + docs/local-development.md

Cuts 0.11.2 around the LOCAL_DEV_GROUPS work plus a small dev-experience
follow-up: every `make local-dev` now boots with two sensible default
mocked groups (Local Dev Engineers + Local Dev Admins on example.com),
so /profile and group-aware code paths render something realistic
without the operator having to discover and set LOCAL_DEV_GROUPS.

Layered so the default lives in the workflow, not the contract:

- scripts/run-local-dev.sh seeds LOCAL_DEV_GROUPS via shell ":="
  syntax — only sets the var when the operator hasn't already.
  Override: LOCAL_DEV_GROUPS='[...]' make local-dev. Disable:
  LOCAL_DEV_GROUPS= make local-dev.
- docker-compose.local-dev.yml swaps the commented JSON example for
  a bare `- LOCAL_DEV_GROUPS` passthrough — the value comes from the
  shell, the compose file just propagates it. Operators running
  `docker compose up` directly without the wrapper script get an
  empty mock (correct: they didn't opt into the make-driven defaults).
- Makefile help line mentions the mocked groups so the behavior is
  visible without grepping.

New docs/local-development.md consolidates dev-onboarding instructions
that were previously scattered across docker-compose.local-dev.yml
inline comments, docs/auth-groups.md "Local-dev mock" section, the
Makefile help text, and CLAUDE.md "First-Time Setup". Single page now
covers TL;DR, what LOCAL_DEV_MODE actually bypasses, group mocking
controls + verification, what is *not* mocked (Cloud Identity, real
OAuth, admin Workspace permissions), and the safety rails that keep
the dev shortcuts off production.

Version bump 0.11.1 → 0.11.2 in pyproject.toml, CHANGELOG cuts
[Unreleased] → [0.11.2] — 2026-04-26 with a fresh empty [Unreleased]
skeleton.

* fix(local-dev): default LOCAL_DEV_GROUPS truncated by shell parameter expansion

Reported by an operator running `make local-dev` against the freshly
released 0.11.2 — the LOCAL_DEV_MODE banner showed:

    LOCAL_DEV_GROUPS is not valid JSON, ignoring:
    Expecting ',' delimiter: line 1 column 70 (char 69)
    LOCAL_DEV_GROUPS is set but produced no valid groups —
    check the WARNING above for the parse error.

Cause: the default value lived inside `${LOCAL_DEV_GROUPS:=…}` parameter
expansion. Bash matches `}` to close the expansion at the *first* `}`
encountered in the body, regardless of context — even one inside a
nested JSON object literal. The two-element JSON array was therefore
truncated to the first group's closing brace, leaving an unparseable
fragment:

    [{"id":"local-dev-engineers@example.com","name":"Local Dev Engineers"

There is no escaping syntax for `}` inside parameter expansion (the
backslash escapes I had only escaped the quotes — `}` reaches bash
literally). Fix: hold the default in a single-quoted variable and
reference it through `${LOCAL_DEV_GROUPS:-$DEFAULT_LOCAL_DEV_GROUPS}`.
The variable's value is opaque to the expansion — no `}` matching
inside it — so the JSON survives intact. Verified with `python -m json`:

    parsed OK: 2 groups: ['local-dev-engineers@example.com',
                          'local-dev-admins@example.com']

Operators on a running 0.11.2 stack: `make local-dev-down && make
local-dev` to pick up the corrected default.

* fix(local-dev): respect LOCAL_DEV_GROUPS= disable path + add 0.11.2 changelog link

Two follow-ups from a Devin code-review pass on PR #70:

- run-local-dev.sh: switch ${LOCAL_DEV_GROUPS:-$DEFAULT} to
  ${LOCAL_DEV_GROUPS-$DEFAULT} (no leading colon). The :- form
  substitutes the default when the variable is unset OR set-but-empty,
  silently overwriting the documented disable knob. Three places
  promise this works — docs/local-development.md, the CHANGELOG entry,
  and the script's own comment — so the bug was an operator-facing
  lie, not just an implementation detail. The bare - form only
  substitutes on unset, so `LOCAL_DEV_GROUPS= make local-dev` now
  reaches the Python parser as "" and short-circuits to []. Verified
  with both empty and unset shells.

- CHANGELOG.md: add the [0.11.2] link reference at the bottom.
  Keep-a-Changelog convention is to mirror every version heading
  with a release-tag link in the footer; the 0.11.2 heading was
  missing its counterpart, breaking the Markdown link rendering on
  GitHub.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-04-26 16:48:55 +02:00

7.1 KiB

Local Development

Single source of truth for working on Agnes against localhost. Covers the dev-mode auth bypass, mocked Google Workspace groups, what isn't mocked, and the safety rails that keep the dev shortcuts off production.

TL;DR

make local-dev

Then open http://localhost:8000. You land on /dashboard already logged in as dev@localhost (role admin) and your /profile shows two mocked Workspace groups. No login screen, no .env file, no SMTP, no GCP project — just code.

What make local-dev actually does:

  • Stacks three Compose files: docker-compose.yml (base) + docker-compose.override.yml (hot-reload + source bind mount) + docker-compose.local-dev.yml (LOCAL_DEV_MODE overlay).
  • Seeds LOCAL_DEV_GROUPS with a sensible default (engineers + admins on example.com) so /profile is non-empty on first boot.
  • Touches an empty .env if missing — Compose validates env_file: paths even for services that never start, and the local-dev overlay drops the env-file requirement for the services that do.

make local-dev-down stops the stack; make local-dev-logs tails it.

What LOCAL_DEV_MODE=1 actually bypasses

The local-dev overlay sets LOCAL_DEV_MODE=1, which flips four switches:

  1. Auth bypass. app/auth/dependencies.py::get_current_user short-circuits to a seeded admin user (dev@localhost by default; override via LOCAL_DEV_USER_EMAIL) before any token check runs. Every protected route — REST and HTML — auto-authenticates.
  2. Magic-link emails skip SMTP. When the email-link auth provider is exercised in dev, the link is logged to stderr and returned in the response body instead of sent over wire. No mail server, no inbox.
  3. Secrets self-seed. JWT_SECRET_KEY and SESSION_SECRET auto-generate into /data/state/ on first boot if not provided. You don't need to manage them manually.
  4. No .env requirement. The overlay declares env_file: [] on the affected services, so the project-level .env doesn't need to exist. Everything dev-relevant is inline in docker-compose.local-dev.yml.

A loud warning banner is logged at startup when LOCAL_DEV_MODE=1:

============================================================
LOCAL_DEV_MODE is ON — authentication is bypassed.
All requests auto-authenticate as: dev@localhost
LOCAL_DEV_GROUPS: mocking 2 group(s) into session: local-dev-engineers@example.com, local-dev-admins@example.com
NEVER enable this in a deployment reachable from the internet.
============================================================

If you don't see that banner at boot, dev mode isn't on — check LOCAL_DEV_MODE=1 made it into the container's env.

Mocking Google Workspace groups

/profile and any future group-aware code path read session.google_groups. In production that field gets populated by the OAuth callback (app/auth/providers/google.py) from a Cloud Identity searchTransitiveGroups call. In dev there's no OAuth round-trip, so the field stays empty unless we mock it.

LOCAL_DEV_GROUPS is a JSON array of objects matching the production shape:

export LOCAL_DEV_GROUPS='[{"id":"engineers@example.com","name":"Engineering"},{"id":"admins@example.com","name":"Admins"}]'

The values flow into session.google_groups on every dev-bypass request, so group-aware code sees something realistic. Same {id, name} shape the OAuth callback writes.

How make local-dev seeds it

scripts/run-local-dev.sh sets a default if you haven't already (engineers + admins on example.com), so first-boot is non-empty. Three ways to control it:

make local-dev                                          # default mock — engineers + admins
LOCAL_DEV_GROUPS='[{"id":"qa@x.com","name":"QA"}]' make local-dev   # custom mock
LOCAL_DEV_GROUPS= make local-dev                         # empty — exercise the no-groups path

Verifying the mock

Two checks:

  1. Boot banner logs the parsed group IDs (or warns loudly if the JSON is malformed):

    LOCAL_DEV_GROUPS: mocking 2 group(s) into session: local-dev-engineers@example.com, local-dev-admins@example.com
    

    A typo (e.g. unbalanced bracket) shows up here — not silently on the first authenticated request.

  2. /profile renders the mocked groups in a list. If you set LOCAL_DEV_GROUPS= (empty), you'll see "No Google groups available".

Edge case: clearing stale groups mid-session

If you previously had LOCAL_DEV_GROUPS set, then unset it and made a request, the dev-bypass path now writes [] into the session — same semantics as the production OAuth callback, which always rewrites session.google_groups on each login. You won't get stuck looking at stale mocked groups after toggling the env var.

What's NOT mocked

LOCAL_DEV_MODE is intentionally narrow. These still need real configuration if you exercise them:

  • Cloud Identity API. No real call ever fires in dev. LOCAL_DEV_GROUPS populates session.google_groups directly without going through _fetch_google_groups. To debug the actual API call, use scripts/debug/probe_google_groups.py against a real OAuth token.
  • Real OAuth round-trip. Google login button is hidden / no-op in dev mode. To test the full OAuth flow, follow docs/auth-google-oauth.md and unset LOCAL_DEV_MODE.
  • Admin Workspace permissions. The mocked groups are not authoritative — they live only in your browser session. They don't grant any real access to anything outside Agnes; they let you exercise group-aware code paths inside the app.
  • PAT (Personal Access Token) flow. PATs work normally in dev mode; the dev bypass only short-circuits cookie/session auth. Token-bearer requests still hit the JWT validation path.

Security model

LOCAL_DEV_MODE=1 is a footgun by design — every protected route auto-authenticates as admin without any check. The codebase has these rails to keep it from leaking into prod:

  • docker-compose.local-dev.yml is a separate overlay, never stacked into docker-compose.prod.yml. Production deployments never see it.
  • The startup banner is loud and unmissableWARNING level, repeated 60-character separator. Anyone reading container logs at startup will spot it immediately.
  • is_local_dev_mode() reads os.environ fresh on every call — no startup-time cache that could be poisoned.
  • LOCAL_DEV_GROUPS is honored only inside the if is_local_dev_mode(): block in get_current_user. Setting it without LOCAL_DEV_MODE=1 does nothing.

If you ever see the dev banner in a real deployment's logs, treat it as a P0 incident: the auth boundary is gone.

  • docs/auth-groups.md — production Google Workspace groups: GCP setup checklist, the security label gotcha, debugging the real Cloud Identity call.
  • docs/auth-google-oauth.md — full Google OAuth setup for non-dev environments (client ID, scopes, redirect URIs).
  • docs/QUICKSTART.md — first-time setup for a real (non-dev) instance.
  • CLAUDE.md — repo-wide engineering conventions (changelog discipline, vendor-agnostic OSS rules, project structure).