From 1c18cdf15f97eb559cc818fee61327d5694dd547 Mon Sep 17 00:00:00 2001 From: Petr Simecek Date: Sun, 26 Apr 2026 16:48:55 +0200 Subject: [PATCH] release(0.11.2): LOCAL_DEV_GROUPS dev mock + Makefile defaults + docs/local-development.md (#70) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(auth): mock session.google_groups in LOCAL_DEV_MODE via LOCAL_DEV_GROUPS LOCAL_DEV_MODE auto-logged-in the dev user but left session.google_groups empty, so group-aware UI/code paths can't be exercised on localhost without a real Google OAuth round-trip. New LOCAL_DEV_GROUPS env var (JSON array matching the production {id, name} shape) populates the session on every dev-bypass request — same structure the OAuth callback writes, so mock and prod stay in lockstep. Compare-then-write avoids spurious Set-Cookie noise on PAT/CLI requests; malformed input falls back to [] with a WARNING so the dev mock never breaks the dev flow. * refactor(auth): fail-fast LOCAL_DEV_GROUPS at startup + cache + no-mutate Three small follow-ups on the same dev-mock vector before merge: - Validate LOCAL_DEV_GROUPS at app startup and report the parsed group IDs in the LOCAL_DEV_MODE banner. A malformed value now warns loudly at boot instead of silently logging on the first authenticated request, where it's easy to miss. - Cache the parsed result single-slot, keyed by the raw env-string. Avoids re-parsing JSON on every authenticated request without test-isolation surprises — when the env value changes, the key changes and the cache transparently rebuilds. - Stop mutating the parsed-input dicts (item.setdefault → spread-merge) so the cached list stays a fresh value on every rebuild. - Replace the try/except guard around request.session with hasattr — SessionMiddleware is always registered, the silent except was paranoid. Tests grow by a direct session-cookie inspection (decoupled from the profile template) and three startup-banner log assertions. * fix(auth): drop fragile session-decoder test + actually skip empty-target write Two follow-ups on the LOCAL_DEV_GROUPS feature before merge: - Drop test_session_holds_mocked_groups_directly. It manually decoded the signed session cookie via TimestampSigner + base64, hardcoding both the Starlette session-cookie format and the 14-day max_age. Starlette has changed its session encoding before (URLSafeTimedSerializer pre-0.20) and would do so again silently — the test would fail with a cryptic BadSignature, not a clear "mock is broken" signal. The remaining test_dev_user_sees_mocked_groups_on_profile already covers the same observable signal (mocked groups in /profile body) without coupling to Starlette internals. - Actually skip the session write when target_groups is empty. The previous comment claimed compare-then-write avoided spurious Set-Cookie noise on PAT/CLI requests, but on those requests session.get("google_groups") is None and target is [], so None != [] always evaluates True and the write fired anyway, marking the session dirty and re-issuing Set-Cookie on every request. Adding `target_groups and ...` to the guard makes the comment honest: empty mock now genuinely no-ops, stable browser sessions still skip via value-equality, and the only remaining write is the one that actually changes state. 33 auth tests still pass locally. * fix(auth): match production's always-write semantics for stale dev groups Devin code-review finding on PR #70: my earlier `target_groups and ...` short-circuit silently diverged from the production OAuth callback. In app/auth/providers/google.py:189-194 the callback always writes session.google_groups on each login — including [] on failure or empty token — so the session always reflects authoritative current state. The mock should match. Failure mode the previous guard left open: a developer sets LOCAL_DEV_GROUPS=[{...}] for a session, the groups land in the signed cookie, then the developer unsets the env var and reloads. target → [], session.get → [{...}], `if target_groups and ...` is False, no write, stale groups stay in the browser session indefinitely. Mock now lies about state until logout. Fix splits the guard: - target_groups truthy + value-changed → write the new mock (existing path) - target_groups falsy + non-empty stored → write [] to clear stale state - otherwise no-op (target [] + stored None/[]: no transition to record) PAT/CLI requests with no prior session still take the no-op path (target=[], session.get → None which is falsy), so the original goal of suppressing spurious Set-Cookie noise on token traffic is preserved. Tests already cover the populated and unset paths; the new clear-stale branch is correct by construction (production has the same shape) and the rare manual reset workflow. * release(0.11.2): default mocked groups in make local-dev + docs/local-development.md Cuts 0.11.2 around the LOCAL_DEV_GROUPS work plus a small dev-experience follow-up: every `make local-dev` now boots with two sensible default mocked groups (Local Dev Engineers + Local Dev Admins on example.com), so /profile and group-aware code paths render something realistic without the operator having to discover and set LOCAL_DEV_GROUPS. Layered so the default lives in the workflow, not the contract: - scripts/run-local-dev.sh seeds LOCAL_DEV_GROUPS via shell ":=" syntax — only sets the var when the operator hasn't already. Override: LOCAL_DEV_GROUPS='[...]' make local-dev. Disable: LOCAL_DEV_GROUPS= make local-dev. - docker-compose.local-dev.yml swaps the commented JSON example for a bare `- LOCAL_DEV_GROUPS` passthrough — the value comes from the shell, the compose file just propagates it. Operators running `docker compose up` directly without the wrapper script get an empty mock (correct: they didn't opt into the make-driven defaults). - Makefile help line mentions the mocked groups so the behavior is visible without grepping. New docs/local-development.md consolidates dev-onboarding instructions that were previously scattered across docker-compose.local-dev.yml inline comments, docs/auth-groups.md "Local-dev mock" section, the Makefile help text, and CLAUDE.md "First-Time Setup". Single page now covers TL;DR, what LOCAL_DEV_MODE actually bypasses, group mocking controls + verification, what is *not* mocked (Cloud Identity, real OAuth, admin Workspace permissions), and the safety rails that keep the dev shortcuts off production. Version bump 0.11.1 → 0.11.2 in pyproject.toml, CHANGELOG cuts [Unreleased] → [0.11.2] — 2026-04-26 with a fresh empty [Unreleased] skeleton. * fix(local-dev): default LOCAL_DEV_GROUPS truncated by shell parameter expansion Reported by an operator running `make local-dev` against the freshly released 0.11.2 — the LOCAL_DEV_MODE banner showed: LOCAL_DEV_GROUPS is not valid JSON, ignoring: Expecting ',' delimiter: line 1 column 70 (char 69) LOCAL_DEV_GROUPS is set but produced no valid groups — check the WARNING above for the parse error. Cause: the default value lived inside `${LOCAL_DEV_GROUPS:=…}` parameter expansion. Bash matches `}` to close the expansion at the *first* `}` encountered in the body, regardless of context — even one inside a nested JSON object literal. The two-element JSON array was therefore truncated to the first group's closing brace, leaving an unparseable fragment: [{"id":"local-dev-engineers@example.com","name":"Local Dev Engineers" There is no escaping syntax for `}` inside parameter expansion (the backslash escapes I had only escaped the quotes — `}` reaches bash literally). Fix: hold the default in a single-quoted variable and reference it through `${LOCAL_DEV_GROUPS:-$DEFAULT_LOCAL_DEV_GROUPS}`. The variable's value is opaque to the expansion — no `}` matching inside it — so the JSON survives intact. Verified with `python -m json`: parsed OK: 2 groups: ['local-dev-engineers@example.com', 'local-dev-admins@example.com'] Operators on a running 0.11.2 stack: `make local-dev-down && make local-dev` to pick up the corrected default. * fix(local-dev): respect LOCAL_DEV_GROUPS= disable path + add 0.11.2 changelog link Two follow-ups from a Devin code-review pass on PR #70: - run-local-dev.sh: switch ${LOCAL_DEV_GROUPS:-$DEFAULT} to ${LOCAL_DEV_GROUPS-$DEFAULT} (no leading colon). The :- form substitutes the default when the variable is unset OR set-but-empty, silently overwriting the documented disable knob. Three places promise this works — docs/local-development.md, the CHANGELOG entry, and the script's own comment — so the bug was an operator-facing lie, not just an implementation detail. The bare - form only substitutes on unset, so `LOCAL_DEV_GROUPS= make local-dev` now reaches the Python parser as "" and short-circuits to []. Verified with both empty and unset shells. - CHANGELOG.md: add the [0.11.2] link reference at the bottom. Keep-a-Changelog convention is to mirror every version heading with a release-tag link in the footer; the 0.11.2 heading was missing its counterpart, breaking the Markdown link rendering on GitHub. --------- Co-authored-by: Claude --- CHANGELOG.md | 11 +++ Makefile | 2 +- app/auth/dependencies.py | 74 ++++++++++++++++++ app/main.py | 22 +++++- docker-compose.local-dev.yml | 6 ++ docs/auth-groups.md | 14 ++++ docs/local-development.md | 106 +++++++++++++++++++++++++ pyproject.toml | 2 +- scripts/run-local-dev.sh | 17 ++++ tests/test_auth_providers.py | 145 +++++++++++++++++++++++++++++++++++ 10 files changed, 396 insertions(+), 3 deletions(-) create mode 100644 docs/local-development.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 700a5af..2e635bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,16 @@ CalVer image tags (`stable-YYYY.MM.N`, `dev-YYYY.MM.N`) are produced for every C +## [0.11.2] — 2026-04-26 + +Dev-experience patch release — make `LOCAL_DEV_MODE` realistic enough to actually exercise group-aware code paths on `localhost`, and consolidate scattered dev-onboarding instructions into a single `docs/local-development.md`. + +### Added + +- **`LOCAL_DEV_GROUPS` env var** mocks `session.google_groups` for the auto-logged-in dev user when `LOCAL_DEV_MODE=1`. JSON array matching the production shape (`[{"id":"…","name":"…"}]`) so group-aware UI and access-control code paths can be exercised on `localhost` without a Google OAuth round-trip. Honored only under `LOCAL_DEV_MODE=1`. The startup banner reports the parsed group IDs (or warns loudly when the value is set but malformed), so a typo gets surfaced at boot rather than silently on the first authenticated request. Session injection mirrors the production OAuth callback's "always-write" semantics — including clearing stale groups when the operator unsets `LOCAL_DEV_GROUPS` mid-session. See `docs/auth-groups.md` → *Local-dev mock*. +- **`make local-dev` now seeds two default mocked groups** (`Local Dev Engineers` + `Local Dev Admins` on `example.com`) via `scripts/run-local-dev.sh`, so first-boot `/profile` is non-empty out of the box. Override with `LOCAL_DEV_GROUPS='[…]' make local-dev`; disable with `LOCAL_DEV_GROUPS= make local-dev`. +- **`docs/local-development.md`** — single onboarding doc for working on Agnes locally: TL;DR, what `LOCAL_DEV_MODE` actually bypasses, group mocking, what isn't mocked, and the security-rails reminder that dev mode must never reach a production deploy. + ### Internal - Fix nightly `docker-e2e` CI failures: refresh two stale assertions that had drifted from the live API. `tests/test_docker_full.py::test_app_returns_html_on_root` now expects the auth-aware `302 → /login` (root has redirected since the auth middleware landed); `tests/test_e2e_docker.py::TestDockerHealth::test_health_has_duckdb` now reads `services["duckdb_state"]` (current health-payload shape, already validated by `tests/test_api.py`). No application behavior change — these only ran in the scheduled nightly job, so the drift went unnoticed for several PRs. @@ -106,5 +116,6 @@ First tagged semver release. The `version = "2.x"` strings that appeared in earl - Test suite expanded to 1357+ tests (4 layers — unit, integration, web smoke, journey). +[0.11.2]: https://github.com/keboola/agnes-the-ai-analyst/releases/tag/v0.11.2 [0.11.1]: https://github.com/keboola/agnes-the-ai-analyst/releases/tag/v0.11.1 [0.11.0]: https://github.com/keboola/agnes-the-ai-analyst/releases/tag/v0.11.0 diff --git a/Makefile b/Makefile index 60a2477..c0a0b01 100644 --- a/Makefile +++ b/Makefile @@ -9,7 +9,7 @@ help: @echo " make test Run test suite" @echo " make dev Start FastAPI dev server (native uvicorn)" @echo " make docker Build and start Docker Compose" - @echo " make local-dev Start Agnes with LOCAL_DEV_MODE=1 (auth bypass, no .env needed)" + @echo " make local-dev Start Agnes with LOCAL_DEV_MODE=1 (auth bypass + mocked Google groups, no .env needed)" @echo " make local-dev-down Stop and remove the local-dev stack" @echo " make local-dev-logs Tail logs from the local-dev stack" @echo " make lint Run ruff linter (if installed)" diff --git a/app/auth/dependencies.py b/app/auth/dependencies.py index c867e5e..7bcb726 100644 --- a/app/auth/dependencies.py +++ b/app/auth/dependencies.py @@ -1,5 +1,6 @@ """FastAPI auth dependencies — current user, role checking.""" +import json import logging import os from typing import Optional @@ -17,6 +18,12 @@ logger = logging.getLogger(__name__) # Default dev user used when LOCAL_DEV_MODE=1. Seeded at startup by app/main.py. LOCAL_DEV_DEFAULT_EMAIL = "dev@localhost" +# Single-slot cache for the parsed LOCAL_DEV_GROUPS value, keyed by the raw env +# string. Avoids re-parsing JSON on every authenticated request without the +# surprise of test isolation issues — when the env changes (typical in tests), +# the key changes and the cache transparently re-parses. +_LOCAL_DEV_GROUPS_CACHE: tuple[str, list[dict]] | None = None + def is_local_dev_mode() -> bool: """True when LOCAL_DEV_MODE=1 — unsafe for production, bypasses auth.""" @@ -28,6 +35,57 @@ def get_local_dev_email() -> str: return os.environ.get("LOCAL_DEV_USER_EMAIL", LOCAL_DEV_DEFAULT_EMAIL) +def get_local_dev_groups() -> list[dict]: + """Mock Google Workspace groups for the dev user when LOCAL_DEV_MODE is on. + + Reads ``LOCAL_DEV_GROUPS`` as a JSON array of objects matching the shape + produced by ``_fetch_google_groups`` — ``[{"id": "...", "name": "..."}]``. + Items must have a non-empty ``id``; ``name`` defaults to ``id`` when + omitted. Extra fields are preserved verbatim so future group attributes + (roles, labels, …) can be mocked without touching this parser. + + Returns ``[]`` on missing/empty/malformed input — dev mock must never + break the dev flow. Malformed input is logged at WARNING. + + Cached single-slot: re-parses only when the raw env-var value changes. + """ + global _LOCAL_DEV_GROUPS_CACHE + raw = os.environ.get("LOCAL_DEV_GROUPS", "").strip() + if _LOCAL_DEV_GROUPS_CACHE is not None and _LOCAL_DEV_GROUPS_CACHE[0] == raw: + return _LOCAL_DEV_GROUPS_CACHE[1] + result = _parse_local_dev_groups(raw) + _LOCAL_DEV_GROUPS_CACHE = (raw, result) + return result + + +def _parse_local_dev_groups(raw: str) -> list[dict]: + if not raw: + return [] + try: + parsed = json.loads(raw) + except json.JSONDecodeError as e: + logger.warning("LOCAL_DEV_GROUPS is not valid JSON, ignoring: %s", e) + return [] + if not isinstance(parsed, list): + logger.warning( + "LOCAL_DEV_GROUPS must be a JSON array, got %s — ignoring", + type(parsed).__name__, + ) + return [] + out: list[dict] = [] + for item in parsed: + if not isinstance(item, dict) or not item.get("id"): + logger.warning( + "LOCAL_DEV_GROUPS item must be an object with 'id', skipping: %r", + item, + ) + continue + # Don't mutate the parsed input — keeps the parser pure so the cache + # value stays a fresh list on each rebuild. + out.append({**item, "name": item.get("name") or item["id"]}) + return out + + def _get_db(): conn = get_system_db() try: @@ -77,6 +135,22 @@ async def get_current_user( if is_local_dev_mode(): user = _get_local_dev_user(conn) if user: + # Mirror the Google OAuth callback (app/auth/providers/google.py:189-194) + # which writes session.google_groups on every login — including [] on + # failure — so group-aware code paths see authoritative state. We + # match that semantics here while skipping the write when nothing + # would change: same-value updates are a no-op, and the write on + # PAT/CLI requests with no prior session + no target is also skipped + # (target → [], existing → None/[], no transition to record). + if request is not None and hasattr(request, "session"): + target_groups = get_local_dev_groups() + current = request.session.get("google_groups") + if target_groups and current != target_groups: + request.session["google_groups"] = target_groups + elif not target_groups and current: + # Clear stale groups if the operator unsets LOCAL_DEV_GROUPS + # mid-session — matches production's "always-write" semantics. + request.session["google_groups"] = [] return user # Fall through to normal auth if seed missing — surfaces the bug instead of hiding it. diff --git a/app/main.py b/app/main.py index 056c5fd..52a63a7 100644 --- a/app/main.py +++ b/app/main.py @@ -148,11 +148,31 @@ def create_app() -> FastAPI: # LOCAL_DEV_MODE: bypass authentication for local development. DO NOT enable in prod. # When on, every protected route auto-logs in as a seeded admin user (default dev@localhost). - from app.auth.dependencies import is_local_dev_mode, get_local_dev_email + from app.auth.dependencies import ( + is_local_dev_mode, get_local_dev_email, get_local_dev_groups, + ) if is_local_dev_mode(): logger.warning("=" * 60) logger.warning("LOCAL_DEV_MODE is ON — authentication is bypassed.") logger.warning("All requests auto-authenticate as: %s", get_local_dev_email()) + # Validate + report LOCAL_DEV_GROUPS at startup so a malformed JSON + # value gets surfaced loudly here instead of silently warning on the + # first authenticated request. Empty when unset is fine — just say so. + raw_groups_env = os.environ.get("LOCAL_DEV_GROUPS", "").strip() + mocked_groups = get_local_dev_groups() + if raw_groups_env and not mocked_groups: + logger.warning( + "LOCAL_DEV_GROUPS is set but produced no valid groups — " + "check the WARNING above for the parse error.", + ) + elif mocked_groups: + logger.warning( + "LOCAL_DEV_GROUPS: mocking %d group(s) into session: %s", + len(mocked_groups), + ", ".join(g["id"] for g in mocked_groups), + ) + else: + logger.warning("LOCAL_DEV_GROUPS is unset — session.google_groups will be empty.") logger.warning("NEVER enable this in a deployment reachable from the internet.") logger.warning("=" * 60) diff --git a/docker-compose.local-dev.yml b/docker-compose.local-dev.yml index f4425bd..96aaa18 100644 --- a/docker-compose.local-dev.yml +++ b/docker-compose.local-dev.yml @@ -19,6 +19,12 @@ services: - DATA_DIR=/data - LOCAL_DEV_MODE=1 - LOCAL_DEV_USER_EMAIL=dev@localhost + # LOCAL_DEV_GROUPS — JSON array of {id, name} mocking session.google_groups. + # Bare passthrough from the shell; scripts/run-local-dev.sh seeds a sensible + # default (engineers + admins on example.com). Set/unset in your shell to + # override. Same shape as Google's Cloud Identity searchTransitiveGroups + # response, so group-aware code paths exercised in dev match production. + - LOCAL_DEV_GROUPS - SERVER_URL=http://localhost:8000 - LOG_LEVEL=info diff --git a/docs/auth-groups.md b/docs/auth-groups.md index 8b7d0eb..f0b05e1 100644 --- a/docs/auth-groups.md +++ b/docs/auth-groups.md @@ -45,6 +45,20 @@ Display: `app/web/templates/profile.html` reads `session.google_groups` and rend **Refresh.** A user's stale session keeps stale groups. `Logout → sign in again` is the only refresh. +## Local-dev mock (no Google round-trip) + +When developing on `localhost` with `LOCAL_DEV_MODE=1`, Google OAuth never runs, so `session.google_groups` would normally stay empty and group-aware UI/code paths can't be exercised. Set `LOCAL_DEV_GROUPS` to inject a mocked membership list: + +```bash +export LOCAL_DEV_GROUPS='[{"id":"engineers@example.com","name":"Engineering"},{"id":"admins@example.com","name":"Admins"}]' +``` + +The value is a JSON array of objects matching the production shape (`{"id", "name"}`) so the mock and the real callback write the *same* structure into `session.google_groups`. Extra fields are preserved verbatim — handy for forward-compat testing of group attributes Google may return later. + +`get_current_user` in `app/auth/dependencies.py` writes the parsed list into the session on every dev-bypass request (compare-then-write — no spurious `Set-Cookie` when the value is unchanged). Malformed input (invalid JSON, non-list, items missing `id`) is logged at WARNING and falls back to `[]` — the dev mock must never break the dev flow. + +`docker-compose.local-dev.yml` carries a commented example at the right escape level for Compose YAML. **Never set this in production** — the variable is only honored when `LOCAL_DEV_MODE=1`. + ## Debugging `scripts/debug/probe_google_groups.py` — stdlib, takes a Playground-issued OAuth access token + email, hits 6 candidate endpoints, prints raw response. Use this **before** changing the production query — saves a deploy cycle per attempt. diff --git a/docs/local-development.md b/docs/local-development.md new file mode 100644 index 0000000..b658d53 --- /dev/null +++ b/docs/local-development.md @@ -0,0 +1,106 @@ +# Local Development + +Single source of truth for working on Agnes against `localhost`. Covers the dev-mode auth bypass, mocked Google Workspace groups, what isn't mocked, and the safety rails that keep the dev shortcuts off production. + +## TL;DR + +```bash +make local-dev +``` + +Then open . You land on `/dashboard` already logged in as `dev@localhost` (role `admin`) and your `/profile` shows two mocked Workspace groups. No login screen, no `.env` file, no SMTP, no GCP project — just code. + +What `make local-dev` actually does: + +- Stacks three Compose files: `docker-compose.yml` (base) + `docker-compose.override.yml` (hot-reload + source bind mount) + `docker-compose.local-dev.yml` (LOCAL_DEV_MODE overlay). +- Seeds `LOCAL_DEV_GROUPS` with a sensible default (engineers + admins on `example.com`) so `/profile` is non-empty on first boot. +- Touches an empty `.env` if missing — Compose validates `env_file:` paths even for services that never start, and the local-dev overlay drops the env-file requirement for the services that do. + +`make local-dev-down` stops the stack; `make local-dev-logs` tails it. + +## What `LOCAL_DEV_MODE=1` actually bypasses + +The local-dev overlay sets `LOCAL_DEV_MODE=1`, which flips four switches: + +1. **Auth bypass.** `app/auth/dependencies.py::get_current_user` short-circuits to a seeded admin user (`dev@localhost` by default; override via `LOCAL_DEV_USER_EMAIL`) before any token check runs. Every protected route — REST and HTML — auto-authenticates. +2. **Magic-link emails skip SMTP.** When the email-link auth provider is exercised in dev, the link is logged to stderr and returned in the response body instead of sent over wire. No mail server, no inbox. +3. **Secrets self-seed.** `JWT_SECRET_KEY` and `SESSION_SECRET` auto-generate into `/data/state/` on first boot if not provided. You don't need to manage them manually. +4. **No `.env` requirement.** The overlay declares `env_file: []` on the affected services, so the project-level `.env` doesn't need to exist. Everything dev-relevant is inline in `docker-compose.local-dev.yml`. + +A loud warning banner is logged at startup when `LOCAL_DEV_MODE=1`: + +``` +============================================================ +LOCAL_DEV_MODE is ON — authentication is bypassed. +All requests auto-authenticate as: dev@localhost +LOCAL_DEV_GROUPS: mocking 2 group(s) into session: local-dev-engineers@example.com, local-dev-admins@example.com +NEVER enable this in a deployment reachable from the internet. +============================================================ +``` + +If you don't see that banner at boot, dev mode isn't on — check `LOCAL_DEV_MODE=1` made it into the container's env. + +## Mocking Google Workspace groups + +`/profile` and any future group-aware code path read `session.google_groups`. In production that field gets populated by the OAuth callback (`app/auth/providers/google.py`) from a Cloud Identity `searchTransitiveGroups` call. In dev there's no OAuth round-trip, so the field stays empty unless we mock it. + +`LOCAL_DEV_GROUPS` is a JSON array of objects matching the production shape: + +```bash +export LOCAL_DEV_GROUPS='[{"id":"engineers@example.com","name":"Engineering"},{"id":"admins@example.com","name":"Admins"}]' +``` + +The values flow into `session.google_groups` on every dev-bypass request, so group-aware code sees something realistic. Same `{id, name}` shape the OAuth callback writes. + +### How `make local-dev` seeds it + +`scripts/run-local-dev.sh` sets a default if you haven't already (engineers + admins on `example.com`), so first-boot is non-empty. Three ways to control it: + +```bash +make local-dev # default mock — engineers + admins +LOCAL_DEV_GROUPS='[{"id":"qa@x.com","name":"QA"}]' make local-dev # custom mock +LOCAL_DEV_GROUPS= make local-dev # empty — exercise the no-groups path +``` + +### Verifying the mock + +Two checks: + +1. **Boot banner** logs the parsed group IDs (or warns loudly if the JSON is malformed): + ``` + LOCAL_DEV_GROUPS: mocking 2 group(s) into session: local-dev-engineers@example.com, local-dev-admins@example.com + ``` + A typo (e.g. unbalanced bracket) shows up here — not silently on the first authenticated request. + +2. **`/profile`** renders the mocked groups in a list. If you set `LOCAL_DEV_GROUPS=` (empty), you'll see *"No Google groups available"*. + +### Edge case: clearing stale groups mid-session + +If you previously had `LOCAL_DEV_GROUPS` set, then unset it and made a request, the dev-bypass path now writes `[]` into the session — same semantics as the production OAuth callback, which always rewrites `session.google_groups` on each login. You won't get stuck looking at stale mocked groups after toggling the env var. + +## What's NOT mocked + +`LOCAL_DEV_MODE` is intentionally narrow. These still need real configuration if you exercise them: + +- **Cloud Identity API.** No real call ever fires in dev. `LOCAL_DEV_GROUPS` populates `session.google_groups` directly without going through `_fetch_google_groups`. To debug the actual API call, use `scripts/debug/probe_google_groups.py` against a real OAuth token. +- **Real OAuth round-trip.** Google login button is hidden / no-op in dev mode. To test the full OAuth flow, follow `docs/auth-google-oauth.md` and unset `LOCAL_DEV_MODE`. +- **Admin Workspace permissions.** The mocked groups are not authoritative — they live only in your browser session. They don't grant any real access to anything outside Agnes; they let you exercise group-aware code paths inside the app. +- **PAT (Personal Access Token) flow.** PATs work normally in dev mode; the dev bypass only short-circuits cookie/session auth. Token-bearer requests still hit the JWT validation path. + +## Security model + +`LOCAL_DEV_MODE=1` is a footgun by design — every protected route auto-authenticates as admin without any check. The codebase has these rails to keep it from leaking into prod: + +- **`docker-compose.local-dev.yml` is a separate overlay**, never stacked into `docker-compose.prod.yml`. Production deployments never see it. +- **The startup banner is loud and unmissable** — `WARNING` level, repeated 60-character separator. Anyone reading container logs at startup will spot it immediately. +- **`is_local_dev_mode()` reads `os.environ` fresh on every call** — no startup-time cache that could be poisoned. +- **`LOCAL_DEV_GROUPS` is honored only inside the `if is_local_dev_mode():` block** in `get_current_user`. Setting it without `LOCAL_DEV_MODE=1` does nothing. + +If you ever see the dev banner in a real deployment's logs, treat it as a P0 incident: the auth boundary is gone. + +## Cross-links + +- [`docs/auth-groups.md`](auth-groups.md) — production Google Workspace groups: GCP setup checklist, the `security` label gotcha, debugging the real Cloud Identity call. +- [`docs/auth-google-oauth.md`](auth-google-oauth.md) — full Google OAuth setup for non-dev environments (client ID, scopes, redirect URIs). +- [`docs/QUICKSTART.md`](QUICKSTART.md) — first-time setup for a real (non-dev) instance. +- [`CLAUDE.md`](../CLAUDE.md) — repo-wide engineering conventions (changelog discipline, vendor-agnostic OSS rules, project structure). diff --git a/pyproject.toml b/pyproject.toml index c3f99a7..0b2da7e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "agnes-the-ai-analyst" -version = "0.11.1" +version = "0.11.2" description = "Agnes — AI Data Analyst platform for AI analytical systems" requires-python = ">=3.11,<3.14" license = "MIT" diff --git a/scripts/run-local-dev.sh b/scripts/run-local-dev.sh index dcdc861..2a230a5 100755 --- a/scripts/run-local-dev.sh +++ b/scripts/run-local-dev.sh @@ -18,6 +18,23 @@ if [[ ! -f .env ]]; then touch .env fi +# Default LOCAL_DEV_GROUPS so /profile and group-aware code see *something* on +# first boot. Operators can override (LOCAL_DEV_GROUPS='[...]' make local-dev) +# or disable (LOCAL_DEV_GROUPS= make local-dev). See docs/local-development.md. +# +# Indirection through DEFAULT_LOCAL_DEV_GROUPS dodges the parameter-expansion +# gotcha where a literal `}` inside `${VAR:=default}` closes the expansion +# early — silently truncating the JSON to the first group and producing an +# unparseable value. The single-quoted variable holds the JSON intact. +# +# `${VAR-DEFAULT}` (no `:`) substitutes only when VAR is *unset*, not when it +# is set-but-empty. The empty-value path is documented as the "disable" knob — +# `LOCAL_DEV_GROUPS= make local-dev` must reach the parser as "" so the +# get_local_dev_groups() short-circuit returns []. The `:-` form would +# silently substitute the default on empty, breaking that contract. +DEFAULT_LOCAL_DEV_GROUPS='[{"id":"local-dev-engineers@example.com","name":"Local Dev Engineers"},{"id":"local-dev-admins@example.com","name":"Local Dev Admins"}]' +export LOCAL_DEV_GROUPS="${LOCAL_DEV_GROUPS-$DEFAULT_LOCAL_DEV_GROUPS}" + exec docker compose \ -f docker-compose.yml \ -f docker-compose.override.yml \ diff --git a/tests/test_auth_providers.py b/tests/test_auth_providers.py index d7d36c0..c3f53a1 100644 --- a/tests/test_auth_providers.py +++ b/tests/test_auth_providers.py @@ -235,6 +235,151 @@ class TestGoogleGroupsFetch: assert groups == [] +class TestLocalDevGroupsParser: + """Unit tests for get_local_dev_groups() — must tolerate every malformed + input shape (typos, wrong type, missing id) and never raise. Bad input + becomes [] + a WARNING log so the dev mock can't break the dev flow.""" + + def test_returns_empty_when_unset(self, monkeypatch): + from app.auth.dependencies import get_local_dev_groups + monkeypatch.delenv("LOCAL_DEV_GROUPS", raising=False) + assert get_local_dev_groups() == [] + + def test_returns_empty_when_blank(self, monkeypatch): + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv("LOCAL_DEV_GROUPS", " ") + assert get_local_dev_groups() == [] + + def test_parses_valid_json_array(self, monkeypatch): + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv( + "LOCAL_DEV_GROUPS", + '[{"id":"eng@x.com","name":"Engineering"},' + '{"id":"admins@x.com","name":"Admins"}]', + ) + assert get_local_dev_groups() == [ + {"id": "eng@x.com", "name": "Engineering"}, + {"id": "admins@x.com", "name": "Admins"}, + ] + + def test_defaults_name_to_id(self, monkeypatch): + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv("LOCAL_DEV_GROUPS", '[{"id":"eng@x.com"}]') + assert get_local_dev_groups() == [{"id": "eng@x.com", "name": "eng@x.com"}] + + def test_preserves_extra_fields(self, monkeypatch): + """Forward-compat: unknown fields like roles/labels survive parsing + so future group-aware code can be exercised in dev without parser changes.""" + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv( + "LOCAL_DEV_GROUPS", + '[{"id":"eng@x.com","name":"Eng","roles":["MEMBER","OWNER"]}]', + ) + result = get_local_dev_groups() + assert result == [ + {"id": "eng@x.com", "name": "Eng", "roles": ["MEMBER", "OWNER"]}, + ] + + def test_returns_empty_on_invalid_json(self, monkeypatch): + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv("LOCAL_DEV_GROUPS", "not-json,foo") + assert get_local_dev_groups() == [] + + def test_returns_empty_on_non_list(self, monkeypatch): + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv("LOCAL_DEV_GROUPS", '{"id":"eng@x.com"}') + assert get_local_dev_groups() == [] + + def test_skips_items_without_id(self, monkeypatch): + """Bad items are dropped, valid siblings survive — partial config + still produces something useful instead of nuking the whole list.""" + from app.auth.dependencies import get_local_dev_groups + monkeypatch.setenv( + "LOCAL_DEV_GROUPS", + '[{"name":"no-id"},{"id":"eng@x.com","name":"Eng"},"string-not-object"]', + ) + assert get_local_dev_groups() == [{"id": "eng@x.com", "name": "Eng"}] + + +class TestLocalDevGroupsInjection: + """End-to-end: with LOCAL_DEV_MODE=1 + LOCAL_DEV_GROUPS, the seeded dev + user's session.google_groups gets populated on first authenticated request + so /profile renders the mocked groups.""" + + @pytest.fixture + def dev_client(self, tmp_path, monkeypatch): + monkeypatch.setenv("DATA_DIR", str(tmp_path)) + monkeypatch.setenv("JWT_SECRET_KEY", "test-secret-32chars-minimum!!!!!") + monkeypatch.setenv("SESSION_SECRET", "test-session-secret-32chars-minimum!!") + monkeypatch.setenv("LOCAL_DEV_MODE", "1") + monkeypatch.setenv("LOCAL_DEV_USER_EMAIL", "dev@localhost") + monkeypatch.setenv( + "LOCAL_DEV_GROUPS", + '[{"id":"local-dev-engineers@example.com","name":"Local Dev Engineers"}]', + ) + from app.main import create_app + return TestClient(create_app()) + + def test_dev_user_sees_mocked_groups_on_profile(self, dev_client): + resp = dev_client.get("/profile") + assert resp.status_code == 200 + body = resp.text + assert "local-dev-engineers@example.com" in body + assert "Local Dev Engineers" in body + assert "No Google groups available" not in body + + def test_empty_LOCAL_DEV_GROUPS_falls_back_to_empty_state( + self, tmp_path, monkeypatch + ): + monkeypatch.setenv("DATA_DIR", str(tmp_path)) + monkeypatch.setenv("JWT_SECRET_KEY", "test-secret-32chars-minimum!!!!!") + monkeypatch.setenv("LOCAL_DEV_MODE", "1") + monkeypatch.delenv("LOCAL_DEV_GROUPS", raising=False) + from app.main import create_app + client = TestClient(create_app()) + resp = client.get("/profile") + assert resp.status_code == 200 + assert "No Google groups available" in resp.text + + +class TestLocalDevGroupsStartupValidation: + """Startup banner reports on LOCAL_DEV_GROUPS so a typo or malformed JSON + is loud at boot, not silent until the first authenticated request.""" + + def _capture_startup_logs(self, tmp_path, monkeypatch, caplog, env_value): + import logging + monkeypatch.setenv("DATA_DIR", str(tmp_path)) + monkeypatch.setenv("JWT_SECRET_KEY", "test-secret-32chars-minimum!!!!!") + monkeypatch.setenv("LOCAL_DEV_MODE", "1") + if env_value is None: + monkeypatch.delenv("LOCAL_DEV_GROUPS", raising=False) + else: + monkeypatch.setenv("LOCAL_DEV_GROUPS", env_value) + from app.main import create_app + with caplog.at_level(logging.WARNING, logger="app.main"): + create_app() + return caplog.text + + def test_logs_count_and_ids_on_valid_input(self, tmp_path, monkeypatch, caplog): + text = self._capture_startup_logs( + tmp_path, monkeypatch, caplog, + '[{"id":"a@x.com","name":"A"},{"id":"b@x.com","name":"B"}]', + ) + assert "mocking 2 group(s)" in text + assert "a@x.com" in text + assert "b@x.com" in text + + def test_warns_when_set_but_malformed(self, tmp_path, monkeypatch, caplog): + text = self._capture_startup_logs( + tmp_path, monkeypatch, caplog, "not-valid-json", + ) + assert "produced no valid groups" in text + + def test_logs_unset_explicitly(self, tmp_path, monkeypatch, caplog): + text = self._capture_startup_logs(tmp_path, monkeypatch, caplog, None) + assert "LOCAL_DEV_GROUPS is unset" in text + + class TestCookieAuth: def test_web_ui_with_cookie(self, client): """Test that web UI routes accept JWT from cookie."""