agnes-the-ai-analyst/app/auth/providers/email.py
Vojtech 2e2e1a1eca
feat(home): state-aware /home + /setup-advanced + schema v26 (#228)
* feat(home+news): state-aware /home + /news + admin-edited news section

Squash of the vr/home-page feature work for clean rebase onto main.
Original 18-commit history preserved in branch backup/vr-home-page-pre-rebase.

What's in this PR:

**State-aware /home page**
- New `/home` route with hero + auto-mode + connectors (Asana / GWS /
  Atlassian) + lookarounds. Onboarded vs not-onboarded state-machine
  branches a single template (`home_not_onboarded.html`); the install
  steps, "Setup a new Claude Code" CTA (90-day PAT mint), and per-
  connector setup prompts hide once `users.onboarded=TRUE`. A
  completion badge replaces them.
- "Mark me as offboarded" button reverses the flag without an SQL UPDATE.
- `users.onboarded BOOLEAN` column added; default FALSE; flipped by the
  CLI's `agnes init` post-success POST and the `/admin/users` API.
- Connector setup prompts pre-check whether the tool is already
  installed/connected before re-running setup.
- GWS scope set widened to include Google Chat (`chat.spaces`,
  `chat.messages`).

**Single template + design tokens**
- `dashboard.html` now extends `base.html` via the new
  `{% block layout %}` opt-out (full-width pages skip the 800px
  `.container`). Net: every page shares one shell.
- `style-custom.css` `:root` extended with `--space-{7,9,10,12}`,
  `--radius-2xl`, `--shadow-{card,elevated}`, `--text-{muted,disabled}`,
  `--focus-ring`, `--transition-*`, `--width-{narrow,app,wide}` so
  inline page styles can migrate incrementally.

**Auth redirects honor AGNES_HOME_ROUTE**
- `safe_next_path` resolves the configured home route when no `default=`
  is passed; OAuth callbacks, magic-link clicks, password form, and
  LOCAL_DEV_MODE shortcuts now land on `/home` (or whatever the operator
  picked) instead of always /dashboard.

**News section + /news permalink + /admin/news editor**
- Schema-bumped `news_template` table (single versioned entity, draft +
  publish gate). `published BOOLEAN` distinguishes draft from public;
  monotonically-increasing `version` per save; rows >30d pruned on
  save except the currently-displayed published version.
- `/home` bottom-of-page renders the latest published intro with a
  "Read more →" link to `/news` (which renders the full body).
- `/admin/news` editor with sandboxed live preview, versions table,
  per-row Unpublish, Format-help cheatsheet.
- `agnes admin news show / draft / edit / publish / unpublish /
  versions / export` (CLI). Talks to the live server via the
  `/api/admin/news/*` endpoints (PAT-authed) — no direct DB access
  so it coexists with a running uvicorn.
- **Optimistic-lock guard**: `agnes admin news publish --version N` and
  PUT/PATCH endpoints accept `expected_version` and 409 with structured
  `{error: "version_conflict", expected, actual, actual_by}` when a
  concurrent admin replaced the draft. Edit refuses to overwrite a
  draft authored by someone else without `--force` or
  `--expect-version`.
- nh3 (Rust-backed ammonia) HTML sanitizer; iframe pre-pass strips
  any iframe whose src is not on the YouTube/Vimeo/Loom allowlist;
  javascript:/data: schemes blocked everywhere.
- Author CSS vocabulary: `.news-hero` (blue gradient hero block),
  `.callout`/`.callout-{info,warn,success,danger}`,
  `.video-embed`, `.news-section`, `.news-grid-{2,3}`, `.news-cta` —
  all consolidated in `style-custom.css` under "News content
  vocabulary (shared)" so /home perex, /news body, and /admin/news
  preview share one source of styling.
- Code-inside-`<pre>` contrast fix (was unreadable amber-on-silver).
- `.news-content` table styling (border, header band, row-hover).

**`scripts/dev/run-local.sh`** — local uvicorn launcher. Pulls Google
OAuth client id/secret from GCP Secret Manager
(`AGNES_OAUTH_GCP_PROJECT`-driven, no vendor defaults), points
`AGNES_CLI_DIST_DIR` at `./dist` so the wheel endpoint resolves, and
`--dev` flips `LOCAL_DEV_MODE=1` + `AGNES_HOME_ROUTE=/home` for one-
command iteration. `LOCAL_DEV_MODE=1` also enables the FastAPI debug
toolbar.

**CLAUDE.md "Run tests before every push" section** codifies
`pytest tests/ -n auto -q` as non-negotiable before each push.

**Tests**: 51 + 14 + 8 = 73 new tests across news-template repo,
sanitizer, API, web, CLI; plus updated home/auth/template tests for
the new shared-shell architecture.

Origin docs (gitignored, customer-fork content):
docs/brainstorms/home-page-requirements.md,
docs/plans/2026-05-07-001-feat-home-page-plan.md.

* feat(cli): agnes onboarded {on,off,status} — self-scoped flag toggle

User-facing equivalent of the in-page "Mark me as (off)boarded" button
on /home. POSTs /api/me/onboarded with {onboarded, source}; --source
overrides the audit-log marker so flips made from the CLI vs the web
button vs agnes init automation stay distinguishable.

`status` reads via /api/me/profile (when present); falls back to a
quick body-marker scan of /home so the read path doesn't write an
audit_log row. PAT-authed via cli.client.api_post — same convention
as agnes admin news / agnes admin add-user etc.

Tests: 5 covering on/off/status round-trip, idempotency, and
audit-log source recording. Full suite holds at 12 pre-existing
failures (same set as before).

* ui(nav+home): primary nav reorg + green What's new band + /marketplace link fix

Primary nav (post-rebase audit + per-user feedback):

- Items: Home → Marketplace → Data Packages → Memory. Admin dropdown
  for admins only. The "Dashboard" label was renamed Home — point still
  resolves through `home_route` so customer instances on /dashboard
  still land there.
- Activity Center moved into the Admin dropdown. Per-team adoption
  analytics is admin-consumed in practice; the route still allows
  any authed user for direct deep-links so existing /home tile +
  bookmarks keep working.
- Memory link added (→ /corporate-memory) — was previously buried in
  the /home "Look around" tiles.
- Setup local agent + My Stack dropped from main nav. Setup is the
  /home install flow's home now; My Stack lives as a tab inside
  /marketplace.

/home tweaks:

- Plugin marketplace tile now points at /marketplace (was /store —
  legacy from before the marketplace rebrand landed in #230).
- "What's new" section header gets a green band (success-flavored
  D1FAE5 background, A7F3D0 border, darker green title) so the
  bottom-of-page news block visibly distinguishes from the blue
  install-hero at the top. Header strip only — body stays white.

Test fix: test_home_route_resolution renamed `dashboard_link_uses_home_route`
→ `home_link_uses_home_route` and asserts `href="/home">Home` instead
of `href="/home">Dashboard` after the label change.

* fix(home): decouple Step 3 + Connect-tools collapse from server onboarded flag

The server-side `users.onboarded` flip happens through two paths:

1. Explicit user click on "Mark me as onboarded" or `agnes onboarded on`.
2. Implicit `agnes init` POST → /api/me/onboarded on success.

Path 2 produced a UX surprise: an analyst running `agnes init` mid-flow
reloaded /home and saw Step 3 (auto-mode) + Connect-your-tools auto-
collapse to summary bars. They were actively working through those
sections — the install POST never signalled "I'm done with the rest
of setup", just "Agnes itself is installed".

Decouple the section-collapse decision from the server flag:

- Step 1 + Step 2 install blocks: still hidden on `onboarded=TRUE`
  (their completion is a hard server signal — Agnes IS installed).
- Step 3 + Connect-your-tools: render flat by default in BOTH states.
  Wrapped in `<details class="setup-collapsible" open>` so the
  browser's native disclosure handles per-section toggle without JS,
  but the `<summary>` is CSS-hidden until the page-level
  `data-setup-minimized="1"` attribute is set on `.home-mock`.
- New "Minimize setup view" toggle inside the blue install-hero,
  rendered only when onboarded. Click flips the data-attr on
  `.home-mock` AND removes the `open` attribute from each
  `<details>`. State persists in `localStorage["agnes_home_setup_minimized"]`
  so the choice survives reloads but is per-device.
- "Show full setup view" (the same button when minimized) re-opens
  both `<details>` and clears localStorage.

When minimized, each `<details>` still has its own native expand/
collapse — click the gray summary bar to peek at one section without
toggling the page-level minimize off.

Tests:
- test_step3_and_connectors_render_flat_when_onboarded_by_default —
  asserts `<details class="setup-collapsible" ... open>` for both
  sections post-onboarding and the absence of any server-rendered
  `data-setup-minimized` attribute on the `.home-mock` root.
- test_minimize_toggle_visible_only_when_onboarded — toggle button
  rendered only when onboarded.

Full pytest holds at 12 pre-existing failures (same set).
2026-05-08 18:28:47 +02:00

269 lines
10 KiB
Python

"""Email magic link auth provider for FastAPI."""
import logging
import os
import secrets
from datetime import datetime, timedelta, timezone
from urllib.parse import quote
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import RedirectResponse
from pydantic import BaseModel
import duckdb
from app.auth.jwt import create_access_token
from app.auth.access import is_user_admin
from app.auth.dependencies import _get_db, is_local_dev_mode
from app.auth.rate_limit import limiter as _rate_limiter
from src.repositories.users import UserRepository
def _role_label(user: dict, conn: duckdb.DuckDBPyConnection) -> str:
"""Display label for the response payload only — `admin` if the user is
in the Admin system group, otherwise `user`. Authorization at runtime
checks `is_user_admin` directly; this label is purely cosmetic."""
return "admin" if is_user_admin(user["id"], conn) else "user"
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/auth/email", tags=["auth"])
MAGIC_LINK_EXPIRY = 3600 # 1 hour
class MagicLinkRequest(BaseModel):
email: str
class MagicLinkVerify(BaseModel):
email: str
token: str
def is_available() -> bool:
# In dev mode the link is rendered to logs + response, so the provider is "available"
# even without SMTP/SendGrid. Keeps the login UI showing the magic-link option.
if is_local_dev_mode():
return True
return bool(os.environ.get("SMTP_HOST") or os.environ.get("SENDGRID_API_KEY"))
def _has_email_transport() -> bool:
return bool(os.environ.get("SMTP_HOST") or os.environ.get("SENDGRID_API_KEY"))
def _build_magic_link(email: str, token: str) -> str:
# URL-encode email: a literal '+' in a query string decodes to space per
# application/x-www-form-urlencoded, which would break addresses like
# "user+tag@gmail.com" on the GET /verify side.
server_url = os.environ.get("SERVER_URL", "http://localhost:8000")
return f"{server_url}/auth/email/verify?email={quote(email, safe='')}&token={token}"
@router.post("/send-link")
@_rate_limiter.limit("5/minute")
async def send_magic_link(
request: Request,
body: MagicLinkRequest,
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
):
"""Send a magic link to the user's email.
When SMTP/SendGrid is not configured, or LOCAL_DEV_MODE=1, the link is
logged to stderr and returned in the response body so a developer can
click it without an email transport.
"""
repo = UserRepository(conn)
user = repo.get_by_email(body.email)
# Always return success to prevent email enumeration
if not user:
return {"message": "If this email is registered, you will receive a login link."}
# Generate token
token = secrets.token_urlsafe(32)
repo.update(
id=user["id"],
reset_token=token,
reset_token_created=datetime.now(timezone.utc),
)
link = _build_magic_link(body.email, token)
send_error: str | None = None
if _has_email_transport():
try:
_send_email(body.email, token)
except Exception as e:
send_error = str(e)
logger.error("Failed to send magic link email to %s: %s", body.email, e)
# Dev fallback: expose the link in logs + response so you can click it without SMTP.
# Scoped strictly to LOCAL_DEV_MODE so test and production behavior are unchanged.
if is_local_dev_mode():
logger.warning("=" * 60)
logger.warning("Magic link for %s (LOCAL_DEV_MODE fallback):", body.email)
logger.warning(" %s", link)
logger.warning("=" * 60)
response: dict = {
"message": "Magic link generated (LOCAL_DEV_MODE) — click dev_link to log in.",
"dev_link": link,
}
if send_error:
response["send_error"] = send_error
return response
return {"message": "If this email is registered, you will receive a login link."}
def _consume_token(conn: duckdb.DuckDBPyConnection, email: str, token: str) -> dict:
"""Validate & consume a magic-link token atomically. Returns the user dict or raises 401.
Uses a "compare-and-swap" pattern: instead of setting reset_token to NULL
directly, we first set it to a unique CONSUMED marker that identifies THIS
consumption attempt, then verify that OUR marker was written. Two concurrent
verifies will both try to write their marker, but only one will succeed
(the WHERE clause checks the original token value); the loser's UPDATE is
a no-op, and the loser sees the winner's marker and fails.
DuckDB doesn't expose affected-row count, so the marker is the only way
to distinguish "I won the race" from "someone else won."
"""
# Compute the TTL cutoff in Python — DuckDB doesn't support
# parameterized INTERVAL arithmetic (?, INTERVAL) in all builds.
cutoff = datetime.now(timezone.utc) - timedelta(seconds=MAGIC_LINK_EXPIRY)
# Unique marker for this consumption attempt — lets us detect who won
# the race without relying on DuckDB rowcount (which returns -1).
consume_id = f"CONSUMED:{secrets.token_hex(16)}"
# Step 1: Atomic compare-and-swap. Only succeeds if the token still
# matches the original value and hasn't expired. On success, writes
# OUR consume_id instead of NULL so we can verify ownership.
# DuckDB raises TransactionContext Error on concurrent row conflicts —
# catch and treat as "someone else won the race."
try:
conn.execute(
"UPDATE users SET reset_token = ?, reset_token_created = NULL "
"WHERE email = ? AND reset_token = ? AND reset_token_created IS NOT NULL "
"AND reset_token_created >= ?",
[consume_id, email, token, cutoff],
)
except Exception as exc:
err = str(exc).lower()
if "conflict" in err or "transaction" in err:
raise HTTPException(status_code=401, detail="Invalid or expired link")
raise
# Step 2: Verify that OUR consume_id was written. If a concurrent
# request won the race, we'll see THEIR consume_id (or NULL if they
# already cleared it in step 3) — either way, we fail.
row = conn.execute(
"SELECT reset_token FROM users WHERE email = ?",
[email],
).fetchone()
if not row or row[0] != consume_id:
raise HTTPException(status_code=401, detail="Invalid or expired link")
# Step 3: Clear the consumed marker. Safe to do unconditionally —
# only the winner reaches here, and the marker is transient.
# If this UPDATE fails (DB error), the marker persists but the user
# can still request a new magic link — not a lockout.
try:
conn.execute(
"UPDATE users SET reset_token = NULL WHERE email = ? AND reset_token = ?",
[email, consume_id],
)
except Exception:
logger.warning("Failed to clear CONSUMED marker for %s — marker will persist", email)
# Fetch the user (token is now cleared, but we need the rest of the fields).
# CAS already validated token + expiry atomically, so no further checks
# needed — re-running them now would always fail because reset_token was
# NULL'd in step 3.
repo = UserRepository(conn)
user = repo.get_by_email(email)
if not user:
raise HTTPException(status_code=401, detail="Invalid link")
return user
@router.post("/verify")
@_rate_limiter.limit("10/minute")
async def verify_magic_link(
request: Request,
body: MagicLinkVerify,
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
):
"""Verify a magic link token and issue JWT (JSON API for programmatic clients).
Rate limited 10/min per IP to slow brute-forcing the 32-byte
``reset_token`` (the same column doubles as the magic-link token).
"""
user = _consume_token(conn, body.email, body.token)
role_label = _role_label(user, conn)
jwt_token = create_access_token(user["id"], user["email"])
return {"access_token": jwt_token, "token_type": "bearer", "email": user["email"], "role": role_label}
@router.get("/verify")
@_rate_limiter.limit("10/minute")
async def verify_magic_link_get(
request: Request,
email: str,
token: str,
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
):
"""Click-through variant — verifies token, sets cookie, redirects to the
operator-configured home route.
This is the URL we embed in outgoing emails (and the dev-fallback link), so
clicking it in a mail client logs the user in without a separate API call.
Rate limited 10/min per IP for the same reason as the POST variant —
don't let the click-through path bypass the brute-force throttle.
"""
user = _consume_token(conn, email, token)
jwt_token = create_access_token(user["id"], user["email"])
# secure=False when DOMAIN is unset so the cookie is actually sent on plain HTTP (dev).
use_secure = os.environ.get("DOMAIN", "") != ""
from app.instance_config import get_home_route
response = RedirectResponse(url=get_home_route(), status_code=302)
response.set_cookie(
key="access_token", value=jwt_token,
httponly=True, max_age=86400, samesite="lax",
secure=use_secure,
)
return response
def _send_email(email: str, token: str):
"""Send magic link email via SMTP or SendGrid."""
link = _build_magic_link(email, token)
sendgrid_key = os.environ.get("SENDGRID_API_KEY")
if sendgrid_key:
import sendgrid
from sendgrid.helpers.mail import Mail
sg = sendgrid.SendGridAPIClient(api_key=sendgrid_key)
message = Mail(
from_email=os.environ.get("EMAIL_FROM_ADDRESS", "noreply@example.com"),
to_emails=email,
subject="Login Link",
html_content=f'<p>Click to login: <a href="{link}">Login</a></p>',
)
sg.send(message)
return
smtp_host = os.environ.get("SMTP_HOST")
if smtp_host:
import smtplib
from email.mime.text import MIMEText
msg = MIMEText(f"Login link: {link}")
msg["Subject"] = "Login Link"
msg["From"] = os.environ.get("SMTP_FROM", "noreply@example.com")
msg["To"] = email
with smtplib.SMTP(smtp_host, int(os.environ.get("SMTP_PORT", "587"))) as s:
if os.environ.get("SMTP_USE_TLS", "true").lower() == "true":
s.starttls()
smtp_user = os.environ.get("SMTP_USER")
if smtp_user:
s.login(smtp_user, os.environ.get("SMTP_PASSWORD", ""))
s.send_message(msg)