* fix: redirect unauthenticated HTML routes to /login (#10) * docs(plan): user mgmt + PAT + CLI distribution implementation plan (#9 #10 #11 #12) * build(docker): produce wheel artifact for /cli/download (#9) * feat(db): schema v5 — users.active + deactivated_at/by (#11) * feat(api): /cli/download wheel + /cli/install.sh with baked server URL (#9) * feat(users): repository supports active flag + count_admins (#11) * feat(ui): /install page with per-deployment install instructions (#9) * feat(api): user PATCH/reset-password/set-password/activate/deactivate (#11) * fix(cli): da login prompts for password and sends it in body (#9) * test(api): safeguard tests for self-deactivate and last admin (#11) * feat(auth): reject requests from deactivated users (#11) * fixup(#10): propagate next through /login buttons + lock down sanitizer tests * feat(cli): da admin set-role/activate/deactivate/reset-password/set-password (#11) * feat(ui): /admin/users management page (#11) * feat(db): schema v6 — personal_access_tokens (#12) * feat(users): access_tokens repository (#12) * feat(auth): JWT carries typ (session|pat) and explicit jti (#12) * feat(auth): reject revoked/expired PATs; update last_used_at (#12) * feat(api): /auth/tokens CRUD + admin revoke; session-only guard (#12) * feat(cli): da auth token create/list/revoke (#12) * feat(ui): /profile page with PAT create/list/revoke (#12) * docs: PAT usage and session/PAT TTL clarification (#12) * feat(auth): PAT first-use-from-new-IP audit + last_used_ip (schema v7) (#12) Closes remaining acceptance gap from issue #12: audit_log entry on first use of a PAT from an IP that differs from the recorded last_used_ip. - schema v7: personal_access_tokens.last_used_ip column - AccessTokenRepository.mark_used now stores the client IP - get_current_user extracts client IP (X-Forwarded-For first hop, fallback to request.client.host) and emits a token.first_use_new_ip audit when the IP changes on a subsequent use (not the very first use) - tests: new-ip audit, same-ip no-op, first-ever-use no-op, schema v7 column * fix: address Devin review findings on PR #28 - app/main.py: exclude /auth/* from HTML redirect handler so JSON endpoints under /auth/ (PAT CRUD used by `da auth token` CLI) keep their 401 JSON contract (Devin #1, bug) - app/api/tokens.py: reject expires_in_days <= 0 explicitly; use `is not None` so 0 no longer silently creates a non-expiring token (Devin #2) - app/api/users.py: validate role against Role enum in create_user to match update_user and prevent 500 on role-protected requests later (Devin #3) - app/web/templates/admin_users.html: escape user-supplied strings before innerHTML; move onclick handlers to addEventListener via data attributes so emails with quotes / HTML no longer break the UI or enable stored XSS (Devin #4) - app/auth/router.py, app/auth/providers/{password,google}.py: reject deactivated users at login instead of issuing a JWT that would then fail on the next request — removes the confusing redirect loop (Devin #5) - CLAUDE.md: document schema v7 instead of stale v4 (Devin #6) - tests/test_web_ui.py: regression test for the /auth/* JSON 401 * feat(web): add /profile and /admin/users links to dashboard nav * feat(web): point setup banner at /install page * chore(web): drop unused setup_instructions context * fix: address Devin review round 2 on PR #28 - app/api/tokens.py: when expires_in_days is None (the "never" option), use a ~100-year JWT expiry so the token doesn't silently die in 24h via the session-default fallback in create_access_token. The real expiry enforcement stays in verify_token's DB-level check (Devin 🔴) - app/web/templates/profile.html: escape t.name and other user-supplied strings via esc() helper before innerHTML, same pattern as admin_users.html. Move revoke onclick to data-attribute + addEventListener (Devin 🟡) - app/api/cli_artifacts.py: use `mktemp -d` with X's at end of template for GNU/BSD portability, place wheel inside the temp dir and clean up with rm -rf (Devin 🚩) * feat(web): redesign /install page; make curl one-liner primary, collapse manual Rebuild the public /install page using the dashboard visual language (shared header, card layout, gradient hero, design tokens from style-custom.css). The page is now anchored on the one-liner install path: curl -fsSL <server>/cli/install.sh | bash is rendered as the primary, prominent step 1, while the old manual wheel-download flow is tucked behind a closed-by-default <details> block for users in restricted/offline environments. Information architecture: hero (server URL + version) -> step 1: quick install (one-liner, big Copy button) -> step 2: create PAT on /profile + export DA_TOKEN / da auth whoami -> step 3: Claude Code / MCP via ~/.config/da/token.json -> collapsed "Manual install" details for download-wheel flow -> footer link to docs/HEADLESS_USAGE.md Every shell snippet has a vanilla-JS "Copy" button that confirms visually ("Copied!" for 1.5s) and falls back to textarea+execCommand on non-secure contexts. No new dependencies, no bundler. The route now also pulls an optional user so the header shows the same nav (Dashboard / Profile / Logout) as dashboard.html when a session exists, while staying fully public when signed out. * fix(cli): use real wheel filename in install.sh (broken pip/uv install) The installer wrote the downloaded wheel as agnes_cli.whl, which lacks a PEP-427 version component — both pip and uv tool install reject it and abort the one-liner. Use curl -OJ so Content-Disposition determines the on-disk filename, then resolve it via glob. Install an EXIT trap to remove the tmpdir even when install fails. * fix(web): correct manual install wheel glob and add PEP 668 / PATH hints - Wheel glob is agnes_the_ai_analyst-*.whl (not agnes-*.whl) — the old pattern never matched the real artefact name from the build. - Add — or — separator between uv tool install and pip install. - Warn that pip install --user is blocked on macOS Homebrew / modern Debian (PEP 668) and recommend uv tool install as the default path. - Both flows now show the ~/.local/bin PATH hint so a fresh shell can find the da binary after install. * fix(web): consistent session.user reference in install header The avatar-letter fallback inside {% if session.user %} was reading user.name / user.email directly, but the route dependency can pass user=None — those references resolved to an empty FlexDict and produced an empty avatar circle. Read everything through session.user to match the guard and the dashboard pattern. * fix(web): point headless usage link at GitHub source /docs/HEADLESS_USAGE.md 404s — no static route serves repo docs. Point the footer link at the rendered markdown on GitHub instead of adding a dedicated docs serving route just for one file. * feat(web): /install hero size, anon sign-in banner, step 2 copy polish - Bump hero h1 from 26px to 30px to match dashboard primary scale. - Anonymous visitors see a small sign-in banner above Step 2 (creating a token requires auth; without the banner the flow appears stuck). - Add an 'After generating your token' section label inside Step 2 so the /profile CTA button no longer looks wedged mid-sentence between adjacent paragraphs. * chore(web): /install a11y + version pill polish - aria-live='polite' on copy buttons so screen readers announce the 'Copied!' state change. - Replace redundant INSTANCE_NAME eyebrow (already in the header logo) with 'Getting started'. - Hide the version pill when AGNES_VERSION is unset/'dev' — avoids the misleading 'vdev' label in local/unbuilt runs. - Manual summary focus-visible outline-offset +2px (was -2px which clipped inside the card), and mark the chevron as decorative. * fix(web): use session.user in dashboard avatar fallback Inside {% if session.user %} guard, the avatar fallback referenced (user.name or user.email). If user is None the block crashes when the profile picture is absent. Align with the guard variable. * fix: address Devin review round 3 on PR #28 - app/api/users.py: stop auto-sending email from reset_password. The magic-link sender would deliver a "Login Link" that — when clicked — consumes the reset_token via verify_magic_link and logs the user in WITHOUT prompting for a new password. Admins now share the raw reset_token from the API response manually, or use set-password directly. email_sent is always False. Documented inline. (Devin 🟡) - app/api/cli_artifacts.py: harden /cli/install.sh generation against shell injection via Host header or AGNES_VERSION. base_url is validated against a strict scheme+host+port regex; version against an alnum + dot/dash/underscore allowlist. Both values are also piped through shlex.quote() as defense in depth. (Devin 🟡) The shared users.reset_token column between magic-link and password- reset flows (Devin 🚩) remains an architectural gap; splitting into separate columns needs schema v8 and is tracked for a follow-up PR. * docs, chore(grpn): manual-deploy helpers + hackathon deploy learnings Adds scripts/grpn/ — Makefile + agnes-auto-upgrade.sh + README for operating Agnes on GRPN's existing foundryai-development VM when the full Terraform flow is blocked by org policies: - iam.disableServiceAccountKeyCreation (org constraint) forbids SA JSON keys, so GCP_SA_KEY-based CI is unavailable - No projectIamAdmin delegation → bootstrap-gcp.sh can't grant roles - Secret Manager IAM bindings require setIamPolicy which editor lacks Helper targets: deploy, deploy-tag, recreate, restart, stop, start, status, version, logs, ps, env, ssh, tunnel, open, bootstrap-admin, set-data-source, install-cron, uninstall-cron. docs/superpowers/plans/2026-04-22-grpn-deploy-learnings.md — running log of all org-policy constraints hit during the hackathon deploy, with workarounds and derived follow-ups (WIF support, external_ip variable, customer onboarding IAM checklist). Not a replacement for the TF flow — stopgap until WIF lands. * fix(web): make header logos clickable links to home * feat(web): one-click "Setup a new Claude Code" button Adds a single-button flow on the dashboard and /install page that generates a fresh personal access token via POST /auth/tokens and copies a complete, paste-ready setup script (server URL, token, install/verify commands) to the clipboard. Falls back to a modal textarea when the clipboard is blocked; redirects to /login on 401; surfaces backend errors inline. - dashboard.html: replaces the top "Set up your local environment" anchor with a real button wired to setupNewClaude(). Removes the duplicate bottom setup banner to keep a single entry point. - install.html: for signed-in users, Step 1 leads with the one-click button and demotes the curl one-liner into a collapsible "Or run manually" aside. Anonymous visitors still see the curl flow plus a sign-in hint. - No new deps. Vanilla JS. Token lives in memory/clipboard only — never rendered into persistent DOM. * feat(cli): add "da auth import-token" for non-interactive PAT login Writes a provided JWT into ~/.config/da/token.json using the canonical {access_token, email, role} shape expected by save_token(). Decodes the token locally to pull email/role claims, verifies it against the server via GET /api/catalog/tables, and refuses to overwrite an existing token file if the server returns 401. --email / --role overrides exist for tokens missing those claims; --skip-verify bypasses the server round-trip for offline / CI scenarios. * test(cli): cover da auth import-token success + 401 + claim-fallback paths Three new tests in TestAuthImportToken: - valid JWT + 200 -> canonical token.json written - 401 from /api/catalog/tables -> exit 1, existing token file untouched - JWT without email/role claims -> refused without overrides, accepted with --email / --role flags * feat(web): update one-click Claude setup instructions — explicit uv install, import-token, skills question Replaces the fragile `cat > token.json <<EOF` clipboard payload with an explicit, auditable sequence: 1. `curl -fsSL /cli/download` + `uv tool install --force` (no opaque `curl | bash`). 2. `da auth import-token --token ...` instead of hand-written JSON. 3. Explicit PATH persistence for zsh/bash. 4. A required question to the user about whether to copy the bundled skills into ~/.claude/skills/agnes/ or pull them on-demand via `da skills show`. 5. A final confirmation step with whoami + version output. Factored both pages to include a shared partial (app/web/templates/_claude_setup_instructions.jinja) so dashboard.html and install.html can never drift apart again. {server_url} and {token} stay as runtime placeholders substituted by renderSetupInstructions(). * feat(ui): modernize /admin/users + unify header nav across pages - New shared partial app/web/templates/_app_header.html — single source of truth for the top navigation. Used by base.html and dashboard.html (which doesn't extend base.html). Active page highlighted via request.url.path. Admin "Users" link gated by session.user.role. - style-custom.css: add .app-header / .app-nav-link / .app-btn-logout / .app-avatar styles (mirrors dashboard's previous inline copy under app-* prefix). Mobile-friendly fallback at <720px. - base.html: include the new partial so every page extending base (admin_users, profile, login_email, error, …) gets the same chrome the dashboard has. - dashboard.html: replace its inline <header class="header"> markup with the shared partial. Inline .header CSS left in place as harmless dead code (separate cleanup PR). - admin_users.html: rewritten with avatars, role pills (color-coded per role), toggle switch for active, search/filter input, toast notifications, modal dialogs replacing alert/confirm/prompt, one-click copy for the reset token, empty / loading states. All XSS-safe via the existing esc() helper + data-attribute event delegation. - tests/test_web_ui.py: smoke test that /admin/users renders the new shared header chrome and the modernized markup. * feat(api): serve CLI wheel at /cli/agnes.whl for direct uv install uv tool install inspects the URL path suffix to recognise a wheel, so /cli/download (which has no .whl suffix) cannot be installed directly. Expose a stable /cli/agnes.whl alias over the same wheel lookup so users can run: uv tool install --force https://<server>/cli/agnes.whl * test(cli): cover da auth import-token --server persisting to config.yaml The server persistence was already implemented in the import-token command (save_config({server}) call) but not covered by tests. Add an explicit test so the one-step setup contract — single import-token call writes both token and server — cannot regress. * feat(web): simpler Claude setup — single uv install URL, single import-token call User feedback: the prior clipboard payload repeated the server URL and token across multiple steps (curl + tmpfile + install + rm + separate seed-config + import-token). Collapse to: 1. uv tool install --force {server_url}/cli/agnes.whl (single URL, direct) 2. da auth import-token --token ... --server ... (one call, persists both) 3. da auth whoami 4. skills (ask user first) 5. confirm uv accepts HTTPS URLs that end in .whl and installs them directly, so the tmpfile dance is unnecessary. import-token --server already persists the server to config.yaml, so no separate printf > config.yaml step. * fix(tests): update admin users heading assertion after template rename The admin_users.html template now uses <h2 class="users-title">Users</h2> instead of <h2>User management</h2>. Update the assertion to match. * feat(ui): unify header across remaining 7 standalone pages These 7 pages render their own full <html> and don't extend base.html, so the previous unification commit only covered base + dashboard. Each had its own ad-hoc <header> markup with inconsistent classes (.top-header / .header / .page-header), inconsistent nav-link sets, and inconsistent avatar/email styling. Replace each inline <header>...</header> block with the shared {% include '_app_header.html' %} so /activity-center, /admin/permissions, /admin/tables, /catalog, /corporate-memory, /corporate-memory/admin, and /install all show the same chrome (Dashboard / Install CLI / Profile / Users / email + avatar / Logout) with the active page highlighted via request.url.path. Old inline header CSS (.header, .top-header, .page-header, .nav-link, etc.) is left in place as harmless dead code; it can be cleaned up in a follow-up sweep. * feat(web): add readable preview of Claude setup payload on dashboard + /install Move the line-by-line setup instructions into app/web/setup_instructions.py as the single source of truth, then render them in two modes from the existing _claude_setup_instructions.jinja partial: - preview_mode=True → visible, read-only <pre><code> block with the real server URL and a clearly-styled placeholder token (never a real one). - preview_mode=False → the JS SETUP_INSTRUCTIONS_TEMPLATE used by the one-click flow (unchanged behaviour). Both /dashboard (env-setup-cta card) and /install (Step 1 card) now show the preview directly under the 'Setup a new Claude Code' button so users can see exactly what will land in their clipboard before they click. * feat(web): update setup instructions — `da diagnose` step, explicit section titles Rework the Claude Code setup payload to: - Give every numbered step an unambiguous verb header ("1) Install the CLI", "2) Log in", "3) Verify the login", "4) Run diagnostics", "5) Skills (ask the user first)", "6) Confirm"). - Add step 4 `da diagnose` as the post-login health check. The CLI already ships this command (cli/commands/diagnose.py); it prints "Overall: healthy" and a list of green checks that map cleanly to next actions. - Ask the skills copy-vs-on-demand question verbatim so Claude Code always prompts the user the same way. - Replace the terse "Confirm" line with a 4-bullet summary (version, whoami, skills choice, diagnose status) so the return message is structured and comparable across setups. * chore(web): remove stale MCP card from /install (no MCP server today) The 'Use with Claude Code / MCP' card (Step 3 on /install) referenced an MCP integration Agnes does not ship. Remove the whole card. The one-click 'Setup a new Claude Code' flow in Step 1 already covers the long-lived client use case and is less confusing than dangling persistence tips for a non-existent integration. * feat(api): include user_email + last_used_ip + user_id in admin tokens list response Adds AdminTokenItem response model (superset of TokenListItem) and AccessTokenRepository.list_all_with_user() joining personal_access_tokens with users to denormalize user_email. Needed for /admin/tokens UI where admins triage tokens across all users. * feat(web): /admin/tokens page — list, filter, search, revoke across all users Adds a new admin-only page with client-side filtering (status, user email, last-used window), column sorting, counts bar (active/revoked/expired), and an inline revoke action. Mirrors the /admin/users visual language. * feat(web): add Tokens nav link for admins + deep-link from admin/users row Admin-only nav entry to /admin/tokens, and a per-row Tokens button on /admin/users that prefills the token page's user filter via ?user=<email>. * test(admin): cover /admin/tokens rendering, filter state, non-admin denial, revoke Verifies admin can render the page (title + JS hooks present), a non-admin is blocked, unauthenticated users are redirected, the admin list response includes user_email / user_id / last_used_ip, and admin can revoke another user's token. * feat(web): modern redesign of /admin/tokens — hero, stat strip, refined table, responsive cards, a11y * feat(web): ditch the table — /admin/tokens as a card stack, modern GitHub-style list Replaces the table-based layout with a stack of self-contained token cards inside a <ul role=list>. Each card is a flex row: avatar + name/meta on the left, last-used block in the middle, status pill + outlined 'Revoke' button on the right. Status and sort controls are pill-shaped toggle chips; user email search has an inline search icon. No <table>/<tr>/<th>/<td> anywhere. Responsive below 720px (card stacks vertically) and 480px (stat chips 2x2). Preserves filter IDs (flt-status, flt-user, flt-last-used) and data-revoke for existing tests. * feat(web): add /tokens (role-aware) — single page for both user PAT CRUD and admin overview - Rename admin_tokens.html -> tokens.html with a new is_admin context flag. - New route GET /tokens: renders the same card-stack UI for everyone. * Admins: loads /auth/admin/tokens, shows owner column + stat strip, keeps the owner-email search box and sort-by-owner chip. * Non-admins: loads /auth/tokens (own tokens only), hides owner column + stat chips, adds a 'New token' CTA in the hero that opens a modal (name + expires_in_days) calling POST /auth/tokens. The raw token is revealed once in a dismissable banner and cleared from the DOM on Hide. - GET /admin/tokens now 302-redirects to /tokens, preserving query string (so the /admin/users deep-link ?user=foo still works). * feat(web): /tokens full-bleed layout to match dashboard width The hero, toolbar, and card list used to sit inside base.html's .container (max-width 800px). Break out with negative horizontal margins so the page spans the viewport like /dashboard does, capped at 1440px for readability on very wide screens with a 24px gutter on each side. - No change to base.html itself. The override is scoped to .tokens-page. - body { overflow-x: hidden; } guards against rare horizontal scrollbars. - < 808px viewport: reset to natural flow (mobile already narrower). - ≥ 1488px viewport: cap to 1440px and re-center. * chore(web): remove /profile template + nav link (redirect /profile -> /tokens) The old /profile PAT CRUD page is now redundant — the modern /tokens page covers both user and admin flows. Delete the template; the router's /profile handler already 302-redirects to /tokens. Nav cleanup: - Remove the 'Profile' link. - Show a single 'Tokens' link to every signed-in user (previously only admins saw it). - Active-state matches /tokens, /admin/tokens, and /profile so the highlight survives the redirect chain. /install CTA now points at /tokens instead of /profile. * test: cover /tokens for admin + non-admin flows, /profile redirect, nav update tests/test_admin_tokens_ui.py - Point admin rendering test at /tokens directly and tighten assertions (admin-only stat strip + owner search, non-admin CTA absent). - Add test_non_admin_can_render_tokens_page: personal body, New-token CTA, create-modal, reveal banner; stat strip + owner search absent. - Add test_admin_tokens_redirects_to_tokens: 302 to /tokens, query string (?user=...) preserved for the /admin/users deep-link. - Add test_profile_redirects_to_tokens: 302 to /tokens. - Add test_non_admin_can_create_pat_via_tokens_page_api: exercises the POST /auth/tokens call that the non-admin create-modal submits. tests/test_pat.py - test_profile_page_renders -> test_profile_page_redirects_to_tokens: assert the 302 + that /tokens lands on the unified non-admin body. tests/test_web_ui.py - admin_users nav assertion: 'Tokens' link present, 'Profile' link absent. - Add test_nav_shows_tokens_link_for_non_admin: non-admins see the same 'Tokens' link (previously only admins did). - Add test_profile_redirects_to_tokens back-compat check. * feat(web): collapse 'What Claude Code will receive' by default The preview block on /dashboard and /install now uses <details>/<summary> so it is hidden by default. Click the chevron/title to expand and review the clipboard payload. Markup stays in the DOM so existing tests that assert on content continue to pass. * fix(web): /tokens width — override .container to 1280px like dashboard The negative-margin full-bleed trick was fragile and pushed content past the right edge on deployed viewports. Replace with a simple max-width override of base.html's .container on this page only, matching /dashboard's 1280px center-column layout. * feat(web): split role-aware /tokens into my_tokens.html + admin_tokens.html * feat(web): router — separate handlers for /tokens (own) and /admin/tokens (all) * feat(web): nav — show Tokens for all, add All tokens for admins * test: cover split token pages (own vs all) + admin access gating * feat(web): move 'My tokens' into a user dropdown menu Replaces the separate Tokens/email/Logout nav trio with a rounded avatar trigger that opens a dropdown containing the user's email, role, a 'My tokens' link, and Logout. Admin-only 'All tokens' stays as a top-level nav item since it's an admin function, not a personal one. Click-outside and Escape close the panel; chevron rotates on open. * fix(api): allow PATs to list/get/revoke their own tokens (CLI flow) The documented 'da auth token list/revoke' CLI flow in docs/HEADLESS_USAGE.md uses a PAT, but the previous dependency (require_session_token) returned 403. Only create_token must be session-only to prevent PAT-spawning-PAT chains; listing and revoking your own tokens is safe with a PAT. * fix(api): cap expires_in_days at 3650 to avoid datetime overflow (500 to 400) Values above ~11 million days overflowed datetime.max in datetime.now(utc) + timedelta(days=...) and surfaced as an unhandled OverflowError → 500. Cap at 10 years with a clear 400 instead; the no-expiry code path is unaffected. * fix(api): relax _SAFE_URL_RE to allow path prefixes, underscores, and IPv6 The previous regex rejected legitimate reverse-proxy base_url values (https://host/agnes/), underscores in Docker Compose hostnames, and IPv6 literals (http://[::1]:8000). Widen the charset and allow an optional trailing path. shlex.quote continues to provide defense-in-depth against any metacharacter that slips through. * fix(web): /login/email and Google OAuth propagate next_path Previously, /login/email silently dropped the ?next=<path> query param so the hidden form field rendered empty and login always landed on /dashboard. Google's button was hard-coded to /auth/google/login, ignoring next entirely. - /login page now appends ?next to the Google button URL - /login/email reads + sanitizes next, passes as template context - google_login stashes sanitized next_path in session['login_next'] - google_callback pops + re-sanitizes and redirects there Sanitization factored into app/auth/_common.safe_next_path. * fix(auth): differentiate argon2 VerifyMismatchError from internal errors in web login The previous except (VerifyMismatchError, Exception) collapsed both cases into the generic 'invalid credentials' redirect, silently hiding corrupted-hash / library errors from ops. Split the two: bad password still gets ?error=invalid; anything else logs via logger.exception and redirects with ?err=auth_internal so ops have a visible signal and users don't retry forever against a broken password_hash column. * docs: correct CLAUDE.md table name (personal_access_tokens) v7 note referenced 'access_tokens.last_used_ip' but the real table is personal_access_tokens (as mentioned two tokens earlier in the same bullet). Same-file consistency fix. * chore(web): clarify admin user-reset UI — encourage Set password over the unused reset_token POST /api/users/{id}/reset-password stores and returns a token but no endpoint consumes it — the magic-link sender would log the user in without prompting for a new password, defeating the reset. - Drop the 'Reset' row action from admin_users so admins aren't pointed at a dead end. - Rewrite the reveal-modal copy to tell admins to use Set password and explicitly note that the magic-link flow isn't available for reset tokens in this build. The API endpoint stays for API-level future use. * test: cover PAT CLI flow, expires_in_days overflow, proxy base_url, next propagation - tests/test_pat.py: PAT can list own tokens (200, was 403); PAT can revoke own tokens (204); create_token returns 400 for expires_in_days > 3650 (was 500 via datetime overflow). - tests/test_cli_artifacts.py: _SAFE_URL_RE accepts reverse-proxy path prefixes, underscores, and IPv6 literals; end-to-end check of cli_install_script with a stubbed base_url that includes a path prefix (Agnes behind /agnes/). - tests/test_web_ui.py: /login propagates ?next to the Google button URL; /login/email renders next in the hidden form field and strips hostile values; unit coverage of safe_next_path. * fix(security): use \Z instead of $ in URL/version allowlists (trailing-\n bypass) Python regex `$` also matches just before a trailing newline, so a Host header or AGNES_VERSION value like "good.example.com\n$(rm -rf /)" would slip past the allowlist. `\Z` anchors to strict end-of-string. shlex.quote downstream remains as defense-in-depth, but the allowlist is now the tight gate it claims to be. * fix(auth): PAT with null expiry omits JWT exp claim (DB is the source of truth) Previously a PAT created with `expires_in_days=null` (user-requested "never expires") set the DB `expires_at` to NULL (correct) but still baked a ~100y `exp` claim into the JWT. That is misleading: the PAT silently did expire eventually, despite the UI and API promising "no expiry". `create_access_token` now accepts `omit_exp=True` to skip the `exp` claim entirely. `app/api/tokens.py` passes that when `expires_in_days is None`. The authoritative expiry check lives in `app/auth/dependencies.py`, which reads `expires_at` from the DB row — unchanged. PyJWT accepts claim-less JWTs indefinitely. * test: cover trailing-newline regex bypass + no-exp JWT for unbounded PAT - test_safe_url_re_rejects_trailing_newline_bypass: asserts both `_SAFE_URL_RE` and `_SAFE_VERSION_RE` reject values with a trailing `\n` (previously accepted because Python `$` matches before `\n`). - test_pat_null_expiry_jwt_has_no_exp_claim: POST /auth/tokens with `expires_in_days=null`, decode the returned JWT, assert `exp` is absent while `typ=pat`, `sub`, and `jti` are still present. - test_pat_with_null_expiry_is_accepted_by_verify_token: verify_token round-trips a claim-less JWT without ExpiredSignatureError. - test_pat_null_expiry_end_to_end_allows_authenticated_request: use the null-expiry PAT against /auth/tokens and confirm it authenticates. * docs(auth): document X-Forwarded-For trust model in _client_ip Deployment runs behind Caddy which strips incoming X-Forwarded-For and sets its own, so the leftmost hop is trustworthy. Clarify that the stored last_used_ip is audit-only and never used for access control — if the app is ever exposed directly, this value becomes client-settable. * docs: /profile → /tokens in install.sh next-steps, CLI error, HEADLESS_USAGE, security skill After splitting PAT management to /tokens (with /profile as a back-compat 302), stale references remained in user-facing text. Update them to the canonical /tokens URL so shell scripts, CLI error hints, docs, and the bundled security skill are all consistent.
20 KiB
Security Fixes for Production Deployment
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Fix all critical and important security findings before deploying to paying customers.
Architecture: Targeted fixes to auth, RBAC, query endpoint, script sandbox, and upload endpoints. No architectural changes — just closing specific holes identified by security review.
Tech Stack: Python 3.13, FastAPI, DuckDB, argon2-cffi, PyJWT
Source: Security posture review 2026-04-09 (findings C1-C3, I1-I5, I10-I11)
File Map
| File | Responsibility | Tasks |
|---|---|---|
app/auth/router.py |
Token endpoint auth | 1 |
app/web/router.py |
Web UI route guards | 2 |
app/api/query.py |
SQL query blocklist | 3 |
app/api/scripts.py |
Script execution RBAC | 4 |
app/api/catalog.py |
Catalog profile access control | 5 |
app/api/upload.py |
Upload path leak fix | 6 |
app/auth/providers/google.py |
Cookie secure flag | 7 |
app/instance_config.py |
Instance name YAML path fix | 8 |
Task 1: Block /auth/token for OAuth-only users (C1)
Users without password_hash (OAuth-only) can get a JWT by just sending their email. This is an account takeover vulnerability.
Files:
-
Modify:
app/auth/router.py:47-56 -
Test:
tests/test_auth_providers.py -
Step 1: Write the failing test
# In tests/test_auth_providers.py, add to TestTokenEndpoint:
def test_token_rejected_for_oauth_only_user(self, client, e2e_env):
"""OAuth-only users (no password_hash) cannot get token via /auth/token."""
from src.db import get_system_db
from src.repositories.users import UserRepository
conn = get_system_db()
repo = UserRepository(conn)
# Create user without password (simulates Google OAuth user)
repo.create(id="oauth-user", email="oauth@test.com", role="analyst")
conn.close()
resp = client.post("/auth/token", json={"email": "oauth@test.com"})
assert resp.status_code == 401
assert "password" in resp.json()["detail"].lower() or "provider" in resp.json()["detail"].lower()
- Step 2: Run test to verify it fails
Run: pytest tests/test_auth_providers.py::TestTokenEndpoint::test_token_rejected_for_oauth_only_user -v
Expected: FAIL — currently returns 200
- Step 3: Fix the auth logic
In app/auth/router.py, replace lines 47-56 with:
# Require authentication proof
if user.get("password_hash"):
# User has password — require and verify it
if not request.password:
raise HTTPException(status_code=401, detail="Password required")
try:
from argon2 import PasswordHasher
ph = PasswordHasher()
ph.verify(user["password_hash"], request.password)
except Exception:
raise HTTPException(status_code=401, detail="Invalid password")
else:
# No password set — user must use their auth provider (Google, magic link)
raise HTTPException(
status_code=401,
detail="This account uses external authentication. Please log in via your configured provider.",
)
Also update the docstring on line 41:
"""Issue a JWT token. Requires password for password-protected accounts.
OAuth-only accounts must use their auth provider instead."""
- Step 4: Run test to verify it passes
Run: pytest tests/test_auth_providers.py -v
Expected: All pass
- Step 5: Verify bootstrap still works
Run: pytest tests/test_bootstrap.py -v
Expected: All pass (bootstrap creates user WITH password or returns token directly)
- Step 6: Commit
git add app/auth/router.py tests/test_auth_providers.py
git commit -m "fix: block /auth/token for OAuth-only users — require password or external provider"
Task 2: Add role checks to web admin pages (C2)
/admin/tables, /admin/permissions, /corporate-memory/admin are accessible to any authenticated user.
Files:
-
Modify:
app/web/router.py:431-452,388-394 -
Test:
tests/test_web_ui.py -
Step 1: Write the failing tests
# In tests/test_web_ui.py, add:
@pytest.fixture
def analyst_cookie(web_client, tmp_path, monkeypatch):
"""Create analyst user (non-admin) and return cookie."""
from src.db import get_system_db
from src.repositories.users import UserRepository
conn = get_system_db()
UserRepository(conn).create(id="analyst1", email="analyst@test.com", name="Analyst", role="analyst")
conn.close()
resp = web_client.post("/auth/token", json={"email": "analyst@test.com"})
# analyst has no password_hash — need to give them one first
# Actually: after Task 1, this will fail. Create with password instead:
from argon2 import PasswordHasher
from src.db import get_system_db as gsdb
c = gsdb()
c.execute("UPDATE users SET password_hash = ? WHERE id = ?",
[PasswordHasher().hash("testpass"), "analyst1"])
c.close()
resp = web_client.post("/auth/token", json={"email": "analyst@test.com", "password": "testpass"})
assert resp.status_code == 200
return {"access_token": resp.json()["access_token"]}
class TestWebUIRBAC:
def test_admin_tables_requires_admin(self, web_client, analyst_cookie):
resp = web_client.get("/admin/tables", cookies=analyst_cookie)
assert resp.status_code == 403
def test_admin_permissions_requires_admin(self, web_client, analyst_cookie):
resp = web_client.get("/admin/permissions", cookies=analyst_cookie)
assert resp.status_code == 403
def test_corporate_memory_admin_requires_km_admin(self, web_client, analyst_cookie):
resp = web_client.get("/corporate-memory/admin", cookies=analyst_cookie)
assert resp.status_code == 403
def test_admin_can_access_admin_tables(self, web_client, admin_cookie):
resp = web_client.get("/admin/tables", cookies=admin_cookie)
assert resp.status_code == 200
def test_admin_can_access_admin_permissions(self, web_client, admin_cookie):
resp = web_client.get("/admin/permissions", cookies=admin_cookie)
assert resp.status_code == 200
- Step 2: Run tests to verify they fail
Run: pytest tests/test_web_ui.py::TestWebUIRBAC -v
Expected: analyst can access admin pages (403 expected, gets 200)
- Step 3: Add role checks to web routes
In app/web/router.py, add import at top:
from src.rbac import Role
from app.auth.dependencies import require_role
Replace line 434 (user: dict = Depends(get_current_user)) in admin_tables:
user: dict = Depends(require_role(Role.ADMIN)),
Replace line 448 in admin_permissions_page:
user: dict = Depends(require_role(Role.ADMIN)),
Replace line 391 in corporate_memory_admin:
user: dict = Depends(require_role(Role.KM_ADMIN)),
- Step 4: Run tests
Run: pytest tests/test_web_ui.py -v
Expected: All pass
- Step 5: Commit
git add app/web/router.py tests/test_web_ui.py
git commit -m "fix: require admin/km_admin role for web admin pages"
Task 3: Expand SQL query blocklist with DuckDB metadata (C3)
The query endpoint doesn't block information_schema, duckdb_tables(), duckdb_columns(), relative paths, or pragma_ functions.
Files:
-
Modify:
app/api/query.py:40-54 -
Test:
tests/test_security.py -
Step 1: Write the failing tests
# In tests/test_security.py, add to TestQuerySecurity:
def test_blocks_information_schema(self, client, auth_headers):
resp = client.post("/api/query", json={"sql": "SELECT * FROM information_schema.tables"}, headers=auth_headers)
assert resp.status_code == 400
def test_blocks_duckdb_tables(self, client, auth_headers):
resp = client.post("/api/query", json={"sql": "SELECT * FROM duckdb_tables()"}, headers=auth_headers)
assert resp.status_code == 400
def test_blocks_duckdb_columns(self, client, auth_headers):
resp = client.post("/api/query", json={"sql": "SELECT * FROM duckdb_columns()"}, headers=auth_headers)
assert resp.status_code == 400
def test_blocks_duckdb_databases(self, client, auth_headers):
resp = client.post("/api/query", json={"sql": "SELECT * FROM duckdb_databases()"}, headers=auth_headers)
assert resp.status_code == 400
def test_blocks_relative_path(self, client, auth_headers):
resp = client.post("/api/query", json={"sql": "SELECT * FROM '../secret.parquet'"}, headers=auth_headers)
assert resp.status_code == 400
def test_blocks_pragma_table_info(self, client, auth_headers):
resp = client.post("/api/query", json={"sql": "SELECT * FROM pragma_table_info('users')"}, headers=auth_headers)
assert resp.status_code == 400
- Step 2: Run to verify failures
Run: pytest tests/test_security.py::TestQuerySecurity -v
Expected: New tests FAIL
- Step 3: Expand the blocklist
In app/api/query.py, replace the blocked list (lines 40-54):
blocked = [
# DDL/DML
"drop ", "delete ", "insert ", "update ", "alter ", "create ",
"copy ", "attach ", "detach ", "load ", "install ",
"export ", "import ", "pragma ", "call ",
# File access functions
"read_csv", "read_json", "read_parquet", "read_text",
"write_csv", "write_parquet", "read_blob", "read_ndjson",
"parquet_scan", "parquet_metadata", "parquet_schema",
"json_scan", "csv_scan",
"query_table", "iceberg_scan", "delta_scan",
"glob(", "list_files",
# URL/path schemes
"'/", '"/','http://', 'https://', 's3://', 'gcs://',
"'../", '"../',
# DuckDB metadata (leaks schema info regardless of RBAC)
"information_schema", "duckdb_tables", "duckdb_columns",
"duckdb_databases", "duckdb_settings", "duckdb_functions",
"duckdb_views", "duckdb_indexes", "duckdb_schemas",
"pragma_table_info", "pragma_storage_info",
# Multiple statements
";",
]
- Step 4: Run tests
Run: pytest tests/test_security.py::TestQuerySecurity -v
Expected: All pass
- Step 5: Commit
git add app/api/query.py tests/test_security.py
git commit -m "fix: block DuckDB metadata functions and relative paths in query endpoint"
Task 4: Restrict script execution to analyst role (I2/I4)
Any authenticated user (including viewers) can deploy and execute scripts.
Files:
-
Modify:
app/api/scripts.py:53-56,72-73,83-84 -
Test:
tests/test_security.py -
Step 1: Write the failing test
# In tests/test_security.py, add:
class TestScriptRBAC:
def test_viewer_cannot_run_scripts(self, client):
"""Viewers should not be able to execute scripts."""
from src.db import get_system_db
from src.repositories.users import UserRepository
from app.auth.jwt import create_access_token
conn = get_system_db()
UserRepository(conn).create(id="viewer1", email="viewer@test.com", role="viewer")
conn.close()
token = create_access_token(user_id="viewer1", email="viewer@test.com", role="viewer")
headers = {"Authorization": f"Bearer {token}"}
resp = client.post("/api/scripts/run", json={
"name": "test", "source": "print('hi')"
}, headers=headers)
assert resp.status_code == 403
def test_viewer_cannot_deploy_scripts(self, client):
from src.db import get_system_db
from src.repositories.users import UserRepository
from app.auth.jwt import create_access_token
conn = get_system_db()
try:
UserRepository(conn).create(id="viewer2", email="viewer2@test.com", role="viewer")
except Exception:
pass
conn.close()
token = create_access_token(user_id="viewer2", email="viewer2@test.com", role="viewer")
headers = {"Authorization": f"Bearer {token}"}
resp = client.post("/api/scripts/deploy", json={
"name": "test", "source": "print('hi')", "schedule": ""
}, headers=headers)
assert resp.status_code == 403
- Step 2: Run to verify failures
Run: pytest tests/test_security.py::TestScriptRBAC -v
Expected: FAIL — viewers get 200
- Step 3: Add role requirements
In app/api/scripts.py, add import:
from app.auth.dependencies import require_role
from src.rbac import Role
Replace get_current_user with require_role(Role.ANALYST) on these endpoints:
-
deploy_script(line 56):user: dict = Depends(require_role(Role.ANALYST)), -
run_ad_hoc(line 73):user: dict = Depends(require_role(Role.ANALYST)), -
run_deployed(line 84):user: dict = Depends(require_role(Role.ANALYST)), -
list_scripts(line 46): keep asget_current_user(read-only, safe for all) -
undeploy_script(line 101):user: dict = Depends(require_role(Role.ADMIN)), -
Step 4: Run tests
Run: pytest tests/test_security.py tests/test_api_scripts.py -v
Expected: All pass
- Step 5: Commit
git add app/api/scripts.py tests/test_security.py
git commit -m "fix: restrict script deploy/execute to analyst role, undeploy to admin"
Task 5: Add access control to catalog profile endpoints (I5)
/api/catalog/profile/{table_name} returns profile data without checking table access.
Files:
-
Modify:
app/api/catalog.py:18-39 -
Test:
tests/test_access_control.py -
Step 1: Write the failing test
# In tests/test_access_control.py, add to TestPrivateTablesRestricted or new class:
class TestCatalogProfileAccessControl:
def test_profile_denied_for_private_table(self, client, e2e_env):
"""Analyst without explicit access should not see profile of private table."""
# Assumes 'private_table' is registered as private in e2e_env
# and the test user doesn't have access
from app.auth.jwt import create_access_token
token = create_access_token(user_id="analyst-no-access", email="noaccess@test.com", role="analyst")
headers = {"Authorization": f"Bearer {token}"}
resp = client.get("/api/catalog/profile/private_table", headers=headers)
assert resp.status_code == 403
-
Step 2: Run to verify failure
-
Step 3: Add access check
In app/api/catalog.py, add import:
from src.rbac import can_access_table
In get_table_profile (line 18), after user: dict = Depends(get_current_user), add:
# Check table-level access
if not can_access_table(user, table_name, conn):
raise HTTPException(status_code=403, detail=f"Access denied to table '{table_name}'")
Add the same check to the /profile/{table_name}/refresh endpoint if it exists.
- Step 4: Run tests
Run: pytest tests/test_access_control.py -v
Expected: All pass
- Step 5: Commit
git add app/api/catalog.py tests/test_access_control.py
git commit -m "fix: add per-table access control to catalog profile endpoints"
Task 6: Stop leaking internal file paths in upload responses (I10)
Upload endpoints return "path": str(target) exposing server filesystem structure.
Files:
-
Modify:
app/api/upload.py:37,59 -
Test:
tests/test_api_complete.py -
Step 1: Write the failing test
# In tests/test_api_complete.py, add to TestUpload:
def test_upload_does_not_leak_absolute_path(self, client, admin_headers):
"""Upload response should not contain absolute filesystem paths."""
import io
resp = client.post(
"/api/upload/session/test-session",
files={"file": ("test.txt", io.BytesIO(b"hello"), "text/plain")},
headers=admin_headers,
)
assert resp.status_code == 200
data = resp.json()
assert not data.get("path", "").startswith("/"), "Response should not leak absolute path"
- Step 2: Run to verify failure
Run: pytest tests/test_api_complete.py::TestUpload::test_upload_does_not_leak_absolute_path -v
Expected: FAIL — returns /data/user_sessions/...
- Step 3: Fix upload responses
In app/api/upload.py, replace "path": str(target) with "filename": filename in both endpoints:
Line 37:
return {"status": "ok", "filename": filename, "size": len(content)}
Line 59:
return {"status": "ok", "filename": filename, "size": len(content)}
- Step 4: Run tests
Run: pytest tests/test_api_complete.py -v
Expected: All pass
- Step 5: Commit
git add app/api/upload.py tests/test_api_complete.py
git commit -m "fix: return filename instead of absolute path in upload responses"
Task 7: Force secure cookie flag in production (I11)
Google OAuth callback sets secure=True only when the request is HTTPS. Behind a TLS-terminating proxy, the app sees HTTP.
Files:
-
Modify:
app/auth/providers/google.py:92-98 -
Step 1: Fix the cookie setting
In app/auth/providers/google.py, replace lines 92-98:
is_production = os.environ.get("TESTING", "").lower() not in ("1", "true")
response = RedirectResponse(url="/dashboard", status_code=302)
response.set_cookie(
key="access_token", value=jwt_token,
httponly=True, max_age=86400, samesite="lax",
secure=is_production, # Always secure in production (behind TLS proxy)
)
Note: max_age reduced from 86400 * 30 (30 days) to 86400 (1 day) to match JWT expiry.
- Step 2: Run auth tests
Run: pytest tests/test_auth_providers.py -v
Expected: All pass
- Step 3: Commit
git add app/auth/providers/google.py
git commit -m "fix: force secure cookie flag in production, align cookie max_age with JWT expiry"
Task 8: Fix instance_config YAML path for instance name (C4)
get_instance_name() reads flat key instance_name but YAML structure is instance.name.
Files:
-
Modify:
app/instance_config.py:48-53 -
Test:
tests/test_instance_config.py -
Step 1: Write the failing test
# In tests/test_instance_config.py, add:
def test_reads_nested_instance_name(self, tmp_path, monkeypatch):
"""get_instance_name should read instance.name from YAML, not flat instance_name."""
monkeypatch.setenv("DATA_DIR", str(tmp_path))
monkeypatch.setenv("TESTING", "1")
monkeypatch.setenv("JWT_SECRET_KEY", "test-secret-key-min-32-characters!!")
config_dir = tmp_path / "config"
config_dir.mkdir(exist_ok=True)
(config_dir / "instance.yaml").write_text(
"instance:\n name: Acme Analytics\n subtitle: Data Team\n"
)
import importlib
import app.instance_config as mod
importlib.reload(mod)
assert mod.get_instance_name() == "Acme Analytics"
assert mod.get_instance_subtitle() == "Data Team"
- Step 2: Run to verify failure
Run: pytest tests/test_instance_config.py::TestInstanceConfig::test_reads_nested_instance_name -v
Expected: FAIL — returns "AI Data Analyst" instead of "Acme Analytics"
- Step 3: Fix the accessor functions
In app/instance_config.py, replace lines 48-53:
def get_instance_name() -> str:
return get_value("instance", "name", default="AI Data Analyst")
def get_instance_subtitle() -> str:
return get_value("instance", "subtitle", default="")
- Step 4: Run tests
Run: pytest tests/test_instance_config.py -v
Expected: All pass
- Step 5: Commit
git add app/instance_config.py tests/test_instance_config.py
git commit -m "fix: get_instance_name reads nested instance.name from YAML"
Execution Order
Tasks are independent and can run in parallel (different files). Recommended order by impact:
- Task 1 (C1) — account takeover via /auth/token
- Task 2 (C2) — admin pages exposed
- Task 3 (C3) — SQL metadata leaks
- Task 4 (I4) — script execution RBAC
- Task 5 (I5) — catalog profile access control
- Task 8 (C4) — instance name config
- Task 6 (I10) — upload path leak
- Task 7 (I11) — cookie secure flag
Verification after all tasks:
pytest tests/ -v --tb=short # All 650+ tests pass