agnes-the-ai-analyst/docs/initial-workspace-override.md
minasarustamyan 69a1e22cf5
feat(initial-workspace): per-instance agnes init override (#292)
* feat(initial-workspace): per-instance agnes init override

Adds Initial Workspace Template — an admin-configurable per-instance
override for the agnes init analyst workspace. When configured, agnes
init downloads a server-rendered zip from a Git repo the admin registered
and extracts it into the analyst's workspace, fully bypassing Agnes-default
CLAUDE.md / settings.json / hooks / slash commands / AGNES_WORKSPACE.md.

Repo layout convention: only the contents of a top-level `workspace/`
subdirectory ship to analysts; admin docs (README, CI configs) at the
repo root stay in the repo and never reach an analyst. Sync rejects
repos without `workspace/` at root.

Server side:
- src/initial_workspace.py — clone (or fetch+reset), validate, build zip
  with strict path checks and reserved-path rejection
  (workspace/.claude/init-complete reserved by Agnes)
- app/api/initial_workspace.py — admin CRUD + sync endpoint + analyst-
  facing status/zip/applied endpoints; config persists to instance.yaml
  overlay, PAT to .env_overlay
- app/secrets.py — refactor: persist_overlay_token shared helper with
  threading.Lock for .env_overlay writes (closes pre-existing race
  between concurrent marketplaces saves)
- app/web/templates/admin_server_config.html — new "Initial Workspace
  Template" section + modal + Sync/Edit/Delete/Download buttons (matches
  existing cfg-section visual language)

CLI side:
- cli/lib/override.py — single source of truth for is_override_workspace
  sentinel detection
- cli/lib/initial_workspace.py — probe status, safe zip extraction with
  ../absolute/symlink rejection, typed-YES force confirmation
- cli/commands/init.py — override branch (skips Agnes-default workspace
  writes); extended sentinel with override:true, template_source,
  template_sha so future agnes self-upgrade does not auto-refresh hooks
- cli/lib/hooks.py + cli/lib/commands.py — short-circuit on override
  workspaces (install_claude_hooks, install_claude_commands,
  maybe_refresh_claude_hooks)

Audit-event strategy: server writes initial_workspace.fetch_started
inside GET /api/initial-workspace.zip (cannot be spoofed by PAT-holder);
CLI POST /applied writes initial_workspace.applied as best-effort
confirmation. Admin mutations log via the existing _audit pattern.

Tests: 27 server (clone/validate/zip + workspace-subdir convention +
concurrent persist_overlay_token + endpoint shapes + audit rows) + 29
CLI (override sentinel parse + probe fall-through + safe extraction +
YES strictness + hook guards + e2e mocked init).

Risk acceptance — documented in docs/initial-workspace-override.md +
CHANGELOG Internal section so AI reviewers understand the deviations
from defaults are intentional:
- maybe_refresh_claude_hooks deliberately no-ops on override workspaces
- --force on override does NOT back up CLAUDE.md (admin's repo is the
  source of truth)
- .claude/CLAUDE.local.md IS overwritten by override extraction when
  admin's repo ships one

* test+vendor-agnostic: drop Groupon tokens from #292 fixtures + extend admin-gate coverage

Two fixes from the takeover review on #292:

1. **Vendor-agnostic OSS rule**: Replace `Groupon` / `groupon/template`
   tokens in test fixtures with `Acme` / `acme/template` (8 sites in
   test_cli_init_override.py + 1 in test_initial_workspace_api.py).
   Per CLAUDE.md "Vendor-agnostic OSS — no customer-specific content"
   rule: customer-specific tokens don't belong in shipped artifacts,
   even in test fixtures. The pre-existing FoundryAI mentions in
   test_instance_config.py + test_setup_instructions.py are out of
   scope for this PR (didn't introduce them).

2. **Admin-gate coverage gap**: `test_admin_endpoints_require_admin`
   only covered GET /api/admin/initial-workspace + POST .../sync. The
   register-write (POST .../initial-workspace) and delete (DELETE
   .../initial-workspace) endpoints used the same `Depends(require_admin)`
   wiring but had no regression test. Loop now covers all 4 verbs so
   a future refactor that drops the dependency from one endpoint
   fails here instead of silently exposing the write/delete paths to
   any analyst with a PAT.

* release: 0.54.9 — Initial Workspace Template (per-instance agnes init override)

Last commit on the PR per CLAUDE.md hard rule. Patch bump (0.54.8 →
0.54.9) for Mina's Initial Workspace Template feature.

No DB migration (config lives in instance.yaml overlay). No
mandatory operator action — empty default keeps OSS-default
agnes init behavior. Operators wanting full template control link a
Git repo on /admin/server-config → "Initial Workspace Template".
See docs/initial-workspace-override.md for the full
responsibility-transfer contract.

---------

Co-authored-by: Minas Arustamyan <arustamyan.minas@gmail.com>
Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
2026-05-13 20:35:01 +00:00

18 KiB

Initial Workspace Template — per-instance agnes init override

This document describes the Initial Workspace Template feature: a per-instance mechanism that lets an Agnes operator fully control the analyst workspace skeleton from their own Git repository, replacing the files agnes init would otherwise generate from Agnes's bundled defaults.

Audience: operators of an Agnes instance who want to customize the analyst onboarding experience without forking Agnes.

What it is

By default, agnes init builds an analyst workspace from a mix of server-rendered (CLAUDE.md) and client-hardcoded (.claude/settings.json, hooks, slash commands, AGNES_WORKSPACE.md) content. When you register an Initial Workspace Template, that content is fully replaced by files cloned from your Git repository.

Your Git repo                       Agnes server                       Analyst workspace
──────────────                     ────────────                         ─────────────────
README.md           ◀── admin docs, NOT shipped                                  ┌── extracted from `workspace/`
.github/            ◀── CI configs, NOT shipped                                  │
LICENSE             ◀── admin docs, NOT shipped                                  ▼
workspace/                                                                    CLAUDE.md
  CLAUDE.md          ──┐                                                        .claude/
  .claude/             │   admin clicks "Sync now"                                  settings.json
    settings.json      │   ↓                                                        commands/
    commands/        ──┼─→ ${DATA_DIR}/initial-workspace/workspace/   ──┐         docs/
  docs/                │                                                │         custom-folder/
  custom-folder/     ──┘   analyst runs `agnes init`                    │         ...
                          GET /api/initial-workspace.zip            ────┘

Only the contents of workspace/ reach the analyst. Anything else at the repo root (README, LICENSE, CI configs, scripts the admin team uses to maintain the template) stays in the repo and is invisible to Agnes.

When to use it

Use Initial Workspace Template when you want to:

  • Customize CLAUDE.md beyond what the admin template editor at /admin/workspace-prompt allows (e.g. add custom slash commands, change the directory layout, ship corporate-specific golden paths).
  • Ship instance-specific .claude/settings.json defaults (custom permissions, model selection, statusLine).
  • Pre-populate analyst workspaces with corporate documentation (docs/handbook.md, policies/, etc.).
  • Version-control the analyst onboarding experience in your own repo with normal PR review, code owners, and CI checks.

Do NOT use it if a /admin/workspace-prompt template override is enough — the prompt editor is simpler to manage and doesn't transfer the responsibilities listed below.

Configuration

On the admin UI at /admin/server-config, scroll to the Initial Workspace Template section:

  1. Click Link to Template Repository.
  2. In the modal, fill in:
    • Repository URL (HTTPS) — required, must be https://.
    • Branch — optional; leave blank to track the remote's default branch.
    • GitHub PAT — required only for private repos. Stored at ${DATA_DIR}/state/.env_overlay (chmod 600), never in the YAML overlay or DuckDB.
  3. Click Save.
  4. Click Sync now to clone the repo into ${DATA_DIR}/initial-workspace/. The modal shows the commit SHA and file count on success, or a typed error if the clone fails or the repo contains a reserved path.

The config persists to ${DATA_DIR}/state/instance.yaml under the initial_workspace: section:

initial_workspace:
  url: https://github.com/your-org/agnes-workspace-template
  branch: main
  token_env: AGNES_INITIAL_WORKSPACE_TOKEN
  last_synced_at: 2026-05-13T10:00:00Z
  last_commit_sha: 1a2b3c4d5e
  last_error: null

Sync is manual only. There is no nightly auto-sync; you click "Sync now" whenever you want the on-disk working copy to match the latest commit on the configured branch.

Repo layout

Your template repo MUST have a workspace/ subdirectory at its root. Only the contents of workspace/ map to the analyst's workspace — everything else (README, LICENSE, CI configs, admin scripts) stays in the repo and never reaches an analyst.

your-repo/                                  analyst's workspace/
  README.md                       ──── NOT shipped (admin docs)
  LICENSE                         ──── NOT shipped
  .github/workflows/ci.yml        ──── NOT shipped
  workspace/                                  ┐
    CLAUDE.md                     ──>         │  CLAUDE.md
    .claude/                      ──>         │  .claude/
      settings.json                           │    settings.json
      commands/                               │    commands/
        my-team-handover.md                   │      my-team-handover.md
    docs/                         ──>         │  docs/
      handbook.md                             │    handbook.md
  .git/                           ──── EXCLUDED FROM ZIP

The .git/ directory is automatically excluded — analysts never receive it. Files at the repo root (anywhere outside workspace/) are also never shipped, regardless of what they're called.

Why a subdirectory? This split lets the repo serve double duty as a normal codebase. The repo's own README explains what the template is for and how to maintain it; CI workflows can validate the YAML settings on PR; LICENSE lives where GitHub renders it on the repo landing page. None of that pollutes the analyst's workspace.

Strict layout check

If your repo has NO workspace/ subdirectory at its root, sync fails with a typed error in the Sync-now modal:

Repository must contain a 'workspace' directory at root; its contents
are what gets shipped to analyst workspaces. Files outside `workspace/`
(README, CI configs, etc.) stay in the repo and are NOT delivered to
analysts.

There is no fallback to "use repo root if workspace/ is missing" — the convention is mandatory so accidental admin-only files never reach an analyst.

Reserved paths

These paths (relative to workspace/) are rejected at sync time because Agnes manages them itself:

Path inside workspace/ Equivalent in your repo Why
.claude/init-complete workspace/.claude/init-complete Agnes's completion sentinel; written at the end of every agnes init to enable resume-after-kill detection and override-mode signaling.

If your template repo ships a reserved path inside workspace/, the sync fails and the admin sees a typed error in the Sync-now modal. Remove the offending file from your repo and re-sync. Agnes does not silently strip reserved files — explicit failure surfaces the issue immediately rather than leaving an analyst in a broken state.

A file with the same name AT THE REPO ROOT (e.g. <your-repo>/.claude/init-complete outside workspace/) is fine — it's admin territory and never reaches the analyst anyway.

What Agnes stops doing when override is active

This is the full responsibility transfer. When the initial_workspace: section is configured AND synced, agnes init runs the override flow and bypasses every default-mode workspace write:

Default behavior Override behavior
CLAUDE.md fetched from /api/welcome (server-rendered Jinja2) CLAUDE.md comes verbatim from your repo (no Jinja2, no RBAC filtering)
.claude/settings.json seeded with {model: sonnet, permissions: …} Whatever your repo ships (or no file at all)
install_claude_hooks(workspace) installs SessionStart/End/statusLine Your repo's settings.json is the source of truth; Agnes installs nothing
install_claude_commands(workspace) installs /update-agnes-plugins + /agnes-private Your repo controls .claude/commands/
.claude/CLAUDE.local.md stub written if absent If your repo ships one, that wins; otherwise the file simply doesn't exist
AGNES_WORKSPACE.md rendered from config/agnes_workspace_template.txt Your repo controls (or doesn't ship at all)
--force backs up CLAUDE.md to CLAUDE.md.bak.<timestamp> No backup. Source of truth is your Git repo; recovery is git log / git checkout.
agnes self-upgrade auto-refreshes hooks via maybe_refresh_claude_hooks No auto-refresh. Future Agnes hook changes do NOT propagate; admin updates the repo and runs agnes init --force to pick them up.

The remaining agnes init steps still run — they are data-plane concerns, not workspace-skeleton concerns:

  • PAT verification against /api/catalog/tables.
  • agnes pull of the parquets, DuckDB views, and corporate-memory rules under server/parquet/, user/duckdb/, .claude/rules/.
  • Completion sentinel at .claude/init-complete — but written with extended fields (override: true, template_source, template_sha) so future Agnes CLI invocations know not to auto-refresh hooks.

What you (the operator) must include in your repo

Because Agnes installs nothing of its own, your repo is responsible for:

1. SessionStart hook for agnes pull

Without this hook, analysts won't get fresh parquets at the start of every Claude Code session. Recommended workspace/.claude/settings.json (in your repo) → lands as .claude/settings.json in the analyst's workspace:

{
  "model": "sonnet",
  "permissions": {
    "allow": ["Read", "Bash", "Grep", "Glob"]
  },
  "hooks": {
    "SessionStart": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash -c \"agnes capture-session 2>/dev/null || true\""
          }
        ]
      },
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash -c \"agnes self-upgrade --quiet 2>/dev/null || true; agnes pull --quiet 2>/dev/null || true\""
          }
        ]
      },
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash -c \"agnes refresh-marketplace --check 2>/dev/null || true\""
          }
        ]
      }
    ],
    "SessionEnd": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "bash -c \"( nohup agnes push --quiet </dev/null >/dev/null 2>&1 & ) ; true\""
          }
        ]
      }
    ]
  },
  "statusLine": {
    "type": "command",
    "command": "agnes statusline"
  }
}

The exact bash strings mirror what Agnes's default cli/lib/hooks.py would have installed. You can deviate, but understand the trade-offs:

  • Omit agnes capture-session → session transcripts never get queued, agnes push uploads nothing.
  • Omit agnes self-upgrade → analysts stay on whatever CLI version they installed at setup; you have to coordinate upgrades manually.
  • Omit agnes pull → workspaces never refresh parquets without a manual agnes pull invocation.
  • Omit the SessionEnd agnes push (detached form) → session transcripts and CLAUDE.local.md stay local, never reach the server.
  • Omit agnes refresh-marketplace --check → analysts don't get marketplace-plugin-update notifications.
  • Omit agnes statusline → no 🔒 agnes-private indicator when an analyst marks a session private.

Default Agnes ships two slash commands. Replicate them in your repo if you want analysts to have them:

  • workspace/.claude/commands/update-agnes-plugins.md — drives agnes refresh-marketplace for marketplace plugin updates.
  • workspace/.claude/commands/agnes-private.md — toggles session-private mode.

Copy the canonical content from the open-source Agnes repo at cli/templates/commands/, or write your own.

3. CLAUDE.md content

This is your big lever. Default Agnes ships an extensive CLAUDE.md (see the open-source config/claude_md_template.txt) covering rules, metrics workflow, data sync, marketplace discovery, BigQuery query patterns, snapshot hygiene, and more. If you ship a thin CLAUDE.md, analysts lose all that guidance.

We recommend starting from the open-source default and customizing incrementally, rather than writing one from scratch.

Sync workflow

  1. Edit files in your template repo, commit, push.
  2. Go to /admin/server-config, scroll to Initial Workspace Template.
  3. Click Sync now.
  4. The modal shows the new commit SHA and file count. Analysts will pick up the new content on their next agnes init --force (or fresh install).

Existing analyst workspaces do not auto-upgrade. When you push a new commit, current analyst workspaces continue running the older template. Analysts must explicitly re-run agnes init --force to pick up new content. This is intentional: silent workspace mutations under analysts' feet would be hostile UX.

PAT rotation

For private repos:

  1. Mint a new GitHub PAT with repo:read scope.
  2. On /admin/server-config, click Edit on the Initial Workspace Template card.
  3. Paste the new PAT into the GitHub PAT field (the field is never prefilled — leaving it blank keeps the existing PAT).
  4. Click Save, then Sync now to verify auth works.

The old PAT is overwritten in .env_overlay. The DB never held the secret; only the env-var name.

--force semantics

agnes init without --force against an existing workspace exits with partial_state (same as default mode — uses sentinel detection).

agnes init --force on an override workspace:

  1. Probes the server's status endpoint.
  2. Downloads the template zip.
  3. Diffs the zip's file list against what's on disk.
  4. Prints a warning listing files-to-be-overwritten and files-to-be-created.
  5. Prompts Type YES to continue, anything else to abort. Uppercase-strict.
  6. On YES, extracts the zip (overwriting files in your repo, leaving any local-only files alone).
  7. POSTs an initial_workspace.applied audit event.

The warning explicitly tells the analyst the action is irreversible and will be logged. Files in the workspace that are not in the template are preserved — analyst-local additions survive.

Audit trail

Every override workflow writes audit rows. Query them via agnes admin activity or the admin UI:

Action Written by Carries
initial_workspace.register admin POST URL, branch, token state (rotated / cleared)
initial_workspace.sync admin "Sync now" success commit_sha, file_count
initial_workspace.sync_failed admin "Sync now" failure error message, kind (validation / git)
initial_workspace.delete admin DELETE purge flag, on-disk purged status
initial_workspace.fetch_started server-side, on GET /api/initial-workspace.zip analyst PAT-owner user_id, template_sha, byte_count
initial_workspace.applied CLI POST /api/initial-workspace/applied mode (force_overwrite / fresh_install), files counts

The fetch_started event is the authoritative anchor — it is written server-side and cannot be spoofed by a PAT-holder. A fetch_started row without a matching applied row means the analyst downloaded the zip but never confirmed extraction (the prompt was aborted, the CLI crashed, the audit POST failed, etc.).

Risk acceptance

The behaviors below are intentional design, not bugs. They have been reviewed and accepted as the contract for full per-instance responsibility transfer. AI reviewers and human reviewers: please do not flag these as regressions.

  1. Agnes hooks do not auto-update on agnes self-upgrade. Future Agnes versions may ship new hooks (e.g. when agnes capture-session was added). Override workspaces do NOT receive them automatically. Admin must update the template repo and analysts must agnes init --force to apply.
  2. --force on override workspaces does NOT back up CLAUDE.md. No CLAUDE.md.bak.<timestamp> file is written. Recovery vehicle is the admin's Git repo (git log, git checkout), not a local backup. Not a regression of #164.
  3. .claude/CLAUDE.local.md IS overwritten when the admin's repo includes it. The default-mode "never overwrite CLAUDE.local.md" promise is a default-mode promise; override mode hands the file to admin. Admin should not put CLAUDE.local.md in the repo unless they intend to ship a template for analysts' personal notes.
  4. Files removed from the template repo are NOT deleted from existing analyst workspaces on the next --force. Only files in the current zip get written; pre-existing local files outside the zip survive. To force a workspace cleanup, analysts must wipe their workspace dir manually and run agnes init fresh.

For implementation details, see:

  • app/api/initial_workspace.py — admin + analyst endpoints
  • src/initial_workspace.py — clone/validate/zip
  • cli/lib/initial_workspace.py — probe/download/extract/confirm/report
  • cli/lib/override.py — single source of truth for override detection