docs(spec+plan): rename CLI binary from da to agnes (BREAKING)
- Spec rev 5: branding consistency. New CLI verbs use agnes prefix (agnes init, agnes pull, agnes push, agnes catalog, agnes status, agnes snapshot create, agnes admin, …). - Plan: add Phase 0 / Task 0 — pyproject.toml [project.scripts] entry rename to "agnes = cli.main:app" + Typer(name="agnes") in cli/main.py. - Legacy command references (da sync, da fetch, da analyst setup, da metrics) keep their da prefix throughout — they're historical artifacts being removed (preserved in CHANGELOG Removed section, _LEGACY_STRINGS constant for admin override scan, etc.). Bulk rename via Python regex with verb whitelist: 286 verb refs rewritten in plan, 265 in spec; 104+72 legacy refs restored to "da" post-pass (false positives where the doc was describing the legacy flow being replaced).
This commit is contained in:
ZdenekSrotyr
parent
fb8f55c335
commit
5e7fa418d1
2 changed files with 370 additions and 281 deletions
|
|
@ -4,11 +4,13 @@
|
|||
|
||||
**Goal:** Replace the interactive `da analyst setup` flow with a single web→paste→done bootstrap. New analyst pastes a clipboard prompt from `/setup?role=analyst` into Claude Code in an empty folder, and ends up with `CLAUDE.md`, `AGNES_WORKSPACE.md`, hooks, fresh data, and DuckDB views — fully ready to query. Drop dead workspace dirs (`data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`). Establish a lazy-mkdir contract so nothing creates empty directories.
|
||||
|
||||
**Architecture:** PAT-only auth. `da init` is a thin orchestrator that auths, fetches `CLAUDE.md` from `/api/welcome`, installs hooks, and calls `cli/lib/pull.py:run_pull` for first data refresh. CLI verbs renamed: `init/pull/push/status/snapshot create` (greenfield, no aliases). Server-side install prompt branches on `role` query param. `cli/lib/` shared library tree separates data primitives from Typer wrappers so `da init` can call them without subprocess. Reader contract: every reader handles missing dirs gracefully (exit 0 empty or exit 1 with friendly hint, never traceback).
|
||||
**Architecture:** PAT-only auth. `agnes init` is a thin orchestrator that auths, fetches `CLAUDE.md` from `/api/welcome`, installs hooks, and calls `cli/lib/pull.py:run_pull` for first data refresh. CLI verbs renamed: `init/pull/push/status/snapshot create` (greenfield, no aliases). Server-side install prompt branches on `role` query param. `cli/lib/` shared library tree separates data primitives from Typer wrappers so `agnes init` can call them without subprocess. Reader contract: every reader handles missing dirs gracefully (exit 0 empty or exit 1 with friendly hint, never traceback).
|
||||
|
||||
**Tech Stack:** Python 3.11+, FastAPI, Typer, Pydantic, DuckDB, httpx, pytest, Hatchling.
|
||||
|
||||
**Spec:** `docs/superpowers/specs/2026-05-04-clean-analyst-bootstrap-design.md` (revision 4, cleared for implementation).
|
||||
**Spec:** `docs/superpowers/specs/2026-05-04-clean-analyst-bootstrap-design.md` (revision 5, cleared for implementation).
|
||||
|
||||
**CLI rename:** As part of this plan, the binary changes from `da` to `agnes`. References to legacy commands (`da sync`, `da fetch`, `da analyst setup`, `da metrics`) keep their `da` prefix throughout this document — they're historical artifacts being removed. New commands and hook strings use `agnes`.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -21,13 +23,13 @@
|
|||
| `cli/lib/__init__.py` | Empty — makes `cli/lib/` a package so Hatchling includes it in the wheel. |
|
||||
| `cli/lib/pull.py` | `run_pull(server_url, token, workspace, *, dry_run) -> PullResult` — pure-function data refresh primitive (manifest, parquet download, DuckDB rebuild, memory bundle write). Lazy mkdir. |
|
||||
| `cli/lib/hooks.py` | `install_claude_hooks(workspace)` — idempotent SessionStart/End hook installer for `<workspace>/.claude/settings.json`. |
|
||||
| `cli/commands/init.py` | `da init` Typer command — auth check, save config, write CLAUDE.md, install hooks, call `run_pull`, write `AGNES_WORKSPACE.md`. |
|
||||
| `cli/commands/pull.py` | `da pull` Typer wrapper around `cli/lib/pull.py:run_pull`. Flags `--quiet`, `--json`, `--dry-run`. |
|
||||
| `cli/commands/push.py` | `da push` Typer command — uploads `user/sessions/*.jsonl` and `.claude/CLAUDE.local.md`. |
|
||||
| `cli/commands/admin_metrics.py` | `da admin metrics {import,export,validate}` sub-Typer (lifted from `cli/commands/metrics.py`). |
|
||||
| `cli/commands/init.py` | `agnes init` Typer command — auth check, save config, write CLAUDE.md, install hooks, call `run_pull`, write `AGNES_WORKSPACE.md`. |
|
||||
| `cli/commands/pull.py` | `agnes pull` Typer wrapper around `cli/lib/pull.py:run_pull`. Flags `--quiet`, `--json`, `--dry-run`. |
|
||||
| `cli/commands/push.py` | `agnes push` Typer command — uploads `user/sessions/*.jsonl` and `.claude/CLAUDE.local.md`. |
|
||||
| `cli/commands/admin_metrics.py` | `agnes admin metrics {import,export,validate}` sub-Typer (lifted from `cli/commands/metrics.py`). |
|
||||
| `config/agnes_workspace_template.txt` | Static client-side template for `AGNES_WORKSPACE.md`. Three placeholders: `{created_at}`, `{server_url}`, `{workspace_path}`. |
|
||||
| `tests/fixtures/analyst_bootstrap.py` | Test fixtures: `fastapi_test_server`, `test_pat`, `test_pat_no_grants`, `zero_grants_workspace`, `web_session`. |
|
||||
| `tests/test_lib_hooks.py` | Tests for `install_claude_hooks` (idempotent, preserves third-party hooks, replaces old `da pull`/`da sync` entries). |
|
||||
| `tests/test_lib_hooks.py` | Tests for `install_claude_hooks` (idempotent, preserves third-party hooks, replaces old `agnes pull`/`da sync` entries). |
|
||||
| `tests/test_lib_pull.py` | Tests for `run_pull` (lazy mkdir, partial failure handling, manifest empty case). |
|
||||
| `tests/test_setup_instructions_analyst.py` | Tests `render_setup_instructions(role="analyst")` produces correct steps. |
|
||||
| `tests/test_tokens_bootstrap_scope.py` | Tests `scope=bootstrap-analyst` PATs are TTL-clamped to ≤ 1 h; `ttl_seconds` upper bound; `ttl_seconds` wins over `expires_in_days`. |
|
||||
|
|
@ -41,18 +43,18 @@
|
|||
|---|---|
|
||||
| `app/api/tokens.py` | `CreateTokenRequest`: add `scope: str = "general"` and `ttl_seconds: Optional[int] = None`. Validate `ttl_seconds <= 315_360_000`. Resolution: `ttl_seconds` wins; fall back to `expires_in_days`. For `scope == "bootstrap-analyst"`, force-clamp resolved TTL ≤ 3600 s. Audit-log includes scope. |
|
||||
| `app/api/claude_md.py` | Add module-level `_LEGACY_STRINGS = ("data/parquet", "da sync", "da fetch", "da analyst setup", "da metrics list", "da metrics show")`. Add helper `_scan_legacy_strings(text) -> list[str]`. Add field `legacy_strings_detected: list[str] = []` to `TemplateGetResponse`. Populate in `admin_get_workspace_template`. |
|
||||
| `app/web/setup_instructions.py` | Add `role: Literal["analyst","admin"] = "admin"` to `resolve_lines()` and `render_setup_instructions()`. Analyst layout: TLS trust (when `ca_pem`) → install `da` → `da init --server-url X --token Y --workspace .` → `da catalog` smoke verify → confirm. Drop for analyst: marketplace, plugins, skills, diagnose, login, whoami. |
|
||||
| `app/web/setup_instructions.py` | Add `role: Literal["analyst","admin"] = "admin"` to `resolve_lines()` and `render_setup_instructions()`. Analyst layout: TLS trust (when `ca_pem`) → install `agnes` → `agnes init --server-url X --token Y --workspace .` → `agnes catalog` smoke verify → confirm. Drop for analyst: marketplace, plugins, skills, diagnose, login, whoami. |
|
||||
| `app/web/router.py` | `setup_page`: read `role` query param (default `"admin"`), pass to `render_setup_instructions(role=...)`. |
|
||||
| `app/web/templates/setup.html` (or wherever `setup_page` renders) | Two role tiles (Analyst / Admin), POST `/auth/tokens` with matching `scope`. |
|
||||
| `app/web/templates/admin_workspace_prompt.html` | Yellow banner above editor when `legacy_strings_detected` non-empty. |
|
||||
| `config/claude_md_template.txt` | Update verb names: `da sync` → `da pull`, `da fetch` → `da snapshot create`, `da metrics list/show` → `da catalog --metrics`, `da analyst setup` → `da init`. Path strings: `data/parquet/` → `server/parquet/`, `data/duckdb/...` → `user/duckdb/analytics.duckdb`. |
|
||||
| `config/claude_md_template.txt` | Update verb names: `da sync` → `agnes pull`, `da fetch` → `agnes snapshot create`, `da metrics list/show` → `agnes catalog --metrics`, `da analyst setup` → `agnes init`. Path strings: `data/parquet/` → `server/parquet/`, `data/duckdb/...` → `user/duckdb/analytics.duckdb`. |
|
||||
| `cli/commands/snapshot.py` | Add `create` subcommand — moves logic from `cli/commands/fetch.py` verbatim. Add `if not db_path.exists(): exit 1` guard before `duckdb.connect()`. |
|
||||
| `cli/commands/catalog.py` | Add `--metrics` flag (replaces `da metrics list`); `--metrics --show <id>` (replaces `da metrics show`). |
|
||||
| `cli/commands/admin.py` | Register the new `admin_metrics_app` sub-Typer. |
|
||||
| `cli/commands/query.py` | Update hint text "Run: da sync" → "Run: da pull" in two places. |
|
||||
| `cli/commands/explore.py` | Update hint text "Run: da sync" → "Run: da pull". |
|
||||
| `cli/commands/query.py` | Update hint text "Run: da sync" → "Run: agnes pull" in two places. |
|
||||
| `cli/commands/explore.py` | Update hint text "Run: da sync" → "Run: agnes pull". |
|
||||
| `cli/main.py` | Drop registrations for `sync`, `analyst`, `metrics`, `fetch`, `status` (existing). Add `init`, `pull`, `push`. Re-register `status` to point at new workspace-status command. |
|
||||
| `CLAUDE.md` (repo root) | Verb + path rewrites throughout. The "Local sync & Claude Code hooks" subsection rewrites verbatim with new commands. The "Querying Agnes data — agent rails" subsection keeps the 0.32.0 `query_mode='materialized'` and `query_mode='remote'` cost-guardrail prose verbatim, just verb-renaming `da fetch` → `da snapshot create`. |
|
||||
| `CLAUDE.md` (repo root) | Verb + path rewrites throughout. The "Local sync & Claude Code hooks" subsection rewrites verbatim with new commands. The "Querying Agnes data — agent rails" subsection keeps the 0.32.0 `query_mode='materialized'` and `query_mode='remote'` cost-guardrail prose verbatim, just verb-renaming `da fetch` → `agnes snapshot create`. |
|
||||
| `CHANGELOG.md` | Entry under `[Unreleased]` per spec preview (Changed/Added/Fixed/Removed/Kept). |
|
||||
| `pyproject.toml` | No change; `cli/lib/__init__.py` triggers Hatchling auto-discovery. |
|
||||
|
||||
|
|
@ -63,14 +65,99 @@
|
|||
| `cli/commands/sync.py` | Replaced by `cli/commands/pull.py` + `cli/commands/push.py` + `cli/lib/pull.py`. |
|
||||
| `cli/commands/fetch.py` | Folded into `cli/commands/snapshot.py:create`. |
|
||||
| `cli/commands/analyst.py` | Replaced by `cli/commands/init.py` + new `cli/commands/status.py` (workspace status). `_install_claude_hooks` lifted to `cli/lib/hooks.py`. |
|
||||
| `cli/commands/metrics.py` | Read paths fold into `da catalog --metrics`; write paths move to `cli/commands/admin_metrics.py`. |
|
||||
| `cli/commands/metrics.py` | Read paths fold into `agnes catalog --metrics`; write paths move to `cli/commands/admin_metrics.py`. |
|
||||
|
||||
### Existing `cli/commands/status.py` rename
|
||||
|
||||
| Action | Detail |
|
||||
|---|---|
|
||||
| Existing `da status` ("System status") | Renamed to `da diagnose system` (subcommand under `diagnose_app`) — its content is a subset of what `da diagnose` already does. |
|
||||
| New `da status` | Workspace status — fresh implementation, replaces `da analyst status`. Lives in `cli/commands/status.py` (overwrite). |
|
||||
| Existing `agnes status` ("System status") | Renamed to `agnes diagnose system` (subcommand under `diagnose_app`) — its content is a subset of what `agnes diagnose` already does. |
|
||||
| New `agnes status` | Workspace status — fresh implementation, replaces `da analyst status`. Lives in `cli/commands/status.py` (overwrite). |
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — CLI binary rename (`da` → `agnes`)
|
||||
|
||||
### Task 0: Rename the CLI entry point
|
||||
|
||||
**Files:**
|
||||
- Modify: `pyproject.toml` (`[project.scripts]`), `cli/main.py` (`Typer(name=...)`)
|
||||
- Test: `tests/test_cli_binary_rename.py` (new)
|
||||
|
||||
**Why first:** Every later task that registers Typer apps, writes hook command strings, or asserts CLI output uses `agnes`. Rename the binary up front so tests in subsequent tasks reference the right name.
|
||||
|
||||
- [ ] **Step 1: Read current entry points**
|
||||
|
||||
```bash
|
||||
grep -n "scripts\|^name\|tool.hatch" pyproject.toml | head
|
||||
grep -n "Typer\|name=\"da\"\|name='da'" cli/main.py
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update `pyproject.toml`**
|
||||
|
||||
In `[project.scripts]`, replace `da = "cli.main:app"` with:
|
||||
|
||||
```toml
|
||||
[project.scripts]
|
||||
agnes = "cli.main:app"
|
||||
```
|
||||
|
||||
Single entry — no `da` alias kept. Greenfield.
|
||||
|
||||
- [ ] **Step 3: Update `cli/main.py`**
|
||||
|
||||
Change the Typer app construction from `name="da"` to `name="agnes"` and update the help string:
|
||||
|
||||
```python
|
||||
app = typer.Typer(
|
||||
name="agnes",
|
||||
help="Agnes — AI Data Analyst CLI",
|
||||
no_args_is_help=True,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Reinstall the editable package**
|
||||
|
||||
```bash
|
||||
uv pip install -e ".[dev]"
|
||||
which agnes
|
||||
agnes --version
|
||||
```
|
||||
|
||||
Expected: `agnes <version>` prints; `da --version` now fails with "command not found".
|
||||
|
||||
- [ ] **Step 5: Write a binary-name regression test**
|
||||
|
||||
```python
|
||||
# tests/test_cli_binary_rename.py
|
||||
"""Confirm the wheel installs the binary as `agnes`, not `da`."""
|
||||
import subprocess
|
||||
|
||||
|
||||
def test_agnes_command_exists():
|
||||
result = subprocess.run(["agnes", "--version"], capture_output=True, text=True)
|
||||
assert result.returncode == 0
|
||||
|
||||
|
||||
def test_da_command_no_longer_works():
|
||||
"""Greenfield: no backward-compat alias."""
|
||||
result = subprocess.run(["bash", "-c", "command -v da"],
|
||||
capture_output=True, text=True)
|
||||
assert result.returncode != 0, "da should NOT be on PATH after rename"
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run the test**
|
||||
|
||||
```bash
|
||||
pytest tests/test_cli_binary_rename.py -v
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add pyproject.toml cli/main.py tests/test_cli_binary_rename.py
|
||||
git commit -m "feat(cli): rename binary from da to agnes (BREAKING)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -341,7 +428,7 @@ def test_scan_finds_all_known_legacy_strings():
|
|||
|
||||
|
||||
def test_scan_returns_empty_for_clean_text():
|
||||
text = "Use `da pull` to refresh, `da snapshot create` for ad-hoc, `server/parquet/`."
|
||||
text = "Use `agnes pull` to refresh, `agnes snapshot create` for ad-hoc, `server/parquet/`."
|
||||
assert _scan_legacy_strings(text) == []
|
||||
|
||||
|
||||
|
|
@ -442,7 +529,7 @@ def test_admin_get_template_returns_legacy_strings_when_override_dirty(web_sessi
|
|||
|
||||
def test_admin_get_template_returns_empty_when_clean(web_session):
|
||||
web_session.put("/api/admin/workspace-prompt-template",
|
||||
json={"content": "Use `da pull` and check `server/parquet/`."})
|
||||
json={"content": "Use `agnes pull` and check `server/parquet/`."})
|
||||
resp = web_session.get("/api/admin/workspace-prompt-template")
|
||||
assert resp.json()["legacy_strings_detected"] == []
|
||||
```
|
||||
|
|
@ -490,15 +577,15 @@ def test_render_analyst_role_basic():
|
|||
)
|
||||
# Required content for analyst role:
|
||||
assert "uv tool install" in text
|
||||
assert "da init" in text
|
||||
assert "agnes init" in text
|
||||
assert "--token" in text and "agnes_pat_TEST" in text
|
||||
assert "--server-url" in text and "https://agnes.example.com" in text
|
||||
assert "da catalog" in text # smoke verify step
|
||||
assert "agnes catalog" in text # smoke verify step
|
||||
# Forbidden content (admin-only):
|
||||
assert "marketplace" not in text
|
||||
assert "claude plugin install" not in text
|
||||
assert "da skills install" not in text # analyst doesn't bulk-install skills
|
||||
assert "da diagnose" not in text # analyst smoke verify is `da catalog`, not diagnose
|
||||
assert "agnes skills install" not in text # analyst doesn't bulk-install skills
|
||||
assert "agnes diagnose" not in text # analyst smoke verify is `agnes catalog`, not diagnose
|
||||
|
||||
|
||||
def test_render_admin_role_unchanged():
|
||||
|
|
@ -509,8 +596,8 @@ def test_render_admin_role_unchanged():
|
|||
wheel_filename="agnes-0.32.0-py3-none-any.whl",
|
||||
# role omitted — defaults to "admin"
|
||||
)
|
||||
assert "da auth import-token" in text # admin uses import-token, not da init
|
||||
assert "da diagnose" in text # admin keeps diagnose
|
||||
assert "agnes auth import-token" in text # admin uses import-token, not agnes init
|
||||
assert "agnes diagnose" in text # admin keeps diagnose
|
||||
|
||||
|
||||
def test_render_analyst_with_ca_pem():
|
||||
|
|
@ -524,7 +611,7 @@ def test_render_analyst_with_ca_pem():
|
|||
)
|
||||
assert "AGNES_CA_PEM" in text # heredoc marker from trust block
|
||||
assert "ca-bundle.pem" in text
|
||||
assert "da init" in text # analyst-specific step still present
|
||||
assert "agnes init" in text # analyst-specific step still present
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run tests to verify they fail**
|
||||
|
|
@ -541,10 +628,10 @@ Insert after `_install_cli_lines` (around line 311 in `setup_instructions.py`):
|
|||
|
||||
```python
|
||||
def _analyst_init_lines(server_url_placeholder: str = "{server_url}") -> list[str]:
|
||||
"""Steps 2-3 — `da init` (auth + workspace bootstrap) + smoke verify.
|
||||
"""Steps 2-3 — `agnes init` (auth + workspace bootstrap) + smoke verify.
|
||||
|
||||
Replaces the admin-flow login + verify steps (today: `da auth import-token`
|
||||
+ `da auth whoami`). `da init` is non-interactive: `--token` carries the PAT,
|
||||
Replaces the admin-flow login + verify steps (today: `agnes auth import-token`
|
||||
+ `agnes auth whoami`). `agnes init` is non-interactive: `--token` carries the PAT,
|
||||
`--server-url` carries the origin. The bootstrap PAT has a 1 h TTL — if the
|
||||
user takes longer than that to paste this prompt, the init call returns 401
|
||||
and the user re-clicks "Generate prompt" on the install page.
|
||||
|
|
@ -552,14 +639,14 @@ def _analyst_init_lines(server_url_placeholder: str = "{server_url}") -> list[st
|
|||
return [
|
||||
"",
|
||||
"2) Bootstrap your analyst workspace in this directory:",
|
||||
f" da init --server-url \"{server_url_placeholder}\" --token \"{{token}}\" --workspace .",
|
||||
f" agnes init --server-url \"{server_url_placeholder}\" --token \"{{token}}\" --workspace .",
|
||||
"",
|
||||
" This authenticates with the PAT, fetches your CLAUDE.md (RBAC-filtered),",
|
||||
" installs Claude Code SessionStart/End hooks (auto-refresh), and runs an",
|
||||
" initial `da pull` so your DuckDB views are ready.",
|
||||
" initial `agnes pull` so your DuckDB views are ready.",
|
||||
"",
|
||||
"3) Verify the data is queryable:",
|
||||
" da catalog",
|
||||
" agnes catalog",
|
||||
"",
|
||||
" This should list the tables your account has grants for. Empty list",
|
||||
" means your admin hasn't granted you access yet — contact them.",
|
||||
|
|
@ -570,8 +657,8 @@ def _analyst_finale_lines(confirm_step_num: str, has_ca: bool) -> list[str]:
|
|||
"""Final Confirm step for analyst role. Shorter than admin: no marketplace,
|
||||
no plugins, no skills."""
|
||||
bullets = [
|
||||
" - `da --version` output",
|
||||
" - First few lines of `da catalog` (tables you can see)",
|
||||
" - `agnes --version` output",
|
||||
" - First few lines of `agnes catalog` (tables you can see)",
|
||||
" - Confirmation that `./CLAUDE.md` and `./AGNES_WORKSPACE.md` exist",
|
||||
" - Confirmation that `./.claude/settings.json` contains SessionStart/End hooks",
|
||||
]
|
||||
|
|
@ -851,7 +938,7 @@ def test_admin_prompt_template_renders_banner_when_legacy_present(web_session):
|
|||
|
||||
def test_admin_prompt_template_no_banner_when_clean(web_session):
|
||||
web_session.put("/api/admin/workspace-prompt-template",
|
||||
json={"content": "Run `da pull` daily."})
|
||||
json={"content": "Run `agnes pull` daily."})
|
||||
resp = web_session.get("/admin/workspace-prompt")
|
||||
assert resp.status_code == 200
|
||||
# The banner div is absent or empty
|
||||
|
|
@ -935,11 +1022,11 @@ wc -l config/claude_md_template.txt
|
|||
- [ ] **Step 2: Apply systematic rewrites**
|
||||
|
||||
Replace throughout the file:
|
||||
- `da sync` → `da pull` (everywhere)
|
||||
- `da analyst setup` → `da init` (everywhere)
|
||||
- `da fetch` → `da snapshot create`
|
||||
- `da metrics list` → `da catalog --metrics`
|
||||
- `da metrics show` → `da catalog --metrics --show`
|
||||
- `da sync` → `agnes pull` (everywhere)
|
||||
- `da analyst setup` → `agnes init` (everywhere)
|
||||
- `da fetch` → `agnes snapshot create`
|
||||
- `da metrics list` → `agnes catalog --metrics`
|
||||
- `da metrics show` → `agnes catalog --metrics --show`
|
||||
- `data/parquet/` → `server/parquet/`
|
||||
- `data/duckdb/` → `user/duckdb/`
|
||||
- `data/metadata/` → (delete references; the path no longer exists)
|
||||
|
|
@ -948,12 +1035,12 @@ Use `sed`:
|
|||
|
||||
```bash
|
||||
sed -i.bak \
|
||||
-e 's|da sync --upload-only|da push|g' \
|
||||
-e 's|da sync|da pull|g' \
|
||||
-e 's|da analyst setup|da init|g' \
|
||||
-e 's|da fetch|da snapshot create|g' \
|
||||
-e 's|da metrics list|da catalog --metrics|g' \
|
||||
-e 's|da metrics show|da catalog --metrics --show|g' \
|
||||
-e 's|da sync --upload-only|agnes push|g' \
|
||||
-e 's|da sync|agnes pull|g' \
|
||||
-e 's|da analyst setup|agnes init|g' \
|
||||
-e 's|da fetch|agnes snapshot create|g' \
|
||||
-e 's|da metrics list|agnes catalog --metrics|g' \
|
||||
-e 's|da metrics show|agnes catalog --metrics --show|g' \
|
||||
-e 's|data/parquet/|server/parquet/|g' \
|
||||
-e 's|data/duckdb/|user/duckdb/|g' \
|
||||
config/claude_md_template.txt
|
||||
|
|
@ -974,7 +1061,7 @@ Expected: no matches.
|
|||
Insert near the top of the rendered template (e.g., after the `# {instance_name}` heading):
|
||||
|
||||
```markdown
|
||||
> Looking for human-readable workspace docs? Open `AGNES_WORKSPACE.md` in this directory — that file documents what `da init` installed, where files live, and how to uninstall.
|
||||
> Looking for human-readable workspace docs? Open `AGNES_WORKSPACE.md` in this directory — that file documents what `agnes init` installed, where files live, and how to uninstall.
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Render the template via `/api/welcome` (manual smoke)**
|
||||
|
|
@ -1020,9 +1107,9 @@ def test_install_creates_settings_file(tmp_path):
|
|||
install_claude_hooks(tmp_path)
|
||||
cfg = _read_settings(tmp_path)
|
||||
assert cfg["hooks"]["SessionStart"]
|
||||
assert "da pull --quiet" in cfg["hooks"]["SessionStart"][0]["hooks"][0]["command"]
|
||||
assert "agnes pull --quiet" in cfg["hooks"]["SessionStart"][0]["hooks"][0]["command"]
|
||||
assert cfg["hooks"]["SessionEnd"]
|
||||
assert "da push --quiet" in cfg["hooks"]["SessionEnd"][0]["hooks"][0]["command"]
|
||||
assert "agnes push --quiet" in cfg["hooks"]["SessionEnd"][0]["hooks"][0]["command"]
|
||||
|
||||
|
||||
def test_install_idempotent(tmp_path):
|
||||
|
|
@ -1047,7 +1134,7 @@ def test_install_replaces_old_da_sync_entries(tmp_path):
|
|||
install_claude_hooks(tmp_path)
|
||||
cfg = _read_settings(tmp_path)
|
||||
assert len(cfg["hooks"]["SessionStart"]) == 1
|
||||
assert "da pull" in cfg["hooks"]["SessionStart"][0]["hooks"][0]["command"]
|
||||
assert "agnes pull" in cfg["hooks"]["SessionStart"][0]["hooks"][0]["command"]
|
||||
assert "da sync" not in cfg["hooks"]["SessionStart"][0]["hooks"][0]["command"]
|
||||
|
||||
|
||||
|
|
@ -1062,10 +1149,10 @@ def test_install_preserves_third_party_hooks(tmp_path):
|
|||
}))
|
||||
install_claude_hooks(tmp_path)
|
||||
cfg = _read_settings(tmp_path)
|
||||
# Third-party SessionStart entry survives; our da pull entry appended
|
||||
# Third-party SessionStart entry survives; our agnes pull entry appended
|
||||
starts = cfg["hooks"]["SessionStart"]
|
||||
assert any("echo hi from another tool" in s["hooks"][0]["command"] for s in starts)
|
||||
assert any("da pull" in s["hooks"][0]["command"] for s in starts)
|
||||
assert any("agnes pull" in s["hooks"][0]["command"] for s in starts)
|
||||
# PreToolUse untouched
|
||||
assert cfg["hooks"]["PreToolUse"][0]["hooks"][0]["command"] == "echo pre"
|
||||
|
||||
|
|
@ -1109,14 +1196,14 @@ touch cli/lib/__init__.py
|
|||
|
||||
Replaces the in-place `_install_claude_hooks` from `cli/commands/analyst.py`
|
||||
(deleted as part of the clean-analyst-bootstrap rewrite). Splits hook
|
||||
installation into a pure-function library so `da init` and any future caller
|
||||
installation into a pure-function library so `agnes init` and any future caller
|
||||
can use it without dragging in the deleted command module.
|
||||
|
||||
Design notes:
|
||||
- Workspace-scoped (`<workspace>/.claude/settings.json`), NOT user-home.
|
||||
The hooks fire only when Claude Code opens this workspace.
|
||||
- Idempotent: second invocation drops a prior `da pull` / `da sync` /
|
||||
`da push` entry (matched by command substring) and appends fresh entries.
|
||||
- Idempotent: second invocation drops a prior `agnes pull` / `da sync` /
|
||||
`agnes push` entry (matched by command substring) and appends fresh entries.
|
||||
Third-party hooks (mixed entries, foreign commands) are left alone.
|
||||
- Uses `\\| true` so the hook never blocks a session on a transient sync error.
|
||||
"""
|
||||
|
|
@ -1127,11 +1214,11 @@ import json
|
|||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_OUR_COMMAND_MARKERS = ("da pull", "da push", "da sync")
|
||||
_OUR_COMMAND_MARKERS = ("agnes pull", "agnes push", "da sync")
|
||||
|
||||
|
||||
def install_claude_hooks(workspace: Path) -> None:
|
||||
"""Install SessionStart→`da pull` and SessionEnd→`da push` hooks.
|
||||
"""Install SessionStart→`agnes pull` and SessionEnd→`agnes push` hooks.
|
||||
|
||||
Idempotent. Workspace-scoped (writes `<workspace>/.claude/settings.json`).
|
||||
Preserves third-party hooks and other event types.
|
||||
|
|
@ -1165,8 +1252,8 @@ def install_claude_hooks(workspace: Path) -> None:
|
|||
existing.remove(entry)
|
||||
existing.append({"hooks": [{"type": "command", "command": command}]})
|
||||
|
||||
_replace_or_add("SessionStart", "da pull --quiet 2>/dev/null || true")
|
||||
_replace_or_add("SessionEnd", "da push --quiet 2>/dev/null || true")
|
||||
_replace_or_add("SessionStart", "agnes pull --quiet 2>/dev/null || true")
|
||||
_replace_or_add("SessionEnd", "agnes push --quiet 2>/dev/null || true")
|
||||
|
||||
settings_path.write_text(json.dumps(cfg, indent=2) + "\n", encoding="utf-8")
|
||||
```
|
||||
|
|
@ -1313,7 +1400,7 @@ Lift the body of today's `cli/commands/sync.py:sync()` into a pure function. Spe
|
|||
|
||||
```python
|
||||
# cli/lib/pull.py
|
||||
"""Pure-function data-refresh primitive — used by `da pull` and `da init`.
|
||||
"""Pure-function data-refresh primitive — used by `agnes pull` and `agnes init`.
|
||||
|
||||
Extracted from `cli/commands/sync.py` (deleted in the clean-bootstrap rewrite).
|
||||
This module has no Typer dependency, no stdout side effects, no exit calls.
|
||||
|
|
@ -1537,7 +1624,7 @@ git commit -m "feat(cli-lib): cli/lib/pull.py:run_pull primitive with lazy mkdir
|
|||
|
||||
## Phase 3 — New CLI commands
|
||||
|
||||
### Task 9: `da pull` Typer wrapper
|
||||
### Task 9: `agnes pull` Typer wrapper
|
||||
|
||||
**Files:**
|
||||
- Create: `cli/commands/pull.py`
|
||||
|
|
@ -1547,7 +1634,7 @@ git commit -m "feat(cli-lib): cli/lib/pull.py:run_pull primitive with lazy mkdir
|
|||
|
||||
```python
|
||||
# tests/test_cli_pull.py
|
||||
"""Tests for `da pull` Typer wrapper."""
|
||||
"""Tests for `agnes pull` Typer wrapper."""
|
||||
|
||||
from typer.testing import CliRunner
|
||||
from cli.commands.pull import pull_app
|
||||
|
|
@ -1575,11 +1662,11 @@ Expected: ImportError.
|
|||
|
||||
```python
|
||||
# cli/commands/pull.py
|
||||
"""`da pull` — refresh registered data into the workspace.
|
||||
"""`agnes pull` — refresh registered data into the workspace.
|
||||
|
||||
Thin Typer wrapper around `cli/lib/pull.py:run_pull`. Used by:
|
||||
- Manual invocation: analyst types `da pull` to force a refresh.
|
||||
- SessionStart hook: `da pull --quiet 2>/dev/null || true` runs at the start
|
||||
- Manual invocation: analyst types `agnes pull` to force a refresh.
|
||||
- SessionStart hook: `agnes pull --quiet 2>/dev/null || true` runs at the start
|
||||
of every Claude Code session in this workspace.
|
||||
"""
|
||||
|
||||
|
|
@ -1610,7 +1697,7 @@ def pull(
|
|||
if not server_url:
|
||||
typer.echo(render_error(0, {"detail": {
|
||||
"kind": "server_unreachable",
|
||||
"hint": "No server configured. Run: da init --server-url <URL> --token <PAT>",
|
||||
"hint": "No server configured. Run: agnes init --server-url <URL> --token <PAT>",
|
||||
}}), err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
|
@ -1618,7 +1705,7 @@ def pull(
|
|||
if not token:
|
||||
typer.echo(render_error(0, {"detail": {
|
||||
"kind": "auth_failed",
|
||||
"hint": "No token. Run: da auth import-token --token <PAT>",
|
||||
"hint": "No token. Run: agnes auth import-token --token <PAT>",
|
||||
}}), err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
|
@ -1669,12 +1756,12 @@ Expected: PASS.
|
|||
|
||||
```bash
|
||||
git add cli/commands/pull.py tests/test_cli_pull.py
|
||||
git commit -m "feat(cli): da pull command (Typer wrapper around lib.pull.run_pull)"
|
||||
git commit -m "feat(cli): agnes pull command (Typer wrapper around lib.pull.run_pull)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 10: `da push` command (extract from `da sync --upload-only`)
|
||||
### Task 10: `agnes push` command (extract from `da sync --upload-only`)
|
||||
|
||||
**Files:**
|
||||
- Create: `cli/commands/push.py`
|
||||
|
|
@ -1719,10 +1806,10 @@ pytest tests/test_cli_push.py -v
|
|||
|
||||
```python
|
||||
# cli/commands/push.py
|
||||
"""`da push` — upload local sessions and CLAUDE.local.md to the server.
|
||||
"""`agnes push` — upload local sessions and CLAUDE.local.md to the server.
|
||||
|
||||
Extracted from today's `da sync --upload-only`. Hook command:
|
||||
`da push --quiet 2>/dev/null || true` (runs at SessionEnd).
|
||||
`agnes push --quiet 2>/dev/null || true` (runs at SessionEnd).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -1752,7 +1839,7 @@ def push(
|
|||
if not server_url or not token:
|
||||
typer.echo(render_error(0, {"detail": {
|
||||
"kind": "auth_failed",
|
||||
"hint": "No server/token configured. Run: da init",
|
||||
"hint": "No server/token configured. Run: agnes init",
|
||||
}}), err=True)
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
|
@ -1809,12 +1896,12 @@ pytest tests/test_cli_push.py -v
|
|||
|
||||
```bash
|
||||
git add cli/commands/push.py tests/test_cli_push.py
|
||||
git commit -m "feat(cli): da push command (extracted from sync --upload-only)"
|
||||
git commit -m "feat(cli): agnes push command (extracted from sync --upload-only)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 11: `da init` — workspace bootstrap orchestrator
|
||||
### Task 11: `agnes init` — workspace bootstrap orchestrator
|
||||
|
||||
**Files:**
|
||||
- Create: `cli/commands/init.py`, `config/agnes_workspace_template.txt`
|
||||
|
|
@ -1831,7 +1918,7 @@ Create `config/agnes_workspace_template.txt`:
|
|||
**Server:** {server_url}
|
||||
**Workspace:** {workspace_path}
|
||||
|
||||
This file documents what `da init` installed on this machine and in this folder.
|
||||
This file documents what `agnes init` installed on this machine and in this folder.
|
||||
Read this when you want to know "what is this thing", "how does it work", or
|
||||
"how do I uninstall it". For Claude Code's instructions, see `CLAUDE.md`.
|
||||
|
||||
|
|
@ -1841,7 +1928,7 @@ Read this when you want to know "what is this thing", "how does it work", or
|
|||
|
||||
| Path | What it is | How to remove |
|
||||
|------|------------|---------------|
|
||||
| `~/.local/bin/da` | The `da` CLI binary | `uv tool uninstall agnes-the-ai-analyst` |
|
||||
| `~/.local/bin/da` | The `agnes` CLI binary | `uv tool uninstall agnes-the-ai-analyst` |
|
||||
| `~/.config/da/config.yaml` | Default Agnes server URL | `rm -rf ~/.config/da/` |
|
||||
| `~/.config/da/token.json` | Personal access token (PAT) | `rm ~/.config/da/token.json` |
|
||||
| `~/.agnes/ca.pem` | Server's CA cert (private CA installs only) | `rm ~/.agnes/ca.pem` |
|
||||
|
|
@ -1860,11 +1947,11 @@ Read this when you want to know "what is this thing", "how does it work", or
|
|||
| `./.claude/CLAUDE.local.md` | Your private notes (uploaded on session end) |
|
||||
| `./.claude/rules/km_*.md` | Server-pushed corporate-knowledge rules (only when granted) |
|
||||
| `./server/parquet/*.parquet` | Synced data — RBAC-filtered subset (only when grants exist) |
|
||||
| `./user/duckdb/analytics.duckdb` | DuckDB views over the parquets — what `da query` reads |
|
||||
| `./user/snapshots/*.parquet` | Ad-hoc materialized snapshots from `da snapshot create` |
|
||||
| `./user/duckdb/analytics.duckdb` | DuckDB views over the parquets — what `agnes query` reads |
|
||||
| `./user/snapshots/*.parquet` | Ad-hoc materialized snapshots from `agnes snapshot create` |
|
||||
| `./user/sessions/*.jsonl` | Captured Claude Code sessions (uploaded on session end) |
|
||||
|
||||
Some folders only exist when they have content — `da pull` and `da push`
|
||||
Some folders only exist when they have content — `agnes pull` and `agnes push`
|
||||
only create them when there's something to write.
|
||||
|
||||
---
|
||||
|
|
@ -1874,10 +1961,10 @@ only create them when there's something to write.
|
|||
Two hooks in `./.claude/settings.json` keep this workspace in sync without
|
||||
you doing anything:
|
||||
|
||||
- **SessionStart** → `da pull --quiet` — new parquets, schema changes, and
|
||||
- **SessionStart** → `agnes pull --quiet` — new parquets, schema changes, and
|
||||
updated rules pull down before Claude Code answers. Failure is silent;
|
||||
your session continues with the last-known data.
|
||||
- **SessionEnd** → `da push --quiet` — your session transcript and
|
||||
- **SessionEnd** → `agnes push --quiet` — your session transcript and
|
||||
`CLAUDE.local.md` ship to the server.
|
||||
|
||||
Both are workspace-scoped — they only run when Claude Code opens this folder.
|
||||
|
|
@ -1888,33 +1975,33 @@ Both are workspace-scoped — they only run when Claude Code opens this folder.
|
|||
|
||||
```bash
|
||||
# Tables you can read (server-side catalog, RBAC-filtered)
|
||||
da catalog
|
||||
da catalog --json | jq '.[] | select(.query_mode=="local")'
|
||||
agnes catalog
|
||||
agnes catalog --json | jq '.[] | select(.query_mode=="local")'
|
||||
|
||||
# Schema and sample
|
||||
da schema opportunity
|
||||
da describe opportunity -n 10
|
||||
agnes schema opportunity
|
||||
agnes describe opportunity -n 10
|
||||
|
||||
# Run a SQL query (DuckDB flavor against local parquets)
|
||||
da query "SELECT count(*) FROM opportunity WHERE stage='Closed Won'"
|
||||
agnes query "SELECT count(*) FROM opportunity WHERE stage='Closed Won'"
|
||||
|
||||
# Remote BigQuery query (server-side, no local materialization)
|
||||
da query --remote "SELECT count(*) FROM web_sessions_example"
|
||||
agnes query --remote "SELECT count(*) FROM web_sessions_example"
|
||||
|
||||
# Materialize a remote subset locally
|
||||
da snapshot create web_sessions_example \
|
||||
agnes snapshot create web_sessions_example \
|
||||
--select event_date,country_code \
|
||||
--where "event_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY)" \
|
||||
--as recent_sessions
|
||||
|
||||
# Manual data refresh (the SessionStart hook does this automatically)
|
||||
da pull
|
||||
agnes pull
|
||||
|
||||
# Workspace status (what's synced, when)
|
||||
da status
|
||||
agnes status
|
||||
|
||||
# Re-generate this workspace from scratch (preserves CLAUDE.local.md)
|
||||
da init --server-url https://agnes.example.com --token <PAT> --force
|
||||
agnes init --server-url https://agnes.example.com --token <PAT> --force
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -1938,11 +2025,11 @@ rm -rf ./CLAUDE.md ./AGNES_WORKSPACE.md ./.claude ./server ./user
|
|||
```
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write failing tests for `da init`**
|
||||
- [ ] **Step 2: Write failing tests for `agnes init`**
|
||||
|
||||
```python
|
||||
# tests/test_cli_init.py
|
||||
"""Tests for `da init` orchestrator command."""
|
||||
"""Tests for `agnes init` orchestrator command."""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
|
@ -1973,7 +2060,7 @@ def test_init_writes_expected_files(tmp_path, monkeypatch):
|
|||
if path == "/api/catalog/tables":
|
||||
resp.json.return_value = []
|
||||
elif path == "/api/welcome":
|
||||
resp.json.return_value = {"content": "# Test CLAUDE.md\n\nUse `da pull`.\n"}
|
||||
resp.json.return_value = {"content": "# Test CLAUDE.md\n\nUse `agnes pull`.\n"}
|
||||
elif path == "/api/sync/manifest":
|
||||
resp.json.return_value = {"tables": []}
|
||||
elif path == "/api/memory/bundle":
|
||||
|
|
@ -1989,7 +2076,7 @@ def test_init_writes_expected_files(tmp_path, monkeypatch):
|
|||
])
|
||||
assert result.exit_code == 0, result.output
|
||||
assert (tmp_path / "CLAUDE.md").exists()
|
||||
assert "da pull" in (tmp_path / "CLAUDE.md").read_text()
|
||||
assert "agnes pull" in (tmp_path / "CLAUDE.md").read_text()
|
||||
assert (tmp_path / ".claude" / "settings.json").exists()
|
||||
assert (tmp_path / ".claude" / "CLAUDE.local.md").exists()
|
||||
assert (tmp_path / "AGNES_WORKSPACE.md").exists()
|
||||
|
|
@ -2057,10 +2144,10 @@ pytest tests/test_cli_init.py -v
|
|||
|
||||
```python
|
||||
# cli/commands/init.py
|
||||
"""`da init` — bootstrap an analyst workspace.
|
||||
"""`agnes init` — bootstrap an analyst workspace.
|
||||
|
||||
Single-paste flow: web user clicks "Generate prompt" on /setup?role=analyst,
|
||||
pastes into Claude Code in an empty folder; Claude runs `da init` (among other
|
||||
pastes into Claude Code in an empty folder; Claude runs `agnes init` (among other
|
||||
steps). Non-interactive: --token + --server-url required.
|
||||
"""
|
||||
|
||||
|
|
@ -2152,7 +2239,7 @@ def init(
|
|||
local_md = workspace / ".claude" / "CLAUDE.local.md"
|
||||
if not local_md.exists():
|
||||
local_md.write_text(
|
||||
"# My Notes\n\nPersonal notes for this workspace. Uploaded on `da push`.\n",
|
||||
"# My Notes\n\nPersonal notes for this workspace. Uploaded on `agnes push`.\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
|
@ -2187,7 +2274,7 @@ def init(
|
|||
typer.echo(f" Rules : {result.rules_count}")
|
||||
typer.echo(f" Workspace: {workspace}")
|
||||
typer.echo("")
|
||||
typer.echo("Try: da catalog")
|
||||
typer.echo("Try: agnes catalog")
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass**
|
||||
|
|
@ -2200,12 +2287,12 @@ pytest tests/test_cli_init.py -v
|
|||
|
||||
```bash
|
||||
git add cli/commands/init.py config/agnes_workspace_template.txt tests/test_cli_init.py
|
||||
git commit -m "feat(cli): da init orchestrator + AGNES_WORKSPACE.md template"
|
||||
git commit -m "feat(cli): agnes init orchestrator + AGNES_WORKSPACE.md template"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 12: New `da status` (workspace status, replaces `da analyst status`)
|
||||
### Task 12: New `agnes status` (workspace status, replaces `da analyst status`)
|
||||
|
||||
**Files:**
|
||||
- Modify (overwrite): `cli/commands/status.py`
|
||||
|
|
@ -2217,7 +2304,7 @@ git commit -m "feat(cli): da init orchestrator + AGNES_WORKSPACE.md template"
|
|||
cat cli/commands/status.py
|
||||
```
|
||||
|
||||
The existing `da status` shows server health. Per spec, this content moves to `da diagnose system` (Task 13); the file is repurposed for workspace status.
|
||||
The existing `agnes status` shows server health. Per spec, this content moves to `agnes diagnose system` (Task 13); the file is repurposed for workspace status.
|
||||
|
||||
- [ ] **Step 2: Write failing tests**
|
||||
|
||||
|
|
@ -2272,7 +2359,7 @@ pytest tests/test_cli_status.py -v
|
|||
|
||||
```python
|
||||
# cli/commands/status.py
|
||||
"""`da status` — workspace status: initialized? data fresh? hooks active?"""
|
||||
"""`agnes status` — workspace status: initialized? data fresh? hooks active?"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
|
@ -2334,7 +2421,7 @@ def status(
|
|||
|
||||
if not initialized:
|
||||
typer.echo("")
|
||||
typer.echo("Run `da init --server-url <URL> --token <PAT>` to bootstrap.")
|
||||
typer.echo("Run `agnes init --server-url <URL> --token <PAT>` to bootstrap.")
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run tests to verify they pass**
|
||||
|
|
@ -2347,18 +2434,18 @@ pytest tests/test_cli_status.py -v
|
|||
|
||||
```bash
|
||||
git add cli/commands/status.py tests/test_cli_status.py
|
||||
git commit -m "feat(cli): da status now shows workspace state (was system health)"
|
||||
git commit -m "feat(cli): agnes status now shows workspace state (was system health)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 13: Move old `da status` content into `da diagnose system`
|
||||
### Task 13: Move old `agnes status` content into `agnes diagnose system`
|
||||
|
||||
**Files:**
|
||||
- Modify: `cli/commands/diagnose.py` (add `system` subcommand with the old status logic)
|
||||
- Test: `tests/test_cli_diagnose_system.py` (new)
|
||||
|
||||
- [ ] **Step 1: Recover old `da status` logic**
|
||||
- [ ] **Step 1: Recover old `agnes status` logic**
|
||||
|
||||
```bash
|
||||
git show HEAD~12:cli/commands/status.py
|
||||
|
|
@ -2394,12 +2481,12 @@ def test_diagnose_system_help():
|
|||
|
||||
```bash
|
||||
git add cli/commands/diagnose.py tests/test_cli_diagnose_system.py
|
||||
git commit -m "refactor(cli): move old `da status` health check to `da diagnose system`"
|
||||
git commit -m "refactor(cli): move old `agnes status` health check to `agnes diagnose system`"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 14: `da snapshot create` — fold `da fetch` into snapshot group
|
||||
### Task 14: `agnes snapshot create` — fold `da fetch` into snapshot group
|
||||
|
||||
**Files:**
|
||||
- Modify: `cli/commands/snapshot.py` (add `create` subcommand)
|
||||
|
|
@ -2419,7 +2506,7 @@ Move the body of `fetch.py:fetch()` into a new `@snapshot_app.command("create")`
|
|||
```python
|
||||
local_db = _local_dir() / "user" / "duckdb" / "analytics.duckdb"
|
||||
if not local_db.exists():
|
||||
typer.echo("Local DuckDB not found. Run: da pull first.", err=True)
|
||||
typer.echo("Local DuckDB not found. Run: agnes pull first.", err=True)
|
||||
raise typer.Exit(1)
|
||||
# (then proceed with duckdb.connect — no longer creates an empty DB)
|
||||
```
|
||||
|
|
@ -2445,7 +2532,7 @@ def test_snapshot_create_no_duckdb_friendly_exit(tmp_path, monkeypatch):
|
|||
runner = CliRunner()
|
||||
result = runner.invoke(snapshot_app, ["create", "any_table", "--as", "x", "--estimate"])
|
||||
assert result.exit_code == 1
|
||||
assert "Run: da pull" in result.output or "Run: da pull" in (result.stderr or "")
|
||||
assert "Run: agnes pull" in result.output or "Run: agnes pull" in (result.stderr or "")
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run tests**
|
||||
|
|
@ -2458,12 +2545,12 @@ pytest tests/test_cli_snapshot_create.py -v
|
|||
|
||||
```bash
|
||||
git add cli/commands/snapshot.py tests/test_cli_snapshot_create.py
|
||||
git commit -m "feat(cli): da snapshot create (folded from da fetch); friendly exit if no DuckDB"
|
||||
git commit -m "feat(cli): agnes snapshot create (folded from da fetch); friendly exit if no DuckDB"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 15: `da catalog --metrics` — fold `da metrics list/show`
|
||||
### Task 15: `agnes catalog --metrics` — fold `da metrics list/show`
|
||||
|
||||
**Files:**
|
||||
- Modify: `cli/commands/catalog.py`
|
||||
|
|
@ -2515,12 +2602,12 @@ def test_catalog_metrics_help():
|
|||
|
||||
```bash
|
||||
git add cli/commands/catalog.py tests/test_cli_catalog_metrics.py
|
||||
git commit -m "feat(cli): da catalog --metrics replaces da metrics list/show"
|
||||
git commit -m "feat(cli): agnes catalog --metrics replaces da metrics list/show"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 16: Move `da metrics import/export/validate` to `da admin metrics`
|
||||
### Task 16: Move `da metrics import/export/validate` to `agnes admin metrics`
|
||||
|
||||
**Files:**
|
||||
- Create: `cli/commands/admin_metrics.py`
|
||||
|
|
@ -2532,7 +2619,7 @@ Lift `import_metrics`, `export_metrics`, `validate_metrics` from `cli/commands/m
|
|||
|
||||
```python
|
||||
# cli/commands/admin_metrics.py
|
||||
"""`da admin metrics {import,export,validate}` — lifted from metrics.py."""
|
||||
"""`agnes admin metrics {import,export,validate}` — lifted from metrics.py."""
|
||||
|
||||
import typer
|
||||
|
||||
|
|
@ -2589,7 +2676,7 @@ def test_admin_metrics_subcommands_present():
|
|||
|
||||
```bash
|
||||
git add cli/commands/admin_metrics.py cli/commands/admin.py tests/test_cli_admin_metrics.py
|
||||
git commit -m "feat(cli): da admin metrics {import,export,validate}"
|
||||
git commit -m "feat(cli): agnes admin metrics {import,export,validate}"
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -2607,10 +2694,10 @@ git commit -m "feat(cli): da admin metrics {import,export,validate}"
|
|||
grep -rn "Run: da sync" cli/
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Replace with "Run: da pull"**
|
||||
- [ ] **Step 2: Replace with "Run: agnes pull"**
|
||||
|
||||
```bash
|
||||
sed -i.bak 's/Run: da sync/Run: da pull/g' cli/commands/query.py cli/commands/explore.py
|
||||
sed -i.bak 's/Run: da sync/Run: agnes pull/g' cli/commands/query.py cli/commands/explore.py
|
||||
rm cli/commands/query.py.bak cli/commands/explore.py.bak
|
||||
```
|
||||
|
||||
|
|
@ -2626,7 +2713,7 @@ Expected: no matches.
|
|||
|
||||
```bash
|
||||
git add cli/commands/query.py cli/commands/explore.py
|
||||
git commit -m "fix(cli): hint text 'Run: da sync' → 'Run: da pull'"
|
||||
git commit -m "fix(cli): hint text 'Run: da sync' → 'Run: agnes pull'"
|
||||
```
|
||||
|
||||
---
|
||||
|
|
@ -2729,13 +2816,13 @@ git commit -m "refactor(cli): drop sync/fetch/analyst/metrics; register init/pul
|
|||
|
||||
```bash
|
||||
sed -i.bak \
|
||||
-e 's|da sync --upload-only|da push|g' \
|
||||
-e 's|da sync|da pull|g' \
|
||||
-e 's|da analyst setup|da init|g' \
|
||||
-e 's|da fetch|da snapshot create|g' \
|
||||
-e 's|da metrics list|da catalog --metrics|g' \
|
||||
-e 's|da metrics show|da catalog --metrics --show|g' \
|
||||
-e 's|da metrics import|da admin metrics import|g' \
|
||||
-e 's|da sync --upload-only|agnes push|g' \
|
||||
-e 's|da sync|agnes pull|g' \
|
||||
-e 's|da analyst setup|agnes init|g' \
|
||||
-e 's|da fetch|agnes snapshot create|g' \
|
||||
-e 's|da metrics list|agnes catalog --metrics|g' \
|
||||
-e 's|da metrics show|agnes catalog --metrics --show|g' \
|
||||
-e 's|da metrics import|agnes admin metrics import|g' \
|
||||
-e 's|data/parquet/|server/parquet/|g' \
|
||||
-e 's|data/duckdb/|user/duckdb/|g' \
|
||||
CLAUDE.md
|
||||
|
|
@ -2744,24 +2831,24 @@ rm CLAUDE.md.bak
|
|||
|
||||
- [ ] **Step 2: Manually rewrite the "Local sync & Claude Code hooks" subsection**
|
||||
|
||||
Find the section. Replace the surrounding prose so it describes `da pull` + `da push` hooks:
|
||||
Find the section. Replace the surrounding prose so it describes `agnes pull` + `agnes push` hooks:
|
||||
|
||||
```markdown
|
||||
### Local sync & Claude Code hooks
|
||||
|
||||
`da pull` is the canonical analyst-side distribution path: pulls the
|
||||
`agnes pull` is the canonical analyst-side distribution path: pulls the
|
||||
RBAC-filtered manifest from the server, downloads parquets whose MD5 changed
|
||||
(skipping `query_mode='remote'` rows), rebuilds local DuckDB views over them.
|
||||
`da push` mirrors it for the upload direction (sessions, CLAUDE.local.md).
|
||||
`agnes push` mirrors it for the upload direction (sessions, CLAUDE.local.md).
|
||||
|
||||
`da init` writes two hooks into `<workspace>/.claude/settings.json`:
|
||||
`agnes init` writes two hooks into `<workspace>/.claude/settings.json`:
|
||||
|
||||
- `SessionStart` → `da pull --quiet` — pulls fresh parquets at the start of every Claude Code session
|
||||
- `SessionEnd` → `da push --quiet` — uploads session jsonl + `CLAUDE.local.md` to the server
|
||||
- `SessionStart` → `agnes pull --quiet` — pulls fresh parquets at the start of every Claude Code session
|
||||
- `SessionEnd` → `agnes push --quiet` — uploads session jsonl + `CLAUDE.local.md` to the server
|
||||
|
||||
Both pass `--quiet` so they don't pollute Claude Code stdout, and trail with `|| true` so a server outage never blocks a session. Workspace-level (not user-home) so the hooks fire only when Claude Code opens this analyst workspace, not in unrelated sessions on the same machine.
|
||||
|
||||
Admin RBAC for auto-sync: `query_mode IN ('local', 'materialized')` plus a `resource_grants` row for one of the analyst's groups → table appears in their manifest → `da pull` downloads it. No per-user sync config; the admin layer is the single source of truth.
|
||||
Admin RBAC for auto-sync: `query_mode IN ('local', 'materialized')` plus a `resource_grants` row for one of the analyst's groups → table appears in their manifest → `agnes pull` downloads it. No per-user sync config; the admin layer is the single source of truth.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify no leftover legacy strings**
|
||||
|
|
@ -2960,7 +3047,7 @@ def test_pat_no_grants(web_session) -> str:
|
|||
|
||||
@pytest.fixture
|
||||
def zero_grants_workspace(tmp_path, fastapi_test_server, test_pat_no_grants) -> Path:
|
||||
"""Run `da init` against a no-grants PAT; return the workspace path."""
|
||||
"""Run `agnes init` against a no-grants PAT; return the workspace path."""
|
||||
workspace = tmp_path / "workspace"
|
||||
workspace.mkdir()
|
||||
result = subprocess.run([
|
||||
|
|
@ -3090,7 +3177,7 @@ git commit -m "test: reader smoke matrix on zero-grants workspace"
|
|||
|
||||
```python
|
||||
# tests/test_clean_install_integration.py
|
||||
"""End-to-end clean-install integration tests for `da init`."""
|
||||
"""End-to-end clean-install integration tests for `agnes init`."""
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
|
|
@ -3133,13 +3220,13 @@ def test_clean_install_minimal_grants(fastapi_test_server, tmp_path, test_pat):
|
|||
assert_no_dead_dirs(workspace)
|
||||
|
||||
settings = json.loads((workspace / ".claude" / "settings.json").read_text())
|
||||
assert any("da pull" in h["hooks"][0]["command"]
|
||||
assert any("agnes pull" in h["hooks"][0]["command"]
|
||||
for h in settings["hooks"]["SessionStart"])
|
||||
assert any("da push" in h["hooks"][0]["command"]
|
||||
assert any("agnes push" in h["hooks"][0]["command"]
|
||||
for h in settings["hooks"]["SessionEnd"])
|
||||
|
||||
claude_md = (workspace / "CLAUDE.md").read_text()
|
||||
assert "da pull" in claude_md
|
||||
assert "agnes pull" in claude_md
|
||||
assert "da sync" not in claude_md
|
||||
|
||||
workspace_md = (workspace / "AGNES_WORKSPACE.md").read_text()
|
||||
|
|
@ -3148,7 +3235,7 @@ def test_clean_install_minimal_grants(fastapi_test_server, tmp_path, test_pat):
|
|||
assert placeholder not in workspace_md, f"placeholder leaked: {placeholder}"
|
||||
assert fastapi_test_server.url in workspace_md
|
||||
assert str(workspace) in workspace_md
|
||||
assert "da pull" in workspace_md
|
||||
assert "agnes pull" in workspace_md
|
||||
|
||||
|
||||
def test_clean_install_zero_grants(fastapi_test_server, tmp_path, test_pat_no_grants):
|
||||
|
|
@ -3188,7 +3275,7 @@ def test_init_force_preserves_local_md(fastapi_test_server, tmp_path, test_pat):
|
|||
|
||||
|
||||
def test_readers_in_pre_init_dir(tmp_path):
|
||||
"""Reader commands in a folder that never had `da init`. Friendly hints, no tracebacks."""
|
||||
"""Reader commands in a folder that never had `agnes init`. Friendly hints, no tracebacks."""
|
||||
for cmd in [["da", "query", "SELECT 1"],
|
||||
["da", "snapshot", "create", "x", "--as", "y", "--estimate"],
|
||||
["da", "explore", "x"],
|
||||
|
|
@ -3227,7 +3314,7 @@ If `docs/RELEASE_CHECKLIST.md` exists, append; otherwise create with header:
|
|||
|
||||
## Bootstrap path changes (mandatory pre-merge)
|
||||
|
||||
For any PR touching the analyst-bootstrap path (`da init`, `cli/lib/pull.py`,
|
||||
For any PR touching the analyst-bootstrap path (`agnes init`, `cli/lib/pull.py`,
|
||||
`cli/lib/hooks.py`, `app/web/setup_instructions.py`, `/api/welcome`), run
|
||||
this protocol locally before requesting review:
|
||||
|
||||
|
|
@ -3239,10 +3326,10 @@ this protocol locally before requesting review:
|
|||
5. `claude` in that folder. Three queries: "what tables can I see",
|
||||
"SELECT count(*) FROM <t>", "show me last 5 rows of <t>". All must work
|
||||
without further intervention.
|
||||
6. `/exit`. Verify SessionEnd hook ran (server-side audit log shows `da push`;
|
||||
6. `/exit`. Verify SessionEnd hook ran (server-side audit log shows `agnes push`;
|
||||
`du -sh /tmp/test-analyst-1/user/sessions/` non-empty).
|
||||
7. Second `claude` in same folder. Verify SessionStart hook fires
|
||||
(`da pull` request in audit log).
|
||||
(`agnes pull` request in audit log).
|
||||
8. Second workspace `/tmp/test-analyst-2` with the same PAT (within TTL).
|
||||
Repeat 3-5. Verify global `~/.config/da/` is not duplicated; the second
|
||||
workspace has its own DuckDB.
|
||||
|
|
@ -3272,27 +3359,27 @@ Open `CHANGELOG.md`, find the topmost `## [Unreleased]` section (it sits above t
|
|||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
- **BREAKING** Analyst bootstrap rewritten end-to-end. `da analyst setup` is removed; replaced by `da init` (non-interactive, requires `--server-url` and `--token`). `da sync` is split into `da pull` (refresh) and `da push` (upload). `da fetch` is folded into `da snapshot create`. `da metrics list/show` is folded into `da catalog --metrics`; `da metrics import/export/validate` move to `da admin metrics {import,export,validate}`. The `da analyst` namespace is removed; the workspace status command is now `da status`. The previous `da status` (server-health overview) becomes `da diagnose system`.
|
||||
- **BREAKING** Analyst bootstrap rewritten end-to-end. `da analyst setup` is removed; replaced by `agnes init` (non-interactive, requires `--server-url` and `--token`). `da sync` is split into `agnes pull` (refresh) and `agnes push` (upload). `da fetch` is folded into `agnes snapshot create`. `da metrics list/show` is folded into `agnes catalog --metrics`; `da metrics import/export/validate` move to `agnes admin metrics {import,export,validate}`. The `da analyst` namespace is removed; the workspace status command is now `agnes status`. The previous `agnes status` (server-health overview) becomes `agnes diagnose system`.
|
||||
- **BREAKING** Workspace layout simplified. Removed: `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`. Canonical paths: `server/parquet/` (synced parquets), `user/duckdb/analytics.duckdb` (DuckDB views), `user/snapshots/` (ad-hoc snapshots), `user/sessions/` (recorded sessions).
|
||||
- The `/setup` web page now branches on a `role` query parameter: `/setup?role=analyst` renders the analyst workspace bootstrap prompt; `/setup?role=admin` renders the admin CLI install prompt. `/install` continues to 302 to `/setup`.
|
||||
- `CLAUDE.md` server-side template + repo-root `CLAUDE.md` updated to reference the new CLI verbs and workspace paths. The admin UI for the `claude_md_template` DB override (`/admin/workspace-prompt`) renders a yellow banner when the saved override contains legacy strings (`data/parquet/`, `da sync`, `da fetch`, `da analyst setup`, `da metrics list/show`); admins re-author and save to clear it. Migration is manual.
|
||||
|
||||
### Added
|
||||
- `AGNES_WORKSPACE.md` — human-readable workspace docs file generated by `da init` in the workspace root. Documents global install, workspace layout, hooks, cheat sheet, uninstall recipe.
|
||||
- `AGNES_WORKSPACE.md` — human-readable workspace docs file generated by `agnes init` in the workspace root. Documents global install, workspace layout, hooks, cheat sheet, uninstall recipe.
|
||||
- PAT request body now accepts `scope: str = "general"` and `ttl_seconds: int | None = None` fields. PATs minted with `scope="bootstrap-analyst"` are TTL-clamped to ≤ 1 h server-side. Existing `expires_in_days` field continues to work; `ttl_seconds` wins when both are set. `ttl_seconds` upper bound is 315_360_000 (matches `expires_in_days <= 3650` cap).
|
||||
- `cli/lib/` shared-library tree, with `cli/lib/pull.py:run_pull` (data-refresh primitive callable from both the Typer wrapper and `da init`) and `cli/lib/hooks.py:install_claude_hooks` (workspace-scoped Claude Code hook installer).
|
||||
- `cli/lib/` shared-library tree, with `cli/lib/pull.py:run_pull` (data-refresh primitive callable from both the Typer wrapper and `agnes init`) and `cli/lib/hooks.py:install_claude_hooks` (workspace-scoped Claude Code hook installer).
|
||||
|
||||
### Fixed
|
||||
- `da pull` (formerly `da sync`) no longer creates `.claude/rules/` when the corporate-memory bundle is empty.
|
||||
- `da pull` no longer creates `server/parquet/` when the manifest is empty.
|
||||
- `da snapshot create` (formerly `da fetch`) no longer materializes an empty `user/duckdb/analytics.duckdb` when run before any `da pull`.
|
||||
- Workspace `da status` reads from the canonical `server/parquet/` and `user/duckdb/analytics.duckdb` paths (was reading legacy `data/parquet/`, `data/metadata/last_sync.json`).
|
||||
- `da init` and `da pull` errors now use the `cli/error_render.py` typed-error renderer (added in 0.32.0), so analyst-facing error UX matches the structured shape `da query --remote` already produces.
|
||||
- `agnes pull` (formerly `da sync`) no longer creates `.claude/rules/` when the corporate-memory bundle is empty.
|
||||
- `agnes pull` no longer creates `server/parquet/` when the manifest is empty.
|
||||
- `agnes snapshot create` (formerly `da fetch`) no longer materializes an empty `user/duckdb/analytics.duckdb` when run before any `agnes pull`.
|
||||
- Workspace `agnes status` reads from the canonical `server/parquet/` and `user/duckdb/analytics.duckdb` paths (was reading legacy `data/parquet/`, `data/metadata/last_sync.json`).
|
||||
- `agnes init` and `agnes pull` errors now use the `cli/error_render.py` typed-error renderer (added in 0.32.0), so analyst-facing error UX matches the structured shape `agnes query --remote` already produces.
|
||||
|
||||
### Removed
|
||||
- `da analyst setup`, `da analyst status`, `da sync`, `da fetch`. See "Changed" above for replacements.
|
||||
- `da metrics` namespace as a top-level group (subcommands moved to `da catalog --metrics` for read-only views and `da admin metrics …` for write operations).
|
||||
- Legacy workspace directories `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`. Existing analyst workspaces should be reinitialized with `da init --server-url ... --token ... --force` (a fresh empty folder is recommended).
|
||||
- `da metrics` namespace as a top-level group (subcommands moved to `agnes catalog --metrics` for read-only views and `agnes admin metrics …` for write operations).
|
||||
- Legacy workspace directories `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`. Existing analyst workspaces should be reinitialized with `agnes init --server-url ... --token ... --force` (a fresh empty folder is recommended).
|
||||
|
||||
### Internal
|
||||
- `cli/lib/__init__.py` (empty) makes `cli/lib/` a proper package picked up by Hatchling for wheel inclusion.
|
||||
|
|
|
|||
|
|
@ -1,10 +1,12 @@
|
|||
# Clean analyst bootstrap — design
|
||||
|
||||
**Date:** 2026-05-04 (revision 4 — round-3 review fixes; cleared for implementation)
|
||||
**Date:** 2026-05-04 (revision 5 — CLI binary renamed from `da` to `agnes` for branding consistency)
|
||||
**Branch:** `zs/clean-analyst-bootstrap-spec`
|
||||
**Status:** Draft (approved by user, pre-implementation)
|
||||
**Successor to:** today's `da analyst setup` flow (interactive email/password) and the empty-folder bug under `da sync`.
|
||||
|
||||
**CLI binary rename:** As part of this rewrite the CLI binary changes from `da` to `agnes`. References to legacy command names (`da sync`, `da fetch`, `da analyst setup`, `da metrics`) keep their `da` prefix throughout this document — they're historical artifacts being removed. New commands use `agnes` (`agnes init`, `agnes pull`, `agnes push`, `agnes catalog`, …).
|
||||
|
||||
## Problem
|
||||
|
||||
A new analyst should be able to:
|
||||
|
|
@ -12,9 +14,9 @@ A new analyst should be able to:
|
|||
1. Sign in to the Agnes web UI.
|
||||
2. Click a button on `/setup?role=analyst`, copy a single Claude-Code-paste prompt to the clipboard.
|
||||
3. In an empty terminal, in an empty folder, paste the prompt into Claude Code.
|
||||
4. Have Claude Code do **all** of the local setup — install the `da` CLI, trust the server's TLS cert (when needed), authenticate, generate `CLAUDE.md`, install Claude Code hooks, pull the RBAC-allowed parquets, build the local DuckDB views, write a human-readable workspace docs file.
|
||||
4. Have Claude Code do **all** of the local setup — install the `agnes` CLI, trust the server's TLS cert (when needed), authenticate, generate `CLAUDE.md`, install Claude Code hooks, pull the RBAC-allowed parquets, build the local DuckDB views, write a human-readable workspace docs file.
|
||||
5. Immediately start asking questions about the data — without ever typing a follow-up command.
|
||||
6. From the second session onwards, have data freshness handled automatically by hooks (no `da pull` ever typed by hand).
|
||||
6. From the second session onwards, have data freshness handled automatically by hooks (no `agnes pull` ever typed by hand).
|
||||
|
||||
Today this flow does not exist. The closest piece (`da analyst setup` in `cli/commands/analyst.py`) is interactive (prompts for email + password), produces a workspace layout that does not match what `da sync` later writes (the `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/` directories it creates are never read by anything; `da sync` writes parquets to a sibling `server/parquet/` and DuckDB to `user/duckdb/analytics.duckdb`), and never registers the SessionStart/End hooks unless the analyst already managed to authenticate.
|
||||
|
||||
|
|
@ -58,11 +60,11 @@ In addition, `da sync` itself creates empty directories (`.claude/rules/` is `mk
|
|||
│ │
|
||||
│ paste prompt; Claude runs: │
|
||||
│ 0. (TLS trust if needed) │
|
||||
│ 1. uv tool install da │
|
||||
│ 2. da init │
|
||||
│ 1. uv tool install <wheel> # binary: agnes │
|
||||
│ 2. agnes init │
|
||||
│ --server-url URL │
|
||||
│ --token PAT │
|
||||
│ 3. da catalog (smoke) │
|
||||
│ 3. agnes catalog (smoke) │
|
||||
└─────────────┬────────────────┘
|
||||
│
|
||||
▼
|
||||
|
|
@ -75,8 +77,8 @@ In addition, `da sync` itself creates empty directories (`.claude/rules/` is `mk
|
|||
│ ├── AGNES_WORKSPACE.md (human docs) │
|
||||
│ ├── .claude/ │
|
||||
│ │ ├── settings.json (model, perms, │
|
||||
│ │ │ hooks: SessionStart→`da pull`, │
|
||||
│ │ │ SessionEnd →`da push`) │
|
||||
│ │ │ hooks: SessionStart→`agnes pull`, │
|
||||
│ │ │ SessionEnd →`agnes push`) │
|
||||
│ │ ├── CLAUDE.local.md (stub) │
|
||||
│ │ └── rules/km_*.md (only if non-empty) │
|
||||
│ ├── server/parquet/*.parquet │
|
||||
|
|
@ -88,80 +90,80 @@ In addition, `da sync` itself creates empty directories (`.claude/rules/` is `mk
|
|||
│ └── sessions/*.jsonl (lazy, on push)│
|
||||
└────────────────────────────────────────────┘
|
||||
|
||||
next session: SessionStart hook → da pull (incremental MD5)
|
||||
SessionEnd hook → da push (sessions + CLAUDE.local.md)
|
||||
next session: SessionStart hook → agnes pull (incremental MD5)
|
||||
SessionEnd hook → agnes push (sessions + CLAUDE.local.md)
|
||||
```
|
||||
|
||||
Single source of truth for data path: `da pull`. `da init` is a thin orchestrator that does auth + writes templates + installs hooks + calls `da pull` once. No data-path code lives in `init`.
|
||||
Single source of truth for data path: `agnes pull`. `agnes init` is a thin orchestrator that does auth + writes templates + installs hooks + calls `agnes pull` once. No data-path code lives in `init`.
|
||||
|
||||
Single source of truth for the install prompt: `app/web/setup_instructions.py`. New `role: Literal["analyst", "admin"]` parameter branches the step list. TLS trust block is the only piece shared between the two roles.
|
||||
|
||||
Single source of truth for `CLAUDE.md` content: server-side `/api/welcome`. `da init` fetches the rendered text rather than rendering from a client-side template. This means admin-published overrides (DB-stored at `claude_md_template` table, exposed via `/api/admin/workspace-prompt-template`) automatically flow to all analysts. Server-side default template (`config/claude_md_template.txt` or equivalent rendering source) and any DB override **both** need their path strings updated as part of this PR — see "Migration of admin override" in Components.
|
||||
Single source of truth for `CLAUDE.md` content: server-side `/api/welcome`. `agnes init` fetches the rendered text rather than rendering from a client-side template. This means admin-published overrides (DB-stored at `claude_md_template` table, exposed via `/api/admin/workspace-prompt-template`) automatically flow to all analysts. Server-side default template (`config/claude_md_template.txt` or equivalent rendering source) and any DB override **both** need their path strings updated as part of this PR — see "Migration of admin override" in Components.
|
||||
|
||||
Config and PAT live globally per user at `~/.config/da/{config.yaml,token.json}`. There is no per-workspace config in this design.
|
||||
|
||||
## New CLI surface
|
||||
|
||||
The CLI is rewritten with mnemonic, non-overlapping verbs. There are no backward-compat aliases. Today's `da analyst *`, `da sync`, `da sync --upload-only`, `da fetch` are removed; `da metrics list/show` folds into `da catalog --metrics`; `da metrics import/export/validate` move under `da admin`. `da skills list/show` survive as analyst discovery commands; bulk-install variants (none today, but spec refuses to add them) stay out.
|
||||
The CLI is rewritten with mnemonic, non-overlapping verbs. There are no backward-compat aliases. Today's `da analyst *`, `da sync`, `da sync --upload-only`, `da fetch` are removed; `da metrics list/show` folds into `agnes catalog --metrics`; `da metrics import/export/validate` move under `agnes admin`. `agnes skills list/show` survive as analyst discovery commands; bulk-install variants (none today, but spec refuses to add them) stay out.
|
||||
|
||||
```
|
||||
WORKSPACE LIFECYCLE
|
||||
da init one-time workspace bootstrap (--server-url, --token, --force, --workspace)
|
||||
da pull refresh registered data (server → workspace) [--quiet, --json, --dry-run]
|
||||
da push upload sessions + notes (workspace → server) [--quiet, --json, --dry-run]
|
||||
da status what's in this workspace, when last synced
|
||||
agnes init one-time workspace bootstrap (--server-url, --token, --force, --workspace)
|
||||
agnes pull refresh registered data (server → workspace) [--quiet, --json, --dry-run]
|
||||
agnes push upload sessions + notes (workspace → server) [--quiet, --json, --dry-run]
|
||||
agnes status what's in this workspace, when last synced
|
||||
|
||||
DATA QUERY
|
||||
da query "SELECT ..." local DuckDB SQL (over server/parquet/* + user/snapshots/*)
|
||||
da query --remote "SELECT ..." server-side BQ passthrough
|
||||
da explore <view> interactive REPL over a single view
|
||||
da disk-info snapshot disk usage summary
|
||||
agnes query "SELECT ..." local DuckDB SQL (over server/parquet/* + user/snapshots/*)
|
||||
agnes query --remote "SELECT ..." server-side BQ passthrough
|
||||
agnes explore <view> interactive REPL over a single view
|
||||
agnes disk-info snapshot disk usage summary
|
||||
|
||||
DISCOVERY
|
||||
da catalog tables I have access to (RBAC-filtered)
|
||||
da catalog --metrics list metric definitions (replaces da metrics list)
|
||||
da catalog --metrics --show <id> show one metric definition (replaces da metrics show)
|
||||
da schema <table> columns + types
|
||||
da describe <table> sample rows
|
||||
da skills list list bundled CLI skill markdown documents
|
||||
da skills show <name> print one skill's content
|
||||
agnes catalog tables I have access to (RBAC-filtered)
|
||||
agnes catalog --metrics list metric definitions (replaces da metrics list)
|
||||
agnes catalog --metrics --show <id> show one metric definition (replaces da metrics show)
|
||||
agnes schema <table> columns + types
|
||||
agnes describe <table> sample rows
|
||||
agnes skills list list bundled CLI skill markdown documents
|
||||
agnes skills show <name> print one skill's content
|
||||
|
||||
SNAPSHOTS (ad-hoc remote materialization)
|
||||
da snapshot create <table> --as <name> [--select ... --where ... --limit ... --order-by ... --estimate / --no-estimate --force]
|
||||
da snapshot list
|
||||
da snapshot drop <name>
|
||||
da snapshot refresh <name> re-run the snapshot's saved query
|
||||
da snapshot prune drop snapshots older than --older-than
|
||||
agnes snapshot create <table> --as <name> [--select ... --where ... --limit ... --order-by ... --estimate / --no-estimate --force]
|
||||
agnes snapshot list
|
||||
agnes snapshot drop <name>
|
||||
agnes snapshot refresh <name> re-run the snapshot's saved query
|
||||
agnes snapshot prune drop snapshots older than --older-than
|
||||
|
||||
AUTH + IDENTITY
|
||||
da auth login interactive login (browser flow). NOT called by da init.
|
||||
da auth import-token <PAT> non-interactive
|
||||
da auth whoami
|
||||
da auth logout
|
||||
da auth token create / list / revoke (today's location; unchanged by this PR)
|
||||
agnes auth login interactive login (browser flow). NOT called by agnes init.
|
||||
agnes auth import-token <PAT> non-interactive
|
||||
agnes auth whoami
|
||||
agnes auth logout
|
||||
agnes auth token create / list / revoke (today's location; unchanged by this PR)
|
||||
|
||||
HEALTH
|
||||
da diagnose health check (server + local)
|
||||
agnes diagnose health check (server + local)
|
||||
|
||||
ADMIN-ADJACENT (kept; not part of analyst flow)
|
||||
da admin metrics import starter-pack import of metric definitions
|
||||
da admin metrics export dump metric definitions to YAML
|
||||
da admin metrics validate validate metric definitions
|
||||
da admin <other> existing admin verbs continue unchanged
|
||||
agnes admin metrics import starter-pack import of metric definitions
|
||||
agnes admin metrics export dump metric definitions to YAML
|
||||
agnes admin metrics validate validate metric definitions
|
||||
agnes admin <other> existing admin verbs continue unchanged
|
||||
```
|
||||
|
||||
Removed:
|
||||
- `da analyst setup`, `da analyst status` — `analyst` namespace had only one user role; replaced by top-level `da init` + `da status`.
|
||||
- `da sync` (and `--upload-only`) — split into `da pull` + `da push`. Hook commands rename accordingly.
|
||||
- `da fetch` — folded into `da snapshot create` with all flags carried over (`--select`, `--where`, `--limit`, `--order-by`, `--as`, `--estimate`, `--no-estimate`, `--force`).
|
||||
- `da metrics list/show` — folded into `da catalog --metrics`.
|
||||
- `da metrics import`, `da metrics export`, `da metrics validate` — relocated to `da admin metrics {import,export,validate}` (admin-only operations).
|
||||
- `da analyst setup`, `da analyst status` — `analyst` namespace had only one user role; replaced by top-level `agnes init` + `agnes status`.
|
||||
- `da sync` (and `--upload-only`) — split into `agnes pull` + `agnes push`. Hook commands rename accordingly.
|
||||
- `da fetch` — folded into `agnes snapshot create` with all flags carried over (`--select`, `--where`, `--limit`, `--order-by`, `--as`, `--estimate`, `--no-estimate`, `--force`).
|
||||
- `da metrics list/show` — folded into `agnes catalog --metrics`.
|
||||
- `da metrics import`, `da metrics export`, `da metrics validate` — relocated to `agnes admin metrics {import,export,validate}` (admin-only operations).
|
||||
|
||||
Surface decisions vs. earlier draft:
|
||||
- `da skills list / show` retained for analyst-side discovery. Skills bundled under `cli/skills/*.md` (e.g., `agnes-data-querying.md`, `agnes-table-registration.md`) carry rails that the rebased main expanded as part of #160 (cost guardrail, registry-gating). Removing them would cost the analyst documentation that the project actively invests in. Bulk install/copy verbs are not added.
|
||||
- `da auth token …` keeps its current location under `auth_app` (today's `cli/commands/auth.py:200-201` registers the sub-Typer there). No move to top-level `da token`. Surface listing reflects that.
|
||||
- `agnes skills list / show` retained for analyst-side discovery. Skills bundled under `cli/skills/*.md` (e.g., `agnes-data-querying.md`, `agnes-table-registration.md`) carry rails that the rebased main expanded as part of #160 (cost guardrail, registry-gating). Removing them would cost the analyst documentation that the project actively invests in. Bulk install/copy verbs are not added.
|
||||
- `agnes auth token …` keeps its current location under `auth_app` (today's `cli/commands/auth.py:200-201` registers the sub-Typer there). No move to top-level `da token`. Surface listing reflects that.
|
||||
|
||||
Reader commands explicitly listed (`da explore`, `da disk-info`, `da snapshot refresh`, `da snapshot prune`, `da skills list/show`) survive unchanged.
|
||||
Reader commands explicitly listed (`agnes explore`, `agnes disk-info`, `agnes snapshot refresh`, `agnes snapshot prune`, `agnes skills list/show`) survive unchanged.
|
||||
|
||||
## Components
|
||||
|
||||
|
|
@ -169,31 +171,31 @@ Reader commands explicitly listed (`da explore`, `da disk-info`, `da snapshot re
|
|||
|
||||
| Component | File | Change |
|
||||
|---|---|---|
|
||||
| Analyst install-prompt branch | `app/web/setup_instructions.py` | Add `role: Literal["analyst","admin"]="admin"` to `resolve_lines()` and `render_setup_instructions()`. Analyst layout: TLS trust block (reused, when `ca_pem` supplied) → install `da` (reused) → `da init --server-url X --token Y --workspace .` → `da catalog` smoke verify → confirm. Drop for analyst: marketplace, plugins, skills, diagnose, login, whoami (all subsumed by `da init`). |
|
||||
| Analyst install-prompt branch | `app/web/setup_instructions.py` | Add `role: Literal["analyst","admin"]="admin"` to `resolve_lines()` and `render_setup_instructions()`. Analyst layout: TLS trust block (reused, when `ca_pem` supplied) → install `agnes` (reused) → `agnes init --server-url X --token Y --workspace .` → `agnes catalog` smoke verify → confirm. Drop for analyst: marketplace, plugins, skills, diagnose, login, whoami (all subsumed by `agnes init`). |
|
||||
| `/setup?role=...` query branching | `app/web/router.py` `setup_page` (line 717) | Read `role` query param, default `"admin"`. Pass to `render_setup_instructions(role=...)`. Existing `/install` 302 redirect to `/setup` is preserved (legacy bookmarks keep working). |
|
||||
| `setup.html` UI | `app/web/templates/setup.html` (or wherever `setup_page` renders) | Two role tiles: "Analyst workspace" / "Admin CLI". PAT mint button per tile, posts to `/auth/tokens` with `scope` matching the tile. Renders the prompt for the selected role. |
|
||||
| PAT scope + TTL clamp | `app/api/tokens.py` (`CreateTokenRequest` Pydantic model + `create_token` route) | Add two fields: `scope: str = "general"` and `ttl_seconds: int \| None = None` (alongside the existing `expires_in_days: Optional[int] = 90` at lines 23-25). Resolution: when `ttl_seconds` is set, it wins; otherwise fall back to `expires_in_days`. **Upper bound:** mirror the existing `expires_in_days <= 3650` cap at line 100 with `ttl_seconds <= 315_360_000` (3650 days × 86400 s) so a hostile client can't bypass the cap by switching field names. For `scope == "bootstrap-analyst"`, server force-clamps the resolved TTL to ≤ 3600 s regardless of request. Audit-log entry includes the scope. The audit log is the only consumer of `scope` in this PR — per-endpoint enforcement is an explicit follow-up. |
|
||||
| Server-side template rewrite | `config/claude_md_template.txt` (or wherever `render_claude_md` reads its default from) | Update path strings: `data/parquet/` → `server/parquet/`, `data/duckdb/...` → `user/duckdb/analytics.duckdb`. Replace `da sync` → `da pull`, `da fetch` → `da snapshot create`, `da metrics list` → `da catalog --metrics`. |
|
||||
| Server-side template rewrite | `config/claude_md_template.txt` (or wherever `render_claude_md` reads its default from) | Update path strings: `data/parquet/` → `server/parquet/`, `data/duckdb/...` → `user/duckdb/analytics.duckdb`. Replace `da sync` → `agnes pull`, `da fetch` → `agnes snapshot create`, `da metrics list` → `agnes catalog --metrics`. |
|
||||
| Admin override migration | `claude_md_template` DB table (schema v23, exposed via `/admin/workspace-prompt` UI and `app/api/claude_md.py` admin CRUD) | Add a module-level constant `_LEGACY_STRINGS = ("data/parquet", "da sync", "da fetch", "da analyst setup", "da metrics list", "da metrics show")` and a helper `def _scan_legacy_strings(text: str) -> list[str]` inside `app/api/claude_md.py`. Add a `legacy_strings_detected: list[str] = []` field to `TemplateGetResponse` (today defined at `app/api/claude_md.py:72-76`); `admin_get_workspace_template` populates it via `_scan_legacy_strings(override.content)`. UI in `app/web/templates/admin_workspace_prompt.html` (file confirmed to exist) renders a yellow banner above the editor when the list is non-empty: "This override references CLI verbs / paths that were renamed in this release. Re-author and Save to clear the warning. See CHANGELOG for the rename list." Migration stays manual — admin re-authors and saves. |
|
||||
| `/api/welcome` content unchanged | `app/api/claude_md.py:91` (`get_welcome`) | No code change — endpoint already serves rendered CLAUDE.md. Spec calls it out so implementer knows `da init`'s producer is here, not in the client. |
|
||||
| Adopt `cli/error_render.py` (added in #160) for client-side errors | server: nothing — client-side only | `cli/error_render.py:render_error(status_code, body)` was introduced in 0.32.0 for typed BQ errors served by `da query --remote` (recognizes `detail.kind` / `detail.reason` shapes; falls back to plain HTTP `{code}: {text}`). The renderer is structurally generic — no BQ-specific code. `da init` and `da pull` are **first-time adopters in the bootstrap path** (today's `sync.py`, `auth.py`, `fetch.py` don't import it). Pattern: synthesize a `{"detail": {"kind": "...", "hint": "...", "message": "..."}}` dict client-side and pass with a chosen `status_code` (0 or `-1` for purely client-side errors with no HTTP origin), exactly as `cli/commands/query.py:152, 165` already does for `RemoteQueryError` translation. New typed kinds added in this PR: `auth_failed`, `server_unreachable`, `manifest_unauthorized`, `disk_full`, `partial_state` — the renderer doesn't gate on a kind allowlist, so no renderer change is needed. No server work; client-side only. |
|
||||
| `/api/welcome` content unchanged | `app/api/claude_md.py:91` (`get_welcome`) | No code change — endpoint already serves rendered CLAUDE.md. Spec calls it out so implementer knows `agnes init`'s producer is here, not in the client. |
|
||||
| Adopt `cli/error_render.py` (added in #160) for client-side errors | server: nothing — client-side only | `cli/error_render.py:render_error(status_code, body)` was introduced in 0.32.0 for typed BQ errors served by `agnes query --remote` (recognizes `detail.kind` / `detail.reason` shapes; falls back to plain HTTP `{code}: {text}`). The renderer is structurally generic — no BQ-specific code. `agnes init` and `agnes pull` are **first-time adopters in the bootstrap path** (today's `sync.py`, `auth.py`, `fetch.py` don't import it). Pattern: synthesize a `{"detail": {"kind": "...", "hint": "...", "message": "..."}}` dict client-side and pass with a chosen `status_code` (0 or `-1` for purely client-side errors with no HTTP origin), exactly as `cli/commands/query.py:152, 165` already does for `RemoteQueryError` translation. New typed kinds added in this PR: `auth_failed`, `server_unreachable`, `manifest_unauthorized`, `disk_full`, `partial_state` — the renderer doesn't gate on a kind allowlist, so no renderer change is needed. No server work; client-side only. |
|
||||
|
||||
### Client-side (CLI, Python)
|
||||
|
||||
| Component | File | Change |
|
||||
|---|---|---|
|
||||
| `da init` (new) | `cli/commands/init.py` (new) | Required args: `--server-url`, `--token`. Optional: `--force`, `--workspace` (default `cwd`). Steps: (1) verify server reachability + PAT validity via `GET /api/catalog/tables` with `Authorization: Bearer <PAT>` — same endpoint `da auth import-token` already uses for this purpose (`cli/commands/auth.py:154`); exercises full PAT validation chain (revocation, expiry, hash) and 401s on bad PAT, unlike `/api/health` which is unauthenticated; (2) save server URL + PAT to `~/.config/da/{config.yaml,token.json}`; (3) `GET /api/welcome` and write its body to `<workspace>/CLAUDE.md`; (4) write `.claude/settings.json` (model, permissions, hooks pointing at `da pull` and `da push`) — delegate hook installation to `cli/lib/hooks.py:install_claude_hooks` (see new module row below); (5) write `.claude/CLAUDE.local.md` (stub, only if absent); (6) call `cli/lib/pull.py:run_pull(server_url, token, workspace)` programmatically (no Typer round-trip); (7) write `AGNES_WORKSPACE.md` from a static client-side template with `{created_at}`, `{server_url}`, `{workspace_path}` substituted. `da init` does NOT call `da auth login`; the PAT from the paste-prompt is the only auth path during bootstrap. Errors are rendered by `cli/error_render.py:render_error()` — `da init` synthesizes `{"detail": {"kind": "...", "hint": "..."}}` dicts client-side (pattern: `cli/commands/query.py:152, 165`); typed kinds: `auth_failed`, `server_unreachable`, `partial_state`, `disk_full`. |
|
||||
| `cli/lib/pull.py` (new module) | `cli/lib/pull.py` + `cli/lib/__init__.py` (new) — establish `cli/lib/` as the shared-library tree | Pure-function refactor of today's `cli/commands/sync.py:sync()` body, minus Typer decorators and stdout. Signature: `def run_pull(server_url: str, token: str, workspace: Path, *, dry_run: bool = False) -> PullResult`. Returns a structured `PullResult` (tables_updated, parquets_total, rules_count, duration_s, errors). Caller decides what to print (`da init` summarizes; `da pull` Typer wrapper prints per `--quiet`/`--json` flags). Tested directly without subprocess. **Packaging:** `cli/lib/__init__.py` (empty file) is required for Hatchling to include the dir in the wheel — `pyproject.toml:packages` already lists `cli`, sub-packages with `__init__.py` are picked up automatically. |
|
||||
| `cli/lib/hooks.py` (new module) | `cli/lib/hooks.py` (new) — replaces `cli/commands/analyst.py:_install_claude_hooks` | `def install_claude_hooks(workspace: Path) -> None`. Idempotent. Reads `<workspace>/.claude/settings.json`, drops any prior entry whose every command is a `da pull`/`da sync`/`da push` invocation (covers both today's hook commands and the new ones during a transition window if anyone runs the new init in a folder that had old hooks), appends fresh entries: `SessionStart → da pull --quiet 2>/dev/null \|\| true`, `SessionEnd → da push --quiet 2>/dev/null \|\| true`. Workspace-level scope (`<workspace>/.claude/settings.json`, not user-home), preserves third-party hooks. Lives next to `cli/lib/pull.py` under the new `cli/lib/__init__.py` package. |
|
||||
| `da pull` (renamed from `da sync`) | `cli/commands/pull.py` (renamed from `cli/commands/sync.py`) | Behavior is today's `da sync` minus the `--upload-only` branch. Lazy-mkdir fixes (see below). Calls `cli/lib/pull.py:run_pull` and prints the result. Flags: `--quiet` (suppress success stdout, used by hook), `--json` (machine output of `PullResult`), `--dry-run` (compute deltas without writing — uses `dry_run=True`). Errors render via `cli/error_render.py`. |
|
||||
| `da push` (extracted from `da sync --upload-only`) | `cli/commands/push.py` (new) | Uploads `user/sessions/*.jsonl` and `.claude/CLAUDE.local.md`. Lazy: skip when nothing to upload (no `user/sessions/` mkdir if no sessions). Same auth as `da pull`. Flags: `--quiet`, `--json`, `--dry-run`. Errors render via `cli/error_render.py`. |
|
||||
| `da snapshot create` (renamed from `da fetch`) | `cli/commands/snapshot.py` | Move logic from `cli/commands/fetch.py` into a `create` subcommand of the existing `snapshot` group. Remove `cli/commands/fetch.py`. Carry over all flags: `--select`, `--where`, `--limit`, `--order-by`, `--as`, `--estimate`, `--no-estimate`, `--force`. Add existence check before opening DuckDB to avoid creating an empty DB file when no `da pull` has run yet (guard: `if not db_path.exists(): typer.echo("Local DuckDB not found. Run: da pull"); raise typer.Exit(1)`). Existing `da snapshot {refresh, prune, list, drop}` are unchanged. |
|
||||
| `da status` (renamed from `da analyst status`) | `cli/commands/status.py` (renamed from analyst.py status fn) | Path refs updated to new layout: `server/parquet/`, `user/duckdb/analytics.duckdb`. Drop `data/metadata/last_sync.json`; use mtime on `user/duckdb/analytics.duckdb` as freshness proxy. |
|
||||
| `agnes init` (new) | `cli/commands/init.py` (new) | Required args: `--server-url`, `--token`. Optional: `--force`, `--workspace` (default `cwd`). Steps: (1) verify server reachability + PAT validity via `GET /api/catalog/tables` with `Authorization: Bearer <PAT>` — same endpoint `agnes auth import-token` already uses for this purpose (`cli/commands/auth.py:154`); exercises full PAT validation chain (revocation, expiry, hash) and 401s on bad PAT, unlike `/api/health` which is unauthenticated; (2) save server URL + PAT to `~/.config/da/{config.yaml,token.json}`; (3) `GET /api/welcome` and write its body to `<workspace>/CLAUDE.md`; (4) write `.claude/settings.json` (model, permissions, hooks pointing at `agnes pull` and `agnes push`) — delegate hook installation to `cli/lib/hooks.py:install_claude_hooks` (see new module row below); (5) write `.claude/CLAUDE.local.md` (stub, only if absent); (6) call `cli/lib/pull.py:run_pull(server_url, token, workspace)` programmatically (no Typer round-trip); (7) write `AGNES_WORKSPACE.md` from a static client-side template with `{created_at}`, `{server_url}`, `{workspace_path}` substituted. `agnes init` does NOT call `agnes auth login`; the PAT from the paste-prompt is the only auth path during bootstrap. Errors are rendered by `cli/error_render.py:render_error()` — `agnes init` synthesizes `{"detail": {"kind": "...", "hint": "..."}}` dicts client-side (pattern: `cli/commands/query.py:152, 165`); typed kinds: `auth_failed`, `server_unreachable`, `partial_state`, `disk_full`. |
|
||||
| `cli/lib/pull.py` (new module) | `cli/lib/pull.py` + `cli/lib/__init__.py` (new) — establish `cli/lib/` as the shared-library tree | Pure-function refactor of today's `cli/commands/sync.py:sync()` body, minus Typer decorators and stdout. Signature: `def run_pull(server_url: str, token: str, workspace: Path, *, dry_run: bool = False) -> PullResult`. Returns a structured `PullResult` (tables_updated, parquets_total, rules_count, duration_s, errors). Caller decides what to print (`agnes init` summarizes; `agnes pull` Typer wrapper prints per `--quiet`/`--json` flags). Tested directly without subprocess. **Packaging:** `cli/lib/__init__.py` (empty file) is required for Hatchling to include the dir in the wheel — `pyproject.toml:packages` already lists `cli`, sub-packages with `__init__.py` are picked up automatically. |
|
||||
| `cli/lib/hooks.py` (new module) | `cli/lib/hooks.py` (new) — replaces `cli/commands/analyst.py:_install_claude_hooks` | `def install_claude_hooks(workspace: Path) -> None`. Idempotent. Reads `<workspace>/.claude/settings.json`, drops any prior entry whose every command is a `agnes pull`/`da sync`/`agnes push` invocation (covers both today's hook commands and the new ones during a transition window if anyone runs the new init in a folder that had old hooks), appends fresh entries: `SessionStart → agnes pull --quiet 2>/dev/null \|\| true`, `SessionEnd → agnes push --quiet 2>/dev/null \|\| true`. Workspace-level scope (`<workspace>/.claude/settings.json`, not user-home), preserves third-party hooks. Lives next to `cli/lib/pull.py` under the new `cli/lib/__init__.py` package. |
|
||||
| `agnes pull` (renamed from `da sync`) | `cli/commands/pull.py` (renamed from `cli/commands/sync.py`) | Behavior is today's `da sync` minus the `--upload-only` branch. Lazy-mkdir fixes (see below). Calls `cli/lib/pull.py:run_pull` and prints the result. Flags: `--quiet` (suppress success stdout, used by hook), `--json` (machine output of `PullResult`), `--dry-run` (compute deltas without writing — uses `dry_run=True`). Errors render via `cli/error_render.py`. |
|
||||
| `agnes push` (extracted from `da sync --upload-only`) | `cli/commands/push.py` (new) | Uploads `user/sessions/*.jsonl` and `.claude/CLAUDE.local.md`. Lazy: skip when nothing to upload (no `user/sessions/` mkdir if no sessions). Same auth as `agnes pull`. Flags: `--quiet`, `--json`, `--dry-run`. Errors render via `cli/error_render.py`. |
|
||||
| `agnes snapshot create` (renamed from `da fetch`) | `cli/commands/snapshot.py` | Move logic from `cli/commands/fetch.py` into a `create` subcommand of the existing `snapshot` group. Remove `cli/commands/fetch.py`. Carry over all flags: `--select`, `--where`, `--limit`, `--order-by`, `--as`, `--estimate`, `--no-estimate`, `--force`. Add existence check before opening DuckDB to avoid creating an empty DB file when no `agnes pull` has run yet (guard: `if not db_path.exists(): typer.echo("Local DuckDB not found. Run: agnes pull"); raise typer.Exit(1)`). Existing `agnes snapshot {refresh, prune, list, drop}` are unchanged. |
|
||||
| `agnes status` (renamed from `da analyst status`) | `cli/commands/status.py` (renamed from analyst.py status fn) | Path refs updated to new layout: `server/parquet/`, `user/duckdb/analytics.duckdb`. Drop `data/metadata/last_sync.json`; use mtime on `user/duckdb/analytics.duckdb` as freshness proxy. |
|
||||
| Lazy-mkdir contract | `cli/commands/pull.py`, `cli/lib/pull.py`, `cli/commands/push.py` | No `mkdir(parents=True, exist_ok=True)` before a conditional write loop. Mkdir only immediately before the first file write. Concretely: `_fetch_and_write_rules` mkdirs `.claude/rules/` only when `mandatory ∪ approved` is non-empty; `parquet_dir` mkdir is inlined into the per-table download loop. |
|
||||
| `da catalog --metrics` flag | `cli/commands/catalog.py` | Add `--metrics` flag (replaces `da metrics list`) and `--metrics --show <id>` (replaces `da metrics show`). Decided shape, not unresolved — implementation should not negotiate. |
|
||||
| `da admin metrics {import,export,validate}` (relocated) | `cli/commands/admin.py` | Add a `metrics` sub-Typer to the existing `admin_app` (which already nests sub-Typers `memory`, `group`, `grant`, `break-glass` per `cli/commands/admin.py:10`). Move `import`, `export`, `validate` from `cli/commands/metrics.py`. Admin-only; not part of analyst flow. |
|
||||
| `agnes catalog --metrics` flag | `cli/commands/catalog.py` | Add `--metrics` flag (replaces `da metrics list`) and `--metrics --show <id>` (replaces `da metrics show`). Decided shape, not unresolved — implementation should not negotiate. |
|
||||
| `agnes admin metrics {import,export,validate}` (relocated) | `cli/commands/admin.py` | Add a `metrics` sub-Typer to the existing `admin_app` (which already nests sub-Typers `memory`, `group`, `grant`, `break-glass` per `cli/commands/admin.py:10`). Move `import`, `export`, `validate` from `cli/commands/metrics.py`. Admin-only; not part of analyst flow. |
|
||||
| Removed (full delete) | `cli/commands/{metrics.py, fetch.py, analyst.py, sync.py}` | Deleted entirely (greenfield). |
|
||||
| Retained | `cli/commands/skills.py` | Kept. `da skills list` and `da skills show` are analyst-side discovery commands. No code change in this PR. |
|
||||
| Retained | `cli/commands/skills.py` | Kept. `agnes skills list` and `agnes skills show` are analyst-side discovery commands. No code change in this PR. |
|
||||
|
||||
### Templates and docs
|
||||
|
||||
|
|
@ -201,7 +203,7 @@ Reader commands explicitly listed (`da explore`, `da disk-info`, `da snapshot re
|
|||
|---|---|---|
|
||||
| Server-side `CLAUDE.md` template | `config/claude_md_template.txt` (and any DB override flagged in admin migration) | Path strings + verb names updated as listed in Server-side table. |
|
||||
| `AGNES_WORKSPACE.md` template (new) | `config/agnes_workspace_template.txt` (new, client-side static asset bundled with the wheel) | Three placeholders: `{created_at}`, `{server_url}`, `{workspace_path}`. Header line uses all three; remaining content is static. Content described in dedicated section below. |
|
||||
| Repo-root `CLAUDE.md` rewrite | `CLAUDE.md` (project root) | Update all references: `da sync` → `da pull`, `da analyst setup` → `da init`, `da metrics list/show` → `da catalog --metrics`, `da fetch` → `da snapshot create`, `data/parquet/` → `server/parquet/`. The "Local sync & Claude Code hooks" subsection and the "Querying Agnes data — agent rails" subsection both need full walk-throughs. The latter was expanded by 0.32.0 (#160) with cost-guardrail / registry-gating prose — those sections stay verbatim, just verb-renamed. The "Business Metrics" subsection's `da metrics import` / `da metrics list` / `da metrics show` examples become `da admin metrics import` and `da catalog --metrics` respectively. |
|
||||
| Repo-root `CLAUDE.md` rewrite | `CLAUDE.md` (project root) | Update all references: `da sync` → `agnes pull`, `da analyst setup` → `agnes init`, `da metrics list/show` → `agnes catalog --metrics`, `da fetch` → `agnes snapshot create`, `data/parquet/` → `server/parquet/`. The "Local sync & Claude Code hooks" subsection and the "Querying Agnes data — agent rails" subsection both need full walk-throughs. The latter was expanded by 0.32.0 (#160) with cost-guardrail / registry-gating prose — those sections stay verbatim, just verb-renamed. The "Business Metrics" subsection's `da metrics import` / `da metrics list` / `da metrics show` examples become `agnes admin metrics import` and `agnes catalog --metrics` respectively. |
|
||||
|
||||
## Web UI flow
|
||||
|
||||
|
|
@ -232,7 +234,7 @@ Legacy `/install` URL: kept as a 302 redirect to `/setup`. All new references in
|
|||
|
||||
## Workspace layout
|
||||
|
||||
Post-`da init`, workspace contains exactly:
|
||||
Post-`agnes init`, workspace contains exactly:
|
||||
|
||||
```
|
||||
<cwd>/
|
||||
|
|
@ -250,8 +252,8 @@ Conditional additions:
|
|||
|
||||
- `./.claude/rules/km_*.md` — only when `/api/memory/bundle` returns ≥ 1 mandatory or approved item.
|
||||
- `./server/parquet/<table>.parquet` — only when `/api/sync/manifest` returns ≥ 1 table the user has grants on.
|
||||
- `./user/snapshots/<name>.parquet` — only after the user runs `da snapshot create <table> --as <name>`.
|
||||
- `./user/sessions/<id>.jsonl` — only after the SessionEnd hook runs `da push` against captured Claude Code sessions.
|
||||
- `./user/snapshots/<name>.parquet` — only after the user runs `agnes snapshot create <table> --as <name>`.
|
||||
- `./user/sessions/<id>.jsonl` — only after the SessionEnd hook runs `agnes push` against captured Claude Code sessions.
|
||||
|
||||
Forbidden under any circumstances (these are the dead paths today's setup creates):
|
||||
|
||||
|
|
@ -266,10 +268,10 @@ Empty folder + Claude Code with paste prompt
|
|||
├─ Step 0 (TLS trust block) — only when server uses private CA
|
||||
│ writes ~/.agnes/{ca.pem, ca-bundle.pem}, appends shell rc block
|
||||
│
|
||||
├─ Step 1 — uv tool install da
|
||||
├─ Step 1 — uv tool install <wheel> # binary: agnes
|
||||
│ writes ~/.local/bin/da
|
||||
│
|
||||
├─ Step 2 — da init --server-url URL --token PAT --workspace .
|
||||
├─ Step 2 — agnes init --server-url URL --token PAT --workspace .
|
||||
│ ├─ verify: GET /api/catalog/tables with Bearer PAT → 200 (PAT-validating endpoint)
|
||||
│ ├─ save: ~/.config/da/{config.yaml, token.json}
|
||||
│ ├─ fetch: GET /api/welcome?server_url=<URL> → write ./CLAUDE.md
|
||||
|
|
@ -277,9 +279,9 @@ Empty folder + Claude Code with paste prompt
|
|||
│ │ installs render the operator-visible URL, not the FastAPI
|
||||
│ │ internal hostname; endpoint default falls back to
|
||||
│ │ request.base_url which equals --server-url in practice)
|
||||
│ ├─ write: ./.claude/settings.json (with hooks SessionStart→`da pull`, SessionEnd→`da push`)
|
||||
│ ├─ write: ./.claude/settings.json (with hooks SessionStart→`agnes pull`, SessionEnd→`agnes push`)
|
||||
│ ├─ write: ./.claude/CLAUDE.local.md (stub, if absent)
|
||||
│ ├─ call: da pull (programmatic — calls cli/lib/pull.py:run_pull)
|
||||
│ ├─ call: agnes pull (programmatic — calls cli/lib/pull.py:run_pull)
|
||||
│ │ ├─ GET /api/sync/manifest → {tables, hashes}
|
||||
│ │ ├─ for each table where local md5 ≠ remote md5:
|
||||
│ │ │ GET /api/data/<id>/download (stream)
|
||||
|
|
@ -292,16 +294,16 @@ Empty folder + Claude Code with paste prompt
|
|||
│ │ if empty: skip mkdir
|
||||
│ └─ write: ./AGNES_WORKSPACE.md (from client-side static template)
|
||||
│
|
||||
├─ Step 3 — da catalog (smoke verify)
|
||||
├─ Step 3 — agnes catalog (smoke verify)
|
||||
│ confirms end-to-end works; prints table count
|
||||
│
|
||||
└─ Step 4 — confirm
|
||||
Claude reports: tables synced, files created, hooks active.
|
||||
|
||||
Subsequent sessions:
|
||||
├─ SessionStart hook fires: da pull --quiet 2>/dev/null || true
|
||||
├─ SessionStart hook fires: agnes pull --quiet 2>/dev/null || true
|
||||
├─ user works
|
||||
└─ SessionEnd hook fires: da push --quiet 2>/dev/null || true
|
||||
└─ SessionEnd hook fires: agnes push --quiet 2>/dev/null || true
|
||||
```
|
||||
|
||||
## Empty-folder discipline
|
||||
|
|
@ -318,31 +320,31 @@ Concretely:
|
|||
|---|---|---|---|
|
||||
| `_fetch_and_write_rules` | `cli/commands/sync.py:222` | `rules_dir.mkdir(parents=True, exist_ok=True)` before iterating | Check `mandatory + approved` first; if empty, return without mkdir. |
|
||||
| Per-table download loop | `cli/commands/sync.py:120, 529` | `parquet_dir.mkdir(parents=True, exist_ok=True)` before loop | Mkdir inlined into the per-file write block; first table triggers mkdir. |
|
||||
| `install_claude_hooks` | `cli/lib/hooks.py` (new; replaces `cli/commands/analyst.py:_install_claude_hooks`, today at line 254) | mkdir `.claude/` | unchanged — `.claude/` always has content (settings.json is load-bearing). Function lifted from the deleted `cli/commands/analyst.py` into a shared library so `da init` (and any future caller) can use it without importing the deleted module. |
|
||||
| `install_claude_hooks` | `cli/lib/hooks.py` (new; replaces `cli/commands/analyst.py:_install_claude_hooks`, today at line 254) | mkdir `.claude/` | unchanged — `.claude/` always has content (settings.json is load-bearing). Function lifted from the deleted `cli/commands/analyst.py` into a shared library so `agnes init` (and any future caller) can use it without importing the deleted module. |
|
||||
| `_rebuild_duckdb_views` | `cli/commands/sync.py:321` | mkdir `user/duckdb/` | unchanged — DuckDB file is opened unconditionally as part of view rebuild; the file is the load-bearing artifact, not just the directory. |
|
||||
| `da push` upload | (new) `cli/commands/push.py` | (n/a) | Mkdir `user/sessions/` only inside the per-session-write branch; `da push` with nothing to upload exits 0 without touching disk. |
|
||||
| `da snapshot create` parquet write | `cli/commands/snapshot.py` | mkdir `user/snapshots/` before write | unchanged (snapshot create is the canonical writer; mkdir on first write is correct). |
|
||||
| `agnes push` upload | (new) `cli/commands/push.py` | (n/a) | Mkdir `user/sessions/` only inside the per-session-write branch; `agnes push` with nothing to upload exits 0 without touching disk. |
|
||||
| `agnes snapshot create` parquet write | `cli/commands/snapshot.py` | mkdir `user/snapshots/` before write | unchanged (snapshot create is the canonical writer; mkdir on first write is correct). |
|
||||
|
||||
### Reader contract
|
||||
|
||||
> Every reader MUST handle missing paths gracefully. "Gracefully" means:
|
||||
> - **Exit 0 with empty / zero output** when missing paths are a natural empty answer (`da disk-info` shows 0; `da status` shows "initialized: no").
|
||||
> - **Exit 1 with friendly hint** when the missing path means a workflow precondition isn't met (`da query`: "Local DuckDB not found. Run: da pull").
|
||||
> - **Exit 0 with empty / zero output** when missing paths are a natural empty answer (`agnes disk-info` shows 0; `agnes status` shows "initialized: no").
|
||||
> - **Exit 1 with friendly hint** when the missing path means a workflow precondition isn't met (`agnes query`: "Local DuckDB not found. Run: agnes pull").
|
||||
> - **Never create the path side-effect-ally** unless this command is the canonical writer for it.
|
||||
|
||||
Audit of current readers (only commands that touch the filesystem are listed; others are server-API only and unaffected):
|
||||
|
||||
| Command | Path it reads | Today's behavior | Change needed |
|
||||
|---|---|---|---|
|
||||
| `da query` | `user/duckdb/analytics.duckdb` | `.exists()` check, friendly "Run: da sync" exit 1 | Update hint text → "Run: da pull". |
|
||||
| `da explore` | same | `.exists()` check, friendly exit | Update hint text. |
|
||||
| `da snapshot create` (was `da fetch`) | same | unconditional `duckdb.connect()` → creates empty DB | Add `.exists()` check + hint "Run: da pull first". |
|
||||
| `da snapshot create` (write side) | `user/snapshots/` | unchanged (writer, mkdir at first write) | unchanged. |
|
||||
| `da disk-info` | `user/snapshots/` | `.exists()` guards around sum/count/free | unchanged. |
|
||||
| `da snapshot list` | `user/snapshots/` | glob safe on missing | unchanged (glob returns empty iterator on missing dir). |
|
||||
| `da snapshot refresh` / `prune` | `user/snapshots/` | glob/.exists() guards | unchanged. |
|
||||
| `da push` | `user/sessions/` | `.exists()` check before iterating | unchanged. |
|
||||
| `da status` | `server/parquet/`, `user/duckdb/...` | path strings reference legacy `data/parquet/` etc. | Update path strings; `.exists()` checks already in place. |
|
||||
| `agnes query` | `user/duckdb/analytics.duckdb` | `.exists()` check, friendly "Run: da sync" exit 1 | Update hint text → "Run: agnes pull". |
|
||||
| `agnes explore` | same | `.exists()` check, friendly exit | Update hint text. |
|
||||
| `agnes snapshot create` (was `da fetch`) | same | unconditional `duckdb.connect()` → creates empty DB | Add `.exists()` check + hint "Run: agnes pull first". |
|
||||
| `agnes snapshot create` (write side) | `user/snapshots/` | unchanged (writer, mkdir at first write) | unchanged. |
|
||||
| `agnes disk-info` | `user/snapshots/` | `.exists()` guards around sum/count/free | unchanged. |
|
||||
| `agnes snapshot list` | `user/snapshots/` | glob safe on missing | unchanged (glob returns empty iterator on missing dir). |
|
||||
| `agnes snapshot refresh` / `prune` | `user/snapshots/` | glob/.exists() guards | unchanged. |
|
||||
| `agnes push` | `user/sessions/` | `.exists()` check before iterating | unchanged. |
|
||||
| `agnes status` | `server/parquet/`, `user/duckdb/...` | path strings reference legacy `data/parquet/` etc. | Update path strings; `.exists()` checks already in place. |
|
||||
|
||||
### Regression guard (test)
|
||||
|
||||
|
|
@ -366,7 +368,7 @@ This guard runs in every clean-install integration test.
|
|||
|
||||
## `AGNES_WORKSPACE.md` content
|
||||
|
||||
Generated by `da init` in the workspace root from a static client-side template (`config/agnes_workspace_template.txt`, bundled with the wheel). Not state — pure documentation. Idempotent overwrite on every `da init` (preserves nothing, regenerates everything).
|
||||
Generated by `agnes init` in the workspace root from a static client-side template (`config/agnes_workspace_template.txt`, bundled with the wheel). Not state — pure documentation. Idempotent overwrite on every `agnes init` (preserves nothing, regenerates everything).
|
||||
|
||||
Three placeholders only: `{created_at}`, `{server_url}`, `{workspace_path}`. Used in the header line "Created: {created_at} · Server: {server_url} · Workspace: {workspace_path}". No email, no user identity, no role. Email is not used anywhere in the analyst CLI flow; PAT identifies the user server-side, and decoded JWT email is informational at best — we drop it from this header for clarity.
|
||||
|
||||
|
|
@ -376,7 +378,7 @@ Sections:
|
|||
2. **What's installed (global, per-user)** — table of paths in `~/.local/bin/`, `~/.config/da/`, `~/.agnes/`, shell rc block. Each row: `path | what it is | how to remove`.
|
||||
3. **What's in this folder** — table of paths in workspace. Each row: `path | what it is`. Notes which dirs are conditional ("only when grants/sessions/etc. exist").
|
||||
4. **How it stays fresh** — explains SessionStart/End hooks: what they run, when, what failure looks like (silent, `|| true`).
|
||||
5. **Cheat sheet** — `da pull`, `da catalog`, `da query`, `da snapshot create`, `da status`, `da init --force` examples.
|
||||
5. **Cheat sheet** — `agnes pull`, `agnes catalog`, `agnes query`, `agnes snapshot create`, `agnes status`, `agnes init --force` examples.
|
||||
6. **Uninstall** — step-by-step recipe to remove the CLI globally, the config dir, the trust artifacts, the rc block, and the workspace itself.
|
||||
|
||||
Approximate size: 3.5 KB, ~100 lines. Disk overhead: nil.
|
||||
|
|
@ -389,14 +391,14 @@ PAT value never appears in `AGNES_WORKSPACE.md` — only its location (`~/.confi
|
|||
|
||||
| Failure | Detection | Behavior |
|
||||
|---|---|---|
|
||||
| Server unreachable during `da init` | `httpx.ConnectError` on `/api/catalog/tables` | exit 1 via `cli/error_render.render_error()` with kind `server_unreachable`, hint: "Cannot reach `<URL>` — check network or server status". |
|
||||
| Server unreachable during `agnes init` | `httpx.ConnectError` on `/api/catalog/tables` | exit 1 via `cli/error_render.render_error()` with kind `server_unreachable`, hint: "Cannot reach `<URL>` — check network or server status". |
|
||||
| PAT expired | `/api/catalog/tables` → 401 | exit 1 via `render_error()` with kind `auth_failed`, hint: "Token expired — get a fresh one at `<URL>/setup?role=analyst`". |
|
||||
| PAT invalid (mis-paste) | 401, JWT decode failure | exit 1 via `render_error()` with kind `auth_failed`, hint: "Token format invalid — re-copy from `/setup`". |
|
||||
| TLS trust failure | curl/wheel install fails with `unknown CA` | exit 1, hint refers user back to paste-prompt step 0. |
|
||||
| Disk full during `da pull` | `OSError(ENOSPC)` on parquet write | atomic rename → partial file deleted; exit 1 with disk-info dump. |
|
||||
| Concurrent `da init` in same folder | sentinel `<cwd>/.claude/.init.lock` | second invocation: "Setup already running" exit 1. |
|
||||
| Partial state (previous `da init` crashed mid-way) | `CLAUDE.md` exists but `.claude/settings.json` missing | `da init` (without `--force`): friendly hint "Workspace partially set up — run `da init --force` to redo". |
|
||||
| `da pull` 401 mid-session (PAT revoked server-side) | response 401 from `/api/sync/manifest` | hook command prints warning, exits 0 (`\|\| true`); session continues with last-known data. Manual `da pull` next time prints actionable hint. |
|
||||
| Disk full during `agnes pull` | `OSError(ENOSPC)` on parquet write | atomic rename → partial file deleted; exit 1 with disk-info dump. |
|
||||
| Concurrent `agnes init` in same folder | sentinel `<cwd>/.claude/.init.lock` | second invocation: "Setup already running" exit 1. |
|
||||
| Partial state (previous `agnes init` crashed mid-way) | `CLAUDE.md` exists but `.claude/settings.json` missing | `agnes init` (without `--force`): friendly hint "Workspace partially set up — run `agnes init --force` to redo". |
|
||||
| `agnes pull` 401 mid-session (PAT revoked server-side) | response 401 from `/api/sync/manifest` | hook command prints warning, exits 0 (`\|\| true`); session continues with last-known data. Manual `agnes pull` next time prints actionable hint. |
|
||||
| Empty manifest | `/api/sync/manifest` → `{"tables": []}` | success, no parquet dir created, no warning (valid state). |
|
||||
| Empty memory bundle | `/api/memory/bundle` → `{"mandatory": [], "approved": []}` | success, no `.claude/rules/` dir (valid state). |
|
||||
| Per-table 5xx mid-pull | per-table 500 from `/api/data/<id>/download` | per-table warn; pull continues; final exit 0 if at least one table succeeded, exit 1 if all failed. |
|
||||
|
|
@ -416,7 +418,7 @@ Verification has three layers: (a) automated reader-smoke matrix that proves no
|
|||
| `fastapi_test_server` | object with `.url`, `.shutdown()` | Starts the FastAPI app in a background thread/subprocess against a `tmp_path`-rooted DATA_DIR. Clean schema (latest version, currently v23), two seeded users (`admin@example.com`, `analyst@example.com`, both with a known test password seeded into the local password provider), two seeded user groups (`Admin`, `Everyone`), three seeded tables in `table_registry` with one `query_mode='local'`, one `query_mode='materialized'`, one `query_mode='remote'`. Manifest + memory + welcome endpoints serve real (test) data. |
|
||||
| `test_pat` | string PAT for `analyst@example.com` | Group membership: `Everyone` only. `resource_grants` for the local + materialized tables (so manifest returns 2 rows for them). Two `mandatory` corporate-memory items granted via group. PAT TTL: 1 h. |
|
||||
| `test_pat_no_grants` | string PAT for `analyst@example.com` | Same user, but `resource_grants` is empty and `corporate_memory` has zero items granted to `Everyone`. Manifest returns `{"tables": []}`; memory bundle returns `{"mandatory": [], "approved": []}`. |
|
||||
| `zero_grants_workspace` | `tmp_path` after running `da init --token <test_pat_no_grants> --server-url <fastapi_test_server.url>` | A fully-bootstrapped workspace where every conditional dir is absent. Used by the reader smoke matrix. The fixture also exposes a sentinel constant `NONEXISTENT_TABLE = "__nonexistent__"` for tests that need a deliberately-unknown table id; readers must produce a friendly exit-1 (no traceback) when given this id. |
|
||||
| `zero_grants_workspace` | `tmp_path` after running `agnes init --token <test_pat_no_grants> --server-url <fastapi_test_server.url>` | A fully-bootstrapped workspace where every conditional dir is absent. Used by the reader smoke matrix. The fixture also exposes a sentinel constant `NONEXISTENT_TABLE = "__nonexistent__"` for tests that need a deliberately-unknown table id; readers must produce a friendly exit-1 (no traceback) when given this id. |
|
||||
| `web_session` | authenticated `httpx.Client` with cookies | Calls `POST /auth/password/login/web` with form fields `email=admin@example.com` and `password=<test_password>` (the test password is seeded into the same `users` row by `fastapi_test_server`). The form-login endpoint sets the session cookie that `POST /auth/tokens` requires (PAT mint route gates on `require_session_token`, see `app/api/tokens.py:88`). Used to mint PATs in PAT-scope tests. Choice rationale: real-endpoint login over dependency-override keeps the auth path under test rather than bypassed. |
|
||||
| `client` | `TestClient(app)` | Plain FastAPI test client with no auth. Used for endpoint-shape tests. |
|
||||
|
||||
|
|
@ -477,13 +479,13 @@ def test_clean_install_minimal_grants(fastapi_test_server, tmp_path, test_pat):
|
|||
assert_no_dead_dirs(tmp_path)
|
||||
# Hooks installed correctly:
|
||||
settings = json.loads((tmp_path / ".claude" / "settings.json").read_text())
|
||||
assert any("da pull" in h["hooks"][0]["command"]
|
||||
assert any("agnes pull" in h["hooks"][0]["command"]
|
||||
for h in settings["hooks"]["SessionStart"])
|
||||
assert any("da push" in h["hooks"][0]["command"]
|
||||
assert any("agnes push" in h["hooks"][0]["command"]
|
||||
for h in settings["hooks"]["SessionEnd"])
|
||||
# CLAUDE.md was fetched from /api/welcome (not local template):
|
||||
claude_md = (tmp_path / "CLAUDE.md").read_text()
|
||||
assert "da pull" in claude_md and "da sync" not in claude_md # post-rewrite content
|
||||
assert "agnes pull" in claude_md and "da sync" not in claude_md # post-rewrite content
|
||||
# AGNES_WORKSPACE.md content asserts (security + placeholder substitution):
|
||||
workspace_md = (tmp_path / "AGNES_WORKSPACE.md").read_text()
|
||||
assert test_pat not in workspace_md, "PAT must not leak into AGNES_WORKSPACE.md"
|
||||
|
|
@ -492,7 +494,7 @@ def test_clean_install_minimal_grants(fastapi_test_server, tmp_path, test_pat):
|
|||
assert "{workspace_path}" not in workspace_md, "placeholder not substituted"
|
||||
assert fastapi_test_server.url in workspace_md
|
||||
assert str(tmp_path) in workspace_md
|
||||
assert "da pull" in workspace_md # cheat sheet uses new verb
|
||||
assert "agnes pull" in workspace_md # cheat sheet uses new verb
|
||||
|
||||
|
||||
def test_clean_install_zero_grants(fastapi_test_server, tmp_path, test_pat_no_grants):
|
||||
|
|
@ -512,11 +514,11 @@ def test_clean_install_zero_grants(fastapi_test_server, tmp_path, test_pat_no_gr
|
|||
|
||||
|
||||
def test_setup_force_preserves_user_files(...):
|
||||
"""`da init --force` regenerates CLAUDE.md and AGNES_WORKSPACE.md
|
||||
"""`agnes init --force` regenerates CLAUDE.md and AGNES_WORKSPACE.md
|
||||
but never touches CLAUDE.local.md."""
|
||||
|
||||
def test_readers_in_pre_setup_dir(tmp_path, test_pat):
|
||||
"""User runs reader commands in a folder that never had `da init`.
|
||||
"""User runs reader commands in a folder that never had `agnes init`.
|
||||
No crash; friendly hints to run init or pull."""
|
||||
```
|
||||
|
||||
|
|
@ -530,15 +532,15 @@ def test_render_setup_instructions_analyst_role():
|
|||
role="analyst",
|
||||
)
|
||||
assert "uv tool install" in text
|
||||
assert "da init" in text
|
||||
assert "agnes init" in text
|
||||
assert "--token" in text and "agnes_pat_TEST" in text
|
||||
assert "--server-url" in text
|
||||
assert "da catalog" in text
|
||||
assert "agnes catalog" in text
|
||||
# Must not contain (admin-only):
|
||||
assert "marketplace" not in text
|
||||
assert "claude plugin install" not in text
|
||||
assert "da skills" not in text
|
||||
assert "da diagnose" not in text
|
||||
assert "agnes skills" not in text
|
||||
assert "agnes diagnose" not in text
|
||||
```
|
||||
|
||||
### 5.4 PAT scope/TTL test
|
||||
|
|
@ -571,31 +573,31 @@ def test_bootstrap_pat_falls_back_to_expires_in_days(web_session):
|
|||
3. Empty terminal in `/tmp/test-analyst-1`. From the web `/setup?role=analyst`, paste prompt.
|
||||
4. `tree -a /tmp/test-analyst-1` and compare with the expected tree from §5.2.
|
||||
5. `claude` in that folder. Three queries: "what tables can I see", "SELECT count(*) FROM <t>", "show me last 5 rows of <t>". All must work without further intervention.
|
||||
6. `/exit`. Verify SessionEnd hook ran (server-side audit log shows `da push`; `du -sh /tmp/test-analyst-1/user/sessions/` non-empty).
|
||||
7. Second `claude` in same folder. Verify SessionStart hook fires (`da pull` request in audit log).
|
||||
6. `/exit`. Verify SessionEnd hook ran (server-side audit log shows `agnes push`; `du -sh /tmp/test-analyst-1/user/sessions/` non-empty).
|
||||
7. Second `claude` in same folder. Verify SessionStart hook fires (`agnes pull` request in audit log).
|
||||
8. Second workspace `/tmp/test-analyst-2` with the same PAT (within TTL). Repeat 3-5. Verify global `~/.config/da/` is not duplicated; the second workspace has its own DuckDB.
|
||||
|
||||
This protocol is documented in `docs/RELEASE_CHECKLIST.md` as a mandatory pre-merge step for changes touching the bootstrap path.
|
||||
|
||||
## Out of scope
|
||||
|
||||
1. **Admin CLI tooling** — `/setup?role=admin` and `da admin *` continue unchanged. The new CLI surface listing in this spec is the *analyst* surface; admin verbs not listed (e.g., `da admin marketplace`, `da admin user`, etc.) are unaffected.
|
||||
1. **Admin CLI tooling** — `/setup?role=admin` and `agnes admin *` continue unchanged. The new CLI surface listing in this spec is the *analyst* surface; admin verbs not listed (e.g., `agnes admin marketplace`, `agnes admin user`, etc.) are unaffected.
|
||||
2. **Migration of existing analyst workspaces** — greenfield; old `data/parquet/` etc. are dead but harmless.
|
||||
3. **Backward-compat aliases** — no `da analyst setup` → `da init` shim, no `da sync` → `da pull` shim. Hard cutover.
|
||||
3. **Backward-compat aliases** — no `da analyst setup` → `agnes init` shim, no `da sync` → `agnes pull` shim. Hard cutover.
|
||||
4. **Multi-user / shared workspace** — `<cwd>` is single-user.
|
||||
5. **Offline initial bootstrap** — `da init` requires server reachability.
|
||||
5. **Offline initial bootstrap** — `agnes init` requires server reachability.
|
||||
6. **PAT auto-refresh / refresh tokens** — bootstrap PAT expires after 1 h; user re-clicks "Generate prompt".
|
||||
7. **Per-endpoint PAT scope enforcement** — `bootstrap-analyst` scope is informational at this stage (audit-trail). Per-endpoint enforcement is a follow-up issue.
|
||||
8. **Web UI redesign** — `/setup?role=...` reuses the existing page shell + JS. No visual redesign.
|
||||
9. **CLI rename adjacent commands** beyond what's listed (e.g., `da auth login` → `da login`) — out of scope.
|
||||
9. **CLI rename adjacent commands** beyond what's listed (e.g., `agnes auth login` → `da login`) — out of scope.
|
||||
10. **Layered per-workspace config** — `<cwd>/.agnes/{config.yaml,token.json}` overrides considered but dropped from this PR (no defined producer; multi-instance is edge case). Captured in Open questions.
|
||||
|
||||
## Open questions / follow-ups
|
||||
|
||||
- **Per-endpoint PAT scope enforcement** — should `scope="bootstrap-analyst"` PATs be restricted to `/api/catalog/tables`, `/api/sync/manifest`, `/api/data/*/download`, `/api/memory/bundle`, `/api/welcome` only, and refused on (e.g.) `/api/admin/*`? Today not enforced. New issue.
|
||||
- **Layered per-workspace config** — supporting multi-instance use cases (one analyst, two Agnes servers) requires a defined producer for `<cwd>/.agnes/`. Options: `da init --per-workspace-config` flag, post-init manual `mkdir`, or `da config init`. Not chosen because no current user has asked for it. New issue if/when needed.
|
||||
- **`da snapshot create --where` SQL flavor** — keep BigQuery flavor (today's `da fetch`) for parity with `da query --remote`, since BQ is the only remote source. Confirmed in this PR; flagged in case a non-BQ remote source is added later.
|
||||
- **Hook performance budget** — `da pull` on a 1.1 GB workspace (real-world example: today's `tmp_oss/server/parquet/`) with all parquets unchanged should complete the manifest comparison in well under 1 s so SessionStart doesn't perceptibly delay the user. If incremental MD5 comparison is too slow at scale, consider a server-side ETag.
|
||||
- **Layered per-workspace config** — supporting multi-instance use cases (one analyst, two Agnes servers) requires a defined producer for `<cwd>/.agnes/`. Options: `agnes init --per-workspace-config` flag, post-init manual `mkdir`, or `da config init`. Not chosen because no current user has asked for it. New issue if/when needed.
|
||||
- **`agnes snapshot create --where` SQL flavor** — keep BigQuery flavor (today's `da fetch`) for parity with `agnes query --remote`, since BQ is the only remote source. Confirmed in this PR; flagged in case a non-BQ remote source is added later.
|
||||
- **Hook performance budget** — `agnes pull` on a 1.1 GB workspace (real-world example: today's `tmp_oss/server/parquet/`) with all parquets unchanged should complete the manifest comparison in well under 1 s so SessionStart doesn't perceptibly delay the user. If incremental MD5 comparison is too slow at scale, consider a server-side ETag.
|
||||
- **Anti-coupling test** — add a test that imports every `cli/commands/*.py` and `cli/lib/*.py` module and asserts no `cli/commands/*` module imports another `cli.commands.*` module except via dispatch (Typer subcommand registration). `cli/lib/*` modules may be imported by command modules; reverse direction (`cli.lib` importing `cli.commands`) is forbidden. Prevents `init` accidentally re-importing `pull`'s Typer wrapper instead of the library function.
|
||||
|
||||
## CHANGELOG entry (preview)
|
||||
|
|
@ -604,29 +606,29 @@ This protocol is documented in `docs/RELEASE_CHECKLIST.md` as a mandatory pre-me
|
|||
## [Unreleased]
|
||||
|
||||
### Changed
|
||||
- **BREAKING** Analyst bootstrap rewritten end-to-end. `da analyst setup` is removed; replaced by `da init` (non-interactive, requires `--server-url` and `--token`). `da sync` is split into `da pull` (refresh) and `da push` (upload). `da fetch` is folded into `da snapshot create`. `da metrics list/show` is folded into `da catalog --metrics`; `da metrics import/export/validate` move to `da admin metrics {import,export,validate}`. The `da analyst` namespace is removed; the workspace status command is now `da status`.
|
||||
- **BREAKING** Analyst bootstrap rewritten end-to-end. `da analyst setup` is removed; replaced by `agnes init` (non-interactive, requires `--server-url` and `--token`). `da sync` is split into `agnes pull` (refresh) and `agnes push` (upload). `da fetch` is folded into `agnes snapshot create`. `da metrics list/show` is folded into `agnes catalog --metrics`; `da metrics import/export/validate` move to `agnes admin metrics {import,export,validate}`. The `da analyst` namespace is removed; the workspace status command is now `agnes status`.
|
||||
- **BREAKING** Workspace layout simplified. Removed: `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`. Canonical paths: `server/parquet/` (synced parquets), `user/duckdb/analytics.duckdb` (DuckDB views), `user/snapshots/` (ad-hoc snapshots), `user/sessions/` (recorded sessions).
|
||||
- The `/setup` web page now branches on a `role` query parameter: `/setup?role=analyst` renders the analyst workspace bootstrap prompt; `/setup?role=admin` renders the admin CLI install prompt. `/install` continues to 302 to `/setup`.
|
||||
- `CLAUDE.md` server-side template + repo-root `CLAUDE.md` updated to reference the new CLI verbs and workspace paths. The admin UI for the `claude_md_template` DB override (`/admin/workspace-prompt`) renders a yellow banner when the saved override contains legacy strings (`data/parquet/`, `da sync`, `da fetch`, `da analyst setup`, `da metrics list/show`); admins re-author and save to clear it. Migration is manual.
|
||||
|
||||
### Added
|
||||
- `AGNES_WORKSPACE.md` — human-readable workspace docs file generated by `da init` in the workspace root. Documents global install, workspace layout, hooks, cheat sheet, uninstall recipe.
|
||||
- `AGNES_WORKSPACE.md` — human-readable workspace docs file generated by `agnes init` in the workspace root. Documents global install, workspace layout, hooks, cheat sheet, uninstall recipe.
|
||||
- PAT request body now accepts `scope: str = "general"` and `ttl_seconds: int | None = None` fields. PATs minted with `scope="bootstrap-analyst"` are TTL-clamped to ≤ 1 h server-side. Existing `expires_in_days` field continues to work; `ttl_seconds` wins when both are set.
|
||||
- `cli/lib/` shared-library tree, with `cli/lib/pull.py:run_pull` (data-refresh primitive callable from both the Typer wrapper and `da init`) and `cli/lib/hooks.py:install_claude_hooks` (workspace-scoped Claude Code hook installer).
|
||||
- `cli/lib/` shared-library tree, with `cli/lib/pull.py:run_pull` (data-refresh primitive callable from both the Typer wrapper and `agnes init`) and `cli/lib/hooks.py:install_claude_hooks` (workspace-scoped Claude Code hook installer).
|
||||
|
||||
### Fixed
|
||||
- `da pull` (formerly `da sync`) no longer creates `.claude/rules/` when the corporate-memory bundle is empty.
|
||||
- `da pull` no longer creates `server/parquet/` when the manifest is empty.
|
||||
- `da snapshot create` (formerly `da fetch`) no longer materializes an empty `user/duckdb/analytics.duckdb` when run before any `da pull`.
|
||||
- Workspace `da status` reads from the canonical `server/parquet/` and `user/duckdb/analytics.duckdb` paths (was reading legacy `data/parquet/`, `data/metadata/last_sync.json`).
|
||||
- `da init` and `da pull` errors now use the `cli/error_render.py` typed-error renderer (added in 0.32.0), so analyst-facing error UX matches the structured shape `da query --remote` already produces.
|
||||
- `agnes pull` (formerly `da sync`) no longer creates `.claude/rules/` when the corporate-memory bundle is empty.
|
||||
- `agnes pull` no longer creates `server/parquet/` when the manifest is empty.
|
||||
- `agnes snapshot create` (formerly `da fetch`) no longer materializes an empty `user/duckdb/analytics.duckdb` when run before any `agnes pull`.
|
||||
- Workspace `agnes status` reads from the canonical `server/parquet/` and `user/duckdb/analytics.duckdb` paths (was reading legacy `data/parquet/`, `data/metadata/last_sync.json`).
|
||||
- `agnes init` and `agnes pull` errors now use the `cli/error_render.py` typed-error renderer (added in 0.32.0), so analyst-facing error UX matches the structured shape `agnes query --remote` already produces.
|
||||
|
||||
### Removed
|
||||
- `da analyst setup`, `da analyst status`, `da sync`, `da fetch`. See "Changed" above for replacements.
|
||||
- `da metrics` namespace as a top-level group (subcommands moved to `da catalog --metrics` for read-only views and `da admin metrics …` for write operations).
|
||||
- Legacy workspace directories `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`. Existing analyst workspaces should be reinitialized with `da init --server-url ... --token ... --force` (a fresh empty folder is recommended).
|
||||
- `da metrics` namespace as a top-level group (subcommands moved to `agnes catalog --metrics` for read-only views and `agnes admin metrics …` for write operations).
|
||||
- Legacy workspace directories `data/parquet/`, `data/duckdb/`, `data/metadata/`, `user/artifacts/`. Existing analyst workspaces should be reinitialized with `agnes init --server-url ... --token ... --force` (a fresh empty folder is recommended).
|
||||
|
||||
### Kept (clarified)
|
||||
- `da skills list` and `da skills show` survive as analyst-side discovery commands. Earlier draft proposed removal; the rebased main strengthened the bundled skill content (#160 cost-guardrail and registry-gating rails) and removing the surface would cost analyst documentation that the project actively maintains.
|
||||
- `da auth token {create,list,revoke}` stays under `da auth` (where it lives today). No top-level `da token` group is added.
|
||||
- `agnes skills list` and `agnes skills show` survive as analyst-side discovery commands. Earlier draft proposed removal; the rebased main strengthened the bundled skill content (#160 cost-guardrail and registry-gating rails) and removing the surface would cost analyst documentation that the project actively maintains.
|
||||
- `agnes auth token {create,list,revoke}` stays under `agnes auth` (where it lives today). No top-level `da token` group is added.
|
||||
```
|
||||
|
|
|
|||
Loading…
Reference in a new issue