agnes-the-ai-analyst/tests/test_setup_page_unified.py
minasarustamyan 19c5a7592a
Session capture queue, private session, and setup-prompt fixes (#242)
* Capture session paths via SessionStart hook + lock parallel pushes

Replace the encoding-based scan of ~/.claude/projects/<encoded-cwd>/ with
a queue file populated by a new `agnes capture-session` SessionStart hook.
The hook reads the documented `transcript_path` field from Claude Code's
hook stdin JSON, sidestepping the cwd-to-folder encoding (which is an
internal implementation detail and varies by Claude Code version).

- New `agnes capture-session` subcommand appends transcript_path to
  <workspace>/.claude/agnes-sessions.txt. Silent on all malformed input
  so a hook chain failure doesn't clutter Claude Code startup.
- `agnes push` now consumes the queue: atomic snapshot rename guards
  against hooks writing during the push window, successful uploads land
  in agnes-sessions-uploaded.txt (TSV: timestamp + path), failed paths
  are requeued.
- Cross-platform single-instance lock via the filelock package (fcntl
  on POSIX, msvcrt on Windows). Concurrent SessionEnd hooks — common
  when the user closes several sessions at once — silent-exit on the
  losing side instead of all racing the upload.
- Recovery: pre-existing snapshot files from a crashed push are picked
  up and processed before the live queue.
- The SessionStart `agnes push` self-heal entry is dropped — it became
  redundant once the queue persists across runs (orphans from headless /
  crashed sessions ship out on the next interactive SessionEnd push).
  Existing workspaces auto-migrate via the marker-based replace logic.
- Legacy encoding scan stays available behind `--legacy-scan` for one-
  off backfills of sessions predating the queue.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Add /agnes-private + statusLine indicator for private sessions

Users handling sensitive data inside Claude Code can now opt a session
out of the Agnes upload pipeline, either proactively (right after session
start) or reactively (mid-session). The `/agnes-private` slash command
runs `agnes mark-private` deterministically via `!`-prefix direct bash —
no AI in the loop. A workspace-installed statusLine surfaces a
`🔒 agnes-private` indicator in Claude Code's status bar so the user
sees the state at a glance.

Authoritative source of "do not upload" is a separate file
`<workspace>/.claude/agnes-sessions-private.txt` (one session_id per
line). Both `capture-session` (queue writer) and `push` (queue reader)
consult the list. This makes the slash-command / SessionStart-hook race
impossible by construction: whichever runs first, the session is correctly
filtered out.

- `agnes mark-private` reads `CLAUDE_CODE_SESSION_ID` from env (set by
  Claude Code in every bash subprocess it spawns — stable documented API)
  and appends to the private list.
- `agnes statusline` reads the session JSON Claude Code pipes on stdin,
  checks the private list, and emits the indicator or nothing. Optimized
  for the high call frequency of statusLine renders.
- `capture-session` extracts session_id from hook stdin and skips queue
  write when the ID is already on the private list (race protection).
- `push` filters snapshot entries by the private list and appends to a
  per-workspace audit log `agnes-sessions-private-skipped.txt`.
- Queue format migrated from `<path>` to `<session_id>\t<path>`; legacy
  one-column lines still parse (empty session_id, still upload, can't be
  marked private retroactively — fine, they pre-date the feature).
- `install_claude_hooks` writes a workspace statusLine unless the user
  already has a custom one (warn + preserve). Idempotent re-init.
- `install_claude_commands` ships `agnes-private.md` alongside
  `update-agnes-plugins.md`. Per-template fallback so a missing template
  doesn't get clobbered with the wrong content.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix setup-prompt + CLAUDE.md marketplace copy + drop skills step

Three issues against the post-PR-#240 / post-PR-#237 state:

1. Setup prompt's marketplace block trailer (both has-stack and
   empty-stack variants) claimed the SessionStart hook keeps the
   marketplace clone in sync via `agnes refresh-marketplace --quiet`
   on every session and that admin grants land automatically — both
   false since PR #237 (0.47.x) moved the install/update path out of
   the hook into the `/update-agnes-plugins` slash command. The hook
   is `--check`-only: detects server-side changes, prompts the user
   to run the slash command, which does the full reconcile
   interactively with output visible in the transcript.

2. The empty-stack variant framed composition as "admin grants only",
   missing the actual three-source served stack:
     (admin RBAC ∩ /marketplace subscriptions)
       ∪ system-mandatory plugins (admin-pinned, auto-applied)
       ∪ Flea market installs (skills/agents bundled, plugins standalone)
   Updated copy spells out all three sources so analysts know where
   their stack picks live, and what the SessionStart hook actually
   does on change detection.

3. CLAUDE.md template's "Agnes Marketplace" section conflated
   eligibility (`resolve_allowed_plugins` — what's listed) with served
   stack (`resolve_user_marketplace` — what actually reaches Claude
   Code). The two are different: a user can be RBAC-eligible for a
   plugin without having subscribed to it on /marketplace. Rewrote
   the section to distinguish the eligibility set from the served
   stack and to describe the `--check`-only hook accurately.

Plus: deleted the setup prompt's interactive Skills step (final step
before Confirm). The named-opinion question — "do you want me to
bulk-copy every skill into ~/.claude/skills/agnes/ or pull on-demand
via `agnes skills show <name>`?" — had no obvious right answer for
new users at the tail end of a wall of technical steps. On-demand
lookup is the one-size-fits-all default; `agnes skills list/show`
remain discoverable and the CLAUDE.md template references specific
skills inline (e.g. agnes-data-querying in the BigQuery section)
where they're relevant. Layout: Confirm shifts from step 9 to step 8.

Tests updated, full setup/marketplace/welcome surface green (115
passed). Remaining full-suite failures are pre-existing (BQ/Keboola
fixtures, Windows charmap collection error in test_v26_keboola_e2e)
— verified against a clean stash, unrelated to this diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix session-queue race + snapshot PID-reuse data loss

Two blocker fixes from the PR #242 review:

1. Concurrent SessionStart hooks could corrupt the queue file on
   Windows. Python's `open(path, "a")` is not atomic there — the CRT
   does not pass FILE_APPEND_DATA to CreateFile, so concurrent
   appenders (user opening several Claude Code windows simultaneously)
   could interleave bytes mid-line. The malformed lines then silently
   fail the parser and the entries are dropped.

   Fix: wrap append_to_queue, requeue_failed, and snapshot_queue in a
   short-lived FileLock on a dedicated `agnes-queue.lock`. Separate
   from `agnes-push.lock` so capture-session hooks don't block on the
   push command. New test_append_concurrent_threads_no_corruption
   reproduces the race with 4 threads x 50 appends.

2. Snapshot filenames embedded only the PID (`agnes-sessions.snapshot.
   <PID>.txt`). After a crashed push left a snapshot on disk and the
   OS recycled the PID for a new push, `os.rename` would atomically
   overwrite the recovery snapshot — every entry in it lost, silently.

   Fix: append a uuid8 hex tail (`agnes-sessions.snapshot.<PID>.
   <uuid8>.txt`). find_recovery_snapshots already globs the prefix
   so it picks up both old and new format. New
   test_snapshot_filename_is_unique_per_call asserts two consecutive
   snapshots under the same PID don't collide.

Targeted tests green (47/47 in session_queue/capture_session/cli_push).
Full suite failures unchanged from baseline (pre-existing BQ/Keboola
fixture issues per CLAUDE.md).

* Auto-refresh workspace hooks + bash-wrap all hook entries (Windows)

Fixes from PR #242 second review (ZdenekSrotyr):

1. `uv.lock` regenerated to include `filelock 3.29.0` (declared in
   pyproject.toml but missing from the lock file — CI's
   lockfile-consistency check would fail; `uv pip install` on a clean
   cache would silently miss the dep).

2. `agnes self-upgrade` now auto-refreshes the workspace Claude Code
   hooks via the new `cli.lib.hooks.maybe_refresh_claude_hooks`. Closes
   the silent-stop migration gap: a v0.48 workspace would auto-upgrade
   the CLI from its existing SessionStart self-upgrade entry but never
   pick up the new `agnes capture-session` SessionStart hook, leaving
   the queue empty and `agnes push` uploading nothing.

   The refresh fires on both the "info is None" fast path (CLI already
   current — catches the second SessionStart after a prior upgrade)
   and the install-success path. Guarded by `workspace_has_agnes_hooks`
   so it never writes `.claude/settings.json` into directories that
   aren't Agnes workspaces (e.g. `agnes self-upgrade` invoked from
   `~/`). Errors are surfaced on stderr but never flip the upgrade exit
   code.

3. All Agnes-managed hooks are now wrapped in `bash -c "..."`. The
   self-upgrade+pull chained SessionStart entry was the only one still
   shipping unwrapped — Claude Code on Windows runs hook commands
   directly without a shell, so the `;` chain + `2>/dev/null` +
   `|| true` shell syntax silently no-op'd on native Windows installs
   without Git Bash on PATH. Workspaces still on the old form
   auto-upgrade via the refresh path above.

Tests: +12 in test_lib_hooks.py (guard semantics, v0.48→v0.49
migration end-to-end, third-party-hook preservation, bash-wrap
invariant). +5 in test_self_upgrade.py (refresh fires on info=None,
fires on install success, skipped on failure, skipped on --check-only,
refresh failure never flips exit code).

130 targeted tests green. The 2 pre-existing Windows path-separator
failures in `test_smoke_test_detects_version_mismatch[uv|pip]` are
unrelated (path mismatch `\fake\uv\bin\agnes` vs `/fake/uv/bin/agnes`
in test asserts, pre-PR baseline).

* CHANGELOG: document PR-242 main features

Closes ZdenekSrotyr #4: the [Unreleased] block was missing entries for
the PR's primary surface — only the post-merge fix bullets and the
unrelated setup-prompt copy change were captured. Adds:

- ### Added: 6 bullets covering the session capture queue + new
  `agnes capture-session` subcommand, `/agnes-private` slash + `agnes
  mark-private`, `agnes statusline` + statusLine wiring, `--legacy-scan`
  opt-in fallback, single-instance push lock, and the new `filelock`
  runtime dep.

- ### Changed: BREAKING bullet on the SessionStart / SessionEnd hook
  wire format change (capture-session as first SessionStart entry,
  push self-heal removed, SessionEnd push detached via nohup, all
  entries bash-wrapped). Folds the prior standalone bash-wrap bullet
  into this consolidated entry — Z's review flagged the layout shift
  as BREAKING, and grouping the related sub-changes makes the
  migration story readable in one place.

- Operator migration is auto-handled by `maybe_refresh_claude_hooks`
  invoked from `agnes self-upgrade` (separate Changed entry below).
  No `agnes init` re-run required. Pre-queue session jsonls on
  upgrading workspaces still need a one-off `agnes push --legacy-scan`
  — flagged in the BREAKING bullet.

No code change; doc only.

* Drop permanent 4xx uploads instead of requeueing forever

Closes ZdenekSrotyr #5. Previously the push retry path requeued any
non-200 response except the literal "file not found on disk", so 401
(token expired), 403 (RBAC denial), 413 (payload too large), 400
(server-side validation) cycled through every push run forever — the
queue grew without bound and each run re-bombarded the server with the
same deterministically-failing upload.

Now 4xx (except 408 Request Timeout + 429 Too Many Requests, which the
HTTP spec marks as transient) is dropped and audit-logged to
`<workspace>/.claude/agnes-sessions-failed.txt`:

    <iso_ts>\t<session_id>\t<status>\t<transcript_path>

5xx and network errors continue to requeue — those reflect server /
transport state that can change between runs, so retry is the right
behavior.

The audit log piggybacks on the push single-instance lock
(agnes-push.lock) — push is the only writer to this file, same as the
existing `mark_uploaded` and `mark_private_skipped` paths, so no
separate filelock is needed.

`agnes push --json` surfaces a new `dropped_permanent` counter; non-
quiet stdout mentions the audit-log path so operators tailing the
output have a pointer to the forensic trail.

Tests: +7 in test_cli_push.py (401/400/403/413 → drop; 408/429 →
requeue; 500/502/503 → requeue; network exception → requeue;
--json `dropped_permanent` counter; stdout audit-log pointer). +1 in
test_session_queue.py (mark_failed_permanent TSV format).

127/129 targeted tests green. The 2 pre-existing Windows
path-separator failures in `test_smoke_test_detects_version_mismatch
[uv|pip]` are unrelated (path mismatch `\fake\uv\bin\agnes` vs
`/fake/uv/bin/agnes` in test asserts, pre-PR baseline).

* Catch OSError in push lock acquisition

Closes ZdenekSrotyr #8. `acquire_or_skip` in `cli/lib/push_lock.py`
previously caught only `filelock.Timeout`. Any `OSError` from
`FileLock.acquire` — read-only filesystem, permission denied on
`.claude/`, disk full, hardware I/O failure — propagated as an
unhandled traceback.

Two visible failure modes:
- SessionEnd hook: `|| true` in the wrapper swallowed the error, so
  daily pushes silently never ran. Operator had no signal.
- Manual `agnes push`: ugly Python traceback dumped to the terminal
  instead of a clean exit.

Now `OSError` is treated the same as `Timeout` — yield `None`, caller
returns cleanly with rc=0. The operator's environment in these
scenarios has bigger problems than missing session uploads, so we
swallow rather than retry-loop or surface a noisy warning.

Test: `test_push_silent_exit_when_filelock_raises_oserror` patches
the `FileLock` used inside `push_lock` to raise OSError on acquire,
verifies push exits 0 with no traceback and the queue is preserved
for the next attempt.

* Address remaining S2 items from PR-242 review

Four items from ZdenekSrotyr's S2 list:

S2.10 — `_install_statusline` truthy check (cli/lib/hooks.py): replace
`if existing:` with explicit `if existing is None or existing == "":`.
Documents and tests the behavior for both edge cases (explicit-null
and empty-string `statusLine`) — both treated as "not configured"
rather than "explicit user opt-out", so we install ours. Two new
tests in test_lib_hooks.py pin the contract.

S2.6 — onboarding docs for /agnes-private. New "Private sessions"
subsection in `config/claude_md_template.txt` (next to Data Sync)
covering the slash command, statusbar indicator, and audit-log
location. One-line tip in `app/web/setup_instructions.py` so the
feature is discoverable at onboarding.

S2.9 — e2e privacy test (tests/test_e2e_privacy.py). Wires
capture_session → mark_private → push against a recording fake
api_post and asserts zero session uploads for the marked one.
Three cases: mark-before-capture (queue write skipped),
mark-after-capture (push-side filter catches it + audit-logs),
control (unmarked sessions upload normally).

David #8 — `--legacy-scan` help text now documents the
private-list gap (legacy entries carry empty session_id, so
the filter is not consulted). The practical impact is bounded —
pre-queue sessions cannot have been marked private since the
private list is a queue-era feature — but the disclaimer in the
help text means an operator running a backfill is not surprised.

68 targeted tests green (3 new e2e + 2 new truthy edge tests +
existing). 2 pre-existing Windows path-separator failures in
test_smoke_test_detects_version_mismatch[uv|pip] unchanged.

Remaining S2 items (statusline mkdir push-back, capture-session
silent-fail follow-up) handled in PR comment + follow-up issue
respectively.

* Address remaining S2 follow-ups (David #8, S2.7, David #11)

Three items left over from Mina's bbf63472 batch — that commit
addressed S2.6/S2.9/S2.10 + documented David #8 in help text but
deferred the actual implementations of S2.7, David #11, and the real
David #8 fix to follow-ups. This commit closes them.

David #8 — `agnes push --legacy-scan` now consults the private list.
Claude Code names jsonls `<session-id>.jsonl`, so the file stem IS
the session id; the legacy-scan path can apply the same private filter
the queue path uses. Both the dry-run and live-upload code paths fixed.
Help text updated (no longer warns the filter is bypassed). Two new
tests in test_cli_push.py cover the upload-skip path + the dry-run
`would_skip_private` segregation.

S2.7 — `statusline`/`is_private` no longer mkdir-pollutes arbitrary
workdirs. Split `_claude_dir` into `_claude_dir_writable` (used only
from `add_private`) and `_claude_dir_readonly` (no mkdir). The
read-only public helpers (`private_list_path`, `read_all_private`,
`is_private`) compose the no-mkdir variant by default; `add_private`
opts in via `writable=True`. Added a process-local mtime-keyed cache
around `read_all_private` so in-process callers (push doing one stat
per upload candidate, future `agnes diagnose`) don't re-parse the
file on every check. Cache eviction on `add_private` so a sub-second
write+read sequence doesn't see stale data even on coarse-mtime
filesystems. Two new tests pin the no-mkdir contract + the
in-same-second add+read consistency.

David #11 — `agnes capture-session` writes a breadcrumb log on every
invocation. New `<workspace>/.claude/agnes-capture-session.log` TSV:
`<iso_ts>\t<outcome>\t<detail>` where outcome covers every silent-
exit path (`ok`, `private_skip`, `empty_stdin`, `bad_json`,
`not_object`, `no_transcript_path`, `stdin_read_error`,
`write_error`). Gives operators a signal to detect "hook fires but
queue stays empty" — without it, an upstream Claude Code stdin-
contract change is invisible because the hook always exits 0. Log
rolls at 256 KiB so it doesn't grow unbounded on long-lived
workspaces. Best-effort: a breadcrumb-write failure is itself
swallowed so the hook contract stays "exit 0 always". Skipped in
non-Agnes workdirs (no `.claude/` exists) so opening Claude Code
in `~/` doesn't pollute it. Five new tests in test_capture_session.py
cover the success / bad_json / no_transcript_path / private_skip /
no-pollute paths.

115 targeted tests green (test_cli_push, test_capture_session,
test_private_list, test_session_queue, test_e2e_privacy,
test_lib_hooks, test_statusline, test_mark_private).

---------

Co-authored-by: Minas Arustamyan <arustamyan.minas@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
2026-05-11 13:31:16 +00:00

128 lines
5.4 KiB
Python

"""Tests for the unified `/setup` route.
The previous `?role=analyst|admin` query parameter is gone. The route
renders a single layout for everyone — admin-vs-analyst is no longer a
branch. The marketplace + plugins block is gated by per-user
`resource_grants` resolved inside `compute_default_agent_prompt`.
"""
import pytest
from fastapi.testclient import TestClient
@pytest.fixture
def client(tmp_path, monkeypatch):
"""TestClient against a freshly-built FastAPI app rooted at tmp_path.
Mirrors the `web_client` fixture in tests/test_web_ui.py — we re-create
the app so the DuckDB singleton picks up the per-test DATA_DIR rather
than leaking state across tests on the same xdist worker.
"""
monkeypatch.setenv("DATA_DIR", str(tmp_path))
monkeypatch.setenv("TESTING", "1")
monkeypatch.setenv("JWT_SECRET_KEY", "test-secret-key-min-32-characters!!")
(tmp_path / "state").mkdir()
(tmp_path / "analytics").mkdir()
(tmp_path / "extracts").mkdir()
from src.db import close_system_db
close_system_db()
from app.main import create_app
app = create_app()
yield TestClient(app)
close_system_db()
def test_setup_page_renders_unified_layout(client):
"""Bare `/setup` (no query param) renders the unified flow:
- `agnes init` is mandatory (subsumes the old admin-only
`agnes auth import-token` + `agnes auth whoami` pair).
- Marketplace block is always emitted: anonymous visitors with no
plugin grants still get the marketplace registration step so
future stack changes land cleanly. Confirm = step 8 in the
post-skills-removal layout.
"""
resp = client.get("/setup", follow_redirects=True)
assert resp.status_code == 200
text = resp.text
# Unified flow markers.
assert "agnes init" in text
# Legacy admin-only login verbs are gone from the rendered prompt.
assert "agnes auth import-token" not in text
# Always-on layout (preflight + marketplace + MCP all unconditional,
# skills removed): Confirm = step 8.
assert "8) Confirm:" in text
def test_setup_page_ignores_role_query_param(client):
"""`?role=...` is no longer accepted by the route signature. FastAPI
ignores unknown query params silently — `/setup?role=admin` still
serves the unified layout. No 422, no redirect, no behavior delta
vs. bare `/setup`."""
bare = client.get("/setup", follow_redirects=True)
with_role = client.get("/setup?role=admin", follow_redirects=True)
assert bare.status_code == 200
assert with_role.status_code == 200
# Both responses contain the unified-flow marker.
assert "agnes init" in bare.text
assert "agnes init" in with_role.text
# Legacy admin-only login verbs are gone from both.
assert "agnes auth import-token" not in bare.text
assert "agnes auth import-token" not in with_role.text
def test_setup_page_renders_marketplace_for_user_with_grants(client, monkeypatch):
"""When the caller has a non-empty served stack, the marketplace block
renders the "install your current stack" copy variant. Confirm stays
at step 8 in the post-skills-removal layout (preflight + marketplace
+ MCP all always-on regardless of stack contents).
Stub `marketplace_filter.resolve_user_marketplace` to return a
plugin so we don't have to seed the full marketplace plumbing in
this test — we're verifying the layout, not the RBAC resolver
itself (covered by `test_marketplace_filter`).
Post-Model B (v28+): the setup page reads from
`resolve_user_marketplace` (which gates on explicit subscriptions)
rather than `resolve_allowed_plugins` (RBAC-only)."""
from app.web.router import get_optional_user
from fastapi import Request
from src import marketplace_filter
async def _admin_user(request: Request): # type: ignore[no-redef]
return {"id": "admin-1", "email": "admin@example.com",
"is_admin": True, "name": "Admin", "groups": ["Admin"]}
monkeypatch.setattr(
marketplace_filter,
"resolve_user_marketplace",
lambda conn, user: [{"manifest_name": "demo-plugin"}],
)
client.app.dependency_overrides[get_optional_user] = _admin_user
try:
resp = client.get("/setup", follow_redirects=True)
finally:
client.app.dependency_overrides.pop(get_optional_user, None)
assert resp.status_code == 200
text = resp.text
# Marketplace block marker. The per-plugin install lines moved inside
# `agnes refresh-marketplace --bootstrap`, so we check the section
# header + the one-liner instead of `claude plugin install <name>@agnes`.
# Non-empty stack → "install your current stack" header variant.
assert "Register the Agnes Claude Code marketplace and install your current stack" in text
assert "agnes refresh-marketplace --bootstrap" in text
# Post-skills-removal layout: Confirm is step 8.
assert "8) Confirm:" in text
# Pre-flight is in the rendered prompt at step 4.
assert "Make sure git and claude are installed" in text
# Atlassian MCP registration is at step 6.
assert "claude mcp add --transport sse atlassian" in text
def test_install_legacy_path_redirects_to_setup(client):
"""`/install` legacy path keeps redirecting to `/setup` (302/307)."""
resp = client.get("/install", follow_redirects=False)
assert resp.status_code in (302, 307)
assert "/setup" in resp.headers["location"]