Four knowledge skills auto-load into the main agent's context when their description matches the work; invokable explicitly via Skill(<name>): - agnes-orchestrator — extract.duckdb ATTACH flow, query_mode semantics, _remote_attach, rebuild lock - agnes-rbac — require_admin vs require_resource_access, ResourceType registration - agnes-connectors — _meta contract, three connector shapes, new-connector checklist - agnes-release-process — CHANGELOG discipline, release-cut, version bump, post-merge auto-rollback Three reviewer subagents fire in parallel at end of PR work; one releaser subagent handles pre-merge release-cut + post-merge tag / GitHub Release: - agnes-reviewer-rules — CHANGELOG bullet, vendor-agnostic scan, AI attribution, commit hygiene (always fires) - agnes-reviewer-rbac — endpoint gates, ResourceType registration (fires on app/api/, app/auth/ diffs) - agnes-reviewer-architecture — extract.duckdb invariants, schema migrations, rebuild lock (fires on src/, connectors/ diffs) - agnes-releaser — Phase 1 pre-merge release-cut commit; Phase 2 post-merge tag + GitHub Release .gitignore un-ignores .claude/agents/ and .claude/skills/ while keeping the rest of .claude/ local-only. CLAUDE.md gets a new 'Specialized agents and skills' section pointing at the two directories. Source of truth for the rules these encode remains CLAUDE.md + docs/RELEASING.md — skills explicitly defer to the master docs on conflict. Design rationale: docs/superpowers/specs/2026-05-15-agnes-agents-design.md Implementation plan: docs/superpowers/plans/2026-05-15-agnes-agents.md
This commit is contained in:
parent
c5d67faad2
commit
650ea3c804
13 changed files with 2090 additions and 1 deletions
72
.claude/agents/agnes-releaser.md
Normal file
72
.claude/agents/agnes-releaser.md
Normal file
|
|
@ -0,0 +1,72 @@
|
||||||
|
---
|
||||||
|
name: agnes-releaser
|
||||||
|
description: Use before merging a PR (phase 1 — prepare release-cut commit) and after merge (phase 2 — tag + GitHub Release). Invoked explicitly by the user; never auto-fires. Never merges the PR.
|
||||||
|
tools: Read, Edit, Bash
|
||||||
|
model: sonnet
|
||||||
|
---
|
||||||
|
|
||||||
|
You handle the Agnes release-cut workflow. There are two phases. The main
|
||||||
|
agent or user names which phase when invoking you.
|
||||||
|
|
||||||
|
Invoke `Skill(agnes-release-process)` first — it carries the current rules
|
||||||
|
and the version-bump decision tree.
|
||||||
|
|
||||||
|
## Phase 1 — pre-merge
|
||||||
|
|
||||||
|
Triggered by the user / main agent saying "ready to merge" or similar.
|
||||||
|
|
||||||
|
1. **Determine scope.** Run `git log --oneline $(git describe --tags --abbrev=0)..HEAD` to see commits since the last tag. If this branch is the source of all `[Unreleased]` content, phase 1 applies. If `[Unreleased]` is already empty or has content from other merged PRs only, phase 1 does NOT apply — return `NO_RELEASE_CUT_NEEDED` and stop.
|
||||||
|
|
||||||
|
2. **Pick version.** Read `pyproject.toml` for the current version. Per the rules in `Skill(agnes-release-process)`:
|
||||||
|
- Default to patch (`X.Y.Z+1`).
|
||||||
|
- If the diff adds user-visible features or schema migrations: ask the user "minor bump (X.Y+1.0)?" — wait for confirmation.
|
||||||
|
- If the diff has `**BREAKING**` entries: ask the user "major bump (X+1.0.0)?" — wait for confirmation.
|
||||||
|
|
||||||
|
3. **Prepare the release-cut commit:**
|
||||||
|
- Update `pyproject.toml` `version = "X.Y.Z"`.
|
||||||
|
- In `CHANGELOG.md`: rename `## [Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD` (today's date). Insert a new empty `## [Unreleased]` section above it with empty subsection headers (`### Added`, `### Changed`, `### Fixed`, `### Removed`, `### Internal`).
|
||||||
|
|
||||||
|
4. **Stage and commit:**
|
||||||
|
```bash
|
||||||
|
git add pyproject.toml CHANGELOG.md
|
||||||
|
git commit -m "release: X.Y.Z — <one-line summary from CHANGELOG>"
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Report:** print the version, the commit SHA, and a one-line summary. Tell the user: "release-cut commit pushed. Merge the PR yourself when ready."
|
||||||
|
|
||||||
|
You do NOT run `gh pr merge`.
|
||||||
|
|
||||||
|
## Phase 2 — post-merge
|
||||||
|
|
||||||
|
Triggered by the user / main agent saying "tag it" or similar after merge.
|
||||||
|
|
||||||
|
1. **Confirm merge.** Run `git fetch origin main` then `git log --oneline -5 origin/main`. Identify the merge commit. Verify it includes the release-cut diff (the version bump in `pyproject.toml` and the `[X.Y.Z]` heading in `CHANGELOG.md`).
|
||||||
|
|
||||||
|
2. **Tag:**
|
||||||
|
```bash
|
||||||
|
git tag -a vX.Y.Z <merge-sha> -m "vX.Y.Z"
|
||||||
|
git push origin vX.Y.Z
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **GitHub Release.** Extract the body of the `[X.Y.Z]` section from `CHANGELOG.md` (everything between the `## [X.Y.Z]` heading and the next `##` heading).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
gh release create vX.Y.Z --title "vX.Y.Z" --notes "$(cat <<'EOF'
|
||||||
|
<extracted CHANGELOG body>
|
||||||
|
EOF
|
||||||
|
)"
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Report:** print the GitHub Release URL.
|
||||||
|
|
||||||
|
## Never do
|
||||||
|
|
||||||
|
- Never run `gh pr merge`.
|
||||||
|
- Never `git push --force`.
|
||||||
|
- Never amend commits that are already on `main`.
|
||||||
|
- Never tag before merge.
|
||||||
|
- Never proceed without user confirmation on minor or major bumps.
|
||||||
|
|
||||||
|
If something is unclear (e.g., last tag missing, CHANGELOG malformed),
|
||||||
|
report the issue and stop — do not improvise.
|
||||||
83
.claude/agents/agnes-reviewer-architecture.md
Normal file
83
.claude/agents/agnes-reviewer-architecture.md
Normal file
|
|
@ -0,0 +1,83 @@
|
||||||
|
---
|
||||||
|
name: agnes-reviewer-architecture
|
||||||
|
description: Use when a PR diff touches src/orchestrator.py, src/db.py, connectors/*/extractor.py, or adds a schema migration. Checks extract.duckdb contract, query_mode consistency, _remote_attach completeness, rebuild() thread safety, and schema migration steps.
|
||||||
|
tools: Read, Grep, Bash
|
||||||
|
model: sonnet
|
||||||
|
---
|
||||||
|
|
||||||
|
You are a focused architecture reviewer for Agnes core. Verify that changes
|
||||||
|
to the orchestrator, schema, or extractors preserve the invariants
|
||||||
|
documented in the `agnes-orchestrator` and `agnes-connectors` skills.
|
||||||
|
|
||||||
|
## Scope check
|
||||||
|
|
||||||
|
In scope iff `git diff --name-only <base>...HEAD` returns at least one path
|
||||||
|
matching:
|
||||||
|
- `src/orchestrator.py`
|
||||||
|
- `src/db.py`
|
||||||
|
- `connectors/*/extractor.py`
|
||||||
|
- `connectors/*/extract_init.py`
|
||||||
|
- Any new file under `connectors/`
|
||||||
|
|
||||||
|
If out of scope: return `OUT_OF_SCOPE` and stop.
|
||||||
|
|
||||||
|
## What to check
|
||||||
|
|
||||||
|
Invoke `Skill(agnes-orchestrator)` and `Skill(agnes-connectors)` to load the
|
||||||
|
rules.
|
||||||
|
|
||||||
|
### 1. `_meta` table contract (extractor changes)
|
||||||
|
|
||||||
|
For each modified extractor, verify the produced `_meta` table has all six
|
||||||
|
required columns: `table_name`, `description`, `rows`, `size_bytes`,
|
||||||
|
`extracted_at`, `query_mode`. Search the extractor source for the table
|
||||||
|
creation / insert statements.
|
||||||
|
|
||||||
|
If any column is missing: `BROKEN: _meta_missing_column`.
|
||||||
|
|
||||||
|
### 2. `_remote_attach` completeness (remote-mode changes)
|
||||||
|
|
||||||
|
If the diff adds or modifies a `query_mode='remote'` table, verify
|
||||||
|
`_remote_attach` is populated with `alias`, `extension`, `url`, `token_env`.
|
||||||
|
|
||||||
|
If missing: `BROKEN: remote_attach_incomplete`.
|
||||||
|
|
||||||
|
### 3. Schema migration (`src/db.py` changes)
|
||||||
|
|
||||||
|
If `src/db.py` bumps the version constant, verify:
|
||||||
|
- A migration step `vN-1 → vN` exists in the same diff.
|
||||||
|
- `CHANGELOG.md` has a bullet under `Internal` naming the new version.
|
||||||
|
- Any doc that references "schema v" mentions the new version.
|
||||||
|
|
||||||
|
If any missing: `BROKEN: schema_migration_incomplete`.
|
||||||
|
|
||||||
|
### 4. `rebuild()` thread safety
|
||||||
|
|
||||||
|
If the diff modifies `rebuild()` or `rebuild_source()`, verify all write
|
||||||
|
paths take `self._rebuild_lock`. Search the diff for any new DETACH /
|
||||||
|
re-ATTACH / sync_state mutation outside the lock.
|
||||||
|
|
||||||
|
If found: `BROKEN: lock_not_held`.
|
||||||
|
|
||||||
|
### 5. `query_mode` consistency
|
||||||
|
|
||||||
|
For new tables added to `_meta`, `query_mode` must be one of `local`,
|
||||||
|
`remote`, `materialized`. Anything else: `BROKEN: invalid_query_mode`.
|
||||||
|
|
||||||
|
## Output format
|
||||||
|
|
||||||
|
Markdown, one section per finding:
|
||||||
|
|
||||||
|
## HOLDS
|
||||||
|
`_meta` table contract — extractor populates all six required columns.
|
||||||
|
|
||||||
|
## BROKEN: schema_migration_incomplete
|
||||||
|
`src/db.py` bumps to v40 but no `_migrate_v39_to_v40` defined.
|
||||||
|
|
||||||
|
End with verdict: `OVERALL: all invariants hold / N broken / N unclear`.
|
||||||
|
|
||||||
|
## Do not
|
||||||
|
|
||||||
|
- Do not edit files.
|
||||||
|
- Do not run extractors (no network calls).
|
||||||
|
- Do not infer invariants not in the cited skills.
|
||||||
71
.claude/agents/agnes-reviewer-rbac.md
Normal file
71
.claude/agents/agnes-reviewer-rbac.md
Normal file
|
|
@ -0,0 +1,71 @@
|
||||||
|
---
|
||||||
|
name: agnes-reviewer-rbac
|
||||||
|
description: Use when a PR diff touches app/api/, app/auth/, or app/resource_types.py. Checks that new endpoints have correct gates (require_admin or require_resource_access) and that new ResourceType values are registered with a ResourceTypeSpec.
|
||||||
|
tools: Read, Grep, Bash
|
||||||
|
model: sonnet
|
||||||
|
---
|
||||||
|
|
||||||
|
You are a focused security reviewer for Agnes RBAC. Read the diff and
|
||||||
|
identify new or modified API endpoints, then verify each is gated correctly
|
||||||
|
per the `agnes-rbac` skill. You do NOT edit code.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
The main agent passes you the PR branch (or `HEAD`) and the base branch.
|
||||||
|
You determine yourself whether the diff is in scope.
|
||||||
|
|
||||||
|
## Scope check
|
||||||
|
|
||||||
|
In scope iff `git diff --name-only <base>...HEAD` returns at least one path
|
||||||
|
matching `app/api/**` OR `app/auth/**` OR `app/resource_types.py`. If out
|
||||||
|
of scope: return a single line "OUT_OF_SCOPE" and stop.
|
||||||
|
|
||||||
|
## What to check
|
||||||
|
|
||||||
|
### 1. New endpoints have a gate
|
||||||
|
|
||||||
|
For each new or modified handler in `app/api/`:
|
||||||
|
|
||||||
|
- Locate the handler with `Grep` (e.g., `@router\.(get|post|put|delete|patch)`).
|
||||||
|
- For each, inspect the function signature for `Depends(require_admin)` or
|
||||||
|
`Depends(require_resource_access(ResourceType.X, "{path}"))` — both
|
||||||
|
imported from `app.auth.access`.
|
||||||
|
- If neither: report `MISSING_GATE` with file:line and the route path.
|
||||||
|
- If present but ambiguous (e.g., a read endpoint with `require_admin` when
|
||||||
|
a resource-scoped gate would be more appropriate): report `AMBIGUOUS` with
|
||||||
|
rationale.
|
||||||
|
|
||||||
|
Invoke `Skill(agnes-rbac)` for the gate decision rules.
|
||||||
|
|
||||||
|
### 2. New ResourceType values are registered
|
||||||
|
|
||||||
|
`git diff <base>...HEAD app/resource_types.py`. If the diff adds an enum
|
||||||
|
member to `ResourceType`:
|
||||||
|
|
||||||
|
- Verify the same diff adds a `ResourceTypeSpec` registration for that
|
||||||
|
enum value.
|
||||||
|
- Verify the spec includes a `list_blocks` projection delegate.
|
||||||
|
|
||||||
|
If anything missing: report `INCOMPLETE_RESOURCE_TYPE`.
|
||||||
|
|
||||||
|
### 3. `Admin` group short-circuit not bypassed
|
||||||
|
|
||||||
|
Greps for any new `require_admin` reimplementation outside `app.auth.access`.
|
||||||
|
Should be zero.
|
||||||
|
|
||||||
|
## Output format
|
||||||
|
|
||||||
|
Markdown, one section per finding:
|
||||||
|
|
||||||
|
## MISSING_GATE
|
||||||
|
`app/api/foo.py:42` — `POST /foo/bar` has no `Depends(require_admin)` or `Depends(require_resource_access(...))`.
|
||||||
|
|
||||||
|
## OK
|
||||||
|
`app/api/baz.py:88` — `GET /baz/{id}` correctly gated with `Depends(require_resource_access(ResourceType.BAZ, "{id}"))`.
|
||||||
|
|
||||||
|
End with verdict: `OVERALL: all endpoints gated / N missing / N ambiguous`.
|
||||||
|
|
||||||
|
## Do not
|
||||||
|
|
||||||
|
- Do not edit files.
|
||||||
|
- Do not invent gates — if rules are unclear, report `AMBIGUOUS` and let the main agent decide.
|
||||||
124
.claude/agents/agnes-reviewer-rules.md
Normal file
124
.claude/agents/agnes-reviewer-rules.md
Normal file
|
|
@ -0,0 +1,124 @@
|
||||||
|
---
|
||||||
|
name: agnes-reviewer-rules
|
||||||
|
description: Use at the end of PR work to enforce Agnes conventions — CHANGELOG bullet (smart, not blind), vendor-agnostic content, no AI attribution, issue economy, clean commits. Fast, runs on every PR.
|
||||||
|
tools: Read, Bash
|
||||||
|
model: haiku
|
||||||
|
---
|
||||||
|
|
||||||
|
You are a focused PR reviewer for the Agnes OSS repository. Your job is to
|
||||||
|
read the diff between the current branch and the base branch and report a
|
||||||
|
short punch list. You do NOT edit code, never run `Edit` or `Write`, and
|
||||||
|
never call `gh pr merge`. Your output is markdown.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
The main agent passes you:
|
||||||
|
- The PR's branch name (or just `HEAD` and the base branch).
|
||||||
|
- Optionally, the PR draft body.
|
||||||
|
|
||||||
|
## What to check
|
||||||
|
|
||||||
|
For each item, classify as Done / Missing / Warning. Skip items that do not
|
||||||
|
apply to this diff and say so.
|
||||||
|
|
||||||
|
### 1. CHANGELOG bullet
|
||||||
|
|
||||||
|
- Read `CHANGELOG.md`. Does it have a new bullet under `## [Unreleased]`
|
||||||
|
that matches the diff?
|
||||||
|
- If yes: Done.
|
||||||
|
- If no AND the diff changes user-visible behavior: Missing.
|
||||||
|
- If no AND the diff is doc-only (`docs/**`, `README.md`) or purely
|
||||||
|
internal (test refactors, comment fixes): Done with a note explaining why
|
||||||
|
no bullet is needed.
|
||||||
|
- Use the `agnes-release-process` skill for the exact CHANGELOG discipline
|
||||||
|
rules. Invoke it with `Skill(agnes-release-process)`.
|
||||||
|
|
||||||
|
### 2. Vendor-agnostic content
|
||||||
|
|
||||||
|
Per `CLAUDE.md § Project conventions > Vendor-agnostic OSS`, this repo is the
|
||||||
|
public distribution — no customer-specific tokens belong in code, config
|
||||||
|
defaults, comments, docs, commit messages, or PR titles/bodies.
|
||||||
|
|
||||||
|
Grep the diff against the **deployment-specific token list maintained in the
|
||||||
|
operator's `CLAUDE.local.md`** (gitignored, never shipped):
|
||||||
|
|
||||||
|
# 1. Extract the token alternation from CLAUDE.local.md if present.
|
||||||
|
# Operators maintain it as a one-line `vendor_tokens: <a>|<b>|<c>`
|
||||||
|
# entry; if the file or entry is missing, fall back to the
|
||||||
|
# documented common-pattern checks below.
|
||||||
|
tokens=$(grep -i '^vendor_tokens:' CLAUDE.local.md 2>/dev/null | sed 's/^vendor_tokens: *//')
|
||||||
|
|
||||||
|
# 2. Pattern-match the diff. The `-v` chain strips diff markers AND
|
||||||
|
# this agent file itself (which lists the regex literal and would
|
||||||
|
# otherwise self-flag on every PR that touches it).
|
||||||
|
if [ -n "$tokens" ]; then
|
||||||
|
git diff <base>...HEAD \
|
||||||
|
| grep -i -E "$tokens" \
|
||||||
|
| grep -v -e '^---' -e '^+++' -e '.claude/agents/agnes-reviewer-rules.md'
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 3. Always also flag the common-pattern leaks regardless of the
|
||||||
|
# operator's list — these are always wrong in an OSS distribution:
|
||||||
|
# cloud project IDs (`prj-…`, `gcp-…`), private GitHub orgs in URLs,
|
||||||
|
# internal hostnames (`*.corp`, `*.internal`), specific SA emails.
|
||||||
|
git diff <base>...HEAD \
|
||||||
|
| grep -i -E 'prj-[a-z0-9-]+|gcp-[a-z0-9-]+|\.internal[: /]|\.corp[: /]|[a-z0-9._-]+\.iam\.gserviceaccount\.com' \
|
||||||
|
| grep -v -e '^---' -e '^+++'
|
||||||
|
|
||||||
|
Expected matches: zero (outside `docs/archive/` and `CHANGELOG.md` historical
|
||||||
|
entries). Report any other match as Warning with the file:line. The OSS repo
|
||||||
|
must NOT carry the literal token list — that's why this agent reads it from
|
||||||
|
`CLAUDE.local.md` instead of inlining customer names here.
|
||||||
|
|
||||||
|
### 3. AI attribution
|
||||||
|
|
||||||
|
Check commit messages and PR body:
|
||||||
|
|
||||||
|
git log --format='%B' <base>..HEAD | grep -i -E 'co-authored-by: claude|generated with claude|claude code'
|
||||||
|
|
||||||
|
Expected: zero matches. Any match is Missing — main agent must remove them
|
||||||
|
before opening the PR.
|
||||||
|
|
||||||
|
### 4. Issue economy
|
||||||
|
|
||||||
|
Read the PR body and any new `TODO`/`FIXME` comments in the diff. Red flag:
|
||||||
|
filing a follow-up issue for something that is either fixable in this PR
|
||||||
|
(≤30 min, ≤1 file) or moot on current `main`. If found, report as Warning
|
||||||
|
with a recommendation: fix now, close as moot, or leave a `TODO` in the
|
||||||
|
touching diff.
|
||||||
|
|
||||||
|
### 5. Commit hygiene
|
||||||
|
|
||||||
|
`git log --oneline <base>..HEAD`. Red flags:
|
||||||
|
- Commit message includes AI attribution (already covered above).
|
||||||
|
- WIP / fixup / squash markers left in messages.
|
||||||
|
- Commits that should have been amended (e.g., "typo", "lint fix" of an
|
||||||
|
immediately preceding commit).
|
||||||
|
|
||||||
|
Report each as Warning, naming the SHA.
|
||||||
|
|
||||||
|
### 6. Release-cut implication
|
||||||
|
|
||||||
|
Invoke `Skill(agnes-release-process)`. Then: would this PR land the only
|
||||||
|
`[Unreleased]` content since the last tag? If yes, the release-cut commit
|
||||||
|
must be the last commit on this PR (version bump + CHANGELOG rename + new
|
||||||
|
empty `[Unreleased]`). Report as Done if present, Missing if not.
|
||||||
|
|
||||||
|
## Output format
|
||||||
|
|
||||||
|
Markdown, one section per check, three-line max per finding:
|
||||||
|
|
||||||
|
## CHANGELOG bullet — Done
|
||||||
|
Bullet under `## [Unreleased] > Added`: "Add foo to bar."
|
||||||
|
|
||||||
|
## Vendor-agnostic content — Warning
|
||||||
|
`docs/operator/runbook.md:42` mentions `<cloud-project-id>` (matched the `prj-…` pattern). Replace with `<project>` placeholder or move to the operator's private infra repo.
|
||||||
|
|
||||||
|
End with a one-line verdict: `OVERALL: ready / needs fixes`.
|
||||||
|
|
||||||
|
## Do not
|
||||||
|
|
||||||
|
- Do not edit files.
|
||||||
|
- Do not run tests.
|
||||||
|
- Do not call `gh pr merge` or push branches.
|
||||||
|
- Do not invent rules that are not in `CLAUDE.md` or the `agnes-release-process` skill.
|
||||||
72
.claude/skills/agnes-connectors.md
Normal file
72
.claude/skills/agnes-connectors.md
Normal file
|
|
@ -0,0 +1,72 @@
|
||||||
|
---
|
||||||
|
name: agnes-connectors
|
||||||
|
description: Rules for the extract.duckdb contract every data source must produce — the _meta table, the _remote_attach mechanism for remote-mode tables, parquet layout, and the pattern for adding a new connector. Use when adding a new data source or modifying an existing extractor in connectors/.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Agnes connectors — the extract.duckdb contract
|
||||||
|
|
||||||
|
Every data source produces the same output:
|
||||||
|
|
||||||
|
/data/extracts/{source_name}/
|
||||||
|
├── extract.duckdb ← _meta table + views
|
||||||
|
└── data/ ← parquet files (local sources only)
|
||||||
|
|
||||||
|
See `CLAUDE.md § Architecture: extract.duckdb Contract` and
|
||||||
|
`docs/architecture.md`.
|
||||||
|
|
||||||
|
## Required `_meta` table
|
||||||
|
|
||||||
|
Every `extract.duckdb` MUST contain a `_meta` table with these columns:
|
||||||
|
|
||||||
|
| column | type | meaning |
|
||||||
|
|---|---|---|
|
||||||
|
| `table_name` | VARCHAR | name used in views |
|
||||||
|
| `description` | VARCHAR | human-readable description |
|
||||||
|
| `rows` | BIGINT | row count at extraction time |
|
||||||
|
| `size_bytes` | BIGINT | parquet size for local mode, 0 for remote |
|
||||||
|
| `extracted_at` | TIMESTAMP | extraction time |
|
||||||
|
| `query_mode` | VARCHAR | one of `local`, `remote`, `materialized` |
|
||||||
|
|
||||||
|
If `_meta` is missing or malformed, `SyncOrchestrator.rebuild()` skips the
|
||||||
|
source with an error logged. Tests for new connectors MUST assert `_meta` is
|
||||||
|
well-formed.
|
||||||
|
|
||||||
|
## Four connector shapes
|
||||||
|
|
||||||
|
- **Batch pull** (Keboola, `query_mode='local'`) — DuckDB extension downloads
|
||||||
|
data to parquet, scheduled. Extractor in
|
||||||
|
`connectors/<name>/extractor.py`.
|
||||||
|
- **Remote attach** (BigQuery, `query_mode='remote'`) — DuckDB BQ extension,
|
||||||
|
no download. Queries hit the upstream at query time. Requires `_remote_attach`.
|
||||||
|
- **Materialized SQL** (`query_mode='materialized'`) — scheduler runs
|
||||||
|
admin-registered SQL through DuckDB and writes the result to a parquet under
|
||||||
|
`/data/extracts/<source>/data/`. Distributed via the same manifest +
|
||||||
|
`agnes pull` flow as `local`. BigQuery cost guardrail:
|
||||||
|
`data_source.bigquery.max_bytes_per_materialize` (default 10 GiB; `0` disables).
|
||||||
|
- **Real-time push** (Jira) — webhooks update parquets incrementally; the
|
||||||
|
webhook handler triggers `rebuild_source('jira')`.
|
||||||
|
|
||||||
|
## `_remote_attach` table (remote mode only)
|
||||||
|
|
||||||
|
For each remote-mode table in `_meta`, the extractor writes a row in
|
||||||
|
`_remote_attach` with `alias`, `extension`, `url`, `token_env`. See the
|
||||||
|
`agnes-orchestrator` skill for how the orchestrator consumes it.
|
||||||
|
|
||||||
|
## Adding a new connector — checklist
|
||||||
|
|
||||||
|
1. Create `connectors/<name>/extractor.py` that emits `extract.duckdb` (+
|
||||||
|
`data/*.parquet` if local) into `/data/extracts/<name>/`.
|
||||||
|
2. Populate `_meta` with one row per table.
|
||||||
|
3. If any table is `query_mode='remote'`, populate `_remote_attach`.
|
||||||
|
4. Register the connector type in the catalog (search for existing
|
||||||
|
`source_type` values to follow the pattern).
|
||||||
|
5. Add a fixture-based test that runs the extractor against a fixture
|
||||||
|
upstream and asserts `_meta` is complete.
|
||||||
|
6. CHANGELOG bullet under `Added` per `agnes-release-process`.
|
||||||
|
|
||||||
|
## Stable infrastructure — do NOT modify
|
||||||
|
|
||||||
|
`connectors/jira/file_lock.py`. (`connectors/jira/transform.py` was
|
||||||
|
previously off-limits but as of 0.54.19 is no longer; it remains
|
||||||
|
sensitive — touch only with end-to-end understanding of the
|
||||||
|
JSON-overlay / parquet-rewrite pipeline.)
|
||||||
75
.claude/skills/agnes-orchestrator.md
Normal file
75
.claude/skills/agnes-orchestrator.md
Normal file
|
|
@ -0,0 +1,75 @@
|
||||||
|
---
|
||||||
|
name: agnes-orchestrator
|
||||||
|
description: Rules for the SyncOrchestrator, the extract.duckdb ATTACH flow, query_mode semantics (local / remote / materialized), and when to call rebuild() vs rebuild_source(). Use when editing src/orchestrator.py, src/db.py, or anything that produces extract.duckdb in connectors/.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Agnes orchestrator
|
||||||
|
|
||||||
|
Source of truth for orchestrator invariants. See `CLAUDE.md § Architecture`
|
||||||
|
and `docs/architecture.md` for the canonical description.
|
||||||
|
|
||||||
|
## ATTACH flow
|
||||||
|
|
||||||
|
`SyncOrchestrator.rebuild()` scans `/data/extracts/*/extract.duckdb`,
|
||||||
|
ATTACHes each into the master `analytics.duckdb`, creates views like
|
||||||
|
`<source>."<bucket>"."<table>"`, and updates `sync_state`.
|
||||||
|
|
||||||
|
Per-source rebuild is `rebuild_source(name)` — used after Jira webhooks where
|
||||||
|
only one source changed. Full `rebuild()` is the fallback when scope is
|
||||||
|
unclear.
|
||||||
|
|
||||||
|
## Thread safety
|
||||||
|
|
||||||
|
All write paths take `self._rebuild_lock` (a `threading.Lock`). New write
|
||||||
|
paths — anything that DETACHes / re-ATTACHes / updates `sync_state` — MUST
|
||||||
|
hold the lock. Read paths must not hold it.
|
||||||
|
|
||||||
|
## query_mode
|
||||||
|
|
||||||
|
Every table has a `query_mode` in its `_meta` row:
|
||||||
|
|
||||||
|
- `local` — batch-pulled to parquet, queried locally. Parquets live under
|
||||||
|
`/data/extracts/<source>/data/`. Synced via `agnes pull`.
|
||||||
|
- `remote` — queried against the upstream (e.g., BigQuery) at query time.
|
||||||
|
No parquet on disk. Requires a `_remote_attach` row in `extract.duckdb`.
|
||||||
|
- `materialized` — admin-registered SQL run by the scheduler. Result lands as
|
||||||
|
a parquet under `/data/extracts/<source>/data/`. Distributed like `local`.
|
||||||
|
|
||||||
|
## `_remote_attach` mechanism
|
||||||
|
|
||||||
|
For `query_mode='remote'` tables, the extractor writes a `_remote_attach`
|
||||||
|
table in `extract.duckdb` with columns:
|
||||||
|
|
||||||
|
| column | meaning |
|
||||||
|
|---|---|
|
||||||
|
| `alias` | name used in the ATTACH statement |
|
||||||
|
| `extension` | DuckDB extension to install + load |
|
||||||
|
| `url` | upstream connection URL |
|
||||||
|
| `token_env` | env var holding the auth token (`''` if extension-specific auth, e.g., BigQuery's GCE metadata server) |
|
||||||
|
|
||||||
|
At query time the orchestrator installs/loads the extension, resolves the
|
||||||
|
token, creates a session-scoped SECRET when required, and ATTACHes the
|
||||||
|
source so views like `kbc."bucket"."table"` resolve.
|
||||||
|
|
||||||
|
## Master DB locations
|
||||||
|
|
||||||
|
- System DB: `${DATA_DIR}/state/system.duckdb` (sync_state, table_registry, users, RBAC).
|
||||||
|
- Analytics DB: `${DATA_DIR}/analytics/server.duckdb` (master views).
|
||||||
|
|
||||||
|
## Schema migrations
|
||||||
|
|
||||||
|
`src/db.py` auto-migrates from `v1 → vN` on startup. Per-version notes live
|
||||||
|
in `CHANGELOG.md`. Adding a schema version means:
|
||||||
|
|
||||||
|
1. Bumping the version constant in `src/db.py`.
|
||||||
|
2. Adding the `vN-1 → vN` migration step.
|
||||||
|
3. Adding a CHANGELOG bullet that names the version.
|
||||||
|
4. Updating documentation that references the schema version (search for
|
||||||
|
"schema v" in `docs/` + `CLAUDE.md`).
|
||||||
|
|
||||||
|
## Files NOT to modify
|
||||||
|
|
||||||
|
`connectors/jira/file_lock.py` and `services/ws_gateway/` — stable
|
||||||
|
infrastructure. (`connectors/jira/transform.py` was previously off-limits
|
||||||
|
but as of 0.54.19 is no longer; it remains sensitive — touch only with
|
||||||
|
end-to-end understanding of the JSON-overlay / parquet-rewrite pipeline.)
|
||||||
64
.claude/skills/agnes-rbac.md
Normal file
64
.claude/skills/agnes-rbac.md
Normal file
|
|
@ -0,0 +1,64 @@
|
||||||
|
---
|
||||||
|
name: agnes-rbac
|
||||||
|
description: Rules for endpoint gating (require_admin vs require_resource_access), ResourceType registration, and the user_groups model. Use when adding or changing endpoints in app/api/, touching app/auth/, or introducing a new resource type.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Agnes access control
|
||||||
|
|
||||||
|
Two-layer model with no role hierarchy. See `CLAUDE.md § Access control` and
|
||||||
|
`docs/RBAC.md`.
|
||||||
|
|
||||||
|
## Tables
|
||||||
|
|
||||||
|
- `user_groups` — named groups. `Admin` (god-mode short-circuit on every
|
||||||
|
authorization check) and `Everyone` (auto-membership) are seeded as
|
||||||
|
`is_system=TRUE`.
|
||||||
|
- `user_group_members` — `(user_id, group_id, source)`. `source` segregates
|
||||||
|
writers so Google's nightly sync does not clobber admin-added members.
|
||||||
|
- `resource_grants` — `(group, resource_type, resource_id)` triples for any
|
||||||
|
entity-scoped grant.
|
||||||
|
|
||||||
|
## Gate decision
|
||||||
|
|
||||||
|
For every new endpoint, pick one:
|
||||||
|
|
||||||
|
- `Depends(require_admin)` — app-level mutations (anything that changes shared
|
||||||
|
state without a per-entity scope: registering tables, creating users,
|
||||||
|
managing groups, server config).
|
||||||
|
- `Depends(require_resource_access(ResourceType.X, "{path}"))` — entity-scoped
|
||||||
|
reads or mutations. The path expression extracts the `resource_id` from the
|
||||||
|
request.
|
||||||
|
|
||||||
|
Both imports live in `app.auth.access`.
|
||||||
|
|
||||||
|
## Adding a new ResourceType
|
||||||
|
|
||||||
|
1. Extend the `ResourceType` `StrEnum` in `app/resource_types.py` with the
|
||||||
|
new value.
|
||||||
|
2. Register a `ResourceTypeSpec` for it in the same file, including a
|
||||||
|
`list_blocks` projection delegate that returns the rows visible to a
|
||||||
|
given caller.
|
||||||
|
3. **No DB migration needed** — `resource_grants` is generic.
|
||||||
|
4. Gate the endpoints that consume the new type with
|
||||||
|
`require_resource_access(ResourceType.NEW, "{path}")`.
|
||||||
|
|
||||||
|
## Admin layer is the source of truth for auto-sync
|
||||||
|
|
||||||
|
For `agnes pull`: `query_mode IN ('local', 'materialized')` plus a
|
||||||
|
`resource_grants` row for one of the analyst's groups → table appears in
|
||||||
|
their manifest. There is no per-user sync config.
|
||||||
|
|
||||||
|
## Auth providers
|
||||||
|
|
||||||
|
Auth providers live in `app/auth/`:
|
||||||
|
|
||||||
|
- **Google OAuth** — sign-in via Google. Workspace group memberships are
|
||||||
|
pulled at sign-in (see `docs/auth-groups.md` for GCP setup checklist + the
|
||||||
|
`security` label gotcha).
|
||||||
|
- **Email magic link** — itsdangerous token.
|
||||||
|
- **Desktop JWT** — for the CLI / API.
|
||||||
|
|
||||||
|
## Admin UI and CLI
|
||||||
|
|
||||||
|
- Admin UI: `/admin/access`.
|
||||||
|
- CLI: `agnes admin group …` and `agnes admin grant …`.
|
||||||
86
.claude/skills/agnes-release-process.md
Normal file
86
.claude/skills/agnes-release-process.md
Normal file
|
|
@ -0,0 +1,86 @@
|
||||||
|
---
|
||||||
|
name: agnes-release-process
|
||||||
|
description: Rules for opening a PR, the CHANGELOG bullet, the release-cut commit, and the post-merge tag + GitHub Release. Use before opening a PR, before merge, when handling a release-cut, and when picking a version bump.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Agnes release process
|
||||||
|
|
||||||
|
Source of truth for the rules in `CLAUDE.md § Release process` and
|
||||||
|
`docs/RELEASING.md`. This skill is invoked by the main agent during planning,
|
||||||
|
by `agnes-reviewer-rules` during review, and by `agnes-releaser` during the
|
||||||
|
release-cut. When the rules below conflict with the master documents above,
|
||||||
|
the master documents win — update this skill.
|
||||||
|
|
||||||
|
## When this skill applies
|
||||||
|
|
||||||
|
- Opening a PR
|
||||||
|
- Reviewing a PR (release-cut implications)
|
||||||
|
- Cutting a release (version bump, CHANGELOG rename)
|
||||||
|
- Post-merge tagging + GitHub Release
|
||||||
|
|
||||||
|
## CHANGELOG discipline
|
||||||
|
|
||||||
|
Every PR that changes **user-visible behavior** MUST add a bullet under
|
||||||
|
`## [Unreleased]` in `CHANGELOG.md`, grouped under Added / Changed / Fixed /
|
||||||
|
Removed / Internal. Breaking changes are prefixed `**BREAKING**`.
|
||||||
|
|
||||||
|
Doc-only PRs (`docs/**`, README) typically do not need a bullet. Apply
|
||||||
|
judgment based on the diff — if the docs change describes new behavior that
|
||||||
|
should have shipped with a code change, the *code* PR carries the bullet.
|
||||||
|
|
||||||
|
The CHANGELOG entry is part of the PR that introduces the change — never a
|
||||||
|
follow-up PR.
|
||||||
|
|
||||||
|
## Release-cut belongs in the PR
|
||||||
|
|
||||||
|
If a PR lands the only `[Unreleased]` content since the last release, the
|
||||||
|
release-cut is the **last commit on that PR**:
|
||||||
|
|
||||||
|
1. Bump `pyproject.toml` (`version = "X.Y.Z"`).
|
||||||
|
2. Rename `## [Unreleased]` to `## [X.Y.Z] - YYYY-MM-DD`.
|
||||||
|
3. Add a new empty `## [Unreleased]` above it.
|
||||||
|
|
||||||
|
The release-cut is never a standalone follow-up PR.
|
||||||
|
|
||||||
|
## Version bump decision
|
||||||
|
|
||||||
|
- **Patch** (X.Y.Z+1): default for bug fixes, internal refactors, doc tweaks,
|
||||||
|
small features that do not change documented behavior.
|
||||||
|
- **Minor** (X.Y+1.0): new user-visible features, new APIs, schema migrations
|
||||||
|
that are backwards-compatible. **Ask the user before picking minor.**
|
||||||
|
- **Major** (X+1.0.0): breaking changes, removed APIs, incompatible schema
|
||||||
|
changes. Requires explicit user confirmation.
|
||||||
|
|
||||||
|
## Post-merge sequence
|
||||||
|
|
||||||
|
After the PR with the release-cut is merged to `main`:
|
||||||
|
|
||||||
|
1. `git tag vX.Y.Z <merge-sha>`
|
||||||
|
2. `git push origin vX.Y.Z`
|
||||||
|
3. `gh release create vX.Y.Z --title "vX.Y.Z" --notes "<CHANGELOG body for [X.Y.Z]>"`
|
||||||
|
|
||||||
|
Never tag or release before merge.
|
||||||
|
|
||||||
|
## Post-merge auto-rollback
|
||||||
|
|
||||||
|
On every `main` push, GitHub Actions `release.yml` builds the `:stable`
|
||||||
|
image and a `smoke-test` job pulls it and runs a docker-compose stack.
|
||||||
|
If the smoke test fails:
|
||||||
|
|
||||||
|
- `rollback-on-smoke-fail` calls `rollback.yml`, which re-points `:stable`
|
||||||
|
to the previous known-good build.
|
||||||
|
- A tracking issue labeled `bug` is opened with the failing image, the
|
||||||
|
commit SHA, the deprecated tag, and the rollback target.
|
||||||
|
|
||||||
|
Success signal after merge: `smoke-test` green AND `rollback-on-smoke-fail`
|
||||||
|
skipped. If rollback fires, the merge shipped a broken image to GHCR —
|
||||||
|
investigate the tracking issue before any further push.
|
||||||
|
|
||||||
|
Manual rollback, forced target, and weekly tag-pruning operator commands
|
||||||
|
live in `docs/RELEASING.md`.
|
||||||
|
|
||||||
|
## Tests before push
|
||||||
|
|
||||||
|
Run `.venv/bin/pytest tests/ --tb=short -n auto -q` before every push.
|
||||||
|
Failures in code you touched: fix before pushing. Failures unrelated:
|
||||||
|
confirm they reproduce on a clean branch, note in the PR body, do not block.
|
||||||
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -1,5 +1,7 @@
|
||||||
# Claude Code
|
# Claude Code
|
||||||
.claude/
|
.claude/*
|
||||||
|
!.claude/agents/
|
||||||
|
!.claude/skills/
|
||||||
CLAUDE.local.md
|
CLAUDE.local.md
|
||||||
|
|
||||||
# Local dev data (copied from server for testing)
|
# Local dev data (copied from server for testing)
|
||||||
|
|
|
||||||
17
CHANGELOG.md
17
CHANGELOG.md
|
|
@ -10,6 +10,23 @@ CalVer image tags (`stable-YYYY.MM.N`, `dev-YYYY.MM.N`) are produced for every C
|
||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
|
### Internal
|
||||||
|
- **Repo-committed Claude Code agents + skills under `.claude/`.**
|
||||||
|
Four knowledge skills (`agnes-orchestrator`, `agnes-rbac`,
|
||||||
|
`agnes-connectors`, `agnes-release-process`) auto-load into the main
|
||||||
|
agent's context when their description matches the work or are
|
||||||
|
invokable explicitly via `Skill(<name>)`. Four specialist subagents
|
||||||
|
(`agnes-reviewer-rules`, `agnes-reviewer-rbac`,
|
||||||
|
`agnes-reviewer-architecture`, `agnes-releaser`) wire into the
|
||||||
|
Agent tool — reviewers fire in parallel at the end of PR work;
|
||||||
|
the releaser handles pre-merge release-cut + post-merge tag /
|
||||||
|
GitHub Release. `.gitignore` un-ignores `.claude/agents/` and
|
||||||
|
`.claude/skills/` while keeping the rest of `.claude/` local-only.
|
||||||
|
Source of truth for the rules these encode remains `CLAUDE.md` +
|
||||||
|
`docs/RELEASING.md`. Design rationale +
|
||||||
|
implementation plan: `docs/superpowers/specs/2026-05-15-agnes-agents-design.md`
|
||||||
|
and `docs/superpowers/plans/2026-05-15-agnes-agents.md`.
|
||||||
|
|
||||||
## [0.54.20] — 2026-05-15
|
## [0.54.20] — 2026-05-15
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
|
||||||
|
|
@ -311,6 +311,15 @@ Full recipe, deploy workflows, manual rollback runbook, weekly tag-housekeeping,
|
||||||
- **Run the full test suite before every push** — `.venv/bin/pytest tests/ --tb=short -n auto -q` (this is what CI runs). Failures in code you touched: fix before pushing. Failures unrelated to your diff: confirm with `git stash` they reproduce on a clean branch, note them in the PR body, don't block on them.
|
- **Run the full test suite before every push** — `.venv/bin/pytest tests/ --tb=short -n auto -q` (this is what CI runs). Failures in code you touched: fix before pushing. Failures unrelated to your diff: confirm with `git stash` they reproduce on a clean branch, note them in the PR body, don't block on them.
|
||||||
- **Watch the post-merge `release.yml` run.** On `main` pushes a `smoke-test` job pulls the just-built `:stable` image and runs a docker-compose stack; if it fails, the `rollback-on-smoke-fail` job calls the reusable `rollback.yml` workflow which re-points `:stable` to the previous known-good build and opens a tracking issue labeled `bug`. Success signal after merge = `smoke-test` green + `rollback-on-smoke-fail` skipped. If the rollback fires, the merge shipped a broken image to GHCR — investigate the tracking issue before any further push (the issue body has the failing image, commit SHA, deprecated tag, and rollback target). Manual rollback / forced target / weekly tag-pruning operator commands are in [`docs/RELEASING.md`](docs/RELEASING.md).
|
- **Watch the post-merge `release.yml` run.** On `main` pushes a `smoke-test` job pulls the just-built `:stable` image and runs a docker-compose stack; if it fails, the `rollback-on-smoke-fail` job calls the reusable `rollback.yml` workflow which re-points `:stable` to the previous known-good build and opens a tracking issue labeled `bug`. Success signal after merge = `smoke-test` green + `rollback-on-smoke-fail` skipped. If the rollback fires, the merge shipped a broken image to GHCR — investigate the tracking issue before any further push (the issue body has the failing image, commit SHA, deprecated tag, and rollback target). Manual rollback / forced target / weekly tag-pruning operator commands are in [`docs/RELEASING.md`](docs/RELEASING.md).
|
||||||
|
|
||||||
|
## Specialized agents and skills
|
||||||
|
|
||||||
|
Two committed locations carry Agnes-specific Claude Code behavior:
|
||||||
|
|
||||||
|
- `.claude/skills/agnes-*.md` — knowledge skills (`agnes-orchestrator`, `agnes-rbac`, `agnes-connectors`, `agnes-release-process`). Loaded into the main agent's context when their description matches the work, or invoked explicitly via `Skill(<name>)`. Read these before editing the corresponding part of the codebase.
|
||||||
|
- `.claude/agents/agnes-*.md` — specialist subagents (`agnes-reviewer-rules`, `agnes-reviewer-rbac`, `agnes-reviewer-architecture`, `agnes-releaser`). Spawned via the Agent tool at the end of PR work (reviewers, in parallel) or explicitly before/after merge (releaser).
|
||||||
|
|
||||||
|
Design rationale: `docs/superpowers/specs/2026-05-15-agnes-agents-design.md`.
|
||||||
|
|
||||||
## Project conventions
|
## Project conventions
|
||||||
|
|
||||||
### Vendor-agnostic OSS — no customer-specific content
|
### Vendor-agnostic OSS — no customer-specific content
|
||||||
|
|
|
||||||
1101
docs/superpowers/plans/2026-05-15-agnes-agents.md
Normal file
1101
docs/superpowers/plans/2026-05-15-agnes-agents.md
Normal file
File diff suppressed because it is too large
Load diff
313
docs/superpowers/specs/2026-05-15-agnes-agents-design.md
Normal file
313
docs/superpowers/specs/2026-05-15-agnes-agents-design.md
Normal file
|
|
@ -0,0 +1,313 @@
|
||||||
|
# Agnes specialized agents — design
|
||||||
|
|
||||||
|
**Status:** approved (brainstorm), not yet implemented
|
||||||
|
**Date:** 2026-05-15
|
||||||
|
**Author:** zsrotyr
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
Working on Agnes through Claude Code, three classes of friction recur:
|
||||||
|
|
||||||
|
1. **Mental model drift.** Claude (and humans) forget how Agnes hangs together — the `extract.duckdb` contract, RBAC layering (`require_admin` vs `require_resource_access`), `query_mode` semantics (local / remote / materialized), the orchestrator's `rebuild()` flow. Edits to one part silently break invariants of another, caught only at code review.
|
||||||
|
2. **Convention enforcement at review.** Several CLAUDE.md rules (CHANGELOG bullet, vendor-agnostic OSS, no AI attribution, issue economy, RBAC gates on new endpoints) are easy to forget. Manual review catches most but not all.
|
||||||
|
3. **Release-cut workflow.** The non-negotiable rules from `docs/RELEASING.md` and `CLAUDE.md § Release process` — release-cut belongs in the PR that earned it, last commit on the PR, post-merge tag + GitHub Release — evolve and are repetitive enough that doing them by hand each time is error-prone.
|
||||||
|
|
||||||
|
The brainstorm explored variants A ("one big architect"), B ("three specialist subagents covering all three concerns"), and C ("knowledge as skills, enforcement/workflow as subagents"). C was selected because the mental model is *not delegatable* — when the main agent writes code it needs the context in its own window, not in a subagent's.
|
||||||
|
|
||||||
|
## Approach
|
||||||
|
|
||||||
|
Four layers, each with a distinct mechanism and lifecycle:
|
||||||
|
|
||||||
|
```
|
||||||
|
Layer 1: KNOWLEDGE SKILLS (.claude/skills/agnes-*.md)
|
||||||
|
- Auto-trigger by description + explicit invocation by main agent
|
||||||
|
- Load into MAIN agent's context
|
||||||
|
- Purpose: main agent knows how Agnes works while writing code
|
||||||
|
|
||||||
|
Layer 2: REVIEWER SUBAGENTS (.claude/agents/agnes-reviewer-*.md)
|
||||||
|
- Spawned via Agent tool at end of PR work
|
||||||
|
- Own context window; return a punch list
|
||||||
|
- Three specialists, fired in parallel by main agent
|
||||||
|
|
||||||
|
Layer 3: RELEASER SUBAGENT (.claude/agents/agnes-releaser.md)
|
||||||
|
- Spawned via Agent tool pre-merge (phase 1) and post-merge (phase 2)
|
||||||
|
- Own context window; produces release-cut commit + tag + GitHub Release
|
||||||
|
|
||||||
|
Layer 4: PERSONAL (~/.claude/agents/<customer>-deploy.md)
|
||||||
|
- Outside this repo (operator's home dir, never committed)
|
||||||
|
- Customer-specific context (customer VMs, gcloud / cloud accounts,
|
||||||
|
private infra repos, deploy targets)
|
||||||
|
```
|
||||||
|
|
||||||
|
Skills (Layer 1) share context with the main agent — when Claude edits
|
||||||
|
`src/orchestrator.py`, the orchestrator skill is loaded so rules stay "in head"
|
||||||
|
across delegation. Subagents (Layers 2 and 3) isolate context — review/release
|
||||||
|
read many files but only a punch list returns to the main conversation.
|
||||||
|
|
||||||
|
Cross-layer sharing happens through skills. The `agnes-release-process` skill is
|
||||||
|
read by the main agent (planning), by `agnes-reviewer-rules` (checking), and by
|
||||||
|
`agnes-releaser` (executing). Single source of truth.
|
||||||
|
|
||||||
|
## Components
|
||||||
|
|
||||||
|
### Layer 1 — Knowledge skills (`.claude/skills/`)
|
||||||
|
|
||||||
|
Four skills. Each file is ~80–120 lines. Files are kept focused; if one grows
|
||||||
|
past ~150 lines that is a signal to split. Skills do not duplicate `CLAUDE.md`
|
||||||
|
content verbatim — they reference (`see CLAUDE.md § Access control`) so master
|
||||||
|
rules have one location.
|
||||||
|
|
||||||
|
#### `agnes-orchestrator`
|
||||||
|
|
||||||
|
- **Description (triggers auto-spawn):** Use when editing `src/orchestrator.py`,
|
||||||
|
`src/db.py`, or anything that produces `extract.duckdb` in `connectors/*/`.
|
||||||
|
Rules for ATTACH flow, `query_mode` semantics, and when `rebuild()` is
|
||||||
|
required.
|
||||||
|
- **Body:** master view lifecycle in `analytics.duckdb`; thread-safety via
|
||||||
|
`_rebuild_lock`; `rebuild_source(name)` vs full `rebuild()` decision;
|
||||||
|
`_remote_attach` reattach flow at query time (extension install + token
|
||||||
|
resolution via `token_env` or extension-specific auth path).
|
||||||
|
|
||||||
|
#### `agnes-rbac`
|
||||||
|
|
||||||
|
- **Description:** Use when adding or changing an endpoint in `app/api/`,
|
||||||
|
touching `app/auth/`, or introducing a new resource type. Enforces gate
|
||||||
|
pattern (`require_admin` vs `require_resource_access`) and `ResourceType`
|
||||||
|
registration.
|
||||||
|
- **Body:** decision tree for picking a gate (app-level mutation vs
|
||||||
|
entity-scoped); how to add a new `ResourceType` (StrEnum value +
|
||||||
|
`ResourceTypeSpec` with `list_blocks` delegate in `app/resource_types.py`, no
|
||||||
|
DB migration); when a grant is needed even for reads; god-mode short-circuit
|
||||||
|
for the `Admin` group.
|
||||||
|
|
||||||
|
#### `agnes-connectors`
|
||||||
|
|
||||||
|
- **Description:** Use when adding a new data source or modifying an existing
|
||||||
|
extractor in `connectors/`. Enforces the `extract.duckdb` contract — `_meta`
|
||||||
|
table, `query_mode` column, parquet layout.
|
||||||
|
- **Body:** required `_meta` columns (`table_name`, `description`, `rows`,
|
||||||
|
`size_bytes`, `extracted_at`, `query_mode`); when a connector is batch-pull
|
||||||
|
vs remote-attach vs real-time push; how to expose `_remote_attach` for remote
|
||||||
|
mode; where the extractor writes (`/data/extracts/{source}/`).
|
||||||
|
|
||||||
|
#### `agnes-release-process`
|
||||||
|
|
||||||
|
- **Description:** Use before opening a PR, before merge, or when handling a
|
||||||
|
release-cut. Rules for CHANGELOG bullet, when the release-cut commit belongs
|
||||||
|
in the PR, version bump decision (patch is default; ask before minor).
|
||||||
|
- **Body:** CHANGELOG discipline (Added / Changed / Fixed / Removed / Internal
|
||||||
|
grouping, `**BREAKING**` prefix); release-cut decision tree (when it is the
|
||||||
|
last commit on the PR); post-merge sequence (tag `vX.Y.Z` on merge commit +
|
||||||
|
`gh release create`); patch / minor / major guidance.
|
||||||
|
|
||||||
|
### Layer 2 — Reviewer subagents (`.claude/agents/`)
|
||||||
|
|
||||||
|
Each subagent has a standard frontmatter (`name`, `description`, `tools`,
|
||||||
|
`model`). All are read-only (no `Edit` / `Write`) — they return punch lists, not
|
||||||
|
code changes.
|
||||||
|
|
||||||
|
#### `agnes-reviewer-rules`
|
||||||
|
|
||||||
|
- **When fired:** every PR, at end of work, before opening the PR.
|
||||||
|
- **Tools:** `Read`, `Bash` (restricted to `git diff`, `git log`, `grep`).
|
||||||
|
- **Model:** Haiku — fast, mostly text work.
|
||||||
|
- **Input from main agent:** PR branch name (or current HEAD), optionally PR
|
||||||
|
draft body.
|
||||||
|
- **Checks:**
|
||||||
|
- CHANGELOG.md has a new bullet under `[Unreleased]` *iff* the PR changes
|
||||||
|
user-visible behavior. Smart, not blind — doc-only PRs typically do not
|
||||||
|
need a bullet; judgment applied based on the diff.
|
||||||
|
- No customer-specific tokens in the diff or PR body (deployment names,
|
||||||
|
internal hostnames, cloud project IDs, internal SA emails). The
|
||||||
|
`agnes-reviewer-rules` subagent loads the operator's token list from
|
||||||
|
`CLAUDE.local.md` (`vendor_tokens:` entry) rather than inlining
|
||||||
|
customer names in the public OSS repo.
|
||||||
|
- Commits do not contain `Co-Authored-By: Claude` or any AI attribution; PR
|
||||||
|
body is the same.
|
||||||
|
- Issue-economy red flags — filing follow-up issues rather than fix-it-now or
|
||||||
|
close-it-as-moot.
|
||||||
|
- Commit messages are clean and concise per project convention.
|
||||||
|
- Consults `agnes-release-process` skill for the release-cut implication of
|
||||||
|
this PR.
|
||||||
|
- **Output:** Done / Missing / Warning punch list.
|
||||||
|
|
||||||
|
#### `agnes-reviewer-rbac`
|
||||||
|
|
||||||
|
- **When fired:** when the diff touches `app/api/`, `app/auth/`, or
|
||||||
|
`app/resource_types.py`.
|
||||||
|
- **Tools:** `Read`, `Grep`, `Bash` (read-only).
|
||||||
|
- **Model:** Sonnet — needs to understand the auth flow.
|
||||||
|
- **Checks:**
|
||||||
|
- New `@router.get/post/...` handlers have `Depends(require_admin)` or
|
||||||
|
`Depends(require_resource_access(ResourceType.X, "..."))`.
|
||||||
|
- New `ResourceType` values have a `ResourceTypeSpec` registration in
|
||||||
|
`app/resource_types.py`.
|
||||||
|
- Consults `agnes-rbac` skill for the gate decision rules.
|
||||||
|
- **Output:** per-endpoint flag — gated correctly / missing gate / ambiguous.
|
||||||
|
|
||||||
|
#### `agnes-reviewer-architecture`
|
||||||
|
|
||||||
|
- **When fired:** when the diff touches `src/orchestrator.py`, `src/db.py`,
|
||||||
|
`connectors/*/extractor.py`, or adds a schema migration.
|
||||||
|
- **Tools:** `Read`, `Grep`, `Bash` (read-only).
|
||||||
|
- **Model:** Sonnet.
|
||||||
|
- **Checks:**
|
||||||
|
- Extractor changes preserve `_meta` table contract.
|
||||||
|
- Remote-attach changes preserve `_remote_attach` columns (`alias`,
|
||||||
|
`extension`, `url`, `token_env`).
|
||||||
|
- Schema bumps in `src/db.py` include the `vN-1 → vN` migration step, a
|
||||||
|
CHANGELOG note, and documentation references that reflect the new version.
|
||||||
|
- Changes to `rebuild()` / `rebuild_source()` hold `_rebuild_lock` on all
|
||||||
|
write paths.
|
||||||
|
- Consults `agnes-orchestrator` and `agnes-connectors` skills.
|
||||||
|
- **Output:** per-invariant punch list — holds / broken / unclear.
|
||||||
|
|
||||||
|
### Layer 3 — Releaser subagent (`.claude/agents/agnes-releaser.md`)
|
||||||
|
|
||||||
|
- **Tools:** `Read`, `Edit`, `Bash` (including `gh`, `git`).
|
||||||
|
- **Model:** Sonnet.
|
||||||
|
|
||||||
|
Two phases, each invoked explicitly by the user (never auto-fired).
|
||||||
|
|
||||||
|
**Phase 1 — pre-merge.** User says "ready to merge".
|
||||||
|
|
||||||
|
1. Consults `agnes-release-process` skill.
|
||||||
|
2. Runs `git log` since the last tag and inspects scope.
|
||||||
|
3. Decision tree: patch (default) vs minor (asks user) vs major (requires
|
||||||
|
explicit confirmation).
|
||||||
|
4. If this PR lands the only `[Unreleased]` content since the last release,
|
||||||
|
prepares the last commit on the PR: bump `pyproject.toml`, rename
|
||||||
|
`[Unreleased]` to `[X.Y.Z] - YYYY-MM-DD`, add a new empty `[Unreleased]`.
|
||||||
|
5. Pushes the prepared commit. **Does not merge** — the user merges via
|
||||||
|
`gh pr merge` themselves.
|
||||||
|
|
||||||
|
**Phase 2 — post-merge.** User says "tag it".
|
||||||
|
|
||||||
|
1. Verifies the merge commit contains the release-cut diff.
|
||||||
|
2. `git tag vX.Y.Z <merge-sha>` and `git push origin vX.Y.Z`.
|
||||||
|
3. `gh release create vX.Y.Z` with body extracted from the `[X.Y.Z]` section of
|
||||||
|
CHANGELOG.
|
||||||
|
4. Returns the GitHub Release URL.
|
||||||
|
|
||||||
|
**Never does:** merges the PR (high-blast-radius); force-pushes; amends
|
||||||
|
published commits.
|
||||||
|
|
||||||
|
### Layer 4 — Personal (`~/.claude/agents/<customer>-deploy.md`)
|
||||||
|
|
||||||
|
Outside this repo. Customer-specific content lives here so the OSS repo stays
|
||||||
|
vendor-agnostic per `CLAUDE.md § Vendor-agnostic OSS`. Operators write their
|
||||||
|
own Layer-4 agent against their deployment.
|
||||||
|
|
||||||
|
- **Tools:** `Read`, `Bash` (including the operator's cloud CLI — `gcloud`,
|
||||||
|
`aws`, etc. — plus `gh`, `git push`).
|
||||||
|
- **Model:** Sonnet.
|
||||||
|
- **Knows** (each item is customer-specific; concrete tokens stay in the
|
||||||
|
operator's home dir / `CLAUDE.local.md`, never committed):
|
||||||
|
- VM ↔ project ↔ zone ↔ cloud-account mapping (one row per deployment).
|
||||||
|
- Per-environment deploy ritual (e.g., force-push to `<branch>` to deploy
|
||||||
|
to `<host>`).
|
||||||
|
- Cross-references to private infra repos and pinned module tag.
|
||||||
|
- Default cloud account selection rules; explicit `--account=` overrides.
|
||||||
|
- Any pre-existing legacy systems still running alongside the deployment.
|
||||||
|
- **Does not** touch the OSS repo, push to public branches, or participate in
|
||||||
|
PR review.
|
||||||
|
|
||||||
|
## End-to-end PR flow
|
||||||
|
|
||||||
|
```
|
||||||
|
1. User asks for feature X.
|
||||||
|
|
||||||
|
2. Main agent creates a worktree (per CLAUDE.local.md), invokes relevant
|
||||||
|
knowledge skills based on what the feature touches:
|
||||||
|
- src/orchestrator.py → Skill(agnes-orchestrator)
|
||||||
|
- app/api/ + app/auth/ → Skill(agnes-rbac)
|
||||||
|
- connectors/ → Skill(agnes-connectors)
|
||||||
|
|
||||||
|
3. Implementation + tests + CHANGELOG bullet (the agnes-release-process skill
|
||||||
|
reminded the main agent it is needed).
|
||||||
|
|
||||||
|
4. Before opening the PR, main agent fires reviewers in parallel — one message,
|
||||||
|
multiple Agent tool calls:
|
||||||
|
- Agent(agnes-reviewer-rules) — always
|
||||||
|
- Agent(agnes-reviewer-rbac) — only if diff touched app/api or app/auth
|
||||||
|
- Agent(agnes-reviewer-architecture) — only if diff touched src/ or connectors/
|
||||||
|
|
||||||
|
5. Main agent aggregates the punch lists, fixes findings, opens the PR.
|
||||||
|
|
||||||
|
6. User says "ready to merge" → Agent(agnes-releaser, phase 1) prepares the
|
||||||
|
release-cut decision and the last commit on the PR.
|
||||||
|
|
||||||
|
7. User confirms the version → releaser pushes the prepared commit. User
|
||||||
|
merges via `gh pr merge` manually.
|
||||||
|
|
||||||
|
8. User says "tag it" → Agent(agnes-releaser, phase 2) creates the tag and the
|
||||||
|
GitHub Release.
|
||||||
|
```
|
||||||
|
|
||||||
|
A separate flow handles personal dev-VM deploys (outside the OSS PR cycle):
|
||||||
|
|
||||||
|
```
|
||||||
|
User says "push to my VM" → main agent invokes Agent(<customer>-deploy) →
|
||||||
|
that personal-layer agent knows what "push to my VM" means in the operator's
|
||||||
|
environment (typically a force-push to a deploy-target branch), confirms with
|
||||||
|
the user (force-push is destructive), pushes.
|
||||||
|
```
|
||||||
|
|
||||||
|
## What lands in the repo
|
||||||
|
|
||||||
|
```
|
||||||
|
.claude/
|
||||||
|
├── agents/
|
||||||
|
│ ├── agnes-reviewer-rules.md
|
||||||
|
│ ├── agnes-reviewer-rbac.md
|
||||||
|
│ ├── agnes-reviewer-architecture.md
|
||||||
|
│ └── agnes-releaser.md
|
||||||
|
└── skills/
|
||||||
|
├── agnes-orchestrator.md
|
||||||
|
├── agnes-rbac.md
|
||||||
|
├── agnes-connectors.md
|
||||||
|
└── agnes-release-process.md
|
||||||
|
|
||||||
|
docs/superpowers/specs/
|
||||||
|
└── 2026-05-15-agnes-agents-design.md (this document)
|
||||||
|
```
|
||||||
|
|
||||||
|
`CLAUDE.md` gets a short paragraph under "Project conventions" pointing at
|
||||||
|
`.claude/agents/` and `.claude/skills/`, noting that subagents and skills exist
|
||||||
|
and how to invoke them.
|
||||||
|
|
||||||
|
The personal layer (`~/.claude/agents/<customer>-deploy.md`) is written
|
||||||
|
separately by each operator and is not part of the in-repo change set.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- **Not a Claude Code "team".** The architecture, review, and release agents
|
||||||
|
work sequentially and do not message each other; a team would add the
|
||||||
|
experimental flag, more tokens, file-conflict risk, and no functional gain.
|
||||||
|
If `agnes-reviewer-*` ever needs to coordinate across four+ parallel reviews,
|
||||||
|
re-evaluate.
|
||||||
|
- **Not a pre-commit hook replacement.** Mechanical checks (CHANGELOG bullet
|
||||||
|
present, AI-attribution scan) could be a `pre-commit` hook in addition; the
|
||||||
|
reviewer agent provides judgment-level checks the hook cannot.
|
||||||
|
- **No auto-merge.** `agnes-releaser` never runs `gh pr merge`. Merge is a
|
||||||
|
visible, user-controlled action.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- **Auto-trigger reliability for knowledge skills.** Claude Code skill
|
||||||
|
auto-spawn is description-driven and not always reliable in large skill
|
||||||
|
catalogs. Mitigation: a one-line pointer at the top of `CLAUDE.md`
|
||||||
|
("when touching X, invoke skill Y") and explicit invocation in the main
|
||||||
|
agent's planning step. If reliability is still low after a few weeks, fall
|
||||||
|
back to a `SessionStart` hook that lists Agnes-specific skills as a reminder.
|
||||||
|
- **Schema migration as a separate skill?** `agnes-orchestrator` covers
|
||||||
|
`src/db.py` migration patterns. If migration gotchas grow (versioning beyond
|
||||||
|
`vN`, multi-step migrations, data backfills), split into a fifth skill
|
||||||
|
`agnes-schema-migration`.
|
||||||
|
|
||||||
|
## Implementation plan
|
||||||
|
|
||||||
|
The follow-up step is to invoke the `writing-plans` skill to turn this spec
|
||||||
|
into a sequenced implementation plan covering: skills first (lowest risk,
|
||||||
|
testable in isolation), then reviewers, then the releaser, then a CLAUDE.md
|
||||||
|
pointer. Personal layer lands separately, outside the repo.
|
||||||
Loading…
Reference in a new issue