a48524509a
CLAUDE.md rewritten (708 -> ~320 lines): four overlapping release sections collapsed to one, stale v1->v35 schema history dropped (it lives in CHANGELOG), marketplace endpoint internals and verbose process sections moved out or tightened. New focused docs: - docs/RELEASING.md - release process, deploy workflows, CI quirks (RELEASE_TEMPLATE.md folded in as an appendix) - docs/marketplace.md - marketplace ingestion + re-serving internals - docs/README.md - documentation index by audience, linked from README.md and CLAUDE.md Archived under docs/archive/: docs/superpowers/ (52 historical planning artifacts), HACKATHON.md, pd-ps-comments.md, security-audit-2026-04.md, future/NOTIFICATIONS.md. Removed the docs/auto-install.md stub. Fixed dangling links in connectors/jira/README.md and dev_docs/README.md, repointed code/doc references to archived paths.
101 lines
5.5 KiB
Markdown
101 lines
5.5 KiB
Markdown
# Marketplace internals
|
|
|
|
How Agnes ingests admin-registered Claude Code marketplaces and re-serves a
|
|
single aggregated, RBAC-filtered marketplace back to user instances. CLAUDE.md
|
|
carries a one-paragraph summary; this doc is the reference.
|
|
|
|
For the *content-authoring* side (cover photos, demo videos, doc links via
|
|
`marketplace-metadata.json`), see [`curated-marketplace-format.md`](curated-marketplace-format.md).
|
|
|
|
## Marketplace repositories (ingestion)
|
|
|
|
Admin-managed git repos cloned nightly to `${DATA_DIR}/marketplaces/<slug>/` so
|
|
FastAPI can read their contents from disk.
|
|
|
|
- Register via `/admin/marketplaces` (admin UI) or `POST /api/marketplaces`.
|
|
- Scheduler calls `POST /api/marketplaces/sync-all` (admin-only, authed via
|
|
`SCHEDULER_API_TOKEN`) at `daily 03:00` UTC. Routing through HTTP keeps the
|
|
app the sole writer to `system.duckdb` — the previous in-process call from the
|
|
scheduler container raced the app's long-lived DB handle and 500-ed on
|
|
`Could not set lock on file`.
|
|
- Manual re-sync from the UI ("Sync now") hits `POST /api/marketplaces/{id}/sync`.
|
|
- PATs for private repos persist to `${DATA_DIR}/state/.env_overlay` (chmod 600)
|
|
as `AGNES_MARKETPLACE_<SLUG>_TOKEN`. DuckDB stores only the env-var name
|
|
(`token_env`), never the secret.
|
|
- Registry lives in DuckDB table `marketplace_registry`.
|
|
- After each successful sync, `src/marketplace.py` parses
|
|
`.claude-plugin/marketplace.json` from the cloned repo and caches the plugin
|
|
list in `marketplace_plugins` (keyed by `(marketplace_id, plugin_name)`).
|
|
- `src/marketplace.py` handles clone/fetch/reset with token redaction in any
|
|
surfaced error message.
|
|
|
|
## Claude Code marketplace endpoint (re-serving)
|
|
|
|
Agnes serves a single aggregated Claude Code marketplace over two channels, both
|
|
gated by PAT auth and filtered per caller:
|
|
|
|
- `GET /marketplace.zip` — deterministic ZIP download with `ETag` /
|
|
`If-None-Match` (304 when content unchanged). Consumed by a client-side
|
|
SessionStart hook.
|
|
- `GET /marketplace.git/*` — git smart-HTTP (dulwich via a2wsgi). Registered in
|
|
Claude Code once, then Claude Code owns the clone/fetch cycle.
|
|
|
|
**Auth:** ZIP uses `Authorization: Bearer <PAT>`. Git uses HTTP Basic where the
|
|
password field carries the PAT (`https://x:<PAT>@host/marketplace.git/`) — git
|
|
CLI does not speak Bearer.
|
|
|
|
**Content:** filtered via `src.marketplace_filter.resolve_allowed_plugins` which
|
|
joins `resource_grants ↔ marketplace_plugins` (matching
|
|
`mp.marketplace_id || '/' || mp.name = rg.resource_id`) scoped to the caller's
|
|
`user_group_members`. Admin is treated as a regular group here — no god-mode
|
|
shortcut for the marketplace feed, so admins curate their own view by granting
|
|
plugins to the Admin group (or any group they belong to).
|
|
|
|
On-disk layout in the served ZIP / git tree uses a slug-prefixed directory
|
|
(`plugins/<slug>-<plugin>/`) so two marketplaces shipping a same-named plugin
|
|
don't overwrite each other's files. The synth marketplace.json's `name` field,
|
|
however, is the plugin's authoritative name from its own
|
|
`.claude-plugin/plugin.json` (with a fallback to the upstream marketplace.json
|
|
`name`) — Claude Code's `/plugin` UI resolves a loaded plugin back to its
|
|
catalog entry by `plugin.json` name, so the catalog entry's `name` must match.
|
|
Same-named plugins from two upstream marketplaces therefore collide in the
|
|
catalog by design; admin RBAC (which grants survive the filter) decides which
|
|
one wins, identical to how Claude Code behaves when a user adds two upstream
|
|
marketplaces with overlapping plugin names directly. `/marketplace/info` exposes
|
|
both `name` and `prefixed_name` so operators can disambiguate.
|
|
|
|
**Cache:** content-addressed bare repos at `${DATA_DIR}/marketplaces/git-cache/`
|
|
keyed by sha256(filtered content). Two users with the same RBAC view share one
|
|
repo; content change → new repo next to the old one. No TTL / prune yet.
|
|
|
|
## User registration inside Claude Code
|
|
|
|
```
|
|
# ZIP channel (typically via a SessionStart hook that unpacks into ./marketplace/)
|
|
curl -H "Authorization: Bearer $AGNES_PAT" https://agnes.example.com/marketplace.zip
|
|
|
|
# Git channel — one-time registration. Two paths; pick the first that works.
|
|
|
|
# (a) Direct registration — preferred when it works.
|
|
/plugin marketplace add https://x:$AGNES_PAT@agnes.example.com/marketplace.git/
|
|
|
|
# (b) Two-step fallback — required when (a) fails. Bun-compiled `claude` on
|
|
# macOS / Windows ignores the OS trust store and CA env vars on the
|
|
# marketplace HTTPS path, so direct add can fail with TLS errors against
|
|
# a private-CA Agnes instance even when system tools work fine. System
|
|
# `git` honors GIT_SSL_CAINFO + the OS trust store, so cloning manually
|
|
# and pointing Claude Code at the local clone sidesteps the Bun TLS path
|
|
# entirely.
|
|
git clone https://x:$AGNES_PAT@agnes.example.com/marketplace.git/ ~/agnes-marketplace
|
|
claude plugin marketplace add ~/agnes-marketplace
|
|
# Optional hardening: strip the PAT from the cloned repo's origin so it
|
|
# doesn't sit in plaintext at ~/agnes-marketplace/.git/config — re-clone via
|
|
# the dashboard's setup flow when the PAT rotates.
|
|
git -C ~/agnes-marketplace remote set-url origin https://agnes.example.com/marketplace.git/
|
|
```
|
|
|
|
The dashboard-served setup payload (see `app/web/setup_instructions.py`) already
|
|
branches between (a) and (b) automatically based on platform when a private CA
|
|
is in play. The block above is the manual equivalent for users registering
|
|
outside that flow (e.g. operators bringing up a new instance, or analysts whose
|
|
first attempt failed and need to retry by hand).
|