agnes-the-ai-analyst/docs/marketplace.md

Marketplace internals

How Agnes ingests admin-registered Claude Code marketplaces and re-serves a single aggregated, RBAC-filtered marketplace back to user instances. CLAUDE.md carries a one-paragraph summary; this doc is the reference.

For the content-authoring side (cover photos, demo videos, doc links via marketplace-metadata.json), see curated-marketplace-format.md.

Marketplace repositories (ingestion)

Admin-managed git repos cloned nightly to ${DATA_DIR}/marketplaces/<slug>/ so FastAPI can read their contents from disk.

• Register via /admin/marketplaces (admin UI) or POST /api/marketplaces.
• Scheduler calls POST /api/marketplaces/sync-all (admin-only, authed via SCHEDULER_API_TOKEN) at daily 03:00 UTC. Routing through HTTP keeps the app the sole writer to system.duckdb — the previous in-process call from the scheduler container raced the app's long-lived DB handle and 500-ed on Could not set lock on file.
• Manual re-sync from the UI ("Sync now") hits POST /api/marketplaces/{id}/sync.
• PATs for private repos persist to ${DATA_DIR}/state/.env_overlay (chmod 600) as AGNES_MARKETPLACE_<SLUG>_TOKEN. DuckDB stores only the env-var name (token_env), never the secret.
• Registry lives in DuckDB table marketplace_registry.
• After each successful sync, src/marketplace.py parses .claude-plugin/marketplace.json from the cloned repo and caches the plugin list in marketplace_plugins (keyed by (marketplace_id, plugin_name)).
• src/marketplace.py handles clone/fetch/reset with token redaction in any surfaced error message.

Claude Code marketplace endpoint (re-serving)

Agnes serves a single aggregated Claude Code marketplace over two channels, both gated by PAT auth and filtered per caller:

• GET /marketplace.zip — deterministic ZIP download with ETag / If-None-Match (304 when content unchanged). Consumed by a client-side SessionStart hook.
• GET /marketplace.git/* — git smart-HTTP (dulwich via a2wsgi). Registered in Claude Code once, then Claude Code owns the clone/fetch cycle.

Auth: ZIP uses Authorization: Bearer <PAT>. Git uses HTTP Basic where the password field carries the PAT (https://x:<PAT>@host/marketplace.git/) — git CLI does not speak Bearer.

Content: filtered via src.marketplace_filter.resolve_allowed_plugins which joins resource_grants ↔ marketplace_plugins (matching mp.marketplace_id || '/' || mp.name = rg.resource_id) scoped to the caller's user_group_members. Admin is treated as a regular group here — no god-mode shortcut for the marketplace feed, so admins curate their own view by granting plugins to the Admin group (or any group they belong to).

On-disk layout in the served ZIP / git tree uses a slug-prefixed directory (plugins/<slug>-<plugin>/) so two marketplaces shipping a same-named plugin don't overwrite each other's files. The synth marketplace.json's name field, however, is the plugin's authoritative name from its own .claude-plugin/plugin.json (with a fallback to the upstream marketplace.json name) — Claude Code's /plugin UI resolves a loaded plugin back to its catalog entry by plugin.json name, so the catalog entry's name must match. Same-named plugins from two upstream marketplaces therefore collide in the catalog by design; admin RBAC (which grants survive the filter) decides which one wins, identical to how Claude Code behaves when a user adds two upstream marketplaces with overlapping plugin names directly. /marketplace/info exposes both name and prefixed_name so operators can disambiguate.

Cache: content-addressed bare repos at ${DATA_DIR}/marketplaces/git-cache/ keyed by sha256(filtered content). Two users with the same RBAC view share one repo; content change → new repo next to the old one. No TTL / prune yet.

User registration inside Claude Code

# ZIP channel (typically via a SessionStart hook that unpacks into ./marketplace/)
curl -H "Authorization: Bearer $AGNES_PAT" https://agnes.example.com/marketplace.zip

# Git channel — one-time registration. Two paths; pick the first that works.

# (a) Direct registration — preferred when it works.
/plugin marketplace add https://x:$AGNES_PAT@agnes.example.com/marketplace.git/

# (b) Two-step fallback — required when (a) fails. Bun-compiled `claude` on
#     macOS / Windows ignores the OS trust store and CA env vars on the
#     marketplace HTTPS path, so direct add can fail with TLS errors against
#     a private-CA Agnes instance even when system tools work fine. System
#     `git` honors GIT_SSL_CAINFO + the OS trust store, so cloning manually
#     and pointing Claude Code at the local clone sidesteps the Bun TLS path
#     entirely.
git clone https://x:$AGNES_PAT@agnes.example.com/marketplace.git/ ~/agnes-marketplace
claude plugin marketplace add ~/agnes-marketplace
# Optional hardening: strip the PAT from the cloned repo's origin so it
# doesn't sit in plaintext at ~/agnes-marketplace/.git/config — re-clone via
# the dashboard's setup flow when the PAT rotates.
git -C ~/agnes-marketplace remote set-url origin https://agnes.example.com/marketplace.git/

The dashboard-served setup payload (see app/web/setup_instructions.py) already branches between (a) and (b) automatically based on platform when a private CA is in play. The block above is the manual equivalent for users registering outside that flow (e.g. operators bringing up a new instance, or analysts whose first attempt failed and need to retry by hand).