docs: consolidate and de-clutter the documentation tree (#306)

CLAUDE.md rewritten (708 -> ~320 lines): four overlapping release
sections collapsed to one, stale v1->v35 schema history dropped (it
lives in CHANGELOG), marketplace endpoint internals and verbose
process sections moved out or tightened.

New focused docs:
- docs/RELEASING.md - release process, deploy workflows, CI quirks
  (RELEASE_TEMPLATE.md folded in as an appendix)
- docs/marketplace.md - marketplace ingestion + re-serving internals
- docs/README.md - documentation index by audience, linked from
  README.md and CLAUDE.md

Archived under docs/archive/: docs/superpowers/ (52 historical
planning artifacts), HACKATHON.md, pd-ps-comments.md,
security-audit-2026-04.md, future/NOTIFICATIONS.md.

Removed the docs/auto-install.md stub. Fixed dangling links in
connectors/jira/README.md and dev_docs/README.md, repointed
code/doc references to archived paths.

.gitignore

@@ -1,5 +1,6 @@
 # Claude Code
 .claude/
+CLAUDE.local.md
 
 # Local dev data (copied from server for testing)
 dev_data/

AGENTS.md

@@ -0,0 +1,7 @@
+# AGENTS.md
+
+Agent instructions for this repository live in **[`CLAUDE.md`](CLAUDE.md)** — read it first.
+
+`CLAUDE.md` is the single source of truth for any AI coding agent working here (not just Claude Code): project structure, the `extract.duckdb` architecture, the data-querying agent rails, and the project conventions — release process, changelog discipline, vendor-agnostic OSS rules, and commit/PR etiquette.
+
+Full documentation index: [`docs/README.md`](docs/README.md).

CHANGELOG.md

@@ -10,6 +10,9 @@ CalVer image tags (`stable-YYYY.MM.N`, `dev-YYYY.MM.N`) are produced for every C
 
 ## [Unreleased]
 
+### Internal
+- Documentation tree cleaned up and consolidated. `CLAUDE.md` rewritten (708 → ~320 lines): the four overlapping release sections, the stale `v1→v35` DuckDB schema history, and the marketplace endpoint internals moved out to focused docs; preachy process sections tightened. New `docs/RELEASING.md` (release process + deploy workflows + CI quirks, with `RELEASE_TEMPLATE.md` folded in as an appendix) and `docs/marketplace.md` (marketplace ingestion + re-serving internals). Historical planning artifacts (`docs/superpowers/`, 52 files) and dated one-off docs (`HACKATHON.md`, `pd-ps-comments.md`, `security-audit-2026-04.md`, `future/NOTIFICATIONS.md`) moved under `docs/archive/`. New `docs/README.md` documentation index organized by audience, linked from `README.md` and `CLAUDE.md`. Removed the `docs/auto-install.md` stub. Fixed dangling doc links in `connectors/jira/README.md` and `dev_docs/README.md`, and repointed code/doc references to the archived paths (or dropped the pointer where the target was already a dead reference on `main`). Added a root `AGENTS.md` pointing to `CLAUDE.md` as the single source of truth for any AI coding agent, and `CLAUDE.local.md` to `.gitignore`.
+
 ## [0.54.14] — 2026-05-14
 
 ### Changed

README.md

@@ -182,12 +182,15 @@ See `config/instance.yaml.example` for all available options.
 
 ## Documentation
 
-- [Hackathon TL;DR](docs/HACKATHON.md) — condensed deploy + dev playbooks (for both humans and AI agents)
+**Full index: [docs/README.md](docs/README.md)** — every doc, organized by audience (analyst / operator / developer).
+
+Key entry points:
+
+- [Quickstart](docs/QUICKSTART.md) — local development setup
 - [Onboarding Guide](docs/ONBOARDING.md) — end-to-end Terraform deployment into a GCP project (recommended for production)
 - [Deployment Guide](docs/DEPLOYMENT.md) — chooses between Terraform and Docker Compose; covers OSS self-host
 - [Configuration Reference](docs/CONFIGURATION.md) — `instance.yaml`, env vars, per-instance options
 - [Architecture](ARCHITECTURE.md) — orchestrator, extractors, DB layout
-- [Quickstart](docs/QUICKSTART.md) — local development
 
 ## Contributing
 

connectors/bigquery/access.py

@@ -1,8 +1,8 @@
 """Single entry point for BigQuery access — config resolution, client construction,
 DuckDB-extension session, and Google-API error translation.
 
-See docs/superpowers/specs/2026-04-29-issue-134-bq-access-unify-design.md for the
-full design rationale.
+See docs/archive/superpowers/specs/2026-04-29-issue-134-bq-access-unify-design.md
+for the full design rationale.
 """
 from __future__ import annotations
 

connectors/jira/README.md

@@ -433,7 +433,7 @@ if not hmac.compare_digest(signature, expected):
 
 ## Schema Reference
 
-See [docs/jira_schema.md](jira_schema.md) for detailed table schemas and example queries.
+The Jira tables and their columns are described in [`docs/DATA_SOURCES.md`](../../docs/DATA_SOURCES.md). At runtime, inspect the live schema with `agnes schema <table>` and `agnes describe <table>`.
 
 ## Historical Backfill
 

dev_docs/README.md

@@ -2,30 +2,32 @@
 
 This folder contains documentation for **developers and server administrators** only.
 
-**⚠️ This folder is NOT synced to analyst machines** - it stays on the server and in the git repository only.
+**⚠️ This folder is NOT synced to analyst machines** — it stays on the server and in the git repository only.
 
 ## Contents
 
 ### Server Administration
-- `server.md` - Data broker server configuration and management
-- `disaster-recovery.md` - Recovery procedures for server failures
-- `security.md` - Security audit report and hardening guidelines
-- `jira.md` - Jira webhook integration and server-side processing
+- `server.md` — data broker server configuration and management
+- `disaster-recovery.md` — recovery procedures for server failures
+- `security.md` — security audit report and hardening guidelines
 
 ### Application Development
-- `desktop-app.md` - macOS desktop app architecture and development
-- `telegram_bot.md` - Telegram notification bot technical docs
-- `design-system.md` - UI/UX design system for web applications
-- `insights.md` - Activity Center dashboard feature documentation
+- `desktop-app.md` — macOS desktop app architecture and development
+- `telegram_bot.md` — Telegram notification bot technical docs
+- `design-system.md` — UI/UX design system for web applications
+- `insights.md` — Activity Center dashboard feature documentation
+- `session_explore.md` — session exploration tooling
 
-### Planning & Meetings
-- `meetings/` - Meeting transcripts, summaries, and notes
+Jira webhook integration and server-side processing is documented in
+[`../connectors/jira/README.md`](../connectors/jira/README.md).
 
 ## For Analysts
 
-If you're an analyst looking for documentation on how to **use** the platform, see the `docs/` folder instead:
-- `docs/GETTING_STARTED.md` - Quick start guide
-- `docs/data_description.md` - Data schemas and table definitions
-- `docs/metrics/` - Business metric definitions
-- `docs/jira_schema.md` - Jira data schema reference
-- `docs/notifications.md` - How to use Telegram notifications in your scripts
+If you're an analyst looking for documentation on how to **use** the platform,
+see the `docs/` folder instead — start at [`../docs/README.md`](../docs/README.md)
+for the full index. Key entry points:
+
+- `docs/QUICKSTART.md` — quick start guide
+- `docs/DATA_SOURCES.md` — data sources and table definitions
+- `docs/metrics/` — business metric definitions
+- `docs/HOWTO/` — task-oriented analyst cookbook

docs/QUICKSTART.md

@@ -86,4 +86,4 @@ Open the project in Claude Code. The CLAUDE.md file will guide the AI assistant
 
 ## Hackathon
 
-See [`HACKATHON.md`](HACKATHON.md) for the deploy-and-develop playbook. Per-developer dev VMs are the supported pattern — point your VM at your branch image with `gcloud compute ssh <vm> --command "sudo sed -i 's/^AGNES_TAG=.*/AGNES_TAG=dev-<slug>/' /opt/agnes/.env && sudo /usr/local/bin/agnes-auto-upgrade.sh"`.
+See [`archive/HACKATHON.md`](archive/HACKATHON.md) for the deploy-and-develop playbook (archived event runbook). Per-developer dev VMs are the supported pattern — point your VM at your branch image with `gcloud compute ssh <vm> --command "sudo sed -i 's/^AGNES_TAG=.*/AGNES_TAG=dev-<slug>/' /opt/agnes/.env && sudo /usr/local/bin/agnes-auto-upgrade.sh"`.

docs/README.md

@@ -0,0 +1,73 @@
+# Agnes documentation
+
+Index of all documentation, organized by who needs it. New here? Start with the
+row that matches your role.
+
+| You are… | Start with |
+|----------|-----------|
+| **Analyst** — using Agnes to query data | [`QUICKSTART.md`](QUICKSTART.md), then [`HOWTO/`](HOWTO/) |
+| **Operator** — deploying & running an instance | [`PLATFORM_SETUP.md`](PLATFORM_SETUP.md) |
+| **Developer** — working on Agnes itself | [`../ARCHITECTURE.md`](../ARCHITECTURE.md) + [`architecture.md`](architecture.md) |
+
+---
+
+## For analysts
+
+Using the platform to analyze data.
+
+- [`QUICKSTART.md`](QUICKSTART.md) — local setup + first sync
+- [`HOWTO/`](HOWTO/) — task-oriented cookbook (querying, snapshots, common workflows)
+- [`DATA_SOURCES.md`](DATA_SOURCES.md) — data source connectors (Keboola, BigQuery, CSV) and how tables surface
+- [`metrics/`](metrics/) — canonical business-metric definitions (YAML)
+- [`HEADLESS_USAGE.md`](HEADLESS_USAGE.md) — PAT auth for CI / headless clients
+
+## For operators
+
+Deploying, configuring, and running an Agnes instance.
+
+- [`PLATFORM_SETUP.md`](PLATFORM_SETUP.md) — **the consolidated operator playbook** (bootstrap, TLS, marketplaces, scheduler, telemetry)
+- [`ONBOARDING.md`](ONBOARDING.md) — end-to-end Terraform deployment into a new GCP project
+- [`DEPLOYMENT.md`](DEPLOYMENT.md) — picks between the Terraform and Docker Compose paths
+- [`CONFIGURATION.md`](CONFIGURATION.md) — `instance.yaml`, env vars, per-instance options
+- [`state-dir.md`](state-dir.md) — persistent data layout (`data` + `state` tiers, mount layouts, migration)
+- [`RBAC.md`](RBAC.md) — access control: groups, members, resource grants
+- [`auth-google-oauth.md`](auth-google-oauth.md) — Google OAuth setup + operator gotchas
+- [`auth-groups.md`](auth-groups.md) — Google Workspace group sync
+- [`admin/query-modes.md`](admin/query-modes.md) — table registration query modes
+- [`agent-setup-prompt.md`](agent-setup-prompt.md) — customize the `/setup` page banner
+- [`agent-workspace-prompt.md`](agent-workspace-prompt.md) — customize the generated analyst `CLAUDE.md`
+- [`initial-workspace-override.md`](initial-workspace-override.md) — per-instance analyst-workspace skeleton override
+- [`curated-marketplace-format.md`](curated-marketplace-format.md) — authoring `marketplace-metadata.json` for curated marketplaces
+- [`observability.md`](observability.md) — PostHog integration (exceptions, tracing, session replay)
+- [`operator/news-content-guide.md`](operator/news-content-guide.md) — editorial guidelines for in-app news content
+
+## For developers
+
+Working on the Agnes codebase.
+
+- [`../ARCHITECTURE.md`](../ARCHITECTURE.md) — high-level system overview (the summary)
+- [`architecture.md`](architecture.md) — detailed architecture reference (module map, extract.duckdb contract, components)
+- [`../CLAUDE.md`](../CLAUDE.md) — project instructions for AI agents working in this repo
+- [`development.md`](development.md) — logging, request correlation, debug toolbar
+- [`local-development.md`](local-development.md) — `LOCAL_DEV_MODE` setup (what's mocked vs. real)
+- [`RELEASING.md`](RELEASING.md) — release process, deploy workflows, CI quirks
+- [`RELEASE_CHECKLIST.md`](RELEASE_CHECKLIST.md) — pre-merge checks for bootstrap-path changes
+- [`testing/`](testing/) — test plans (clean-analyst bootstrap, VM test)
+- [`marketplace.md`](marketplace.md) — Claude Code marketplace ingestion + re-serving internals
+- [`STORE_GUARDRAILS.md`](STORE_GUARDRAILS.md) — flea-market upload guardrails (static checks + LLM review)
+- [`corporate-memory-governance.md`](corporate-memory-governance.md) — knowledge-distribution governance design
+- [`ADR-corporate-memory-v1.md`](ADR-corporate-memory-v1.md) — ADR: corporate-memory v1 decisions
+- [`llm-routing.md`](llm-routing.md) — design: provider-agnostic LLM routing
+- [`sample-data.md`](sample-data.md) — sample data generator (e-commerce schema, size presets)
+- [`theme-reference.html`](theme-reference.html) — web UI theme/color reference
+- [`../dev_docs/`](../dev_docs/) — **server/developer-internal docs** (not synced to analyst machines): server ops, disaster recovery, security audit, desktop app, design system, Telegram bot
+
+Code-adjacent READMEs: [`../connectors/jira/README.md`](../connectors/jira/README.md),
+[`../services/corporate_memory/README.md`](../services/corporate_memory/README.md),
+[`../scripts/README.md`](../scripts/README.md).
+Agent skill files: [`../cli/skills/`](../cli/skills/).
+
+## Other
+
+- [`../CHANGELOG.md`](../CHANGELOG.md) — full change history (Keep-a-Changelog format)
+- [`archive/`](archive/) — historical planning artifacts and superseded docs; not maintained, see [`archive/README.md`](archive/README.md)

docs/RELEASE_CHECKLIST.md

@@ -16,7 +16,7 @@ requesting review:
    click the analyst tile and copy the paste prompt.
 4. Paste into Claude Code and let it run. `tree -a /tmp/test-analyst-1` and
    compare with the expected tree from the design spec
-   (`docs/superpowers/specs/2026-05-04-clean-analyst-bootstrap-design.md` §5.2).
+   (`docs/archive/superpowers/specs/2026-05-04-clean-analyst-bootstrap-design.md` §5.2).
 5. `claude` in that folder. Three queries:
    - "What tables can I see?"
    - "SELECT count(\*) FROM <t>" (a table from the catalog)

docs/RELEASE_TEMPLATE.md

@@ -1,37 +0,0 @@
-# Release Notes Template
-
-Use this template when adding a new entry to `CHANGELOG.md`.
-
----
-
-## stable-YYYY.MM.N
-
-**Image:** `ghcr.io/keboola/agnes-the-ai-analyst:stable-YYYY.MM.N`
-**Digest:** `sha256:...` (from `docker inspect --format='{{index .RepoDigests 0}}'`)
-**Date:** YYYY-MM-DD
-
-### Added
-- Feature description
-
-### Changed
-- Change description
-
-### Fixed
-- Bug fix description
-
-### Breaking Changes
-- Description of breaking change
-- **Migration guide:** Steps to upgrade from previous version
-
-### Deprecated
-- Description of deprecated feature (will be removed in YYYY.MM.N)
-
----
-
-## Guidelines
-
-- Every merge to `main` creates a new `stable-YYYY.MM.N` release
-- Include the image digest for verification with `cosign verify`
-- Breaking changes require `BREAKING:` prefix in commit message
-- Migration guides must include exact commands or config changes
-- If a release deprecates the previous stable, note it explicitly

docs/RELEASING.md

@@ -0,0 +1,270 @@
+# Releasing & deploying
+
+The full release process for Agnes. CLAUDE.md carries the short version; this
+doc is the operational reference. Read it linearly the first few times — once
+internalized, the order matters less, but the **non-obvious gotchas never go
+away**.
+
+## Changelog discipline — non-negotiable
+
+**Every PR that adds, removes, or changes user-visible behavior MUST update
+`CHANGELOG.md` in the same PR.** No exceptions, no follow-ups, no "I'll do it
+after merge". User-visible = anything an operator, end-user, or downstream
+integrator can observe: CLI flags / output / exit codes, REST endpoints /
+payloads / status codes, web UI, `instance.yaml` schema, env vars,
+`extract.duckdb` contract, Docker / compose / Caddyfile knobs, default
+behaviors, breaking changes, security fixes.
+
+**How:**
+- Add a bullet under the topmost `## [Unreleased]` heading (create one if
+  missing — it sits above the latest released version).
+- Group by `### Added` / `### Changed` / `### Fixed` / `### Removed` /
+  `### Internal` (Keep-a-Changelog sections).
+- Mark breaking changes with `**BREAKING**` at the start of the bullet —
+  operators grep for that string before bumping the pin.
+- Reference the relevant doc/runbook if one exists (e.g.
+  `see docs/auth-groups.md`), don't restate it.
+- Internal-only changes (refactors, test additions, dependency bumps without
+  behavior change) go under `### Internal` — still log them, just keep them
+  terse.
+
+Reviewers should bounce PRs that touch user-visible behavior without a
+changelog update — same way they'd bounce a PR with no test changes for new
+logic.
+
+## Release-cut belongs to the PR — non-negotiable
+
+**The version bump + CHANGELOG rename + new empty `[Unreleased]` are the LAST
+commit on the PR that earned the version. Never a standalone follow-up PR.**
+
+When a PR lands the only `[Unreleased]` content (or is the last in a queue of
+in-flight feature PRs), the release-cut MUST ship as part of the same merge.
+Standalone release-cut PRs add review-overhead PRs to history with no behavior
+change of their own and pollute `git log` with bookkeeping commits separated
+from the work that earned them.
+
+**Mandatory checklist before approving / enabling auto-merge on ANY PR:**
+
+1. **Stop.** Will this PR land alone in `[Unreleased]` (no other in-flight PRs
+   queued behind it)?
+2. **If yes**, the release-cut is REQUIRED in the same PR before merge. BEFORE
+   pushing the final commit:
+   - Bump `pyproject.toml` to `X.Y.Z`
+   - Rename `## [Unreleased]` → `## [X.Y.Z] — YYYY-MM-DD`, add a new empty
+     `## [Unreleased]` on top
+   - Either squash these into the consolidation commit OR add as a separate
+     `release: X.Y.Z` commit on the same branch
+3. **THEN** push, approve, enable auto-merge.
+4. After auto-merge fires: tag `vX.Y.Z` against the merge commit + create a
+   GitHub Release. Done — one PR, one merge, one release.
+
+**Failure mode to avoid:** enabling auto-merge on the feature PR thinking "I'll
+add the release-cut after." Auto-merge fires faster than the second commit
+lands. The window closes; the only fix is a standalone release-cut PR — exactly
+what this rule prohibits.
+
+**Acceptable standalone release-cut** (rare): only when `[Unreleased]`
+accumulated bullets from MULTIPLE already-merged PRs AND no further
+behavior-change PR is queued — i.e. the cut is the only outstanding work and
+there's no PR to attach it to.
+
+## Release workflow — concrete recipe
+
+### Happy path (8 steps)
+
+```bash
+# 1. Branch from a fresh checkout. iCloud Drive worktrees randomly hang
+#    on git operations — use a fresh shallow clone in /tmp instead.
+cd /tmp && git clone --depth 50 --branch main \
+  https://github.com/keboola/agnes-the-ai-analyst.git agnes-<topic>
+cd agnes-<topic> && git checkout -b zs/<branch-name>
+
+# 2. Make the change + tests. Run the AREA pytest while iterating
+#    (e.g. `pytest tests/test_X.py -p no:xdist -q`).
+
+# 3. Add a CHANGELOG bullet under [Unreleased].
+#    Group: Added | Changed | Fixed | Removed | Internal
+#    Mark BREAKING with **BREAKING** prefix.
+
+# 4. Commit the change(s). Multiple logical commits OK; release-cut
+#    will be a SEPARATE last commit (next step). DO NOT bundle the
+#    release-cut into the same commit as the change — it pollutes
+#    the SHA that auto-close keywords reference and makes revert
+#    targeted at the change-only difficult.
+
+# 5. Run the full pytest suite locally:
+#    `pytest tests/ -p no:xdist -q` (or `-n auto` if xdist works).
+#    Pre-existing fails (e.g. test_readers_in_pre_init_dir under
+#    subprocess timeout) are OK to ignore; verify by reverting your
+#    diff and reproducing on bare main.
+
+# 6. Release-cut commit (LAST commit on the PR per the rule above):
+#    - Bump pyproject.toml: version = "X.Y.Z"
+#    - Rename `## [Unreleased]` → `## [X.Y.Z] — YYYY-MM-DD`
+#    - Add a fresh empty `## [Unreleased]` line above
+#    Commit message: `release: X.Y.Z — <one-line summary>`
+
+# 7. Push branch + open PR + enable auto-merge SQUASH:
+#    git push -u origin HEAD
+#    gh pr create --repo keboola/agnes-the-ai-analyst \
+#      --head <branch> --title "<...>" --body "<...>"
+#    gh pr merge <N> --repo keboola/agnes-the-ai-analyst \
+#      --squash --auto --delete-branch
+
+# 8. After auto-merge fires (poll or `Monitor`):
+#    git fetch origin --tags
+#    git tag vX.Y.Z <merge-sha>
+#    git push origin vX.Y.Z
+#    gh release create vX.Y.Z --repo keboola/agnes-the-ai-analyst \
+#      --title "vX.Y.Z — <...>" --notes "<copy-paste from CHANGELOG>"
+```
+
+### Picking the next version
+
+`pyproject.toml`'s current `version` is the **next-release target** (post-cut
+from the previous release). Pre-1.0 we patch-bump for everything that doesn't
+break operator-facing APIs:
+
+- `instance.yaml` schema additions, new env vars, new endpoints → patch (e.g.
+  0.54.3 → 0.54.4)
+- New CLI subcommands, BREAKING removals, schema migrations → still patch within
+  the current 0.5x cycle (no minor bumps cut today)
+- The CHANGELOG `**BREAKING**` marker is what operators grep for; the version
+  number is secondary
+
+Always check `git tag -l "v0.X*"` before naming — if `v0.54.0` is already
+tagged, the next one is `v0.54.1`, even if `pyproject.toml` still says `0.54.0`
+from a stale post-cut commit (we've shipped that race before).
+
+### Authoring expectations on the PR
+
+- **Self-PRs** (you're both author and reviewer): GitHub forbids self-approve.
+  If branch protection requires N approving reviews (we don't today —
+  `required_approving_review_count = 0`), you need someone else to approve. With
+  our current 0-review setup, self-PRs can still merge automatically once
+  required CI passes.
+- **Other people's PRs you're taking over**: dismiss any prior
+  CHANGES_REQUESTED reviews (yours or someone else's) before auto-merge can
+  fire. `gh pr review <N> --approve --body "..."` after pushing your fixes.
+- **Devin Review**: not a required check today; runs in parallel and posts a
+  comment. Don't wait on it for merge unless the human reviewer explicitly asks.
+
+### CI quirks you WILL hit
+
+- **`gh pr checks` glosses CANCELLED as `fail`.** When you force-push (rebase,
+  amend), GitHub auto-cancels the in-flight `Release` workflow run on the older
+  SHA. Those cancelled jobs show up as "fail" in the PR's check summary and tab
+  forever, even after newer runs succeed. **Look at the conclusion column, not
+  just the count.** Rule of thumb: if the same check name appears with both
+  `pass` and `fail` rows, the `fail` row is from an older auto-cancelled SHA.
+  Verify with `gh api repos/keboola/agnes-the-ai-analyst/commits/<sha>/check-runs`
+  — the raw API distinguishes `cancelled` from `failure` truthfully.
+- **Branch protection's "strict" mode caches cancelled `test` as blocking** even
+  after newer `test` runs succeed. Symptom: `mergeable_state: blocked` despite
+  all required checks green on the latest SHA. Fix: re-run the cancelled
+  `Release` workflow run (`gh run rerun <run-id>`); once its `test` job lands as
+  success, the block clears. We've hit this on PRs #273, #281, #285, #286.
+- **Required checks** (per branch protection): `test` + `docker-build` only.
+  Other workflows (`cli-wheel-clean-install`, `build-and-push`,
+  `Release`-pipeline, Devin Review) are advisory — green/red doesn't gate merge.
+- **`enforce_admins: true`** in branch protection means `--admin` flag on
+  `gh pr merge` does NOT bypass. Don't try; just fix the underlying block.
+
+### Recovery when something derails
+
+- **Force-pushed and lost auto-merge?** GitHub *usually* preserves auto-merge
+  across force-pushes for the same PR; if it cleared, just re-run
+  `gh pr merge <N> --squash --auto --delete-branch`.
+- **Release-cut commit forgot to land?** That's the failure mode the
+  "Release-cut belongs to the PR" rule prevents. If it happens anyway: open a
+  follow-on PR with ONLY the release-cut commit, ship it, and write up why in
+  your post-mortem comment.
+- **Wrong version number tagged?** `git tag -d vX.Y.Z && git push --delete
+  origin vX.Y.Z` then re-tag against the right SHA. Update the GitHub Release if
+  you already created it.
+
+## Deploy workflows
+
+Two separate release.yml-style workflows produce GHCR images. Pick the one that
+matches what you're shipping.
+
+### `release.yml` — auto-build on every push
+
+Runs on **every** push to **every** branch.
+- Push to `main` → `:stable`, `:stable-YYYY.MM.N` (CalVer).
+- Push to non-main `<prefix>/<branch>` → `:dev`, `:dev-YYYY.MM.N`,
+  `:dev-<branch-slug>`, and (when prefix isn't a Git Flow convention)
+  `:dev-<prefix>-latest` alias.
+
+VMs that pin to a floating tag (`:dev`, `:dev-<prefix>-latest`) auto-upgrade
+within ~5 min via the cron in `agnes-auto-upgrade.sh`. Convenient for
+per-developer dev VMs; **footgun for shared dev VMs** (last pusher wins,
+regardless of who).
+
+### `keboola-deploy.yml` — tag-triggered, explicit deploy only
+
+Runs **only** on git tags matching `keboola-deploy-*`. Publishes:
+- `:keboola-deploy-<git-tag-suffix>` — immutable, tied to the exact commit
+- `:keboola-deploy-latest` — floating alias the consumer pins to
+
+**Operator workflow:**
+```bash
+git checkout <commit-or-branch>
+git tag keboola-deploy-<descriptive-name>
+git push origin keboola-deploy-<descriptive-name>
+# → workflow builds + publishes both tags
+# → VM cron picks up :keboola-deploy-latest within ~5 min
+# → manual cron trigger (skip the wait): sudo /usr/local/bin/agnes-auto-upgrade.sh on the VM
+```
+
+Use this when the consumer (e.g. a customer dev VM) needs
+**deploy-when-I-decide** semantics — no surprise rollouts from upstream branch
+pushes by other contributors. The infra repo pins
+`image_tag = "keboola-deploy-latest"` on the relevant VM.
+
+### Module versioning
+
+The customer-instance Terraform module under `infra/modules/customer-instance/`
+is published as `infra-vMAJOR.MINOR.PATCH` git tags (separate from app CalVer
+tags). Bump on any module-API change; downstream infra repos pin to the tag in
+their `source = "github.com/keboola/agnes-the-ai-analyst//infra/modules/customer-instance?ref=infra-v1.X.Y"`.
+
+After merging a module change to `main`:
+```bash
+git tag infra-vX.Y.Z origin/main
+git push origin infra-vX.Y.Z
+```
+
+### Replacing a VM after a startup-script change
+
+Module sets `lifecycle { ignore_changes = [metadata_startup_script] }` on
+`google_compute_instance.vm` so normal `terraform apply` doesn't churn running
+VMs. To propagate a startup-script update, trigger the consumer's apply workflow
+manually with the VM resource address — typical workflow_dispatch input is
+`recreate_targets='module.agnes.google_compute_instance.vm["<vm-name>"]'`.
+
+## Appendix: CHANGELOG entry skeleton
+
+Copy this when adding to `## [Unreleased]` in `CHANGELOG.md`. Drop the sections
+you don't need; keep the Keep-a-Changelog order.
+
+```markdown
+### Added
+- New feature description.
+
+### Changed
+- Change description. **BREAKING** prefix + migration steps if operator-facing.
+
+### Fixed
+- Bug fix description.
+
+### Removed
+- **BREAKING** removed feature — what replaces it.
+
+### Internal
+- Refactors, test additions, dependency bumps with no behavior change.
+```
+
+At release-cut time `## [Unreleased]` is renamed to `## [X.Y.Z] — YYYY-MM-DD`
+and a fresh empty `## [Unreleased]` is added on top. CI publishes the matching
+`stable-YYYY.MM.N` image tag for the merge commit (see Deploy workflows above).

docs/archive/README.md

@@ -0,0 +1,31 @@
+# Archived documentation
+
+Historical artifacts kept for reference but **not maintained**. Nothing here is
+current guidance — paths, line numbers, and APIs cited inside these files
+reflect the state of the repo when they were written. For live docs, start at
+[`../README.md`](../README.md).
+
+## Contents
+
+- **`superpowers/`** — implementation plans (`plans/`) and design specs
+  (`specs/`) from past development sprints. Each file is a point-in-time
+  planning artifact for a feature that has since shipped (or been dropped).
+  Useful as a record of *why* something was built a certain way; useless as a
+  guide to *what the code does now* — read the code and `CHANGELOG.md` for that.
+- **`HACKATHON.md`** — condensed deploy + dev playbook written for a hackathon
+  sprint. Superseded by [`../QUICKSTART.md`](../QUICKSTART.md),
+  [`../DEPLOYMENT.md`](../DEPLOYMENT.md), and [`../ONBOARDING.md`](../ONBOARDING.md).
+- **`NOTIFICATIONS.md`** — early spec for Telegram notifications via scripts +
+  crontab. The feature shipped as a service; see
+  [`../../dev_docs/telegram_bot.md`](../../dev_docs/telegram_bot.md).
+- **`pd-ps-comments.md`** — review notes on the corporate-memory v1 branch.
+  Accepted decisions live in [`../ADR-corporate-memory-v1.md`](../ADR-corporate-memory-v1.md).
+- **`security-audit-2026-04.md`** — point-in-time security audit snapshot.
+  Findings addressed in later releases; see `CHANGELOG.md`.
+
+## Policy
+
+Don't edit archived files to "fix" stale references — rewriting historical
+planning artifacts is revisionist and loses the record of what was actually
+decided when. If an archived doc is genuinely worthless, delete it; otherwise
+leave it as-is.

docs/archive/security-audit-2026-04.md

@@ -1,5 +1,10 @@
 # Security audit — Agnes AI Data Analyst
 
+> **Archived.** This is a point-in-time audit snapshot. Findings have been
+> triaged and addressed in subsequent releases — check `CHANGELOG.md` (search
+> for the relevant file/endpoint) for the fixes that landed since 2026-04-22.
+> Kept for historical reference; not a live tracker.
+
 **Date:** 2026-04-22
 **Branch audited:** `main` at commit `cbb7733`
 **Method:** parallel review passes over four scope areas — (1) secrets/SQLi/authz/SSRF, (2) auth flows & route wiring, (3) templates & UI wiring/XSS, (4) data layer & config & dead code. Findings deduped across the passes, severities adjusted to real-world exploitability.

docs/auto-install.md

@@ -1,5 +0,0 @@
-# Auto-Install Guide
-
-For deployment instructions, see [DEPLOYMENT.md](DEPLOYMENT.md).
-
-For local development setup, see the [README](../README.md#development).

docs/marketplace.md

@@ -0,0 +1,101 @@
+# Marketplace internals
+
+How Agnes ingests admin-registered Claude Code marketplaces and re-serves a
+single aggregated, RBAC-filtered marketplace back to user instances. CLAUDE.md
+carries a one-paragraph summary; this doc is the reference.
+
+For the *content-authoring* side (cover photos, demo videos, doc links via
+`marketplace-metadata.json`), see [`curated-marketplace-format.md`](curated-marketplace-format.md).
+
+## Marketplace repositories (ingestion)
+
+Admin-managed git repos cloned nightly to `${DATA_DIR}/marketplaces/<slug>/` so
+FastAPI can read their contents from disk.
+
+- Register via `/admin/marketplaces` (admin UI) or `POST /api/marketplaces`.
+- Scheduler calls `POST /api/marketplaces/sync-all` (admin-only, authed via
+  `SCHEDULER_API_TOKEN`) at `daily 03:00` UTC. Routing through HTTP keeps the
+  app the sole writer to `system.duckdb` — the previous in-process call from the
+  scheduler container raced the app's long-lived DB handle and 500-ed on
+  `Could not set lock on file`.
+- Manual re-sync from the UI ("Sync now") hits `POST /api/marketplaces/{id}/sync`.
+- PATs for private repos persist to `${DATA_DIR}/state/.env_overlay` (chmod 600)
+  as `AGNES_MARKETPLACE_<SLUG>_TOKEN`. DuckDB stores only the env-var name
+  (`token_env`), never the secret.
+- Registry lives in DuckDB table `marketplace_registry`.
+- After each successful sync, `src/marketplace.py` parses
+  `.claude-plugin/marketplace.json` from the cloned repo and caches the plugin
+  list in `marketplace_plugins` (keyed by `(marketplace_id, plugin_name)`).
+- `src/marketplace.py` handles clone/fetch/reset with token redaction in any
+  surfaced error message.
+
+## Claude Code marketplace endpoint (re-serving)
+
+Agnes serves a single aggregated Claude Code marketplace over two channels, both
+gated by PAT auth and filtered per caller:
+
+- `GET /marketplace.zip` — deterministic ZIP download with `ETag` /
+  `If-None-Match` (304 when content unchanged). Consumed by a client-side
+  SessionStart hook.
+- `GET /marketplace.git/*` — git smart-HTTP (dulwich via a2wsgi). Registered in
+  Claude Code once, then Claude Code owns the clone/fetch cycle.
+
+**Auth:** ZIP uses `Authorization: Bearer <PAT>`. Git uses HTTP Basic where the
+password field carries the PAT (`https://x:<PAT>@host/marketplace.git/`) — git
+CLI does not speak Bearer.
+
+**Content:** filtered via `src.marketplace_filter.resolve_allowed_plugins` which
+joins `resource_grants ↔ marketplace_plugins` (matching
+`mp.marketplace_id || '/' || mp.name = rg.resource_id`) scoped to the caller's
+`user_group_members`. Admin is treated as a regular group here — no god-mode
+shortcut for the marketplace feed, so admins curate their own view by granting
+plugins to the Admin group (or any group they belong to).
+
+On-disk layout in the served ZIP / git tree uses a slug-prefixed directory
+(`plugins/<slug>-<plugin>/`) so two marketplaces shipping a same-named plugin
+don't overwrite each other's files. The synth marketplace.json's `name` field,
+however, is the plugin's authoritative name from its own
+`.claude-plugin/plugin.json` (with a fallback to the upstream marketplace.json
+`name`) — Claude Code's `/plugin` UI resolves a loaded plugin back to its
+catalog entry by `plugin.json` name, so the catalog entry's `name` must match.
+Same-named plugins from two upstream marketplaces therefore collide in the
+catalog by design; admin RBAC (which grants survive the filter) decides which
+one wins, identical to how Claude Code behaves when a user adds two upstream
+marketplaces with overlapping plugin names directly. `/marketplace/info` exposes
+both `name` and `prefixed_name` so operators can disambiguate.
+
+**Cache:** content-addressed bare repos at `${DATA_DIR}/marketplaces/git-cache/`
+keyed by sha256(filtered content). Two users with the same RBAC view share one
+repo; content change → new repo next to the old one. No TTL / prune yet.
+
+## User registration inside Claude Code
+
+```
+# ZIP channel (typically via a SessionStart hook that unpacks into ./marketplace/)
+curl -H "Authorization: Bearer $AGNES_PAT" https://agnes.example.com/marketplace.zip
+
+# Git channel — one-time registration. Two paths; pick the first that works.
+
+# (a) Direct registration — preferred when it works.
+/plugin marketplace add https://x:$AGNES_PAT@agnes.example.com/marketplace.git/
+
+# (b) Two-step fallback — required when (a) fails. Bun-compiled `claude` on
+#     macOS / Windows ignores the OS trust store and CA env vars on the
+#     marketplace HTTPS path, so direct add can fail with TLS errors against
+#     a private-CA Agnes instance even when system tools work fine. System
+#     `git` honors GIT_SSL_CAINFO + the OS trust store, so cloning manually
+#     and pointing Claude Code at the local clone sidesteps the Bun TLS path
+#     entirely.
+git clone https://x:$AGNES_PAT@agnes.example.com/marketplace.git/ ~/agnes-marketplace
+claude plugin marketplace add ~/agnes-marketplace
+# Optional hardening: strip the PAT from the cloned repo's origin so it
+# doesn't sit in plaintext at ~/agnes-marketplace/.git/config — re-clone via
+# the dashboard's setup flow when the PAT rotates.
+git -C ~/agnes-marketplace remote set-url origin https://agnes.example.com/marketplace.git/
+```
+
+The dashboard-served setup payload (see `app/web/setup_instructions.py`) already
+branches between (a) and (b) automatically based on platform when a private CA
+is in play. The block above is the manual equivalent for users registering
+outside that flow (e.g. operators bringing up a new instance, or analysts whose
+first attempt failed and need to retry by hand).

services/session_processors/verification.py

@@ -82,7 +82,7 @@ class VerificationProcessor:
 
             # Confidence is computed in code from (source_type, detection_type).
             # The LLM is not trusted to set its own credibility — see Q3 in
-            # docs/pd-ps-comments.md and the ADR.
+            # docs/archive/pd-ps-comments.md and the ADR.
             detection_type = v.get("detection_type")
             try:
                 confidence_value = compute_confidence("user_verification", detection_type)

services/verification_detector/schemas.py

@@ -2,7 +2,7 @@
 
 Confidence is intentionally NOT part of this schema. It is derived in code from
 (source_type, detection_type) via services.corporate_memory.confidence — the LLM
-is not trusted to set its own credibility (see docs/pd-ps-comments.md Q3).
+is not trusted to set its own credibility (see docs/archive/pd-ps-comments.md Q3).
 """
 
 VERIFICATION_SCHEMA: dict = {

src/orchestrator_security.py

@@ -4,9 +4,7 @@ The orchestrator reads `_remote_attach` rows that connectors write into their
 `extract.duckdb`, then calls `INSTALL`, `LOAD`, and `ATTACH` based on those
 values. Treating the connector as adversarial (compromised image, supply-chain,
 malicious fork) means the orchestrator picks **what** can be installed and
-**which** env vars can be referenced — not the connector. See
-`docs/superpowers/plans/2026-04-27-issue-81-trust-boundary.md` for the full
-threat model.
+**which** env vars can be referenced — not the connector.
 """
 
 from __future__ import annotations