agnes-the-ai-analyst/tests/test_store_entity_versions.py
Vojtech fb6e930bc9
feat(store-guardrails): per-component description quality + plain-language UX (#276)
* feat(store-guardrails): enforce per-component description quality

Two-tier hard guardrail on flea-market submissions. Empty / placeholder /
single-word descriptions now block before any LLM call; vague-but-passes-
floor descriptions block on the substantive LLM review layer.

Tier 1 — inline mechanical check (src/store_guardrails/content_check.py).
Walks the baked plugin tree, evaluates each component (plugin manifest,
agents, skills, commands) plus the submission-level form description
against a 60-char / 25-char (commands) / 5-distinct-word / 200-char-body
floor with a placeholder denylist (TODO, TBD, {{var}}, etc.). Floors
calibrated against real ecosystem norms: Claude / superpowers /
compound-engineering skill packs cluster 150–220 chars, npm / Docker /
VS Code at 100–120. InlineResult.passed now ANDs in content.status.

Tier 2 — LLM review extension (prompts.py + llm_review.py). System
prompt gains a content-quality criterion; REVIEW_JSON_SCHEMA carries a
content_quality {verdict, issues[]} object alongside the existing
security findings. is_safe() requires content_quality.verdict == 'pass'.
Single LLM call covers both dimensions. MAX_RESPONSE_TOKENS bumped
2000 → 2500 for the extra payload. Verdicts missing content_quality
treated as pass (backwards compat with already-recorded rows).

Submitter UX:
- /store/new wizard now carries a "Before you upload — what passes
  review" collapsible disclosure on both step 1 and step 2 with the
  bar + patterns that work. Live char counter on the description
  field. Per-component preview table (green/red dots from the new
  summarize_for_preview helper) renders after the ZIP /preview round
  trip, scoping each finding to its file.
- New /store/examples page with rejected/passes pairs for skill /
  agent / plugin / command plus a "Why these limits" research table.
  Anchored sections (#skill / #agent / #plugin / #command) so the
  rejection banner can deep-link by component_type.
- Quarantine banner _content_findings.html groups findings by file
  (one "See <type> example ↗" per component, not per field) and
  translates field codes (frontmatter.description / body / etc.) to
  plain-English labels. _content_howto_fix.html surfaces a static
  "Re-upload as new version" + "See examples" action row beneath any
  content failure on the entity detail page.
- _parse_frontmatter moved to src/store_guardrails/_frontmatter.py so
  the new check module shares the parser without inverting the
  app → src dependency direction.

Tests:
- New tests/test_store_guardrails_content.py (29 cases) covering
  every failure code per component type plus submission-level checks
  and the summarize_components / summarize_for_preview helpers.
- Extended test_store_guardrails_inline.py for the new
  InlineResult.content field + aggregate behaviour.
- Extended test_store_guardrails_llm.py for the new
  content_quality verdict pathways (fail blocks, missing field passes).
- Backfilled fixture descriptions across test_store_api.py,
  test_store_entity_versions.py, test_store_put_atomic.py,
  test_admin_store_submissions.py, test_marketplace_api.py,
  test_marketplace_v32_endpoints.py so existing happy-path tests
  clear the new 60-char floor.

* fix(content-guardrail): align agents walker with preview + drop import-time .format()

Two cleanups from the takeover review on #276 (vr/guardrails-content).

1) `_iter_components` for agents now skips files lacking frontmatter
   (no `name` AND no `description`). Pre-fix the walker greedily
   evaluated every `*.md` under `agents/` — `agents/README.md` and
   helper docs got flagged as "frontmatter.description empty"
   rejections. Worse: `summarize_for_preview` for `type=agent` ALREADY
   filters the same shape, so the upload preview gave a green dot
   while the post-bake check gave a red rejection on submit. Two new
   regression tests in TestAgentsWalkerSkipsNonAgentFiles pin both
   shapes (README + _NOTES.md) so the preview/check parity stays
   aligned.

2) `body_too_short` hints now use the same runtime-kwarg substitution
   pattern as every other hint in the table. Pre-fix the skill +
   agent body_too_short hints called `.format(min_chars=_MIN_BODY_CHARS)`
   at module-load time, but the call site `_hint_for(type_,
   "body_too_short")` didn't pass `min_chars=`, so the format() was
   just baking the constant at import. Cosmetic inconsistency; pass
   `min_chars=_MIN_BODY_CHARS` at the call site instead and let
   `_hint_for` do the substitution like it does for `too_short`.

Verified end-to-end:
- New TestAgentsWalkerSkipsNonAgentFiles cases fail on the unfixed
  walker (verified by reverting to the pre-fix file and re-running);
  pass cleanly after the fix.
- Full content-guardrail suite: 25/25 (23 existing + 2 new).
- Full pytest: 4189 passed, 25 skipped.

* release: 0.53.5 — content guardrail (flea-market submitter UX) + catalog ENTITY column + BQ hint dispatch

Bundles three threads landed in [Unreleased]:
- Vojta's flea-market content guardrail (two-tier mechanical + LLM)
- Zdeněk's `agnes catalog` ENTITY column replacement for FLAVOR
- Zdeněk's `/api/query` remote_estimate_failed hint dispatch fix

Plus the takeover hygiene from #276 review (agents walker preview/check
parity + body_too_short hint runtime kwarg consistency) and the
backslash-escape fix follow-up to v0.53.4 #275.

No DB migration; no API change. Patch upgrade lands transparently.
Upload form's new "Before you upload" disclosure + per-component preview
table appear on the next dev-VM auto-pull. Quarantine banner now groups
findings by file with "See <type> example ↗" deep-links to the new
/store/examples reference page.

---------

Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
2026-05-12 21:48:27 +02:00

1007 lines
42 KiB
Python

"""v37 flea-market edit feature with version history.
Covers:
* Bundle update bumps version_no + appends version_history entry.
* Metadata-only edit doesn't bump version.
* Type change rejected with 400 type_locked.
* Block-while-pending: 409 prior_version_pending.
* Display name change renames the on-disk slug for live + version dirs.
* Restore copies a prior version forward as v<max+1>; live + history
reflect the new version; original version row keeps its own verdict.
* Restore re-runs guardrails (blocked path leaves live untouched).
* Versions card on detail page renders for owner/admin only.
"""
from __future__ import annotations
import io
import json
import zipfile
from pathlib import Path
import pytest
from argon2 import PasswordHasher
from fastapi.testclient import TestClient
from app.utils import get_store_dir
from src.db import close_system_db, get_system_db
from src.repositories.store_entities import StoreEntitiesRepository
from src.repositories.users import UserRepository
# Strong default description that clears the content guardrail's
# per-component bar (30 chars + 4 distinct words, no placeholder
# leftovers). Tests don't assert on its contents — they just need a
# value that passes review so we can exercise the edit/version path.
_OK_DESC = "Use when validating store version edit flow across every guardrail tier"
@pytest.fixture
def web_client(tmp_path, monkeypatch):
monkeypatch.setenv("DATA_DIR", str(tmp_path))
monkeypatch.setenv("TESTING", "1")
monkeypatch.setenv("JWT_SECRET_KEY", "test-secret-key-min-32-characters!!")
(tmp_path / "state").mkdir()
(tmp_path / "analytics").mkdir()
(tmp_path / "extracts").mkdir()
close_system_db()
from app.main import create_app
app = create_app()
yield TestClient(app)
close_system_db()
def _create_user(client, email, password="UserPass1!"):
ph = PasswordHasher()
conn = get_system_db()
user_id = email.split("@")[0]
UserRepository(conn).create(
id=user_id, email=email, name=user_id, password_hash=ph.hash(password),
)
conn.close()
r = client.post("/auth/token", json={"email": email, "password": password})
assert r.status_code == 200, r.text
return user_id, {"access_token": r.json()["access_token"]}
def _create_admin(client, email="admin-edit@x.com"):
from tests.helpers.auth import grant_admin
user_id, cookies = _create_user(client, email, password="AdminPass1!")
conn = get_system_db()
grant_admin(conn, user_id)
conn.close()
return user_id, cookies
def _make_skill_zip(skill_name: str, body: str = "Body line explaining the skill. " * 12) -> bytes:
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
zf.writestr(
f"{skill_name}/SKILL.md",
f"---\nname: {skill_name}\ndescription: Use when verifying clean-bundle edits across the version-history lifecycle\n---\n\n"
+ body,
)
return buf.getvalue()
def _make_eval_skill_zip(skill_name: str) -> bytes:
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
zf.writestr(
f"{skill_name}/SKILL.md",
f"---\nname: {skill_name}\ndescription: Use when verifying static-security rejects eval-using upload bundles cleanly\n---\n\n"
+ ("Body line explaining the skill. " * 12),
)
zf.writestr(f"{skill_name}/run.sh", "#!/bin/sh\neval $1\n")
return buf.getvalue()
def _upload_clean(client, cookies, name="ed1"):
r = client.post(
"/api/store/entities",
files={"file": ("s.zip", _make_skill_zip(name), "application/zip")},
data={"type": "skill", "description": _OK_DESC}, cookies=cookies,
)
assert r.status_code == 201, r.text
return r.json()["id"]
class TestEditFeature:
def test_metadata_only_edit_no_version_bump(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "metaowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="metaedit")
r = web_client.put(
f"/api/store/entities/{eid}",
data={"description": "Updated description text"},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
entity = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert entity["description"] == "Updated description text"
assert entity["version_no"] == 1
assert len(entity["version_history"]) == 1
def test_bundle_edit_bumps_version_and_appends_history(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "bundleowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="bundleedit")
# PUT with new bundle bytes.
new_zip = _make_skill_zip("bundleedit", body="V2 body. " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", new_zip, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
entity = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert entity["version_no"] == 2
assert len(entity["version_history"]) == 2
v1, v2 = entity["version_history"]
assert v1["n"] == 1
assert v2["n"] == 2
assert v2["hash"] != v1["hash"], "v2 hash must differ from v1"
def test_type_change_rejected(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "typeowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="typelock")
r = web_client.put(
f"/api/store/entities/{eid}",
data={"type": "agent", "description": _OK_DESC},
cookies=owner_cookies,
)
assert r.status_code == 400, r.text
assert r.json()["detail"]["code"] == "type_locked"
def test_block_while_prior_pending_409(self, web_client):
"""Manually flip the entity to visibility=pending + create a
pending submission, then attempt edit → 409."""
from src.repositories.store_submissions import StoreSubmissionsRepository
owner_id, owner_cookies = _create_user(web_client, "blockowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="blockpending")
conn = get_system_db()
# Force pending state.
conn.execute(
"UPDATE store_entities SET visibility_status = 'pending' WHERE id = ?",
[eid],
)
StoreSubmissionsRepository(conn).create(
submitter_id=owner_id, submitter_email="blockowner@x.com",
type="skill", name="blockpending", version="2.0.0",
status="pending_llm", entity_id=eid,
inline_checks={"manifest": {"status": "pass"}},
)
conn.close()
r = web_client.put(
f"/api/store/entities/{eid}",
data={"description": "Trying to edit"},
cookies=owner_cookies,
)
assert r.status_code == 409, r.text
assert r.json()["detail"]["code"] == "prior_version_pending"
def test_name_change_renames_baked_slug(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "renameowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="oldname")
r = web_client.put(
f"/api/store/entities/{eid}",
data={"name": "newname"},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
entity = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert entity["name"] == "newname"
plugin_dir = Path(get_store_dir()) / eid / "plugin"
new_skill_dir = plugin_dir / "skills" / "newname-by-renameowner"
old_skill_dir = plugin_dir / "skills" / "oldname-by-renameowner"
assert new_skill_dir.is_dir(), "renamed slug missing on disk"
assert not old_skill_dir.exists(), "old slug must be gone"
class TestRestoreVersion:
def test_restore_creates_new_version_with_old_bundle(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "restoreowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="restoreme")
# Edit to v2.
v2_zip = _make_skill_zip("restoreme", body="VERSION-2-BODY " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2_zip, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
# Capture v1 + v2 hashes from history.
conn = get_system_db()
entity = StoreEntitiesRepository(conn).get(eid)
conn.close()
v1_hash = entity["version_history"][0]["hash"]
v2_hash = entity["version_history"][1]["hash"]
assert v1_hash != v2_hash
# Restore v1 → creates v3 with v1's bundle hash.
r = web_client.post(
f"/api/store/entities/{eid}/versions/1/restore",
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
entity = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert entity["version_no"] == 3
assert len(entity["version_history"]) == 3
v3 = entity["version_history"][2]
assert v3["n"] == 3
assert v3["hash"] == v1_hash, (
"restored bundle should hash identically to v1 — same bytes"
)
def test_restore_already_current_400(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "alreadyowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="already")
r = web_client.post(
f"/api/store/entities/{eid}/versions/1/restore",
cookies=owner_cookies,
)
assert r.status_code == 400, r.text
assert r.json()["detail"]["code"] == "already_current"
def test_restore_unknown_version_404(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "unknownver@x.com")
eid = _upload_clean(web_client, owner_cookies, name="ukver")
r = web_client.post(
f"/api/store/entities/{eid}/versions/99/restore",
cookies=owner_cookies,
)
assert r.status_code == 404, r.text
assert r.json()["detail"]["code"] == "version_not_found"
def test_non_owner_non_admin_cannot_restore(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "owrestore@x.com")
eid = _upload_clean(web_client, owner_cookies, name="ownedver")
v2 = _make_skill_zip("ownedver", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
_, snoop_cookies = _create_user(web_client, "snoopver@x.com")
r = web_client.post(
f"/api/store/entities/{eid}/versions/1/restore",
cookies=snoop_cookies,
)
assert r.status_code in (403, 404), r.text
class TestEditPage:
def test_edit_page_renders_for_owner(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "editpage@x.com")
eid = _upload_clean(web_client, owner_cookies, name="editrender")
r = web_client.get(
f"/marketplace/flea/{eid}/edit", cookies=owner_cookies,
)
assert r.status_code == 200
assert "edit-form" in r.text
assert "editrender" in r.text
def test_edit_page_404_for_non_owner(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "owneredit@x.com")
eid = _upload_clean(web_client, owner_cookies, name="snoopedit")
_, snoop_cookies = _create_user(web_client, "snoopedit@x.com")
r = web_client.get(
f"/marketplace/flea/{eid}/edit", cookies=snoop_cookies,
)
assert r.status_code == 404
def test_versions_card_renders_for_owner(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "vowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="vcard")
r = web_client.get(
f"/marketplace/flea/{eid}", cookies=owner_cookies,
)
assert r.status_code == 200
assert "versions-card" in r.text
assert "Versions (1)" in r.text
def test_versions_card_hidden_for_non_owner(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "vowner2@x.com")
eid = _upload_clean(web_client, owner_cookies, name="vhide")
_, other_cookies = _create_user(web_client, "vother@x.com")
r = web_client.get(
f"/marketplace/flea/{eid}", cookies=other_cookies,
)
assert r.status_code == 200
assert "versions-card" not in r.text
class TestInstallerAlwaysGetsLatestApproved:
"""Critical contract: existing installers continue receiving the
last APPROVED version through the review window of a new edit, and
NEVER receive an unapproved version. If the new version is blocked,
they keep the prior approved one. If approved, they advance.
Implemented via deferred promotion: PUT/restore append the new
version to history at status='pending_llm' but DO NOT swap live
or bump entity.version_no. runner.run_llm_review's approval branch
promotes; on block, nothing changes.
"""
def _install_as_user(self, web_client, owner_cookies, eid):
"""Install as a separate consumer user, return their list_for_user
rows post-install (mirrors what marketplace.zip serves)."""
from src.repositories.user_store_installs import UserStoreInstallsRepository
installer_id, installer_cookies = _create_user(web_client, "installer@x.com")
r = web_client.post(
f"/api/store/entities/{eid}/install",
cookies=installer_cookies,
)
assert r.status_code == 200, r.text
return installer_id, installer_cookies
def test_pending_review_does_not_break_existing_installer(self, web_client, monkeypatch):
"""Initial upload runs with guardrails OFF (lands approved).
Then we flip guardrails ON and PUT a new bundle. The new
version should defer promotion: existing installer must
continue seeing v1 + entity.version_no=1, not get hidden by a
flipped visibility or a half-promoted live dir."""
from app import instance_config as ic
# Stub LLM scheduling so the BG path never actually runs.
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
owner_id, owner_cookies = _create_user(web_client, "stickyowner@x.com")
# Phase 1: guardrails OFF → initial upload lands approved.
eid = _upload_clean(web_client, owner_cookies, name="sticky")
# Capture v1 hash + size baseline.
conn = get_system_db()
ent = StoreEntitiesRepository(conn).get(eid)
v1_hash = ent["version"]
v1_size = ent["file_size"]
conn.close()
# Install as a different user.
installer_id, _ = self._install_as_user(web_client, owner_cookies, eid)
# Phase 2: flip guardrails ON for the PUT call. Now an edit
# defers promotion until LLM approves.
# Patch where update_entity looks it up — `from app.instance_config
# import get_guardrails_enabled` binds the symbol into app.api.store,
# so patching the source module isn't enough.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
# PUT a new bundle. Guardrails on → submission lands at
# pending_llm; promotion deferred.
v2_zip = _make_skill_zip("sticky", body="V2 BODY " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2_zip, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
# Installer's list_for_user must STILL see entity with v1 hash
# + size + visibility='approved'. The pending edit must not
# have hidden the entity from them.
from src.repositories.user_store_installs import UserStoreInstallsRepository
conn = get_system_db()
installs = UserStoreInstallsRepository(conn).list_for_user(installer_id)
ids = {r["id"] for r in installs}
assert eid in ids, (
"installer lost access to the entity during the LLM "
"review window — list_for_user filter excluded it"
)
row = next(r for r in installs if r["id"] == eid)
assert row["version"] == v1_hash, (
f"installer should still get v1 hash {v1_hash[:8]} but "
f"got {row['version'][:8]}"
)
assert row["file_size"] == v1_size, "size shouldn't change pre-promotion"
assert row["visibility_status"] == "approved", (
"entity must stay 'approved' through the review window so "
"existing installers continue serving"
)
# Entity row's version_no must NOT have bumped yet.
ent = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert ent["version_no"] == 1, (
f"version_no must stay 1 during pending review; got {ent['version_no']}"
)
# But version_history MUST have v2 entry tracked (with the
# new hash) so admin can see what's in flight.
history_n = [int(e["n"]) for e in ent["version_history"]]
assert 2 in history_n, "v2 entry must be in history despite no promotion"
def test_blocked_new_version_keeps_installer_on_prior(self, web_client, monkeypatch):
"""Mock the LLM to BLOCK the v2 review. Installer must keep
v1; entity.version_no must stay at 1; live plugin/ must hold
v1's bytes."""
from app import instance_config as ic
# Mock the runner's LLM call to return a high-risk verdict.
def mock_review_bundle(*args, **kwargs):
return {
"risk_level": "high",
"summary": "mock block",
"findings": [{"severity": "high", "category": "test",
"file": "x", "explanation": "mock"}],
"template_placeholders_found": 0,
"reviewed_by_model": "mock-model",
"error": None,
}
monkeypatch.setattr(
"src.store_guardrails.llm_review.review_bundle",
mock_review_bundle,
)
# Phase 1: initial upload guardrails OFF.
owner_id, owner_cookies = _create_user(web_client, "blockowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="blocksticky")
# Phase 2: switch guardrails ON before PUT.
# Patch where update_entity looks it up — `from app.instance_config
# import get_guardrails_enabled` binds the symbol into app.api.store,
# so patching the source module isn't enough.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
conn = get_system_db()
v1_hash = StoreEntitiesRepository(conn).get(eid)["version"]
conn.close()
installer_id, _ = self._install_as_user(web_client, owner_cookies, eid)
# Edit. Inline checks pass; LLM mocked to block.
v2_zip = _make_skill_zip("blocksticky", body="v2-content " * 80)
# Run the LLM synchronously by calling runner directly after
# the PUT (the BG task may not have fired in TestClient).
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2_zip, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
# Find the just-created submission + run runner against it.
from src.repositories.store_submissions import StoreSubmissionsRepository
from src.store_guardrails.runner import run_llm_review
from app.utils import get_store_dir
conn = get_system_db()
sub_id = StoreSubmissionsRepository(conn).latest_for_entity(eid)["id"]
conn.close()
run_llm_review(
sub_id,
plugin_dir=Path(get_store_dir()) / eid / "versions" / "v2" / "plugin",
conn_factory=get_system_db,
api_key_loader=lambda: "sk-test",
model_loader=lambda: "claude-haiku-4-5-20251001",
)
# Installer must STILL see v1.
from src.repositories.user_store_installs import UserStoreInstallsRepository
conn = get_system_db()
installs = UserStoreInstallsRepository(conn).list_for_user(installer_id)
ent = StoreEntitiesRepository(conn).get(eid)
conn.close()
row = next(r for r in installs if r["id"] == eid)
assert row["version"] == v1_hash, (
f"after v2 blocked, installer must still get v1 hash; got {row['version'][:8]}"
)
assert ent["version_no"] == 1, (
f"version_no must stay at 1 after a blocked verdict; got {ent['version_no']}"
)
def test_approved_new_version_promotes_to_installer(self, web_client):
"""Default test path: guardrails OFF → guardrails-disabled
promote-inline branch fires immediately. Installer's next
list_for_user reflects v2."""
owner_id, owner_cookies = _create_user(web_client, "promoowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="promosticky")
conn = get_system_db()
v1_hash = StoreEntitiesRepository(conn).get(eid)["version"]
conn.close()
installer_id, _ = self._install_as_user(web_client, owner_cookies, eid)
v2_zip = _make_skill_zip("promosticky", body="promo-v2 " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2_zip, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
# Installer should now see v2 hash.
from src.repositories.user_store_installs import UserStoreInstallsRepository
conn = get_system_db()
installs = UserStoreInstallsRepository(conn).list_for_user(installer_id)
ent = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert ent["version_no"] == 2
row = next(r for r in installs if r["id"] == eid)
assert row["version"] != v1_hash, "installer should advance to v2"
class TestAdminAccess:
"""Admin can edit + restore on entities they don't own (parity with
the existing admin override path)."""
def test_admin_can_edit_non_owned_entity_metadata(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "adminedit-owner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="adminedit")
_, admin_cookies = _create_admin(web_client)
r = web_client.put(
f"/api/store/entities/{eid}",
data={"description": "moderated by admin"},
cookies=admin_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
ent = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert ent["description"] == "moderated by admin"
assert ent["version_no"] == 1
def test_admin_can_restore_non_owned_entity(self, web_client):
owner_id, owner_cookies = _create_user(web_client, "adminrestore-owner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="adminrestore")
v2 = _make_skill_zip("adminrestore", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
_, admin_cookies = _create_admin(web_client)
r = web_client.post(
f"/api/store/entities/{eid}/versions/1/restore",
cookies=admin_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
ent = StoreEntitiesRepository(conn).get(eid)
conn.close()
assert ent["version_no"] == 3
class TestRestoreDeferredPromotion:
"""Restore endpoint mirrors PUT semantics: live + version_no stay
on prior current until LLM approves the restored copy."""
def test_restore_with_guardrails_on_does_not_promote_until_approved(
self, web_client, monkeypatch,
):
"""Owner restores v1 → restored bytes baked into v3 dir.
Until LLM approves, live + version_no stay at v2."""
owner_id, owner_cookies = _create_user(web_client, "restoreowner-defer@x.com")
eid = _upload_clean(web_client, owner_cookies, name="restoredefer")
v2 = _make_skill_zip("restoredefer", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
# v2 promoted (guardrails off). Now flip on for the restore.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
r = web_client.post(
f"/api/store/entities/{eid}/versions/1/restore",
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
conn = get_system_db()
ent = StoreEntitiesRepository(conn).get(eid)
conn.close()
# version_no must STAY at 2 until LLM approves the v3 (restored)
# copy. version_history has 3 entries; current is still 2.
assert ent["version_no"] == 2, (
f"restore must defer promotion when guardrails on; "
f"version_no={ent['version_no']}"
)
history_n = sorted([int(e["n"]) for e in ent["version_history"]])
assert history_n == [1, 2, 3]
class TestEditPageBanner:
"""Detail page banner during pending edit review must surface
the version under review + the prior version still serving."""
def test_banner_shows_version_n_under_review(self, web_client, monkeypatch):
from src.repositories.store_submissions import StoreSubmissionsRepository
owner_id, owner_cookies = _create_user(web_client, "bannerowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="bannerver")
# Switch guardrails on; stub LLM scheduler so v2 stays pending.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
v2 = _make_skill_zip("bannerver", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
# v2 review pending. visibility on entity stays approved (per
# the deferred-promotion fix). But banner partial reads sub
# status — for an in-flight edit submission the banner won't
# render unless visibility != approved. Lock the scenario:
# ensure entity stays at version_no=1 + visibility approved.
conn = get_system_db()
ent = StoreEntitiesRepository(conn).get(eid)
sub = StoreSubmissionsRepository(conn).latest_for_entity(eid)
conn.close()
assert ent["version_no"] == 1
assert ent["visibility_status"] == "approved"
assert sub["status"] == "pending_llm"
# Detail page renders. Banner partial only fires when
# visibility_status != approved, so for a deferred-edit case
# the marketplace detail does NOT render the quarantine
# banner — that's correct UX (consumers see the entity as
# approved and operational). Owner-facing review status
# surfaces via the Edit button being disabled.
r = web_client.get(
f"/marketplace/flea/{eid}", cookies=owner_cookies,
)
assert r.status_code == 200
# Edit button should reflect the in-flight review (locked).
assert "review in flight" in r.text or "Edit (review in flight)" in r.text
class TestAuditLogPerVersion:
"""Each edit / restore writes audit rows carrying the version_no
in params, so the entity timeline can attribute events to the
right version."""
def test_edit_audit_carries_version_no(self, web_client):
from src.repositories.audit import AuditRepository
owner_id, owner_cookies = _create_user(web_client, "auditver@x.com")
eid = _upload_clean(web_client, owner_cookies, name="auditver")
v2 = _make_skill_zip("auditver", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
conn = get_system_db()
rows = AuditRepository(conn).query_for_resources(
[f"store_entity:{eid}"], limit=20,
)
conn.close()
# store.entity.update event for the edit must carry version_no
# in its params.
update_rows = [r for r in rows if r.get("action") == "store.entity.update"]
assert update_rows, "missing store.entity.update audit"
assert any(
(r.get("params") or {}).get("version_no") == 2
for r in update_rows
), "update audit must carry version_no=2 in params"
def test_restore_audit_carries_versions(self, web_client):
from src.repositories.audit import AuditRepository
owner_id, owner_cookies = _create_user(web_client, "auditrestore@x.com")
eid = _upload_clean(web_client, owner_cookies, name="auditrest")
v2 = _make_skill_zip("auditrest", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
web_client.post(
f"/api/store/entities/{eid}/versions/1/restore",
cookies=owner_cookies,
)
conn = get_system_db()
rows = AuditRepository(conn).query_for_resources(
[f"store_entity:{eid}"], limit=20,
)
conn.close()
restore_rows = [r for r in rows if r.get("action") == "store.entity.restore"]
assert restore_rows, "missing store.entity.restore audit"
params = restore_rows[0].get("params") or {}
assert params.get("restored_from_version_no") == 1
assert params.get("new_version_no") == 3
class TestPRReviewFixes:
"""Locks in the fixes called out in the PR #239 review."""
def test_block_while_pending_fires_for_v2_edit_under_deferred_promotion(
self, web_client, monkeypatch,
):
"""#1 — v2+ edit during in-flight LLM review must 409 even
though entity.visibility_status is still 'approved'."""
from src.repositories.store_submissions import StoreSubmissionsRepository
owner_id, owner_cookies = _create_user(web_client, "blockv2@x.com")
eid = _upload_clean(web_client, owner_cookies, name="blockv2")
# Switch guardrails on for the edit so promotion defers.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
v2 = _make_skill_zip("blockv2", body="v2 " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
# Entity should remain at 'approved' visibility — v2 in flight.
conn = get_system_db()
ent = StoreEntitiesRepository(conn).get(eid)
sub = StoreSubmissionsRepository(conn).latest_for_entity(eid)
conn.close()
assert ent["visibility_status"] == "approved"
assert sub["status"] == "pending_llm"
# Second concurrent edit MUST be blocked.
v3 = _make_skill_zip("blockv2", body="v3 " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v3.zip", v3, "application/zip")},
cookies=owner_cookies,
)
assert r.status_code == 409, r.text
assert r.json()["detail"]["code"] == "prior_version_pending"
def test_name_change_with_bundle_does_not_rename_live_until_promote(
self, web_client, monkeypatch,
):
"""#2 — name + bundle in same PUT must NOT rename live until
the LLM approves and promotion runs. Existing installer keeps
getting the prior bundle under the prior slug."""
from app.utils import get_store_dir
owner_id, owner_cookies = _create_user(web_client, "rename-defer@x.com")
eid = _upload_clean(web_client, owner_cookies, name="origname")
plugin_dir = Path(get_store_dir()) / eid / "plugin"
old_skill_dir = plugin_dir / "skills" / "origname-by-rename-defer"
assert old_skill_dir.is_dir()
# Enable guardrails so promotion defers.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
v2 = _make_skill_zip("origname", body="v2 " * 80)
r = web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
data={"name": "newname"},
cookies=owner_cookies,
)
assert r.status_code == 200, r.text
# Live skill dir MUST still hold the old slug — promotion
# hasn't fired since we stubbed _schedule_llm_review.
assert old_skill_dir.is_dir(), (
"live skill dir was renamed before LLM approval — "
"violates deferred-promotion contract"
)
new_skill_dir = plugin_dir / "skills" / "newname-by-rename-defer"
assert not new_skill_dir.exists(), (
"live shouldn't have the renamed slug yet"
)
# Version dir HAS been renamed (so promotion will land on the
# new slug).
v2_dir = (
Path(get_store_dir()) / eid / "versions" / "v2" / "plugin"
/ "skills" / "newname-by-rename-defer"
)
assert v2_dir.is_dir(), "version dir should carry the new slug"
def test_v2_approval_logs_approved_not_skipped(
self, web_client, monkeypatch,
):
"""#3 — v2+ approvals must log store.submission.approved, NOT
store.submission.bg_verdict_skipped. Pre-fix the runner used
the visibility-flip return value to gate the audit; under
deferred promotion v2+ never flips visibility (already
'approved'), so the wrong audit was emitted."""
from src.repositories.audit import AuditRepository
from src.repositories.store_submissions import StoreSubmissionsRepository
from src.store_guardrails.runner import run_llm_review
from app.utils import get_store_dir
owner_id, owner_cookies = _create_user(web_client, "auditv2@x.com")
eid = _upload_clean(web_client, owner_cookies, name="auditv2")
# Enable guardrails for the edit. Stub LLM scheduler so we
# control when the runner fires.
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
# Mock the LLM call to return a safe verdict.
monkeypatch.setattr(
"src.store_guardrails.llm_review.review_bundle",
lambda *a, **kw: {
"risk_level": "safe", "summary": "ok",
"findings": [], "template_placeholders_found": 0,
"reviewed_by_model": "mock", "error": None,
},
)
v2 = _make_skill_zip("auditv2", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
# Manually fire the runner against the v2 dir.
conn = get_system_db()
sub_id = StoreSubmissionsRepository(conn).latest_for_entity(eid)["id"]
conn.close()
run_llm_review(
sub_id,
plugin_dir=Path(get_store_dir()) / eid / "versions" / "v2" / "plugin",
conn_factory=get_system_db,
api_key_loader=lambda: "sk-test",
model_loader=lambda: "claude-haiku-4-5-20251001",
)
# Audit log must contain store.submission.approved with
# promoted_to_version_no=2; NO bg_verdict_skipped.
conn = get_system_db()
rows = AuditRepository(conn).query_for_resources(
[f"store_submission:{sub_id}"], limit=20,
)
conn.close()
actions = [r.get("action") for r in rows]
assert "store.submission.approved" in actions, (
f"v2 approval missing approved audit; got {actions}"
)
assert "store.submission.bg_verdict_skipped" not in actions, (
f"v2 approval should NOT log bg_verdict_skipped; got {actions}"
)
approved_row = next(
r for r in rows if r.get("action") == "store.submission.approved"
)
params = approved_row.get("params") or {}
assert params.get("promoted_to_version_no") == 2
def test_bg_verdict_skipped_when_admin_archives_during_review(
self, web_client, monkeypatch,
):
"""Negative: when admin DOES archive mid-review, the runner
correctly logs bg_verdict_skipped (not approved)."""
from src.repositories.audit import AuditRepository
from src.repositories.store_submissions import StoreSubmissionsRepository
from src.repositories.store_entities import StoreEntitiesRepository
from src.store_guardrails.runner import run_llm_review
from app.utils import get_store_dir
owner_id, owner_cookies = _create_user(web_client, "archmid@x.com")
eid = _upload_clean(web_client, owner_cookies, name="archmid")
monkeypatch.setattr("app.api.store.get_guardrails_enabled", lambda: True)
monkeypatch.setattr(
"app.api.store._schedule_llm_review",
lambda *a, **kw: None,
)
monkeypatch.setattr(
"src.store_guardrails.llm_review.review_bundle",
lambda *a, **kw: {
"risk_level": "safe", "summary": "ok",
"findings": [], "template_placeholders_found": 0,
"reviewed_by_model": "mock", "error": None,
},
)
v2 = _make_skill_zip("archmid", body="v2 " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
# Admin archives BEFORE runner fires.
conn = get_system_db()
StoreEntitiesRepository(conn).archive(eid, by_user_id="admin-x")
sub_id = StoreSubmissionsRepository(conn).latest_for_entity(eid)["id"]
conn.close()
run_llm_review(
sub_id,
plugin_dir=Path(get_store_dir()) / eid / "versions" / "v2" / "plugin",
conn_factory=get_system_db,
api_key_loader=lambda: "sk-test",
model_loader=lambda: "claude-haiku-4-5-20251001",
)
conn = get_system_db()
rows = AuditRepository(conn).query_for_resources(
[f"store_submission:{sub_id}"], limit=20,
)
conn.close()
actions = [r.get("action") for r in rows]
assert "store.submission.bg_verdict_skipped" in actions, (
f"archive-during-review must log bg_verdict_skipped; got {actions}"
)
assert "store.submission.approved" not in actions, (
f"archive-during-review must NOT log approved; got {actions}"
)
class TestAdminQueueShowsVersion:
def test_admin_queue_shows_v_no_after_name(self, web_client):
"""v# column derives version_no from entity.version_history by
matching submission.version (hash) against the entry hashes."""
owner_id, owner_cookies = _create_user(web_client, "vqowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="vqcol")
v2 = _make_skill_zip("vqcol", body="v2 body. " * 80)
web_client.put(
f"/api/store/entities/{eid}",
files={"file": ("v2.zip", v2, "application/zip")},
cookies=owner_cookies,
)
_, admin_cookies = _create_admin(web_client)
r = web_client.get(
"/api/admin/store/submissions", cookies=admin_cookies,
)
assert r.status_code == 200
items = {it["name"]: it for it in r.json()["items"]}
assert "vqcol" in items
# version_no derived for the v2 row should be 2.
# The list returns rows newest-first; pick the v2 (current).
v2_row = next(
it for it in r.json()["items"]
if it.get("entity_id") == eid and it.get("version_no") == 2
)
assert v2_row["version_no"] == 2
assert v2_row["entity_version_no"] == 2
def test_admin_detail_shows_version_no(self, web_client):
"""Detail page renders v# under the Status / Entity-lifecycle
block + a separate Bundle hash row."""
from src.repositories.store_submissions import StoreSubmissionsRepository
owner_id, owner_cookies = _create_user(web_client, "vdowner@x.com")
eid = _upload_clean(web_client, owner_cookies, name="vdrow")
conn = get_system_db()
sub_id = StoreSubmissionsRepository(conn).latest_for_entity(eid)["id"]
conn.close()
_, admin_cookies = _create_admin(web_client)
r = web_client.get(
f"/admin/store/submissions/{sub_id}", cookies=admin_cookies,
)
assert r.status_code == 200
body = r.text
assert "<dt>Version</dt>" in body
assert "<dt>Bundle hash</dt>" in body
assert "v1" in body