fb6e930bc9
* feat(store-guardrails): enforce per-component description quality
Two-tier hard guardrail on flea-market submissions. Empty / placeholder /
single-word descriptions now block before any LLM call; vague-but-passes-
floor descriptions block on the substantive LLM review layer.
Tier 1 — inline mechanical check (src/store_guardrails/content_check.py).
Walks the baked plugin tree, evaluates each component (plugin manifest,
agents, skills, commands) plus the submission-level form description
against a 60-char / 25-char (commands) / 5-distinct-word / 200-char-body
floor with a placeholder denylist (TODO, TBD, {{var}}, etc.). Floors
calibrated against real ecosystem norms: Claude / superpowers /
compound-engineering skill packs cluster 150–220 chars, npm / Docker /
VS Code at 100–120. InlineResult.passed now ANDs in content.status.
Tier 2 — LLM review extension (prompts.py + llm_review.py). System
prompt gains a content-quality criterion; REVIEW_JSON_SCHEMA carries a
content_quality {verdict, issues[]} object alongside the existing
security findings. is_safe() requires content_quality.verdict == 'pass'.
Single LLM call covers both dimensions. MAX_RESPONSE_TOKENS bumped
2000 → 2500 for the extra payload. Verdicts missing content_quality
treated as pass (backwards compat with already-recorded rows).
Submitter UX:
- /store/new wizard now carries a "Before you upload — what passes
review" collapsible disclosure on both step 1 and step 2 with the
bar + patterns that work. Live char counter on the description
field. Per-component preview table (green/red dots from the new
summarize_for_preview helper) renders after the ZIP /preview round
trip, scoping each finding to its file.
- New /store/examples page with rejected/passes pairs for skill /
agent / plugin / command plus a "Why these limits" research table.
Anchored sections (#skill / #agent / #plugin / #command) so the
rejection banner can deep-link by component_type.
- Quarantine banner _content_findings.html groups findings by file
(one "See <type> example ↗" per component, not per field) and
translates field codes (frontmatter.description / body / etc.) to
plain-English labels. _content_howto_fix.html surfaces a static
"Re-upload as new version" + "See examples" action row beneath any
content failure on the entity detail page.
- _parse_frontmatter moved to src/store_guardrails/_frontmatter.py so
the new check module shares the parser without inverting the
app → src dependency direction.
Tests:
- New tests/test_store_guardrails_content.py (29 cases) covering
every failure code per component type plus submission-level checks
and the summarize_components / summarize_for_preview helpers.
- Extended test_store_guardrails_inline.py for the new
InlineResult.content field + aggregate behaviour.
- Extended test_store_guardrails_llm.py for the new
content_quality verdict pathways (fail blocks, missing field passes).
- Backfilled fixture descriptions across test_store_api.py,
test_store_entity_versions.py, test_store_put_atomic.py,
test_admin_store_submissions.py, test_marketplace_api.py,
test_marketplace_v32_endpoints.py so existing happy-path tests
clear the new 60-char floor.
* fix(content-guardrail): align agents walker with preview + drop import-time .format()
Two cleanups from the takeover review on #276 (vr/guardrails-content).
1) `_iter_components` for agents now skips files lacking frontmatter
(no `name` AND no `description`). Pre-fix the walker greedily
evaluated every `*.md` under `agents/` — `agents/README.md` and
helper docs got flagged as "frontmatter.description empty"
rejections. Worse: `summarize_for_preview` for `type=agent` ALREADY
filters the same shape, so the upload preview gave a green dot
while the post-bake check gave a red rejection on submit. Two new
regression tests in TestAgentsWalkerSkipsNonAgentFiles pin both
shapes (README + _NOTES.md) so the preview/check parity stays
aligned.
2) `body_too_short` hints now use the same runtime-kwarg substitution
pattern as every other hint in the table. Pre-fix the skill +
agent body_too_short hints called `.format(min_chars=_MIN_BODY_CHARS)`
at module-load time, but the call site `_hint_for(type_,
"body_too_short")` didn't pass `min_chars=`, so the format() was
just baking the constant at import. Cosmetic inconsistency; pass
`min_chars=_MIN_BODY_CHARS` at the call site instead and let
`_hint_for` do the substitution like it does for `too_short`.
Verified end-to-end:
- New TestAgentsWalkerSkipsNonAgentFiles cases fail on the unfixed
walker (verified by reverting to the pre-fix file and re-running);
pass cleanly after the fix.
- Full content-guardrail suite: 25/25 (23 existing + 2 new).
- Full pytest: 4189 passed, 25 skipped.
* release: 0.53.5 — content guardrail (flea-market submitter UX) + catalog ENTITY column + BQ hint dispatch
Bundles three threads landed in [Unreleased]:
- Vojta's flea-market content guardrail (two-tier mechanical + LLM)
- Zdeněk's `agnes catalog` ENTITY column replacement for FLAVOR
- Zdeněk's `/api/query` remote_estimate_failed hint dispatch fix
Plus the takeover hygiene from #276 review (agents walker preview/check
parity + body_too_short hint runtime kwarg consistency) and the
backslash-escape fix follow-up to v0.53.4 #275.
No DB migration; no API change. Patch upgrade lands transparently.
Upload form's new "Before you upload" disclosure + per-component preview
table appear on the next dev-VM auto-pull. Quarantine banner now groups
findings by file with "See <type> example ↗" deep-links to the new
/store/examples reference page.
---------
Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
80 lines
3.5 KiB
HTML
80 lines
3.5 KiB
HTML
{# Inline content-check findings rendered inside _quarantine_banner.html.
|
|
Expects `ic.content.issues` in scope — each issue has file, field,
|
|
code, hint, optional name. Grouped by file. Field + code names
|
|
translated to plain English so a non-developer reader still
|
|
understands what the problem is and which line to edit.
|
|
#}
|
|
{% set issues = ic.content.issues %}
|
|
|
|
{# {file → [issue, ...]} grouping preserving first-seen order. #}
|
|
{% set grouped = namespace(map={}, order=[]) %}
|
|
{% for issue in issues %}
|
|
{% set _ = grouped.map.setdefault(issue.file, []) %}
|
|
{% if issue.file not in grouped.order %}{% set _ = grouped.order.append(issue.file) %}{% endif %}
|
|
{% set _ = grouped.map[issue.file].append(issue) %}
|
|
{% endfor %}
|
|
|
|
{# Field-label → plain English. Defined inline so the template stays
|
|
the single source of truth for what the submitter sees. #}
|
|
{% set field_label = {
|
|
'frontmatter.description': 'Description (the line at the top of the file, after `description:`)',
|
|
'plugin.json.description': 'Description (the `description` field in `.claude-plugin/plugin.json`)',
|
|
'description': 'Description (on the upload form)',
|
|
'body': 'Content (the rest of the file after the description line)',
|
|
} %}
|
|
{% set code_label = {
|
|
'empty': 'is missing',
|
|
'too_short': 'is too short',
|
|
'low_word_count': 'needs more distinct words',
|
|
'placeholder_text': 'still has placeholder text (TODO, template, etc.)',
|
|
'body_too_short': 'is too short',
|
|
} %}
|
|
{% set component_label = {
|
|
'skill': 'skill',
|
|
'agent': 'agent',
|
|
'plugin': 'plugin',
|
|
'command': 'command',
|
|
'submission': 'description',
|
|
} %}
|
|
|
|
<div style="margin-top: 10px; font-weight: 600;">What needs fixing</div>
|
|
<div style="font-size: 12px; opacity: 0.75; margin-bottom: 6px;">
|
|
Each block below is one part of your upload that needs more text.
|
|
Open the example to see exactly which line / field to change.
|
|
</div>
|
|
<ul style="list-style: none; padding-left: 0;">
|
|
{% for file in grouped.order[:12] %}
|
|
{% set file_issues = grouped.map[file] %}
|
|
{% set first = file_issues[0] %}
|
|
{% set anchor = first.component_type or 'skill' %}
|
|
{% set anchor_label = component_label.get(anchor, 'description') %}
|
|
<li style="margin-bottom: 12px; padding: 8px 10px; background: rgba(0,0,0,0.03); border-radius: 6px;">
|
|
<div>
|
|
<strong>
|
|
{% if anchor == 'submission' %}Description on the upload form
|
|
{% else %}{{ anchor_label|capitalize }}{% if first.name %} <span style="opacity: 0.7;">— {{ first.name }}</span>{% endif %}
|
|
{% endif %}
|
|
</strong>
|
|
<span style="opacity: 0.6; font-size: 11px;">
|
|
{% if anchor != 'submission' %}<code>{{ file }}</code>{% endif %}
|
|
</span>
|
|
<a href="/store/examples#{{ anchor }}" target="_blank" rel="noopener"
|
|
style="margin-left: 8px; color: inherit; text-decoration: underline; font-weight: 600; font-size: 12px;">
|
|
See {{ anchor_label }} example ↗
|
|
</a>
|
|
</div>
|
|
<ul style="margin: 6px 0 0 0; padding-left: 18px; font-size: 12px;">
|
|
{% for issue in file_issues %}
|
|
<li style="margin-bottom: 6px;">
|
|
<strong>{{ field_label.get(issue.field, issue.field) }}</strong>
|
|
<span style="opacity: 0.8;">{{ code_label.get(issue.code, issue.code|replace('_', ' ')) }}.</span>
|
|
{% if issue.hint %}<div style="opacity: 0.85; margin-top: 3px;">{{ issue.hint }}</div>{% endif %}
|
|
</li>
|
|
{% endfor %}
|
|
</ul>
|
|
</li>
|
|
{% endfor %}
|
|
</ul>
|
|
{% if grouped.order|length > 12 %}
|
|
<div style="font-size: 12px; opacity: 0.7;">… and {{ grouped.order|length - 12 }} more.</div>
|
|
{% endif %}
|