* fix(store): surface review failures + harden publish gate
Four independent fixes to the flea-market submission pipeline, all surfaced
by an admin upload that landed at status='approved' without an LLM review.
1. LLM truncation no longer pins submissions in review_error.
- Raised MAX_RESPONSE_TOKENS 2500 → 6000 in llm_review.py
- Added one-shot retry-with-doubled-budget in anthropic_provider.py
(capped at 4× initial)
2. Flea detail page surfaces the latest submission's failure verdict even
when a previously-approved version is still serving (deferred-promotion
path). The _quarantine_banner gate widened from `visibility != approved`
to also fire on `blocked_inline / blocked_llm / review_error`, with copy
that distinguishes the v2+ edit case ("Latest edit failed review —
previously approved version (vN) keeps serving") from the initial-upload
quarantine wording.
3. Restore button + endpoint no longer allow restoring a version that was
never approved. Added StoreEntitiesRepository.get_with_version_approvals
joining store_submissions, gated the UI button on submission_status in
('approved', None), rendered status pills for non-restorable rows, and
added a 400 version_not_approved guard in POST /restore.
4. **BREAKING (operator-facing)**: publish gate is now fail-CLOSED on
misconfig. The previous get_guardrails_enabled() silently fell back to
"disabled, auto-approve everything" when guardrails.enabled=true in YAML
but no ANTHROPIC_API_KEY was in env. Split into:
- get_guardrails_enabled() (intent — YAML)
- get_guardrails_llm_provider_ready() (readiness — env)
Three-state matrix:
enabled=false → auto-approve (unchanged)
enabled=true + ready=true → normal pipeline (unchanged)
enabled=true + ready=false (NEW) → submissions hold at pending_llm
awaiting admin retry or override
(was: silent auto-approve)
Admin "Retry review" eligibility broadened to include pending_llm.
Boot-time WARNING banner surfaces the misconfig in app/main.py.
docs/STORE_GUARDRAILS.md updated with the three-state matrix.
Operators relying on the auto-fallback for local-dev no-LLM setups must
now explicitly set `guardrails.enabled: false` in instance.yaml.
Tests: 4623 passed. Added TestPublishGateFailClosed (4 tests) and
TestRestoreVersion::test_restore_rejects_* (3 tests). conftest.py adds an
autouse fixture defaulting guardrails OFF so legacy tests don't need to
know about the new toggle.
* fix(store): admin override promotes v2+ edits to current
The override handler at app/api/admin.py:3708 only flipped submission
status → 'overridden' and entity visibility → 'approved'. Under the v37+
deferred-promotion model that's insufficient for v2+ edits / restores:
the new bundle sits in versions/v<N>/plugin/ and the entity row stays at
the prior approved version_no + hash + on-disk live bundle. Installers
kept getting the OLD bytes the admin had just intended to replace.
Mirror the runner.run_llm_review auto-approval branch: look up the
submission's version_hash in entity.version_history, and if its `n`
differs from entity.version_no, promote_version + _swap_live_to_version.
Initial v1 overrides are unaffected — the loop finds n=1 == version_no
and skips promotion.
Tests:
- test_override_v2_edit_promotes_to_current: stage v1 approved + v2
blocked_llm; override the v2 sub; assert entity.version_no=2,
entity.version flips off the v1 hash, and the live plugin/ dir
mirrors versions/v2/plugin/.
- test_override_v1_initial_upload_no_promote: regression guard so the
promote loop doesn't accidentally bump a v1 override.
Audit log gains a promoted_to_version_no field on the override action.
* fix(store): retry/rescan review staged bundle; override forward-only
Two adversarial-review findings from a Codex pass on the publish-gate
work.
C1. Admin retry + rescan were passing live `plugin/` to the LLM. For a
v2+ submission held at `pending_llm` / `blocked_llm` / `review_error`,
live still holds the prior approved version's bytes — so the LLM
reviewed the WRONG bytes, and the runner's hash-match promotion in
`run_llm_review` would then advance the entity to staged bytes that
were never actually reviewed. Resolve the staged
`<entity>/versions/v<N>/plugin/` from the submission's
`version_history` entry, with a fall-back to live for legacy pre-v37
rows that never seeded a versions/ dir. Helpers
`_submission_plugin_dir` and `_version_no_for_submission` added to
`app/api/store.py` so override / retry / rescan share one path.
H1. Override's promote loop used `target != current`, which would
silently demote the live bundle when admin overrode a stale v2
submission while v3 was already approved + live. Changed to
`target > current` so override flips status + visibility on the row
regardless, but on-disk promotion only fires forward. Same `>`
defensive guard applied in `runner.run_llm_review` so a late LLM
verdict racing with a newer approval can't demote either.
Tests:
- TestAdminRetryReviewsStagedBundle::test_retry_v2_blocked_passes_staged_dir_not_live
- TestAdminRetryReviewsStagedBundle::test_rescan_v2_blocked_passes_staged_dir_not_live
- TestOverrideForwardOnly::test_override_stale_v2_does_not_demote_when_v3_current
* review polish: CHANGELOG drift, override eligibility, defensive copy
Three small additions on top of the retry/rescan staged-bundle fix:
1. CHANGELOG: the PR's bullets had drifted into the released
[0.54.17] section during rebase (context-match landed them next
to already-released content). Moved them up to [Unreleased] where
they belong; [0.54.17] now holds only what was actually released
(refresh-marketplace ls-remote, /me/activity hero, CI sharding +
workflow polish).
2. app/api/admin.py: admin override eligibility now accepts
pending_llm alongside blocked_inline + blocked_llm + review_error.
Closes a UX gap from the new fail-CLOSED behavior: under
enabled-but-not-ready, a known-good submission would otherwise
sit indefinitely until the admin set credentials AND clicked
Retry. Override already routes through version_history (and is
now forward-only on promote), so it stays safe for v2+ deferred-
promotion submissions.
3. src/repositories/store_entities.py: get_with_version_approvals
defensively copies each version_history entry before annotating
with submission_status. self.get() re-parses JSON each call today
so this is belt-and-suspenders against any future caching layer
leaking the annotated key into a subsequent plain get() call.
Tests: 112 passed (focused on test_store_entity_versions +
test_admin_store_submissions, covering the retry/rescan staged-
bundle fix the author shipped + this polish).
---------
Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
329 lines
13 KiB
HTML
329 lines
13 KiB
HTML
{# Shared quarantine banner partial.
|
|
|
|
Surfaces submission status (under review / quarantined / hidden /
|
|
override-applied) to the entity owner + admins. Self-guarded so it's
|
|
safe to {% include %} from any detail page — renders nothing when
|
|
the entity is approved or the viewer isn't owner/admin.
|
|
|
|
Required scope:
|
|
entity — store_entities row (must carry visibility_status,
|
|
visibility_status; entity.id surfaces in admin
|
|
detail link)
|
|
quarantine_sub — latest store_submissions row for entity, or None
|
|
is_owner — bool, viewer == entity.owner_user_id
|
|
is_admin — bool, viewer is in Admin group
|
|
|
|
Mirror of the version that previously lived in store_detail.html.
|
|
Wording stays consistent with the per-status messaging the user
|
|
approved earlier — only the rendering location changed.
|
|
#}
|
|
|
|
{# Gate widened for failure surfacing: under deferred promotion
|
|
(v37+), a v2+ edit can leave the entity approved at the prior
|
|
version while the latest submission landed in `review_error` /
|
|
`blocked_llm` / `blocked_inline`. The original
|
|
`visibility_status != 'approved'` gate silently hid those
|
|
failures from the owner. Render the banner whenever EITHER the
|
|
entity itself is non-approved OR the latest submission carries
|
|
a *failure* verdict the owner needs to see. Pending edits keep
|
|
the original behavior — Edit button locks instead, no banner. #}
|
|
{% if (is_owner or is_admin) and (
|
|
entity.visibility_status != 'approved'
|
|
or (quarantine_sub and quarantine_sub.status in [
|
|
'blocked_inline', 'blocked_llm', 'review_error',
|
|
])
|
|
) %}
|
|
<style>
|
|
.vis-banner {
|
|
margin: 12px 0 16px 0;
|
|
padding: 14px 18px;
|
|
border-radius: 10px;
|
|
font-size: 14px;
|
|
border: 1px solid;
|
|
}
|
|
.vis-banner.pending { background: #fef3c7; color: #92400e; border-color: #fde68a; }
|
|
.vis-banner.blocked { background: #fee2e2; color: #991b1b; border-color: #fecaca; }
|
|
.vis-banner.hidden { background: #e5e7eb; color: #374151; border-color: #d1d5db; }
|
|
.vis-banner h3 { margin: 0 0 6px 0; font-size: 15px; font-weight: 600; }
|
|
.vis-banner ul { margin: 6px 0 0 0; padding-left: 20px; font-size: 13px; }
|
|
.vis-banner code { background: rgba(0,0,0,0.06); padding: 1px 6px; border-radius: 4px; font-size: 12px; }
|
|
.vis-banner .actions { margin-top: 10px; }
|
|
.vis-banner .actions a {
|
|
display: inline-block; padding: 5px 12px; border-radius: 6px;
|
|
background: rgba(0,0,0,0.08); color: inherit; text-decoration: none;
|
|
font-size: 12px; font-weight: 500;
|
|
}
|
|
</style>
|
|
|
|
{% set sub = quarantine_sub %}
|
|
{% set st = sub.status if sub else entity.visibility_status %}
|
|
{% set bcls = 'pending' if st in ['pending_inline','pending_llm','pending']
|
|
else ('blocked' if st in ['blocked_inline','blocked_llm','review_error']
|
|
else 'hidden') %}
|
|
|
|
<div class="vis-banner {{ bcls }}">
|
|
{% if st == 'pending_llm' or st == 'pending_inline' or st == 'pending' %}
|
|
{% set _is_edit_review = entity.version_no and entity.version_no > 1 %}
|
|
{% if _is_edit_review %}
|
|
<h3>⟳ Version {{ entity.version_no }} under review</h3>
|
|
<div>
|
|
Your edit is being checked. The previously approved version
|
|
(v{{ entity.version_no - 1 }}) keeps serving to existing
|
|
installers until v{{ entity.version_no }} passes review. The
|
|
page refreshes automatically when the verdict lands.
|
|
</div>
|
|
{% else %}
|
|
<h3>⟳ Under review</h3>
|
|
<div>
|
|
Your submission is being checked. It is hidden from the public
|
|
Store and from anyone else's view until all checks pass. Page
|
|
refreshes automatically when the verdict lands — usually a few
|
|
seconds.
|
|
</div>
|
|
{% endif %}
|
|
|
|
{% elif st == 'blocked_inline' %}
|
|
{% set _is_edit_review = entity.visibility_status == 'approved' %}
|
|
{% if _is_edit_review %}
|
|
<h3>⚠ Latest edit failed automated checks</h3>
|
|
<div>
|
|
Your latest edit failed at least one automated check. The
|
|
previously approved version (v{{ entity.version_no }}) keeps
|
|
serving to existing installers. Fix the issues below and
|
|
re-upload, or wait for an admin to resolve the quarantine.
|
|
</div>
|
|
{% else %}
|
|
<h3>⚠ Quarantined — automated checks failed</h3>
|
|
<div>
|
|
Your submission failed at least one automated check and has been
|
|
quarantined. It is hidden from the public Store and from every
|
|
other user; nobody can install it. Fix the issues below and
|
|
re-upload to retry, or wait for an admin to resolve the
|
|
quarantine.
|
|
</div>
|
|
{% endif %}
|
|
{% if sub and sub.inline_checks %}
|
|
{% set ic = sub.inline_checks %}
|
|
{% if ic.manifest and ic.manifest.issues %}
|
|
<ul>
|
|
{% for issue in ic.manifest.issues %}<li>manifest: <code>{{ issue }}</code></li>{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% if ic.static_security and ic.static_security.findings %}
|
|
<ul>
|
|
{% for f in ic.static_security.findings[:6] %}
|
|
<li>security: <code>{{ f.file }}:{{ f.line }}</code> — {{ f.reason }}</li>
|
|
{% endfor %}
|
|
{% if ic.static_security.findings|length > 6 %}
|
|
<li><em>… and {{ ic.static_security.findings|length - 6 }} more</em></li>
|
|
{% endif %}
|
|
</ul>
|
|
{% endif %}
|
|
{% if ic.content and ic.content.issues %}
|
|
{% include "_content_findings.html" with context %}
|
|
{% endif %}
|
|
{% endif %}
|
|
|
|
{% elif st == 'blocked_llm' %}
|
|
{% set _is_edit_review = entity.visibility_status == 'approved' %}
|
|
{% if _is_edit_review %}
|
|
<h3>⚠ Latest edit failed review</h3>
|
|
<div>
|
|
The reviewer flagged your latest edit for security risk and/or
|
|
weak component descriptions. The previously approved version
|
|
(v{{ entity.version_no }}) keeps serving to existing installers.
|
|
Address the findings below and re-upload, or wait for an admin
|
|
to resolve the quarantine.
|
|
</div>
|
|
{% else %}
|
|
<h3>⚠ Quarantined — review flagged issues</h3>
|
|
<div>
|
|
The reviewer flagged this submission for security risk and/or
|
|
weak component descriptions. It is hidden from the public Store
|
|
and from every other user; nobody can install it. Address the
|
|
findings below and re-upload, or wait for an admin to resolve
|
|
the quarantine.
|
|
</div>
|
|
{% endif %}
|
|
{% if sub and sub.llm_findings %}
|
|
{% if sub.llm_findings.summary %}
|
|
<div style="margin-top: 6px;"><em>{{ sub.llm_findings.summary }}</em></div>
|
|
{% endif %}
|
|
{% if sub.llm_findings.findings %}
|
|
<div style="margin-top: 8px; font-weight: 600;">Security findings</div>
|
|
<ul>
|
|
{% for f in sub.llm_findings.findings[:6] %}
|
|
<li>[{{ f.severity }}] <code>{{ f.file }}</code> — {{ f.explanation }}</li>
|
|
{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% if sub.llm_findings.content_quality and sub.llm_findings.content_quality.issues %}
|
|
<div style="margin-top: 10px; font-weight: 600;">Description quality — reviewer suggestions</div>
|
|
<ul>
|
|
{% for issue in sub.llm_findings.content_quality.issues[:8] %}
|
|
<li>
|
|
<code>{{ issue.file }}</code> — {{ issue.issue }}
|
|
{% if issue.hint %}
|
|
<div style="margin: 4px 0 0 0; font-size: 12px; opacity: 0.85;">
|
|
<strong>Rewrite:</strong> {{ issue.hint }}
|
|
</div>
|
|
{% endif %}
|
|
</li>
|
|
{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% endif %}
|
|
|
|
{% elif st == 'review_error' %}
|
|
{% set _is_edit_review = entity.visibility_status == 'approved' %}
|
|
{% if _is_edit_review %}
|
|
<h3>⚠ Latest edit failed review</h3>
|
|
<div>
|
|
The security reviewer couldn't complete its check on your latest
|
|
edit. The previously approved version (v{{ entity.version_no }})
|
|
keeps serving to existing installers. No action needed from you —
|
|
an admin will retry.
|
|
</div>
|
|
{% else %}
|
|
<h3>⚠ Under review — security check errored</h3>
|
|
<div>
|
|
The security reviewer couldn't complete its check. The submission
|
|
stays hidden until an admin retries. No action needed from you.
|
|
</div>
|
|
{% endif %}
|
|
{% if sub and sub.llm_findings and sub.llm_findings.error %}
|
|
<div style="margin-top: 6px; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px; word-break: break-word;">
|
|
Error: {{ sub.llm_findings.error }}
|
|
</div>
|
|
{% endif %}
|
|
{# Surface any inline-check findings that were captured before the
|
|
LLM step errored — gives the submitter something concrete to
|
|
look at instead of a bare "errored" message. #}
|
|
{% if sub and sub.inline_checks %}
|
|
{% set ic = sub.inline_checks %}
|
|
{% if ic.static_security and ic.static_security.findings %}
|
|
<ul>
|
|
{% for f in ic.static_security.findings[:6] %}
|
|
<li>security: <code>{{ f.file }}:{{ f.line }}</code> — {{ f.reason }}</li>
|
|
{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% endif %}
|
|
|
|
{% elif st == 'overridden' %}
|
|
<h3>✓ Admin override applied</h3>
|
|
<div>This submission was force-published by an admin.</div>
|
|
{% if sub and sub.override_reason %}
|
|
<div style="margin-top: 6px; font-size: 13px;">
|
|
<em>Override reason:</em> {{ sub.override_reason }}
|
|
</div>
|
|
{% endif %}
|
|
|
|
{% else %}
|
|
{# Fallback for hidden / unexpected lifecycle states. Surface
|
|
whatever verdict context the submission row carries so an
|
|
admin doesn't see a bare "Hidden" with no actionable detail. #}
|
|
<h3>Hidden</h3>
|
|
<div>
|
|
This entity is not visible in the public Store
|
|
(<code>visibility_status = "{{ entity.visibility_status }}"</code>).
|
|
</div>
|
|
{% if sub and sub.inline_checks %}
|
|
{% set ic = sub.inline_checks %}
|
|
{% if ic.manifest and ic.manifest.issues %}
|
|
<ul>
|
|
{% for issue in ic.manifest.issues %}<li>manifest: <code>{{ issue }}</code></li>{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% if ic.static_security and ic.static_security.findings %}
|
|
<ul>
|
|
{% for f in ic.static_security.findings[:6] %}
|
|
<li>security: <code>{{ f.file }}:{{ f.line }}</code> — {{ f.reason }}</li>
|
|
{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% endif %}
|
|
{% if sub and sub.llm_findings %}
|
|
{% if sub.llm_findings.summary %}
|
|
<div style="margin-top: 6px;"><em>{{ sub.llm_findings.summary }}</em></div>
|
|
{% endif %}
|
|
{% if sub.llm_findings.findings %}
|
|
<ul>
|
|
{% for f in sub.llm_findings.findings[:6] %}
|
|
<li>[{{ f.severity }}] <code>{{ f.file }}</code> — {{ f.explanation }}</li>
|
|
{% endfor %}
|
|
</ul>
|
|
{% endif %}
|
|
{% endif %}
|
|
{% endif %}
|
|
|
|
{# How-to-fix panel — render once below the per-tier findings whenever
|
|
content-quality issues exist on either tier. Same guidance regardless
|
|
of whether the inline mechanical check or the LLM substantive check
|
|
rejected the submission. #}
|
|
{% if sub and ((sub.inline_checks and sub.inline_checks.content and sub.inline_checks.content.issues)
|
|
or (sub.llm_findings and sub.llm_findings.content_quality and sub.llm_findings.content_quality.issues)) %}
|
|
{% include "_content_howto_fix.html" with context %}
|
|
{% endif %}
|
|
|
|
{% if is_admin and sub %}
|
|
<div class="actions">
|
|
<a href="/admin/store/submissions/{{ sub.id }}">Open submission detail →</a>
|
|
</div>
|
|
{% endif %}
|
|
</div>
|
|
|
|
{# Auto-refresh while the verdict is pending. Banner copy promises
|
|
"page refreshes automatically when the verdict lands" — this is
|
|
what does it. Polls the owner-accessible flea detail endpoint and
|
|
reloads when EITHER visibility flips off 'pending' OR the
|
|
submission verdict flips off 'pending_inline' / 'pending_llm'.
|
|
Both signals are needed because `blocked_llm` keeps the entity at
|
|
`visibility_status='pending'` (admin can override → publish), so
|
|
visibility alone doesn't fire. Only emits the script while the
|
|
verdict itself is still pending; terminal states render the
|
|
final banner copy and don't need to reload. #}
|
|
{% if quarantine_sub and quarantine_sub.status in ['pending_inline', 'pending_llm'] %}
|
|
<script>
|
|
(function () {
|
|
const entityId = {{ entity.id|tojson }};
|
|
const initialSubStatus = {{ quarantine_sub.status|tojson }};
|
|
const initialVisibility = {{ entity.visibility_status|tojson }};
|
|
let attempts = 0;
|
|
async function tick() {
|
|
attempts++;
|
|
try {
|
|
const r = await fetch(`/api/marketplace/flea/${entityId}/detail`, {
|
|
credentials: 'same-origin',
|
|
headers: {'Accept': 'application/json'},
|
|
});
|
|
if (r.ok) {
|
|
const data = await r.json();
|
|
const subFlipped = data.submission_status
|
|
&& data.submission_status !== initialSubStatus
|
|
&& data.submission_status !== 'pending_inline'
|
|
&& data.submission_status !== 'pending_llm';
|
|
const visFlipped = data.visibility_status
|
|
&& data.visibility_status !== initialVisibility;
|
|
if (subFlipped || visFlipped) {
|
|
window.location.reload();
|
|
return;
|
|
}
|
|
} else if (r.status === 404) {
|
|
// Entity might have been archived/deleted — reload so the
|
|
// page refetches and renders the new state (or a 404).
|
|
window.location.reload();
|
|
return;
|
|
}
|
|
} catch (e) { /* network blip; keep polling */ }
|
|
// First 30 attempts at 3s = 90s of fast polling, then back off
|
|
// to 10s. Same cadence as admin detail polling so an LLM review
|
|
// on Sonnet/Opus has room to land.
|
|
const next = attempts < 30 ? 3000 : 10000;
|
|
setTimeout(tick, next);
|
|
}
|
|
setTimeout(tick, 3000);
|
|
})();
|
|
</script>
|
|
{% endif %}
|
|
{% endif %}
|