Commit graph

3 commits

Author SHA1 Message Date
ZdenekSrotyr
1b0329e8c5
UI design system unification — one stylesheet, canonical primitives, nav fix (#284)
* docs(plan): design-system unification plan (post-review revisions)

Plan covers consolidating two CSS files into one, introducing
canonical primitives (.btn family, .search-input, .filter-bar,
.page-header, .data-table, .empty-state, .toast, .stat-card,
.tab-strip), unifying the top-nav Admin trigger with sibling
links, and migrating 41 templates that today carry inline
<style> blocks.

Post-review revisions: nav fix moved to first commit (user
complaint lands first); sticky-header and dark-mode skeleton
tasks dropped (defer to follow-up PRs); contract test class
detection tokenizes class="..." attributes properly; baseline
screenshot loop added to Task 0; vendor-token grep widened.

* fix(nav): unify Admin trigger with sibling nav links

The top-nav Admin entry is a <button class="app-nav-link
app-nav-menu-trigger">, siblings are <a class="app-nav-link">.
.app-nav-menu-trigger used to override .app-nav-link with
"color: inherit; font: inherit", resetting font-size from 13px
back to body default and color from --text-secondary to body
color. Active state diverged too: .is-active on links used
--primary blue, [aria-expanded=true] on the button used
--border-light grey.

Fix: expand .app-nav-link so it covers <button>-element resets
(font-family: inherit, border: 0, background: transparent,
cursor: pointer, display: inline-flex for chevron alignment).
Add [aria-expanded="true"] as another active-state selector
so the dropdown's open state highlights identically to .is-active
on links. Delete the now-redundant .app-nav-menu-trigger rules
that stripped button chrome.

Extract the inline <script> from _app_header.html into a new
app/web/static/app.js (loaded by base.html only — base_login.html
has no nav). Sets up window.appUI.wireDropdown for both the user
menu and the Admin dropdown via DOMContentLoaded.

* style(css): consolidate style.css into style-custom.css + add cache-bust

One stylesheet for the whole web UI:
- style.css (1086 lines, legacy Google-inspired tokens + components)
  absorbed into style-custom.css under a labeled block, placed after
  the modern :root + body so style-custom's component rules continue
  to override the legacy ones (preserves the original cascade order
  that came from loading style.css first).
- style.css deleted; <link> dropped from base.html + base_login.html.
- static_url() now appends ?v=<mtime> to /static/<path>. Cheap
  per-request os.stat — auto-invalidates browser + proxy caches on
  redeploy without operator intervention. Mtime survives across
  uvicorn restarts as long as the file content is unchanged.

Legacy classes (.btn, .card, .login-*, .badge, .code-block, .flash,
.form-group, .username-box, .btn-copy, .auth-tabs, .divider, etc.)
still render — they live in style-custom.css now. Login pages,
error page, password setup, and the dashboard's Claude Code Setup
card all kept working in browser smoke.

* test(design): contract test for design-system invariants

7 structural invariants enforced from this commit onwards:
- style.css must stay deleted
- no template links style.css via static_url
- exactly one bare :root block in style-custom.css
- canonical primitives declared (.btn, .btn-primary, .search-input,
  .filter-bar, .page-header, .data-table, .empty-state, .toast, …)
- no deprecated class names in templates (.users-table, .gp-table,
  .marketplaces-table, .audit-table, .users-search, .marketplaces-search,
  .modal-btn, .btn-primary-v2, …)
- app.js loaded by base.html, NOT by base_login.html
- 3 helper-level unit tests for the class-attribute tokenizer
  (multi-line attrs, Jinja-conditional fragments, false-positive prose)

Two of the assertions intentionally start FAILING after this commit
(missing primitives + legacy class refs in 7 admin templates) and
will turn green as Tasks 4–7 add primitives and Tasks 8–15 migrate
the templates.

* feat(css): canonical button family + legacy token aliases

Adds at top of :root: legacy token aliases (--bg, --card-bg, --text,
--text-light, --secondary, --radius) pointing at modern equivalents.
Absorbed style.css rules referenced these names; without aliases
they fell back to 'unset'. Aliases live until Task 16 alongside
their absorbed rules.

Appends canonical .btn variants at end of file (last cascade):
  .btn-primary + .btn-primary-v2 + .modal-btn.primary (alias group)
  .btn-secondary + .btn-secondary-v2 + .modal-btn:not(.primary):not(.danger)
  .btn-ghost + .btn-ghost-v2
  .btn-danger + .modal-btn.danger
  .btn-lg
  .btn:disabled + .btn:focus-visible (focus ring via --focus-ring)

Existing absorbed .btn, .btn-primary, .btn-secondary, .btn-sm rules
remain — the canonical block adds the missing variants + selector-list
aliases so .modal-btn and v2 markup keep rendering until migration
tasks swap them out.

Contract test: .btn-danger now declared (one less missing primitive).
Browser smoke: /admin/tokens hero + filter pills + empty state render
correctly with the absorbed style.css rules now backed by real tokens.

* feat(css): form-control primitives — .search-input + .filter-bar + .filter-pill + .form-input

Canonical filter bar shape: 36px-height inputs (matches button height
for vertical rhythm), 28px pills with .is-active state, consistent
focus ring via --focus-ring token.

Selector-list aliases for legacy per-page classes:
- .users-search / .marketplaces-search / .kb-search → .search-input
- .filters-card → .filter-bar
- .pill[aria-pressed="true"] also matches the .filter-pill active state

.form-input added as a sibling of .search-input for forms — same
baseline height + radius + focus treatment, with textarea.form-input
auto-sizing to min 96px and using the mono font (matches CSV/SQL
pasted-snippet patterns on /admin/agent-prompt + /admin/workspace-prompt).

Contract test: .search-input + .filter-bar + .filter-pill now declared.

* feat(css): .page-header primitive + variants + .tab-strip

Canonical page-header pattern with title (22px) + optional subtitle +
optional eyebrow + right-aligned actions slot. Two modifiers:
- .page-header--hero: gradient background (primary→primary-dark),
  28px white title, semi-transparent subtitle/eyebrow. For
  /marketplace, /store, /profile-style pages that already use this
  layout via per-page inline <style>. Migration tasks delete the
  duplicated rules.
- .page-header--compact: 18px title for dense admin index pages.

.tab-strip + .tab-strip__item — the secondary tab row pattern used by
/marketplace?tab=flea and similar. .is-active / [aria-selected=true]
both flip the active treatment (primary color + bottom border).

Contract test: .page-header / __title / __subtitle / __actions all
now declared (4 fewer missing primitives).

* feat(css+js): .data-table + .empty-state + .toast + .stat-card primitives

Last primitive batch. All 8 canonical-primitives invariants in
test_design_system_contract.py now green; only the template-migration
test fails (expected — Tasks 8–15).

.data-table (+ --compact modifier): selector-list aliases for legacy
per-page table classes (.users-table, .gp-table, .marketplaces-table,
.audit-table) so existing markup keeps rendering until migration.
Compact modifier shrinks padding + font for dense lists (audit log).

.empty-state with __icon / __title / __description / __actions —
replaces the ad-hoc 'no results' rendering scattered across pages
(corporate_memory, admin_users, admin_marketplaces, etc.).

.toast / .toast-container — paired with window.appToast({kind, msg,
timeout}) appended to app.js. Bottom-right stacked, click-to-dismiss,
auto-dismiss after 4s by default. Kind 'success' / 'warning' / 'error'
/ 'info' shows a 3px colored left border.

.stat-card (+ --accent variant) + .stat-row grid — for the dashboard
metric tile row.

* style(templates): migrate 8 templates off deprecated class names

Mechanical class-attribute rewrite via tokenizer (preserves Jinja
conditionals + multi-line attrs):

  modal-btn primary    -> btn btn-primary
  modal-btn danger     -> btn btn-danger
  modal-btn            -> btn btn-secondary
  users-table          -> data-table
  gp-table             -> data-table
  marketplaces-table   -> data-table
  audit-table          -> data-table
  users-search         -> search-input
  marketplaces-search  -> search-input

8 templates touched: admin_groups, admin_marketplaces, admin_tokens,
admin_users, admin_welcome, admin_workspace_prompt, my_tokens,
corporate_memory_admin. 43 lines updated total.

Inline <style> blocks in these templates still define rules for the
old class names — those rules no longer match anything and become
dead code, removed in Task 16's alias cleanup along with the
selector-list aliases in style-custom.css.

Contract test (tests/test_design_system_contract.py) now fully green:
9/9 invariants enforced from this commit onward.

* feat(css): extend .data-table selector list to 13 more bespoke -table classes

Visual unification of remaining tables across the codebase without
per-template edits. The .data-table baseline rules (uppercase header
tracking, 12px padding, hover state, border-radius) now apply to:

  .ad-table / .ea-table / .md-table / .members-table /
  .obs-table / .overview-stats-table / .registry-table /
  .sample-table / .sched-table / .sess-table / .sub-table /
  .subs-table / .ud-table

These class names live in 12 templates (activity_center, admin_access,
admin_group_detail, admin_scheduler_runs, admin_sessions,
admin_store_submissions, admin_tables, admin_usage, admin_user_detail,
catalog, me_debug, profile_sessions) that have their own per-page
<style> blocks. Per-page rules with higher specificity still win for
their custom needs (column widths, etc.) — this commit only sets a
shared baseline so every table renders with the same chrome.

Contract test stays green: 9/9 invariants enforced.

* style(css): remove now-unused legacy class aliases

Phase A renamed 8 templates off these names; no markup references
them any more, so the selector-list memberships are dead weight.
Removed from style-custom.css:

  .btn-primary-v2 / .btn-secondary-v2 / .btn-ghost-v2
  .modal-btn / .modal-btn.primary / .modal-btn.danger /
  .modal-btn:not(.primary):not(.danger)
  .users-search / .marketplaces-search / .kb-search
  .users-table / .gp-table / .marketplaces-table / .audit-table
  .filters-card

37 lines smaller. Contract test catches any reintroduction.

KEPT aliases (still in untouched template markup):
- .pill (marketplace_plugin_detail.html, marketplace.html — these
  pages weren't part of Phase A's deprecated-class sweep; their
  own .pill CSS rules still apply)
- All .data-table family extensions (.ad-table, .ea-table, .md-table,
  .members-table, .obs-table, .overview-stats-table, .registry-table,
  .sample-table, .sched-table, .sess-table, .sub-table, .subs-table,
  .ud-table) — these still render data tables in 12 templates;
  selector-list aliasing keeps them visually unified with .data-table
  baseline.
- Legacy token aliases (--bg / --text / --text-light / --secondary /
  --card-bg / --radius) — still resolve absorbed style.css rules.

Templates' inline <style> blocks still contain dead rules for the
renamed classes (.users-search, .modal-btn, etc.); harmless but
bloat. Optional follow-up: a separate sweep can drop those.

* docs(changelog): design-system unification under [Unreleased]

* feat(css): unify page-shell width — .container baseline 1280px + modifiers

Inventory found 30+ unique max-width values across templates (280px
login → 1600px admin/tables). The legacy .container default was 800px,
which made every admin page set its own wider inline override —
30+ ad-hoc widths drifted as a result.

Canonical: .container max-width = var(--width-app) (1280px). Pages
that need a different shape opt in via modifiers:

  .container--narrow → var(--width-narrow)  (800px) — long-form text,
                                                     setup wizards
  .container--wide   → var(--width-wide)    (1400px) — admin lists,
                                                     marketplace grids
  .container--full   → max-width: none — hero / landing

Pages that already set a NARROWER inline max-width (setup, login flows
inside .login-card, etc.) still render at their narrower size — the
inline override beats the new canonical 1280px. The visible change
hits the ~20 admin pages currently rendering at 800px via the legacy
default, which jump to 1280px and pick up consistent breathing room.

Spacing also normalized: padding 24px 20px → var(--space-6) var(--space-5).

* fix(home+catalog): gut dashboard sections + remove confusing toggle + fix table count

Dashboard /home cleanup:
- Remove 'Your Data' card — Data Packages is already a top-nav entry,
  so duplicating data sources on the landing page just adds noise.
- Remove 'Account' card — group memberships + scripts + last sync
  belong on /profile, not on the welcome screen.
- Remove entire right-column (Corporate Memory + Activity Center
  widgets) — both surfaces have dedicated admin pages reachable from
  the Admin dropdown.
- Keep stats row (Tables/Columns/Rows/Data Size/Unstructured),
  env-setup-CTA, and Notifications card.

/catalog cleanup:
- Strip the 'Always included' badge + the locked toggle-switch from
  Core Business Data and Business Metrics cards. The toggle was
  always 'checked disabled' — it visually looked like a switch but
  could not be toggled, which was confusing. The 'Always included'
  copy itself was redundant once the toggle was gone. Agnes Internal
  already rendered without these, so the three cards are now visually
  consistent.

Catalog data_stats fix:
- 'total_tables' was len(sync_state) — counted only tables that had
  ever synced, so a 30-row table_registry with 0 ever synced rendered
  as '0 tables'. Switched to len(tables) — the registered
  business-data table list — so the count reflects what's actually
  available, not what's been touched.

* fix(home): real stat numbers + drop unstructured tile + cleanup dead CSS

Dashboard stats were hardcoded zeros (columns: 0, size_display:
'0 MB', unstructured_display: '0 MB') and the table counter pulled
from sync_state (synced) instead of table_registry (registered).
On a fresh deployment with 30 registered tables and 0 ever synced,
the page rendered '0 / 0 / 0 / 0 MB / 0 MB' — useless.

Now:
- Tables: COUNT(*) FROM table_registry WHERE source_type != 'internal'.
  Matches the /catalog Core Business Data counter.
- Columns: SUM(sync_state.columns). Zero only when nothing's synced yet.
- Rows: unchanged (SUM(sync_state.rows), already correct).
- Data Size: SUM(sync_state.file_size_bytes), human-formatted via
  inline _fmt_bytes helper (KB/MB/GB).
- Unstructured: tile dropped — was always '0 MB' and had no source.
- last_updated: now derived from sync_state max(last_sync), wasn't set
  before so the 'Synced …' tag never rendered.

Dashboard.html cleanup: ~725 lines of orphan inline <style> removed —
.section-title, .data-source*, .toggle-switch*, .catalog-cta*,
.memory-card / .memory-stat / .memory-description / .memory-footer
/ .btn-memory, .activity-card / .activity-stat / .activity-text
/ .btn-activity, .account-grid / .account-row / .account-scripts
/ .badge-role / .badge-group / .cron-line, .badge-included /
.badge-beta / .badge-demo. All matched markup deleted in the
previous commit; the CSS was dead code until now.

* ui(catalog): rename page heading 'Data Catalog' → 'Data Packages'

The top-nav entry says 'Data Packages' but the page itself said
'Data Catalog' — confusing two-name product. Aligns the heading and
<title> with the nav label. Subtitle trimmed too: 'manage your
subscriptions' was a vestige of the toggle UI that just got removed,
replaced with a one-liner describing what the page is for.

Two other 'Data Catalog' strings stay: they live inside the table-
profiler overlay JS and refer to an EXTERNAL catalog system (e.g.
OpenMetadata / Atlan) that an operator may link to per table — that
is a generic term for any external data-catalog product, not our
page name.

* fix(nav): dropdown clicks always work + mutual-exclusion close

Two bugs in the wireDropdown helper:

1. Clicking trigger B while trigger A's menu was open left both open.
   e.stopPropagation() in trigger.click prevented the document-click
   handler from firing, so trigger A's open menu had no way to learn
   that something else was clicked. Net effect: state diverged across
   the two dropdowns the more you clicked.

2. The target-vs-trigger equality check (e.target !== trigger) was
   strict. Clicking the chevron <svg> inside the button reports the
   svg or its <path> child as e.target — not the button — so removing
   stopPropagation alone would trip the close branch in the same
   click that just opened the panel.

Fix both at once: drop e.stopPropagation() AND switch the doc-handler
guard to trigger.contains(e.target). Now any click outside both the
trigger subtree and the panel subtree closes; any click on another
trigger closes via the OTHER dropdown's doc handler; clicks inside
the trigger (button OR svg child) are fully ignored by the doc
handler and only the trigger's own toggle handler fires.

* feat(ui): canonical blue-gradient hero on every admin page

The UI had a per-page hero pattern on ~10 onboarding/marketing pages
(admin_tokens / profile / install / setup_advanced / marketplace /
my_tokens / store_upload / home_*), each with its own ad-hoc CSS
(.tokens-hero, .profile-hero, .install-hero, .upload-hero, …). The
admin section's index + detail pages had plain H1/H2 with their own
.users-title / .gp-title / .obs-title / .cfg-title / … inline styling.
Net effect: half the app felt like a product, half felt like a
spreadsheet.

Now:
- .page-header--hero CSS upgraded to match the look analysts already
  liked from admin_tokens: 28px/32px/24px padding, 14px radius, soft
  primary-tinted box-shadow (0 4px 16px rgba(0,115,209,0.2)), 28px
  semibold title, optional uppercase eyebrow + 13.5px subtitle.
  Narrow-viewport breakpoint included.
- New _page_hero.html partial wraps the boilerplate. Usage:
    {% set page_hero_eyebrow  = "Users & Access" %}
    {% set page_hero_title    = "Users" %}
    {% set page_hero_subtitle = "…" %}
    {% include "_page_hero.html" %}
- 15 admin templates migrated to it: admin_users / admin_groups /
  admin_marketplaces / admin_access / admin_sessions /
  admin_session_detail / admin_store_submissions /
  admin_scheduler_runs / admin_usage / admin_user_detail /
  admin_welcome / admin_workspace_prompt / admin_server_config /
  activity_center / admin/news_editor. Each gets a grouped eyebrow
  (Users & Access / Data / Agent Experience / Activity Center /
  Server) matching the Admin dropdown sections so the page identity
  is unambiguous at a glance.

Legacy *-title H2/H1 + adjacent subtitle paragraphs deleted; their
per-page CSS rules are dead now (harmless, retire in a follow-up
sweep alongside other inline-style cleanup the reviewers flagged).

admin_tables.html intentionally NOT migrated — it's a standalone
HTML page that doesn't extend base.html; a separate refactor.

Test: test_admin_users_page_renders_for_admin assertion updated
from .users-title to .page-header__title + .page-header--hero (the
canonical pair). All other web/template tests stay green.

* refactor(ui): dedup _humanbytes, drop 267 lines of dead inline CSS

(1) _humanbytes consolidation:
- Add TB branch + optional precision param (default 2 preserves existing
  Store detail callers; dashboard uses precision=1 for headline tiles).
- Delete inline _fmt_bytes from dashboard handler — was a copy of
  _humanbytes with different rounding. One canonical helper now.

(2) Dead inline-CSS sweep across 17 migrated templates:
- Conservative regex: a CSS rule is deleted only when its primary class
  matches one of the known-dead names AND that name is NOT referenced
  from any class= attribute in the same file's markup.
- Per-file 'in-use' guard saved several false positives that the deny
  list would have nuked (e.g. .users-toolbar, .gp-search, .obs-subtitle,
  .marketplaces-toolbar are still in use; only .users-table, .users-search,
  .users-title, .modal-btn, etc. that have NO markup left went away).
- Removed: -267 lines across admin_users (-42), admin_marketplaces (-45),
  admin_groups (-31), my_tokens (-38), admin_tokens (-29), admin_access
  (-9), admin_user_detail (-6), admin_welcome (-8), admin_workspace_prompt
  (-8), admin_server_config (-2), admin_sessions (-1), admin_session_detail
  (-1), admin_usage (-1), admin_store_submissions (-3), admin_scheduler_runs
  (-3), activity_center (-4), corporate_memory_admin (-36).

Contract test stays green (9/9); all web/template/render/user_management
tests pass.

* feat(ui): canonical hero on /catalog (Data Packages)

Same .page-header--hero treatment as the admin pages — Data eyebrow,
Data Packages title, Browse-the-data-sources subtitle. Removes the
ad-hoc .page-title block (h1 / p / wrapper-div) and its CSS rules
(now dead, 3 rule blocks deleted).

* fix(nav): load app.js from _app_header.html — works on standalone pages

The previous nav-fix commit moved the inline dropdown script from
_app_header.html into app/web/static/app.js + added <script src=…>
to base.html. That broke EVERY page that includes _app_header.html
WITHOUT extending base.html (catalog, corporate_memory*,
admin_tables, install). They got the nav markup but no JS → both
Admin and AD dropdowns dead on those pages.

Fix: emit the <script src=app.js defer> directly inside the
_app_header.html partial. Any page that includes the header now
gets the script automatically — base.html-extenders AND standalone
HTML pages alike. base.html's duplicate <script> line removed.

Also fixes the wide-hero on /catalog: .page-header--hero now sets
its own max-width: var(--width-app) (1280px) so standalone pages
without a .container parent don't render the gradient edge-to-edge.
catalog's .source-cards bumped from 900px → 1280px to match the
hero, otherwise the page reads two-tier (wide blue band, narrow
content) which the user flagged.

Verified locally via agent-browser: Admin + AD dropdowns now click
through on /catalog, /admin/tables, /corporate-memory.

* docs(plan): standalone pages → base.html framework migration plan

Plan + Plan-agent review (8 must-fix items applied) for converting
the 5 templates that ship their own <html><head><body> scaffold
(catalog, install, corporate_memory, corporate_memory_admin,
admin_tables) to extend base.html. Root cause of yesterday's
'dropdown dead on /catalog' regression: shared infrastructure in
base.html doesn't propagate to standalones.

* feat(base): body_attrs block + migrate install.html to extend base

base.html: new {% block body_attrs %}{% endblock %} slot so pages
that need <body> attributes (admin_tables has data-source-type)
can carry them through extends.

install.html: convert from standalone <html><head><body> scaffold
to {% extends "base.html" %} with title / body_attrs / head_extra
/ layout / scripts blocks. Drops:
- <!DOCTYPE>, <html>, </html>, <head>, </head>
- <meta charset>, <meta viewport>
- Duplicate <link rel="stylesheet" href="...style-custom.css">
  (base.html already provides one)
- <body> opening + closing tags
- Leading _app_header.html include + _version_badge.html include
  (base.html handles both)

Preserves per-page CSS (in head_extra), per-page JS (in scripts),
the Inter font preconnect (kept inline; not hoisted to base in
this PR — separate decision).

Pilots the migration recipe before the 4 larger pages.

* refactor(memory): extend base.html

Same recipe as install.html. corporate_memory.html now inherits
<html>/<head>/<body> + nav + app.js script tag from base.html.
Page-specific CSS and JS preserved in head_extra + scripts blocks.

* refactor(memory-admin): extend base.html

Same recipe as install/corporate_memory. Curation page now in the
shared rendering pipeline.

* refactor(catalog): extend base.html

catalog.html had the most complexity: 7 head-level assets (chart.js,
Prism, prism-sql, metric_modal.css link + 2 preconnects + Inter
stylesheet), 5 body-level <script> blocks including a <script type=
"module"> for the metric modal, 2 duplicate style-custom.css links
in <head>. The migration script preserved all of them — head-level
externals hoisted to {% block head_extra %} in source order, body
scripts relocated to {% block scripts %} in source order (so chart.js
loads before the IIFE that builds Chart instances), duplicate
style-custom.css links dropped (base.html provides one).

* refactor(admin-tables): extend base.html + carry data-source-type

The biggest of the 5 standalones at 3563 lines. <body data-source-
type="{{ data_source_type }}"> attribute carried through via the
new {% block body_attrs %} slot (admin_tables JS reads
document.body.dataset.sourceType to switch between keboola and
bigquery rendering paths).

* release: 0.54.10 — UI design system unification + homepage status frame + initial workspace override + store guardrails

Co-Authored-By: zdenek.srotyr <zdenek.srotyr@keboola.com>

* refactor(web): migrate remaining templates to canonical design primitives

- admin_group_detail: .data-table, .btn family, appToast(), remove duplicate table/button/toast CSS
- admin_store_submission_detail: .data-table, .btn family, appToast(), remove duplicate btn/toast CSS
- profile_sessions: .data-table, _page_hero.html, remove duplicate table/title CSS
- me_debug: .data-table, .btn family, remove duplicate table/button CSS
- marketplace: .btn-primary/.btn-secondary, remove duplicate button CSS
- store_edit: remove duplicate .btn-primary/.btn-link CSS, canonical button classes
- store_upload: remove duplicate .btn-primary/.btn-secondary/.btn-link CSS

Co-Authored-By: zdenek.srotyr <zdenek.srotyr@keboola.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-05-14 13:28:03 +02:00
Vojtech
d6ad08f107
Flea-market upload guardrails + soft delete + JOIN-based admin queue (#233)
* feat(store): flea-market upload guardrails + soft delete + JOIN-based admin queue

Adds an end-to-end guardrails pipeline for store uploads (manifest +
static-security + LLM review), persists blocked bundles for forensics,
introduces soft-delete (Archive) semantics, consolidates the legacy
/store/{id} surface into /marketplace/flea/{id}, and reworks the admin
queue so lifecycle filters read live entity visibility via LEFT JOIN
rather than a denormalized submission column.

Schema v29 → v35:
  * v29 store_submissions table + store_entities.visibility_status
  * v30 file_size, bundle_sha256, bundle_purged_at on submissions
  * v31 reshape store_submissions (drop legacy unique on entity_id)
  * v32 store_entities.archived_at/by + 'archived' visibility value
  * v33 drop store_submissions.retry_count (unused)
  * v34 ensure idx_store_submissions_entity exists post column-drop
  * v35 broaden visibility_status enum + JOIN architecture cutover

Pipeline (src/store_guardrails/):
  * Inline checks: manifest_check, static_scan, quality_check
  * LLM review configurable haiku|sonnet|opus (default haiku)
  * BackgroundTasks-driven async path with structured-output JSON
  * Per-submitter daily quota (default 50)
  * 30-day TTL purge job (POST /api/admin/run-blocked-purge)
  * Bundle SHA256 + size persisted; sha256 survives purge for forensics

Visibility model:
  * pending | approved | hidden | archived
  * _enforce_visibility returns 404 (no leak) for non-owner non-admin
  * Owner sees own non-approved entries via include_owner_id widening
  * Install refused with 409 entity_not_approved when not approved

Soft-delete (DELETE /api/store/entities/{id}):
  * Default = soft (visibility_status='archived'); existing installs
    keep getting served the bundle so users don't lose the plugin
  * ?hard=true admin-only: drops bundle + cascades user_store_installs
  * Hard-delete preserves entity_id on submission as tombstone so
    audit_log linkage survives for the activity timeline

Admin queue lifecycle (the JOIN refactor):
  * Verdict (store_submissions.status) is immutable forensic record
  * Lifecycle (store_entities.visibility_status) is live state
  * /admin/store/submissions Archived chip translates to
    `e.visibility_status='archived'` via LEFT JOIN — any path that
    flips visibility surfaces in the queue immediately
  * Detail page renders Status (verdict) and Entity lifecycle side by
    side so admins see "approved at review, now archived" at a glance

URL consolidation:
  * /store/{id} deleted (no redirect, stale bookmarks 404)
  * /marketplace/flea/{id} is the canonical detail surface
  * Three in-tree callers (upload-success, my-stack card, store
    listing card) updated to point at the new URL
  * Quarantine banner extracted to _quarantine_banner.html partial,
    self-guarded, included from both flea detail templates
  * Banner JS auto-refreshes when the verdict lands by polling
    /api/marketplace/flea/{id}/detail (visibility_status +
    submission_status — the latter is needed because blocked_llm
    keeps the entity at visibility_status='pending')

Audit log resource format:
  * runner.py emits prefixed `store_submission:{id}` (post-fix)
  * Detail-page timeline query handles three patterns: prefixed
    submission, helper-emitted `store_entity:{sub_id}`, and bare-id
    legacy rows — all surface in the activity timeline

UX fixes:
  * Owner sees Under review / Quarantined / Hidden banner with status
  * Install button gray-disabled (not blue) when non-approved
  * Owner cannot delete quarantined entries (403); admin can
  * Admin queue: filter chips, sortable columns, paging, page-size
  * Auto-refresh queue every 5s while pending rows are visible
  * Store upload page file picker no longer opens twice (label →
    input default action collided with explicit JS handler)

Tests: 168 passed across the guardrails suites (admin submissions,
store API, inline / LLM / purge guardrails, store repositories,
marketplace filter, schema version). New regression coverage
includes: archive surfaces via JOIN even when API path is bypassed;
deleted submission renders activity timeline (tombstone); flea
detail surfaces submission_status only for owner/admin; detail page
renders Entity lifecycle row; audit log resource format covers both
helper and runner paths.

* fix(store-guardrails): PR #233 follow-up — prompt injection, atomic PUT, BG race, schema, reaper, sort whitelist

Addresses 9 of the 23 findings from the PR #233 review (spec at
docs/superpowers/specs/2026-05-09-pr233-guardrails-fixes-spec.md).
Merge-gate items #1-#6 plus high-value mediums #7, #9-#12, #23.
Architectural items (#8 enum split, #14 factory) and pure
maintainability (#15-#22) deferred to follow-ups.

Security:
* #1 prompt injection — SYSTEM_PROMPT now passed via the SDK's
  dedicated system= parameter; bundle wrapped in <bundle>...</bundle>
  sentinels declared data-only by the system prompt; literal
  sentinel strings in user content are escaped so an adversarial
  README can't forge a close tag.
* #6 static scan honesty — module docstring + admin copy + docs
  declare static scan as signal not gate; .md/.txt/.rst/.html/.json/
  .yaml/.yml/.toml skipped to avoid false positives on prose.
  AST mode for Python deferred (separate flag, FP comparison work).

Correctness:
* #2 PUT atomicity — bundles bake into plugin.staging-<rand>/
  alongside live, atomic-rename on success; failed checks leave
  live tree byte-for-byte intact.
* #3 BG-task race — set_visibility_if_pending guards verdict flips
  to the (pending, hidden) review window; admin archives during
  review survive; skipped flips audit-logged.
* #4 v35 NOT NULL/DEFAULT — schema v35→v36 re-applies them on
  store_entities.visibility_status. CHECK constraint enforced
  application-side (DuckDB ADD CHECK on existing column unsupported).
* #7 stuck-review reaper — reap_stuck_llm_reviews flips pending_llm
  rows older than guardrails.stuck_review_grace_seconds (default
  1800) to review_error. Scheduler runs every 15 min via new
  /api/admin/run-reap-stuck-reviews. Set knob to 0 to disable.
* #9 quota counter — count_blocked_for_submitter_since now counts
  blocked_inline + blocked_llm + review_error so a submitter
  triggering only LLM-blocked verdicts is bounded.
* #10 missing risk_level — surfaces as review_error with
  error='missing_risk_level' instead of silently defaulting to
  'medium' (which looked like a model-decided block).
* #11 archived_at clear — set_visibility nulls archived_at +
  archived_by when transitioning out of 'archived' so a future
  read doesn't show stale archive forensics on an approved row.

Maintainability:
* #12 FSM doc comment — accurate insert/transition/lifecycle
  description in src/db.py near store_submissions schema.
* #23 sort-key whitelist — admin queue rejects unknown sort keys
  with 400 invalid_sort_key; substring-replace footgun removed.

Deferred (separate PRs):
* #5 quota race — proper fix requires asyncio.Lock spanning the
  full pipeline; threading.Lock blocks event loop, DuckDB MVCC
  doesn't help. API-level slowapi bounds worst case for now.
* #6 part 3 (AST static scan), #8 (enum split), #13 (import
  bundle docs), #14 (factory consolidation), #15-#22 (maint).

Tests:
* New: tests/test_store_guardrails_prompt_injection.py (corpus +
  trust-boundary invariants), tests/test_store_put_atomic.py,
  tests/test_store_guardrails_reaper.py.
* Extended: test_store_guardrails_llm.py (system param, missing
  risk_level, BG race), test_admin_store_submissions.py (quota
  counter widening, sort whitelist 400), test_store_repositories.py
  (un-archive metadata clear), test_db_schema_version.py (v36).
* Full suite: 3738 passed; 17 pre-existing baseline failures
  unchanged (db migration tests, cli binary rename, catalog export,
  user mgmt v5 backfill — confirmed by stash + rerun on clean tree).
2026-05-09 17:32:53 +04:00
ZdenekSrotyr
e86dd5edc5 fix(anthropic): strict json_schema (additionalProperties=false) + add /admin/scheduler-runs UI
E2E test on a real BQ deploy showed every verification-extraction call
fails with HTTP 400 invalid_request_error: "output_config.format.schema:
For 'object' type, 'additionalProperties' must be explicitly set to false".
The Anthropic structured-output API now requires the field on every object
node in the json_schema. Fix: connectors/llm/anthropic_provider.py wraps
the caller-supplied schema through a recursive _strict_json_schema()
walker that adds the field where missing (preserving any explicit
override), then passes the strict variant to the API. Six unit tests in
TestStrictJsonSchema pin the recursion across nested objects, array items,
and the no-mutation invariant.

Adds /admin/scheduler-runs — a read-only admin page that surfaces the
last 200 audit-log entries from scheduler-driven actions. New
AuditRepository.query_actions(actions, limit) helper, new admin nav
entry. Failed scheduler ticks (HTTP 401, network errors) don't reach
the audit_log; the page calls that out with a hint to set
SCHEDULER_API_TOKEN if no rows show up.
2026-05-05 08:00:57 +02:00