AI-Cognitive-Leap/agnes-the-ai-analyst
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
agnes-the-ai-analyst/app/web/templates/activity_center.html
ZdenekSrotyr 64cf78860d
feat(stack): unified Browse + My Stack for Data Packages and Memory (v49 schema) (#333) 
* feat(unified-stack): Browse + My Stack + Recipes + RBAC matrix (v49–v55)

Squash of 94 commits spanning the v49 → v55 unified-stack rewrite.
Full per-feature breakdown lives in CHANGELOG.md under [Unreleased].
Major buckets:

* v49 schema — first-class user_groups + user_group_members +
  resource_grants; admin can CRUD groups and grants; Google
  Workspace nightly sync writes into the new tables.
* v49 data_packages — admin-curated bundles of tables, RBAC-gated,
  first-class section on /catalog Browse + My Stack.
* v49 memory_domains — row-backed (replaces hardcoded VALID_DOMAINS
  enum); admin can CRUD; grants follow the same shape as tables and
  packages.
* v50 cover_image_url + admin sidebar collapsibles + per-row Mode
  tooltip + admin queue domain badges + admin "+ New Item" seed flow.
* v51 lifecycle status (prod/poc/coming-soon/draft) + category +
  palette swatches on admin modals.
* v52 per-table detail page /catalog/t/<id>.
* v53 Recipes — admin-curated SQL templates as a second tab on
  /catalog with full Edit/Delete admin affordances.
* v54 soft-delete (deleted_at) + Undo toast for packages, memory
  domains, and recipes; hard_delete() retained as escape hatch.
* v55 Recipes RBAC — ResourceType.RECIPE registered, inline Group
  Access matrix on Create + Edit Recipe modals (mirrors the Memory
  Domain pattern).
* Activity Center per-resource filter (resource_prefix LIKE-anchored
  on audit_log.resource); admin nav g+letter keyboard shortcuts;
  loadAdminTablesLayout N+1 → single endpoint; /api/memory 30s
  page-level cache.
* CI hardening — Keboola legacy tests pytest.importorskip; perf-
  smoke threshold widened to stop cold-cache flake.

5002 tests passing, 35 skipped.

* feat(p2 backlog): Cmd-K palette + suggest-a-domain + nightly E2E + v55 schema

10-item P2 sweep on top of the unified-stack squash. New behaviour:

* Cmd-K admin command palette (base.html) — fuzzy-search overlay over
  admin + user-facing routes. Arrows/Enter to navigate, Esc to close.
* Stack-tabs digit shortcuts — 1/2/3 switch Browse / My Stack /
  Recipes on /catalog + /corporate-memory.
* Friendlier non-admin empty state on /corporate-memory, plus a
  "Suggest a domain" CTA → POST /api/memory-domain-suggestions, admin
  queue with approve/reject. Backed by a new memory_domain_suggestions
  table (schema v55).
* /admin/corporate-memory 7-tab strip grouped under Moderation /
  Catalog parent labels.
* Bulk-assign table → package dropdown annotates each option with
  "(N of M tables already in)" so the existing distribution is visible
  before picking a target.
* GET /api/memory + /tree accept is_required filter; admin status
  dropdowns route the "Required" sentinel onto it (status no longer
  holds 'mandatory' post-v49, so the old dropdown returned nothing).
* chip-input.js is now opt-in per template via {% block extra_scripts %}
  instead of loaded globally on every page from base.html.
* Edit-modal close helpers consolidated onto _closeEditModalById();
  docs the per-source-type modal architecture decision.
* New .github/workflows/e2e-nightly.yml runs agent-browser smoke
  scripts (scripts/e2e/smoke_*.sh) against a docker-compose stack
  nightly at 04:30 UTC; failures open an agent-browser-nightly issue.

5012 tests passing, 35 skipped.

* fix(visual audit): 6 page regressions on memory + data-package surfaces

agent-browser walkthrough of every memory + data-package page in the PR
turned up 6 real bugs. Fixes:

1. Admin memory modals were dead. Duplicate `let _cmdNewDomainId`
   declarations from the deprecated step-2 RBAC stubs in
   admin_corporate_memory.html collided with the live state vars
   declared earlier in the same <script> → SyntaxError on parse →
   the entire second script block silently failed → every inline
   onclick= handler defined there (`+ New Memory Domain`, Edit, etc.)
   was a no-op. Removed the duplicate stubs.

2. /catalog/t/<table_id> + /catalog/r/<slug> rendered unstyled.
   Both templates injected their CSS via {% block head %} but
   base.html exposes {% block head_extra %} — wrong block name
   meant <style> rules never reached the rendered HTML. Renamed
   to head_extra. Hero card, section cards, dark SQL block, proper
   full-width inputs all now render as designed.

3. L49 leak — "MANDATORY" KPI label + "Make Mandatory" row buttons
   on /admin/corporate-memory still used the old word. Renamed to
   "Required" / "Mark as Required" so UI matches the data model
   (v49 split moved the Required tier onto the orthogonal
   is_required boolean; status no longer holds 'mandatory').

4. Activity Center Resource dropdown didn't know the v55
   `memory_domain_suggestion:` namespace — added it.

5. Tab strip on /admin/corporate-memory wrapped text 2× per button
   on narrow viewports after the L50 MODERATION/CATALOG group
   labels pushed total width past most viewports. Switched the
   strip to flex-wrap:nowrap + overflow-x:auto with
   white-space:nowrap + flex-shrink:0 on every direct child so the
   tabs stay one row and slide horizontally when they overflow.

5012 tests passing, 35 skipped.

* rebase-cleanup: align with main's 0.54.25-27 API design + comment fix

Three follow-on fixes after rebasing onto origin/main (0.54.27):

* admin_tables.html: dropped a stray nested ``{% if data_source_type
  == 'keboola' %}`` around ``prefillFromKeboolaTable`` (main never had
  it; the outer Phase F2 guard already covers it) and reworded a JS
  comment that contained literal ``{% %}`` tokens which Jinja was
  parsing as a real tag → unbalanced if/endif → 30 template render
  failures across the suite.
* /api/stack/subscription/{type}/{id}: DELETE now returns 204 instead
  of 200 per the 0.54.26 design rules. CLI client + parity tests
  updated to accept 2xx / assert 204.
* Memory-domain suggestion approve/reject paths added to
  ``_VERB_PATH_ALLOWLIST`` — they are pending → approved/rejected
  state-machine transitions (approve also creates the real
  memory_domains row as a side effect), so the RPC shape is
  intentional rather than a missed PATCH refactor.

5035 tests passing, 35 skipped.

* fix(catalog_table_detail): real polish pass — hero glyph, dedup pills, rows/size meta, scoped sync CTA

The previous fix only got the block-name typo so the existing CSS rendered.
The actual layout was still wireframe-tier on close inspection:

* No cover glyph in the hero (a flat white card with title + meta line);
  data-package + memory-domain detail pages both have a colored icon
  square. Restored parity — table.icon emoji if set, otherwise initials
  on a colored square using table.color.
* "INTERNAL" pill rendered twice for agnes_audit etc. — the mode pill
  and the source-type pill happened to be identical strings. Now skip
  the source pill when it matches the mode (`internal == internal`).
* Bucket / source_table code chip showed `Agnes Internal.audit_log` for
  internal rows — meaningless to a user. Hidden when source_type is
  internal.
* `pairs_well_with` admin input was a comma-separated `<input>` always
  visible. Wrapped all 4 sections in an Edit-on-demand toggle: read-
  only display by default, "+ Add" / "Edit" button on the right edge
  of each section header reveals the inline form, Cancel hides it.
* "Trigger sync now" was a cramped link squashed into the empty-state
  flex row (visible as `Tr…` overflow before). Promoted to a proper
  btn-primary button under the empty-state copy. Hidden entirely for
  internal tables (which are server-managed — no upstream to pull).
* Hero meta now surfaces row count + payload size (when sync_state has
  them) + last sync timestamp on a single line — was missing from the
  original.
* Mode pills colored by tier (local=green, remote=amber, materialized=
  blue, internal=gray) so the basic fact about a table reads at a
  glance, not from upper-cased ALL-CAPS text alone.

* tests(v56): TDD baseline for extended data-packages content + per-table docs

68 failing tests across 8 files spec the v56 surface before any
implementation lands:

* test_schema_v55_to_v56_migration.py — schema bump, additive ALTERs
  on data_packages + table_registry, idempotency, sequential-upgrade
  preservation
* test_data_packages_repo_v56.py — repo create/update/get/list for
  owner_name, owner_team, tags, long_description, when_to_use,
  when_not_to_use, example_questions (JSON list round-trip, empty
  defaults, partial-update preservation)
* test_table_registry_v56_docs.py — update_docs for grain, platforms,
  partition_col, history, gotchas; preserves v52 docs columns
* test_api_data_packages_v56.py — PUT/POST/GET for all new fields,
  field-level validation (tag count, bullet length, description size),
  virtual badge derivation (curated/new)
* test_api_registry_docs_v56.py — PATCH /api/admin/registry/{id}/docs
  for v56 fields, validation, RBAC unchanged
* test_web_catalog_package_detail_v56.py — /catalog/p/<slug> rewrite
  asserts on rendered owner line, tag pills, badges, What it is,
  Use it when, Skip it when, Example questions, per-table extended
  detail in collapsible row, key-gotcha distinctness, admin-only Edit
* test_web_stack_card_v56_metadata.py — Browse-grid card additions
  (owner chip, tag chips, badges) without breaking back-compat for
  rows missing the new fields
* test_data_packages_no_vendor_content.py — CI guard: scans app/ +
  src/ + cli/ + config/ + scripts/ for Groupon-specific tokens from
  the colleague's spec MD; fails if any leak into OSS surfaces
* test_db_schema_version.py — bumped 55 → 56 with rationale

Plus updates schema-version assertion to 56. Implementation lands in
subsequent commits (schema migration → repo → API → templates).

* feat(v56): schema + repo for extended data-packages content

Schema additions (ALTER ADD COLUMN IF NOT EXISTS — additive + idempotent):

* data_packages: owner_name, owner_team, tags, long_description,
  when_to_use, when_not_to_use, example_questions (JSON-as-VARCHAR for
  the lists)
* table_registry: grain, platforms, partition_col, history, gotchas
  (extends the v52 sample_questions / things_to_know / pairs_well_with
  docs surface with structured per-table content)

Repo extensions:

* DataPackagesRepository.create + update accept the new fields with
  the same Optional-is-no-op contract as v51 (pass an empty list to
  clear a JSON column)
* _decode_row decodes the new JSON-list columns to Python lists; NULL
  rounds back to [] so callers don't branch
* TableRegistryRepository.update_docs grew the v56 fields alongside
  the existing v52 ones — single PATCH can write either tier
  atomically
* TableRegistryRepository._decode_row picks up platforms + gotchas in
  the same NULL-tolerant decoder

22 repo + migration tests passing. API + UI land in subsequent commits.

* feat(v56): API surface for extended data-packages + per-table docs

CreateDataPackageRequest + UpdateDataPackageRequest grew the v56 fields
(owner_name, owner_team, tags, long_description, when_to_use,
when_not_to_use, example_questions) with per-field validators that
match the Foundry spec checklist:

  * tags: ≤8 entries × ≤30 chars
  * long_description: ≤4000 chars
  * use/skip: ≤8 bullets × ≤200 chars
  * example_questions: ≤12 × ≤200 chars

_serialize emits all v56 fields plus a virtual ``badges`` list derived
server-side at render time (no DB column needed): "curated" when the
creator is in the Admin group, "new" within 30 days of created_at.
Backdating created_at or admin-status changes pick up automatically.

PATCH /api/admin/registry/{id}/docs extended with v56 structured
per-table fields (grain, platforms, partition_col, history, gotchas).
gotchas: list of {key: bool, body: str} Pydantic models with the same
≤8 cap; first key=true entry becomes the Key gotcha on the rendered
package detail page. PATCH echoes the fresh state so callers can
re-render without a second GET.

26 API tests passing (16 data-packages + 10 registry-docs).

* feat(v56): /catalog/p/<slug> rewrite + Browse-grid card augmentation

The third (and final) v56 commit lights up the UI surfaces backed by
the schema + API commits earlier in this PR:

* /catalog/p/<slug> template rebuilt around the Foundry spec's
  section ladder — hero (icon + name + badges + owner + tags +
  description + meta + Add-to-stack), "What it is" markdown body,
  paired "Use it when / Skip it when" panels, "Tables in this
  package" with collapsible per-table extended detail (grain /
  platforms / partition_col / history / gotchas + sample questions),
  and an "Example questions you can ask Claude" prompt panel. Each
  section guarded by ``{% if pkg.<field> %}`` — empty content fields
  hide the section entirely (no "No X yet" placeholder noise on the
  public-facing drilldown).
* router catalog_package_detail hydrates per-table v56 fields onto
  the tables list + derives the virtual badges (curated / new)
  server-side from creator-in-Admin + 30-day created_at.
* StackResolver.ResourceEntry grew owner_name / owner_team / tags /
  badges; _fetch_entries pulls the v56 columns + computes badges
  once per fetch using a single Admin-group SELECT.
* _data_package_entry_dict adapter passes the new fields through to
  the macro; tags are merged source-type pills + admin-authored
  category tags per the spec convention.
* _stack_card.html renders the v56 badges (top-left, data-badge=
  hooks) + the owner chip (data-card-owner hook) without breaking
  back-compat — pre-v56 rows render unchanged.
* Admin PUT handler strips the v56 docs fields from the
  read-modify-write merged dict so register() doesn't blow up
  with the now-larger row shape (same pattern as the v52 docs
  fields stripping).

5115 tests passing (+98 v56 + 18 fixed regressions from the merged-
register PUT path), 35 skipped.

* fix(rbac): Edit-on-package + Group-access 'required' persistence + CI vendor guard

Three related bugs reported on the merged-with-main branch:

1. Clicking Edit on a Data Package card landed on /admin/tables with
   a `#<pkg.id>` hash that nothing listened to — admin saw the global
   table listing, not the editor for that specific package. Added a
   `?edit_package=<pkg_id>` query-param handler in admin_tables.html
   (analog to the existing `?edit=<table_id>` and `?assign_to=<pkg_id>`
   patterns) that calls openEditDataPackageModal on DOMContentLoaded
   after a 250ms layout settle. Updated the package-detail Edit link
   to use the new query param.

2. Setting Group Access to 'required' didn't persist — re-opening
   the modal showed 'available'. Root cause was the v49
   ``resource_grants.requirement`` enum existing in the DB but the
   POST /api/admin/grants endpoint not surfacing it: ``CreateGrantRequest``
   declared only group_id + resource_type + resource_id, so Pydantic
   silently dropped the matrix's ``requirement: 'required'`` payload
   and the new row landed at the DB column default ('available').
   Plumbed ``requirement`` through ``CreateGrantRequest`` →
   ``ResourceGrantsRepository.create`` so the value persists in one
   round-trip. Plus a UNIQUE-constraint race in the matrix
   diff-apply: DELETE-old + POST-new ran in parallel via
   ``Promise.allSettled``, so POST could fire first and trip the
   unique check before DELETE freed the slot. Switched to sequential
   (await all deletes; then await all writes) across all three
   matrices (Edit Data Package, Edit Memory Domain, Edit Recipe).

3. CI vendor-content guard ``test_no_groupon_specific_strings_in_oss``
   tripped on two of my own docstrings: a "Foundry Data team" mention
   in two src/db.py comments + an ``s1_session_landings`` example in
   cli/skills/agnes-table-registration.md. Rephrased the comments to
   "extended-descriptions admin spec" and replaced the example with
   a generic ``events_daily`` table name.

5164 tests passing, 35 skipped (+4 regression tests pinning the POST
/api/admin/grants requirement contract). Vendor guard back to green.

* fix(catalog): admin Browse path drops v58 card fields

The /catalog and /memory admin god-mode branch built ResourceEntry
instances inline from pkg_repo.list() / domains_repo.list() and skipped
owner_name, owner_team, tags, and derived badges (curated/new). Visible
symptom: a package with an owner + tags rendered with the v56 chrome
for non-admin viewers but as a bare card for admins.

Adds StackResolver.browse_admin(user_id, resource_type) — admin god-mode
Browse that walks the full table but routes through the same
_fetch_entries enrichment pass as browse(), so admin + non-admin Browse
stay visually consistent. Both /catalog and /corporate-memory routes
switch to it.

Regression test in tests/test_stack_resolver_browse_admin.py covers:
owner/tags propagation, new/curated badge derivation, in_stack from
admin subscriptions, all-packages-regardless-of-grants, and the
ValueError for unsupported resource types.

* fix(catalog): three /catalog tab-strip UX bugs

1. Required Remove → red toast
   browse_admin passed empty required_ids to _fetch_entries, so the
   admin's own required grants surfaced as 'available' and the macro
   rendered an actionable Remove button that POST /unsubscribe 400'd
   on. Now derives required_ids from the admin's own groups so
   Required packages render with the disabled "In stack (required)"
   button. Regression test in test_stack_resolver_browse_admin.py.

2. Remove green-toasts but card stays until refresh
   The My-Stack empty-state placeholder was only emitted server-side
   when stack_entries was empty at render time. Removing the last
   card left the tab completely blank — users read that as "Remove
   didn't work, let me refresh". Both grid + empty-state are now
   always rendered with one of them initially hidden; the JS swaps
   visibility on add/remove instead of injecting DOM. Same fix in
   /corporate-memory.

3. "What are Recipes?" + ambiguous (admin) suffix
   Recipes tab now carries its own curator-block explainer (the
   shared one was moved inside Browse view so it doesn't bleed
   across tabs). The grey "(admin)" suffix becomes a yellow
   .admin-only-hint chip with a title tooltip — visibility hint is
   now unambiguous: yellow chip = "only you see this", non-admins
   don't see the affordance at all.

* schema: renumber v51..v58 → v52..v59 to make room for main's v51

Main 0.54.29 introduced a NEW v51 (table_registry.bq_fqn — issue #343)
that releases ahead of this branch. The unified-stack chain v51..v58
shifts up by one so main's v51 stays as the released schema and ours
become v52..v59. Function names, internal version bumps, dispatch
ladder thresholds, and the migration-test references all move
together. Subsequent merge with main lands the bq_fqn column at the
freed v51 slot.

* fix(seed): seed admin lands in BOTH Admin AND Everyone groups

The LOCAL_DEV_MODE / SEED_ADMIN_EMAIL bootstrap only added the seed
user to Admin. Everyone-scoped grants — the canonical "every-user-
sees-this" pattern for Required onboarding — didn't surface for the
seed admin's own /catalog because they weren't in Everyone. Symptom:
admin grants a Required-tier package to Everyone, then sees it on
/catalog still rendered with an "Add to stack" button (because the
admin's resolved required_ids was empty for that package).

The dual-membership keeps Admin (authorization) and Everyone
(default-grant target) intentionally separate per the design comment
on UserRepository.create — every membership remains traceable to a
concrete row, just now with a system_seed row in Everyone too. Both
INSERTs go through UserGroupMembersRepository.add_member which is
idempotent on (user_id, group_id), so re-fires on every lifespan
startup don't duplicate rows.

Regression test in test_main_seed_admin_everyone.py.

* style: unify admin-only hints across marketplace + memory detail pages

Replaces three stale ``(admin)`` parentheticals with the same yellow
``admin-only`` chip introduced for /catalog tab actions. Same tooltip
copy ("Visible only to admins — analysts won't see this …") so the
visibility hint is unmistakable wherever it appears:

- Hard delete on marketplace_plugin_detail (admin-only destructive
  action — same gating as the original suffix conveyed).
- Hard delete on marketplace_item_detail (same).
- Edit link on memory_domain_detail (title-attr only before; now a
  visible chip too).

Non-admin viewers never saw these affordances — the gates are
unchanged. Pure styling pass for consistency.

* fix(catalog): exclude soft-deleted data packages + memory domains from Browse

``StackResolver._fetch_entries`` and ``browse_admin`` were querying
data_packages / memory_domains without a ``deleted_at IS NULL`` guard.
A package soft-deleted via /admin/* (v54 soft-delete contract) stayed
visible on /catalog and /memory until either an Undo or a hard delete
— directly contradicting the soft-delete UX which is supposed to
remove the affordance immediately and only retain the row for the
Undo window.

The repository accessors (DataPackagesRepository.list,
MemoryDomainsRepository.list, list_packages_of_table, etc.) already
filter deleted rows; this commit brings the resolver's direct SQL in
line with that contract.

Regression test in test_stack_resolver_browse_admin.py.

* fix(catalog): Add/Remove updates full card chrome, not just button

The previous _applyStackChange flipped only the footer button label —
the card border (.is-in-stack class), top-right "In stack" badge, and
button color class (--add / --remove) stayed at their server-rendered
state. After Add the user saw the button checkmark but the rest of
the card still looked like "available, not in stack". They read this
as "the change didn't take — let me refresh".

This commit makes the optimistic update mirror what the server-side
macro renders for the new state:

* ``c.classList.toggle('is-in-stack', becameInStack)`` — flips the
  border + visual state class.
* Top-right ``.stack-card__req-badge--instack`` badge is injected on
  Add, removed on Remove (skipped when ``data-requirement='required'``
  — that slot is owned by the Required badge).
* Button text is "Remove" / "+ Add to stack" matching the macro
  (was "✓ In stack" which was visually nice but inconsistent).
* Button color class --add / --remove swaps so the destructive Remove
  tint kicks in immediately.

The clone-into-My-Stack path applies the same updates so the new card
in My Stack reads identically to a server-rendered in_stack card.
Mirrored in /corporate-memory.

* fix(memory): four Devin-review bugs on /memory drill-down + manifest

PR #333 Devin review surfaced four real bugs that ship a broken
/memory experience even though the unit tests passed.

1. Manifest md5 omits is_required + content (app/api/sync.py:836-840)
   _build_memory_domains_section hashed only (id|title|status) per
   item. _build_per_domain_markdown routes items between "## Required"
   and "## Approved" by is_required and embeds full content — so an
   admin edit of either dimension left the manifest md5 unchanged,
   `agnes pull` skipped the re-fetch, and the analyst kept a stale
   bundle.md. Now both fields participate in the hash.

2. required_count always 0 (src/repositories/memory_domains.py)
   list_items_of_domain only SELECTed (id, title, status) so the
   `it.get("is_required")` in the manifest builder always evaluated
   to None → required_count = 0 regardless of actual state. The
   manifest builder advertised a count it could never compute. Now
   projects is_required + content too (required by fix 1 anyway).

3. Vote URL 404 (memory_domain_detail.html:289-290)
   Constructed `/api/memory/items/{id}/vote` but the route is
   `/api/memory/{id}/vote`. Every upvote/downvote button was a
   silent no-op.

4. Dismiss/undismiss URL + method both wrong (memory_domain_detail.html:296-305)
   Constructed `/api/memory/items/{id}/dismiss` (extra /items/) and
   /undismiss (no such route — undismiss is DELETE on /dismiss).
   Both buttons silently 404'd. Now POST + DELETE on
   `/api/memory/{id}/dismiss` per app/api/memory.py:635/675.

* fix: multi-agent reviewer findings — vendor-token scrubs + manifest md5 predicate + soft-delete filter

Three reviewer findings from the multi-agent review on PR #333,
fixed in-place per CLAUDE.md issue-economy rule.

Reviewer-rules (Important — vendor-agnostic OSS):
- app/main.py:218 comment: replaced 'foundryai-prod' with generic
  'a customer prod instance' phrasing. Public OSS repo must not
  carry customer-specific tokens (CLAUDE.md § Project conventions).
- tests/test_table_registry_v56_docs.py:70 fixture string:
  replaced "user_brand_affiliation = 'groupon'" with 'acme' on
  the same rule.

Reviewer-architecture (closes still-unresolved Devin 🚩 ANALYSIS):
- app/api/sync.py _build_memory_domains_section: md5 hash loop now
  filters items to the SAME predicate the bundle renderer uses
  (is_required OR status='approved'). Pre-fix the hash iterated ALL
  items but _build_per_domain_markdown only rendered the union of
  required items + approved-non-required items — so an admin edit
  to a pending/rejected non-required item flipped the md5 against
  an identical-bytes bundle, triggering a wasteful re-fetch on
  every analyst's next 'agnes pull'. The earlier commit fixed the
  hash-input fields (is_required + content); this closes the
  set-of-items asymmetry Devin separately flagged.

Reviewer-RBAC (minor cleanup):
- app/resource_types.py _data_package_blocks and _memory_domain_blocks
  now filter 'WHERE deleted_at IS NULL' (v54 soft-delete column) so
  the /admin/access UI doesn't surface soft-deleted entities as
  grantable. Mirrors the existing filter on _recipe_blocks. No
  security leak pre-fix (resolver double-filters and re-checks at
  serve time), just UI cleanliness.
- app/services/stack_resolver.py add_to_stack: docstring note
  added explaining that authorization is enforced at the API layer
  (app/api/stack.py can_access gate), not at the resolver. The
  initial review suggested adding a defensive 403 here, but that
  broke 5 existing tests that legitimately call add_to_stack
  directly without setting up grants first; the docstring captures
  the contract instead. stack() already intersects subscriptions
  with current available_ids on every read, so a 'zombie' row from
  a misuse never leaks into the user-facing manifest.

* release: 0.55.0 — unified Browse + My Stack (Data Packages + Memory), schema v48→v59, 3 BREAKING
2026-05-19 15:00:15 +02:00

811 lines
36 KiB
HTML
Raw Blame History

{% extends "base.html" %}
{% block title %}Activity — Admin{% endblock %}
{% block content %}
<div class="obs-page page-shell">
<!-- TOP BAR --------------------------------------------------------- -->
<header class="obs-topbar">
<div class="obs-topbar-left">
{% set page_hero_eyebrow = "Activity Center" %}
{% set page_hero_title = "Audit log" %}
{% set page_hero_subtitle = "Every admin and user-mutating action — RBAC writes, sync triggers, marketplace edits, …" %}
{% include "_page_hero.html" %}
<div class="obs-narrative" id="obs-narrative">Loading…</div>
</div>
<div class="obs-topbar-right">
<label class="obs-window">
<span class="obs-label">Time range</span>
<select id="obs-window">
<option value="60">Last 1h</option>
<option value="360">Last 6h</option>
<option value="1440" selected>Last 24h</option>
<option value="10080">Last 7d</option>
<option value="43200">Last 30d</option>
</select>
</label>
<label class="obs-live" title="Re-fetches KPIs + table every 30s. Independent of the time range — the range still controls what counts as 'recent'.">
<input type="checkbox" id="obs-live" />
<span>Auto-refresh (30s)</span>
<span class="obs-live-dot" id="obs-live-dot" aria-hidden="true"></span>
</label>
</div>
</header>
<!-- KPI CARDS ------------------------------------------------------- -->
<section class="obs-kpis" aria-label="Key metrics">
<button class="obs-kpi" data-filter="" type="button" aria-label="All events in window — click to reset filters">
<span class="obs-kpi-label">Events</span>
<span class="obs-kpi-value" id="kpi-events">—</span>
</button>
<button class="obs-kpi" data-filter="users" type="button" aria-label="Active users — click to focus user filter">
<span class="obs-kpi-label">Active users</span>
<span class="obs-kpi-value" id="kpi-users">—</span>
</button>
<button class="obs-kpi" data-filter="errors" type="button" aria-label="Errors — click to filter to errors only">
<span class="obs-kpi-label">Error rate</span>
<span class="obs-kpi-value" id="kpi-errors">—</span>
<span class="obs-kpi-sub" id="kpi-errors-sub">—</span>
</button>
<button class="obs-kpi" data-filter="slow" type="button" aria-label="Latency p95 — click to sort table by duration desc">
<span class="obs-kpi-label">p95 latency</span>
<span class="obs-kpi-value" id="kpi-p95">—</span>
</button>
</section>
<!-- FILTER BAR ------------------------------------------------------ -->
<section class="obs-filters" aria-label="Filter audit log">
<div class="obs-filter">
<label for="f-user">User</label>
<select id="f-user" class="obs-select"><option value="">Any</option></select>
</div>
<div class="obs-filter">
<label for="f-action">Action</label>
<select id="f-action" class="obs-select"><option value="">Any</option></select>
</div>
<div class="obs-filter">
<label for="f-result">Result</label>
<select id="f-result" class="obs-select">
<option value="">Any</option>
<option value="success">success only</option>
<option value="error%">errors only</option>
</select>
</div>
<div class="obs-filter">
<label for="f-source">Source</label>
<select id="f-source" class="obs-select"><option value="">Any</option></select>
</div>
<div class="obs-filter">
<label for="f-resource">Resource</label>
<select id="f-resource" class="obs-select" aria-label="Filter by resource type">
<option value="">Any</option>
<option value="table:">Tables</option>
<option value="knowledge_item:">Knowledge items</option>
<option value="marketplace:">Marketplaces</option>
<option value="store_submission:">Store submissions</option>
<option value="store_entity:">Store entities</option>
<option value="store_upload:">Store uploads</option>
<option value="user:">Users</option>
<option value="token:">Tokens</option>
<option value="job:">Scheduled jobs</option>
<option value="memory_domain_suggestion:">Memory domain suggestions</option>
</select>
</div>
<div class="obs-filter obs-filter-grow">
<label for="f-q">Search params</label>
<input id="f-q" type="search" placeholder="token, table id, correlation…" />
</div>
<div class="obs-filter-actions">
<button id="f-reset" class="obs-btn" type="button">Reset</button>
</div>
</section>
<!-- AUDIT TABLE ----------------------------------------------------- -->
<section class="obs-table-wrap">
<table class="obs-table" id="obs-table">
<thead>
<tr>
{# Sort is page-local: server returns rows ordered by timestamp DESC
for keyset pagination; clicking another header re-sorts only the
rows currently loaded. Adding a `Load all` step would let the
user sort across the full result set but isn't wired yet. #}
<th data-sort="timestamp" aria-sort="descending"><button type="button">Time <span class="obs-sort-arrow"></span></button></th>
<th data-sort="user"><button type="button">User <span class="obs-sort-arrow"></span></button></th>
<th data-sort="source"><button type="button">Source <span class="obs-sort-arrow"></span></button></th>
<th data-sort="action"><button type="button">Action <span class="obs-sort-arrow"></span></button></th>
<th data-sort="resource"><button type="button">Resource <span class="obs-sort-arrow"></span></button></th>
<th data-sort="duration_ms"><button type="button">Duration <span class="obs-sort-arrow"></span></button></th>
<th data-sort="result"><button type="button">Result <span class="obs-sort-arrow"></span></button></th>
</tr>
</thead>
<tbody id="obs-rows">
<tr><td colspan="7" class="obs-empty">Loading…</td></tr>
</tbody>
</table>
<div class="obs-table-footer">
<span id="obs-row-count" class="obs-row-count">—</span>
<button id="obs-loadmore" class="obs-btn" type="button" hidden>Load more</button>
</div>
</section>
</div>
<!-- DETAIL SIDE PANEL --------------------------------------------------- -->
<aside class="obs-panel" id="obs-panel" hidden aria-label="Event detail">
<header class="obs-panel-head">
<h2 id="obs-panel-title">Event</h2>
<button class="obs-btn" id="obs-panel-close" type="button" aria-label="Close">✕</button>
</header>
<dl class="obs-panel-meta" id="obs-panel-meta"></dl>
<h3>Params</h3>
<pre class="obs-panel-pre" id="obs-panel-params">—</pre>
<h3 class="obs-panel-result-h">Result</h3>
<pre class="obs-panel-pre" id="obs-panel-result">—</pre>
<div class="obs-panel-actions">
<button class="obs-btn" id="obs-panel-filter-user" type="button">Filter to this user</button>
<button class="obs-btn" id="obs-panel-filter-action" type="button">Filter to this action</button>
<button class="obs-btn" id="obs-panel-filter-resource" type="button" hidden>Filter to this resource type</button>
</div>
</aside>
<style>
:root {
--obs-bg: var(--surface, #fff);
--obs-fg: var(--text-primary, #111827);
--obs-muted: var(--text-secondary, #6b7280);
--obs-border: var(--border, #e5e7eb);
--obs-soft: var(--border-light, #f3f4f6);
--obs-ok: #065f46;
--obs-ok-bg: #d1fae5;
--obs-err: #991b1b;
--obs-err-bg: #fee2e2;
--obs-warn: #92400e;
--obs-warn-bg: #fef3c7;
--obs-link: #2563eb;
}
.obs-page { font: 14px/1.5 var(--font-sans, system-ui); color: var(--obs-fg); }
/* Top bar -------------------------------------------------------------- */
.obs-topbar {
display: flex; align-items: flex-start; justify-content: space-between;
gap: 24px; padding: 12px 0 20px;
border-bottom: 1px solid var(--obs-border);
margin-bottom: 16px;
}
.obs-topbar-left { flex: 1; min-width: 0; }
.obs-narrative { font-size: 16px; color: var(--obs-fg); line-height: 1.45; }
.obs-narrative b { font-weight: 600; }
.obs-narrative a { color: var(--obs-link); text-decoration: none; border-bottom: 1px dashed transparent; cursor: pointer; }
.obs-narrative a:hover { border-bottom-color: var(--obs-link); }
.obs-topbar-right { display: flex; align-items: center; gap: 12px; flex-wrap: wrap; }
.obs-label { font-size: 11px; text-transform: uppercase; letter-spacing: 0.4px;
color: var(--obs-muted); display: block; margin-bottom: 2px; }
.obs-window select { padding: 6px 8px; border: 1px solid var(--obs-border); border-radius: 6px; background: var(--obs-bg); font: inherit; }
.obs-live { display: flex; align-items: center; gap: 6px; font-size: 13px; color: var(--obs-muted); cursor: pointer; }
.obs-live-dot { width: 8px; height: 8px; border-radius: 50%; background: #d1d5db; transition: background 0.2s; }
.obs-live input:checked ~ .obs-live-dot { background: #10b981; animation: obs-pulse 2s ease-in-out infinite; }
@keyframes obs-pulse {
0%, 100% { box-shadow: 0 0 0 0 rgba(16, 185, 129, 0.6); }
50% { box-shadow: 0 0 0 4px rgba(16, 185, 129, 0); }
}
/* Saved views dropdown -------------------------------------------------- */
.obs-views { position: relative; }
.obs-btn {
border: 1px solid var(--obs-border); background: var(--obs-bg); color: var(--obs-fg);
padding: 6px 12px; border-radius: 6px; font: inherit; cursor: pointer;
}
.obs-btn:hover { background: var(--obs-soft); }
.obs-btn-primary { background: var(--obs-link); color: #fff; border-color: var(--obs-link); }
.obs-btn-primary:hover { filter: brightness(1.07); background: var(--obs-link); }
.obs-chev { font-size: 10px; color: var(--obs-muted); }
.obs-views-panel {
position: absolute; right: 0; top: calc(100% + 6px);
background: var(--obs-bg); border: 1px solid var(--obs-border); border-radius: 8px;
box-shadow: 0 8px 24px rgba(0,0,0,0.08);
width: 280px; padding: 8px; z-index: 50;
}
.obs-views-panel hr { border: 0; border-top: 1px solid var(--obs-soft); margin: 6px 0; }
.obs-views-list { max-height: 280px; overflow-y: auto; }
.obs-view-item {
display: flex; align-items: center; justify-content: space-between;
padding: 6px 8px; border-radius: 4px; cursor: pointer;
}
.obs-view-item:hover { background: var(--obs-soft); }
.obs-view-name { flex: 1; min-width: 0; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.obs-view-del { opacity: 0; transition: opacity 0.15s; color: var(--obs-err); border: 0; background: none; cursor: pointer; padding: 2px 6px; }
.obs-view-item:hover .obs-view-del { opacity: 1; }
.obs-views-empty { padding: 12px 8px; color: var(--obs-muted); font-size: 13px; text-align: center; }
/* KPI cards ------------------------------------------------------------ */
.obs-kpis { display: grid; grid-template-columns: repeat(4, 1fr); gap: 12px; margin-bottom: 16px; }
.obs-kpi {
text-align: left; padding: 14px 16px; border: 1px solid var(--obs-border); border-radius: 8px;
background: var(--obs-bg); cursor: pointer; transition: border-color 0.15s, background 0.15s;
display: flex; flex-direction: column; gap: 4px;
}
.obs-kpi:hover { border-color: var(--obs-link); background: var(--obs-soft); }
.obs-kpi-label { font-size: 11px; text-transform: uppercase; letter-spacing: 0.4px; color: var(--obs-muted); }
.obs-kpi-value { font-size: 22px; font-weight: 600; font-variant-numeric: tabular-nums; color: var(--obs-fg); }
.obs-kpi-sub { font-size: 12px; color: var(--obs-muted); font-variant-numeric: tabular-nums; }
/* Filter bar ----------------------------------------------------------- */
.obs-filters {
display: flex; gap: 12px; align-items: flex-end; flex-wrap: wrap;
padding: 12px 14px; background: var(--obs-soft); border-radius: 8px;
margin-bottom: 12px;
}
.obs-filter { display: flex; flex-direction: column; gap: 4px; min-width: 140px; }
.obs-filter-grow { flex: 1; min-width: 200px; }
.obs-filter label {
font-size: 11px; text-transform: uppercase; letter-spacing: 0.4px; color: var(--obs-muted);
}
.obs-select, .obs-filter input {
padding: 6px 8px; border: 1px solid var(--obs-border); border-radius: 6px; background: var(--obs-bg); font: inherit;
}
.obs-filter input { min-width: 200px; }
.obs-filter-actions { display: flex; align-items: flex-end; }
/* Audit table ---------------------------------------------------------- */
.obs-table-wrap {
border: 1px solid var(--obs-border); border-radius: 8px; overflow: hidden; background: var(--obs-bg);
}
.obs-table { width: 100%; border-collapse: collapse; font-size: 13px; }
.obs-table thead th {
text-align: left; padding: 10px 12px; background: var(--obs-soft);
border-bottom: 1px solid var(--obs-border);
font-weight: 600; color: var(--obs-muted); font-size: 11px;
text-transform: uppercase; letter-spacing: 0.4px; white-space: nowrap;
}
.obs-table thead th button {
background: none; border: 0; padding: 0; font: inherit; color: inherit; cursor: pointer;
display: inline-flex; align-items: center; gap: 4px; text-transform: inherit; letter-spacing: inherit;
}
.obs-table thead th[aria-sort="ascending"] .obs-sort-arrow::before { content: "▲"; }
.obs-table thead th[aria-sort="descending"] .obs-sort-arrow::before { content: "▼"; }
.obs-sort-arrow { font-size: 9px; color: var(--obs-muted); }
.obs-table tbody td {
padding: 8px 12px; border-bottom: 1px solid var(--obs-soft); vertical-align: top;
}
.obs-table tbody tr:last-child td { border-bottom: none; }
.obs-table tbody tr { cursor: pointer; }
.obs-table tbody tr:hover { background: var(--obs-soft); }
.obs-table tbody tr.is-selected { background: #eff6ff; }
.obs-time { white-space: nowrap; font-variant-numeric: tabular-nums; color: var(--obs-muted); }
.obs-user-primary { color: var(--obs-fg); }
.obs-user-id { color: var(--obs-muted); font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 11px; }
.obs-action-cell { font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px; }
.obs-dur-cell { text-align: right; font-variant-numeric: tabular-nums; color: var(--obs-muted); }
.obs-result-ok { color: var(--obs-ok); background: var(--obs-ok-bg); padding: 2px 6px; border-radius: 4px; font-size: 11px; }
.obs-result-err { color: var(--obs-err); background: var(--obs-err-bg); padding: 2px 6px; border-radius: 4px; font-size: 11px; }
.obs-source-pill { display: inline-block; padding: 2px 6px; border-radius: 4px; font-size: 11px;
text-transform: lowercase; letter-spacing: 0.2px; background: var(--obs-soft); color: var(--obs-muted); }
.obs-source-pill.is-scheduler { background: #ede9fe; color: #5b21b6; }
.obs-source-pill.is-cli { background: #dbeafe; color: #1e40af; }
.obs-source-pill.is-web { background: #dcfce7; color: #166534; }
.obs-empty { padding: 40px 16px; text-align: center; color: var(--obs-muted); font-size: 13px; }
.obs-table-footer {
display: flex; justify-content: space-between; align-items: center;
padding: 10px 14px; border-top: 1px solid var(--obs-border); background: var(--obs-soft);
}
.obs-row-count { color: var(--obs-muted); font-size: 12px; }
/* Detail side panel ---------------------------------------------------- */
.obs-panel {
position: fixed; top: 0; right: 0; bottom: 0; width: min(520px, 50vw);
background: var(--obs-bg); border-left: 1px solid var(--obs-border);
box-shadow: -8px 0 24px rgba(0,0,0,0.06); padding: 20px; overflow-y: auto;
z-index: 100;
}
.obs-panel-head { display: flex; align-items: center; justify-content: space-between; margin: 0 0 16px; }
.obs-panel-head h2 { margin: 0; font-size: 16px; font-weight: 600; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
.obs-panel-meta { margin: 0 0 16px; display: grid; grid-template-columns: 110px 1fr; gap: 6px 12px; font-size: 13px; }
.obs-panel-meta dt { color: var(--obs-muted); }
.obs-panel-meta dd { margin: 0; word-break: break-word; }
.obs-panel h3 { font-size: 11px; text-transform: uppercase; letter-spacing: 0.4px; color: var(--obs-muted); margin: 16px 0 6px; }
.obs-panel-pre {
background: var(--obs-soft); border-radius: 6px; padding: 10px 12px;
margin: 0; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; font-size: 12px;
white-space: pre-wrap; word-break: break-word; max-height: 280px; overflow: auto;
}
.obs-panel-actions { display: flex; gap: 8px; margin-top: 16px; flex-wrap: wrap; }
/* Save-view modal ------------------------------------------------------ */
/* `display: flex` would override the [hidden] attribute (UA stylesheet
only sets `display: none` at lower specificity than author CSS), so the
modal would render on page load. Gate display on the absence of the
attribute. */
.obs-modal {
position: fixed; inset: 0; background: rgba(17, 24, 39, 0.4);
align-items: center; justify-content: center; z-index: 200;
}
.obs-modal:not([hidden]) { display: flex; }
.obs-modal[hidden] { display: none; }
/* Same trap on the side panel + saved-views dropdown, prophylactically. */
.obs-panel[hidden],
.obs-views-panel[hidden] { display: none; }
.obs-modal-body {
background: var(--obs-bg); border-radius: 12px; padding: 24px; width: min(420px, 90vw);
box-shadow: 0 24px 48px rgba(0,0,0,0.18);
}
.obs-modal-body h2 { margin: 0 0 8px; font-size: 18px; }
.obs-modal-help { margin: 0 0 12px; color: var(--obs-muted); font-size: 13px; }
.obs-modal-body label { display: block; font-size: 12px; color: var(--obs-muted); margin-bottom: 4px; }
.obs-modal-body input[type=text] {
width: 100%; padding: 8px 10px; border: 1px solid var(--obs-border); border-radius: 6px; font: inherit;
box-sizing: border-box;
}
.obs-modal-actions { display: flex; justify-content: flex-end; gap: 8px; margin-top: 16px; }
</style>
<script>
(function() {
// ---------------------------------------------------------------------
// State + URL sync
// ---------------------------------------------------------------------
const state = readUrl();
let nextCursor = null;
let livePollId = null;
function readUrl() {
const p = new URLSearchParams(location.search);
return {
window: parseInt(p.get('window'), 10) || 1440,
user_id: p.get('user_id') || '',
action_prefix: p.get('action_prefix') || '',
result: p.get('result') || '',
source: p.get('source') || '',
resource_prefix: p.get('resource_prefix') || '',
q: p.get('q') || '',
sort: p.get('sort') || 'timestamp:desc',
live: p.get('live') === '1',
};
}
function writeUrl() {
const p = new URLSearchParams();
if (state.window !== 1440) p.set('window', state.window);
if (state.user_id) p.set('user_id', state.user_id);
if (state.action_prefix) p.set('action_prefix', state.action_prefix);
if (state.result) p.set('result', state.result);
if (state.source) p.set('source', state.source);
if (state.resource_prefix) p.set('resource_prefix', state.resource_prefix);
if (state.q) p.set('q', state.q);
if (state.sort && state.sort !== 'timestamp:desc') p.set('sort', state.sort);
if (state.live) p.set('live', '1');
const qs = p.toString();
history.replaceState(null, '', qs ? '?' + qs : location.pathname);
}
// ---------------------------------------------------------------------
// DOM refs
// ---------------------------------------------------------------------
const $ = id => document.getElementById(id);
const winSel = $('obs-window');
const live = $('obs-live');
const fUser = $('f-user');
const fAct = $('f-action');
const fRes = $('f-result');
const fSrc = $('f-source');
const fResource = $('f-resource');
const fQ = $('f-q');
const fReset = $('f-reset');
const tbody = $('obs-rows');
const more = $('obs-loadmore');
const rowCount = $('obs-row-count');
const narrative = $('obs-narrative');
// Apply state -> controls
winSel.value = String(state.window);
live.checked = state.live;
fRes.value = state.result;
fResource.value = state.resource_prefix;
fQ.value = state.q;
// ---------------------------------------------------------------------
// Fetch helpers
// ---------------------------------------------------------------------
async function get(url) {
const r = await fetch(url, { credentials: 'same-origin' });
if (!r.ok) throw new Error(r.status + ' ' + r.statusText);
return r.json();
}
function buildTimelineUrl(cursor) {
const p = new URLSearchParams();
p.set('since_minutes', state.window);
if (state.user_id) p.set('user_id', state.user_id);
if (state.action_prefix) p.set('action_prefix', state.action_prefix);
if (state.result) {
const v = state.result === 'success' ? 'success' : state.result; // server uses LIKE
p.set('result_pattern', v);
}
if (state.resource_prefix) p.set('resource_prefix', state.resource_prefix);
if (state.q) p.set('q', state.q);
if (cursor) { p.set('cursor_ts', cursor.ts); p.set('cursor_id', cursor.id); }
p.set('limit', '50');
return '/api/admin/activity?' + p.toString();
}
// ---------------------------------------------------------------------
// Facets — populate dropdowns
// ---------------------------------------------------------------------
async function loadFacets() {
const d = await get('/api/admin/observability/facets?since_minutes=' + state.window);
fillSelect(fUser, d.users || [], state.user_id, u => ({
value: u.id, label: (u.label || u.id) + ' (' + u.count + ')',
}));
fillSelect(fAct, (d.actions || []).map(a => a.value), state.action_prefix, v => ({
value: v, label: v + ' (' + (d.actions.find(a => a.value === v) || {}).count + ')',
}));
fillSelect(fSrc, (d.sources || []).map(s => s.value), state.source, v => ({
value: v, label: v + ' (' + (d.sources.find(s => s.value === v) || {}).count + ')',
}));
}
function fillSelect(sel, items, current, mk) {
while (sel.options.length > 1) sel.remove(1);
items.forEach(it => {
const m = typeof it === 'string' ? mk(it) : mk(it);
const o = document.createElement('option');
o.value = m.value; o.textContent = m.label;
sel.appendChild(o);
});
if (current) sel.value = current;
}
// ---------------------------------------------------------------------
// KPIs
// ---------------------------------------------------------------------
async function loadKpis() {
const d = await get('/api/admin/observability/kpis?since_minutes=' + state.window);
$('kpi-events').textContent = d.events_total.toLocaleString();
$('kpi-users').textContent = d.active_users.toLocaleString();
$('kpi-errors').textContent = (d.error_rate * 100).toFixed(2) + '%';
$('kpi-errors-sub').textContent = d.errors.toLocaleString() + ' errors';
$('kpi-p95').textContent = d.p95_duration_ms == null ? '—' : d.p95_duration_ms + ' ms';
renderNarrative(d);
}
function renderNarrative(d) {
const win = winLabel(state.window);
const filterBits = [];
if (state.user_id) filterBits.push('user filter');
if (state.action_prefix) filterBits.push('action: ' + escapeHtml(state.action_prefix));
if (state.source) filterBits.push('source: ' + escapeHtml(state.source));
if (state.result) filterBits.push('result: ' + escapeHtml(state.result));
if (state.resource_prefix) filterBits.push('resource: ' + escapeHtml(state.resource_prefix.replace(/:$/, '')));
if (state.q) filterBits.push('search: "' + escapeHtml(state.q) + '"');
const filtered = filterBits.length ? ' · filters: ' + filterBits.join(', ') : '';
if (d.events_total === 0) {
narrative.innerHTML = '<b>No activity</b> in ' + win + filtered + '. Lower the time window or reset filters.';
return;
}
narrative.innerHTML =
'<b>' + d.events_total.toLocaleString() + '</b> events in ' + win +
' · <b>' + d.active_users + '</b> users' +
' · <b>' + (d.error_rate * 100).toFixed(2) + '%</b> errors' +
(d.p95_duration_ms != null ? ' · p95 <b>' + d.p95_duration_ms + ' ms</b>' : '') +
filtered + '.';
}
function winLabel(m) {
if (m <= 60) return 'last 1h';
if (m <= 360) return 'last 6h';
if (m <= 1440) return 'last 24h';
if (m <= 10080) return 'last 7d';
return 'last 30d';
}
// ---------------------------------------------------------------------
// Table
// ---------------------------------------------------------------------
async function loadTable(append) {
if (!append) {
tbody.innerHTML = '<tr><td colspan="7" class="obs-empty">Loading…</td></tr>';
nextCursor = null;
}
const d = await get(buildTimelineUrl(append ? nextCursor : null));
const rows = d.rows || [];
// Client-side source classification + sort.
rows.forEach(r => { r._source = classifySource(r); });
// result filter: 'error%' or 'success' — already done server-side via
// result_pattern, but 'error%' is a LIKE wildcard; trust the server.
// source filter is client-side because the API doesn't know about
// our union rule yet.
let filtered = rows;
if (state.source) filtered = filtered.filter(r => r._source === state.source);
if (state.action_prefix && fAct.value && !state.action_prefix.endsWith('%')) {
// exact-match dropdown selection: prefer ==
filtered = filtered.filter(r => (r.action || '').startsWith(state.action_prefix));
}
// Sort (client side for the current page; server is already DESC by time).
// Sort key 'user' / 'source' map to derived display values; the rest read
// straight off the row.
const [sortKey, sortDir] = (state.sort || 'timestamp:desc').split(':');
const keyFor = r => {
if (sortKey === 'user') return (r.user_email || r.user_name || r.user_id || '').toLowerCase();
if (sortKey === 'source') return (r._source || '').toLowerCase();
const v = r[sortKey];
return (typeof v === 'string') ? v.toLowerCase() : v;
};
filtered.sort((a, b) => {
let av = keyFor(a), bv = keyFor(b);
if (av == null) av = '';
if (bv == null) bv = '';
if (av < bv) return sortDir === 'asc' ? -1 : 1;
if (av > bv) return sortDir === 'asc' ? 1 : -1;
return 0;
});
if (!append) tbody.innerHTML = '';
if (filtered.length === 0 && !append) {
tbody.innerHTML = '<tr><td colspan="7" class="obs-empty">No events match the current filters.</td></tr>';
more.hidden = true;
rowCount.textContent = '0 rows';
return;
}
const frag = document.createDocumentFragment();
filtered.forEach(r => frag.appendChild(renderRow(r)));
tbody.appendChild(frag);
nextCursor = d.next_cursor;
more.hidden = !nextCursor;
rowCount.textContent = tbody.children.length + ' rows shown' + (nextCursor ? ' (more available)' : '');
}
function classifySource(r) {
if (r.client_kind) return r.client_kind;
const SCHED = ['run_session_collector','run_verification_detector','run_corporate_memory','marketplace.sync_all'];
if (SCHED.indexOf(r.action) !== -1) return 'scheduler';
if (!r.user_id) return 'system';
return 'other';
}
function renderRow(r) {
const tr = document.createElement('tr');
tr.dataset.rowJson = JSON.stringify(r);
const td = (cls, html) => {
const t = document.createElement('td');
if (cls) t.className = cls;
if (html != null) t.innerHTML = html;
return t;
};
tr.appendChild(td('obs-time', escapeHtml(formatTime(r.timestamp))));
// User cell: prefer "name (id-prefix)" or "email (id-prefix)" when the
// server enriched the row from the users table; fall back to a truncated
// id when the user has been deleted or the row belongs to a system actor.
let userHtml = '—';
if (r.user_id) {
const idPrefix = r.user_id.slice(0, 8) + '…';
const primary = r.user_name || r.user_email;
if (primary) {
userHtml = '<span class="obs-user-primary">' + escapeHtml(primary)
+ '</span> <span class="obs-user-id">(' + escapeHtml(idPrefix) + ')</span>';
} else {
userHtml = '<code title="' + escapeHtml(r.user_id) + '">' + escapeHtml(idPrefix) + '</code>';
}
}
tr.appendChild(td('', userHtml));
const src = r._source || 'other';
tr.appendChild(td('', '<span class="obs-source-pill is-' + src + '">' + escapeHtml(src) + '</span>'));
tr.appendChild(td('obs-action-cell', escapeHtml(r.action || '')));
tr.appendChild(td('', escapeHtml(r.resource || '—')));
tr.appendChild(td('obs-dur-cell', r.duration_ms != null ? escapeHtml(r.duration_ms + ' ms') : '—'));
const result = r.result || '';
let resultHtml = '—';
if (result) {
const isErr = result.toLowerCase().startsWith('error');
resultHtml = '<span class="' + (isErr ? 'obs-result-err' : 'obs-result-ok') + '">' + escapeHtml(result) + '</span>';
}
tr.appendChild(td('', resultHtml));
tr.addEventListener('click', () => openPanel(r));
return tr;
}
function formatTime(ts) {
if (!ts) return '';
const d = new Date(ts);
if (isNaN(d.getTime())) return ts;
// Compact: HH:MM:SS for today, DD.MM HH:MM otherwise.
const now = new Date();
if (d.toDateString() === now.toDateString()) {
return d.toTimeString().slice(0, 8);
}
return d.toLocaleString(undefined, { day: '2-digit', month: '2-digit', hour: '2-digit', minute: '2-digit' });
}
function truncMiddle(s, n) {
if (!s || s.length <= n) return s;
const half = Math.floor((n - 1) / 2);
return s.slice(0, half) + '…' + s.slice(-half);
}
function escapeHtml(s) {
if (s == null) return '';
return String(s).replace(/[&<>"']/g, c => ({'&':'&amp;','<':'&lt;','>':'&gt;','"':'&quot;',"'":'&#39;'}[c]));
}
// ---------------------------------------------------------------------
// Detail panel
// ---------------------------------------------------------------------
const panel = $('obs-panel');
const panelTitle = $('obs-panel-title');
const panelMeta = $('obs-panel-meta');
const panelP = $('obs-panel-params');
const panelR = $('obs-panel-result');
function openPanel(r) {
panelTitle.textContent = r.action || 'event';
panelMeta.innerHTML = [
meta('When', formatTime(r.timestamp)),
meta('User', r.user_id || '—'),
meta('Source', r._source || '—'),
meta('Resource', r.resource || '—'),
meta('Duration', r.duration_ms != null ? r.duration_ms + ' ms' : '—'),
meta('Result', r.result || '—'),
r.client_ip ? meta('Client IP', r.client_ip) : '',
r.correlation_id ? meta('Correlation', r.correlation_id) : '',
].join('');
panelP.textContent = prettyJson(r.params);
panelR.textContent = prettyJson(r.result_payload || r.params_before || '—');
panel.hidden = false;
$('obs-panel-filter-user').onclick = () => {
state.user_id = r.user_id || '';
fUser.value = state.user_id;
panel.hidden = true; writeUrl(); refreshAll();
};
$('obs-panel-filter-action').onclick = () => {
state.action_prefix = r.action || '';
fAct.value = state.action_prefix;
panel.hidden = true; writeUrl(); refreshAll();
};
// "Filter to this resource type": derives the prefix (everything before
// the first ":") and only shows when the row's resource has that shape.
// Singleton resources like "manifest" or "auth" don't carry a prefix,
// so the button stays hidden — pivoting on them isn't useful.
const resourceBtn = $('obs-panel-filter-resource');
const colonIdx = (r.resource || '').indexOf(':');
if (colonIdx > 0) {
const prefix = r.resource.slice(0, colonIdx + 1);
const opt = fResource.querySelector('option[value="' + prefix + '"]');
if (opt) {
resourceBtn.hidden = false;
resourceBtn.onclick = () => {
state.resource_prefix = prefix;
fResource.value = prefix;
panel.hidden = true; writeUrl(); refreshAll();
};
} else {
resourceBtn.hidden = true;
}
} else {
resourceBtn.hidden = true;
}
document.querySelectorAll('.obs-table tbody tr.is-selected').forEach(t => t.classList.remove('is-selected'));
event.currentTarget.classList.add('is-selected');
}
function meta(label, value) {
return '<dt>' + escapeHtml(label) + '</dt><dd>' + escapeHtml(value || '—') + '</dd>';
}
function prettyJson(v) {
if (v == null) return '—';
if (typeof v === 'string') {
try { return JSON.stringify(JSON.parse(v), null, 2); } catch (_) { return v; }
}
try { return JSON.stringify(v, null, 2); } catch (_) { return String(v); }
}
$('obs-panel-close').addEventListener('click', () => panel.hidden = true);
// Saved-views UI was removed on user feedback (the modal pop-open on load
// bug was the trigger; the underlying value-add wasn't there). The
// /api/admin/observability/views CRUD + DuckDB table remain in case we
// bring it back behind an opt-in toggle. See PR thread.
// ---------------------------------------------------------------------
// KPI quick-filter buttons
// ---------------------------------------------------------------------
document.querySelectorAll('.obs-kpi').forEach(b => {
b.addEventListener('click', () => {
const k = b.dataset.filter;
if (k === 'errors') {
state.result = 'error%'; fRes.value = 'error%';
} else if (k === 'users') {
fUser.focus();
return;
} else if (k === 'slow') {
state.sort = 'duration_ms:desc';
updateSortIndicators();
} else if (k === '') {
Object.assign(state, { user_id:'', action_prefix:'', result:'', source:'', resource_prefix:'', q:'' });
fUser.value=''; fAct.value=''; fRes.value=''; fSrc.value=''; fResource.value=''; fQ.value='';
}
writeUrl(); refreshAll();
});
});
// ---------------------------------------------------------------------
// Sort
// ---------------------------------------------------------------------
document.querySelectorAll('.obs-table thead th[data-sort]').forEach(th => {
th.addEventListener('click', () => {
const key = th.dataset.sort;
const [curKey, curDir] = (state.sort || 'timestamp:desc').split(':');
const newDir = (curKey === key && curDir === 'desc') ? 'asc' : 'desc';
state.sort = key + ':' + newDir;
updateSortIndicators();
writeUrl();
loadTable(false);
});
});
function updateSortIndicators() {
const [k, dir] = (state.sort || 'timestamp:desc').split(':');
document.querySelectorAll('.obs-table thead th[data-sort]').forEach(th => {
const isActive = th.dataset.sort === k;
th.setAttribute('aria-sort', isActive ? (dir === 'asc' ? 'ascending' : 'descending') : 'none');
});
}
updateSortIndicators();
// ---------------------------------------------------------------------
// Wiring
// ---------------------------------------------------------------------
winSel.addEventListener('change', () => {
state.window = parseInt(winSel.value, 10);
writeUrl(); refreshAll();
});
fUser.addEventListener('change', () => { state.user_id = fUser.value; writeUrl(); refreshOnFilter(); });
fAct.addEventListener('change', () => { state.action_prefix = fAct.value; writeUrl(); refreshOnFilter(); });
fRes.addEventListener('change', () => { state.result = fRes.value; writeUrl(); refreshOnFilter(); });
fSrc.addEventListener('change', () => { state.source = fSrc.value; writeUrl(); refreshOnFilter(); });
fResource.addEventListener('change', () => { state.resource_prefix = fResource.value; writeUrl(); refreshOnFilter(); });
let qDebounce;
fQ.addEventListener('input', () => {
clearTimeout(qDebounce);
qDebounce = setTimeout(() => { state.q = fQ.value; writeUrl(); refreshOnFilter(); }, 250);
});
fReset.addEventListener('click', () => {
Object.assign(state, { user_id:'', action_prefix:'', result:'', source:'', q:'' });
fUser.value=''; fAct.value=''; fRes.value=''; fSrc.value=''; fQ.value='';
writeUrl(); refreshAll();
});
more.addEventListener('click', () => loadTable(true));
live.addEventListener('change', () => {
state.live = live.checked; writeUrl();
if (state.live) { livePollId = setInterval(refreshLight, 30000); }
else { clearInterval(livePollId); livePollId = null; }
});
function refreshLight() {
// Only KPIs + table. Facets don't change every 30s usefully.
loadKpis().catch(()=>{});
loadTable(false).catch(()=>{});
}
function refreshOnFilter() {
loadKpis().catch(()=>{});
loadTable(false).catch(()=>{});
}
function refreshAll() {
loadKpis().catch(()=>{});
loadFacets().catch(()=>{});
loadTable(false).catch(()=>{});
}
// boot
refreshAll();
if (state.live) { livePollId = setInterval(refreshLight, 30000); }
})();
</script>
{% endblock %}
Reference in a new issue View git blame Copy permalink