b4d3c576af
* docs(spec): admin observability spec + Activity Center MVP plan
Parent spec (480 lines) + executable plan (2295 lines, 14 TDD tasks).
Covers Activity Center rebuild (/admin/activity), with /admin/sessions
and /admin/feedback deferred to follow-up plans.
Already incorporates reviewer-pass revisions across three angles
(security, production resilience, code architecture):
- _get_db import path corrected to app.auth.dependencies
- Test fixtures aligned with seeded_app / admin_user / get_system_db
- All new audit writes wrapped in try/except + logger.exception
- Filename sanitization on session uploads
- DuckDB DESC index behavior documented; upgrade window flagged
- Migration idempotency + evolved-DB test cases
- reveal_raw + shared-cache multi-worker explicitly deferred
Targets schema v40 (audit_log gains params_before, client_ip,
client_kind, correlation_id + 3 indices).
* feat(db): schema v40 — audit_log gains params_before, client_ip, client_kind, correlation_id + 3 indices
* chore(test): clean up Task 1 — drop unused import, rename stale test
* feat(audit): AuditRepository.log() accepts params_before/client_ip/client_kind/correlation_id
* test(audit): strengthen params_before assertion to round-trip JSON content
* feat(audit): AuditRepository.query() rich filters + keyset cursor pagination
* feat(sync): SyncStateRepository.list_recent() cross-table feed
* feat(audit): POST /api/sync/trigger writes audit_log row
* feat(audit): POST /api/scripts/run-due writes audit_log row
* feat(audit): POST /api/upload/sessions writes audit_log row + sanitizes filename
* feat(audit): GET /api/data/{table_id}/download writes audit_log row
* feat(activity): /api/admin/activity timeline + /health + /sync endpoints
* feat(ui): /admin/activity rebuilt — health pulse, timeline, sync grid; /activity-center → 308 redirect
BREAKING: removed demo executive-pulse / maturity-roadmap content from activity_center.html.
The page now reflects real audit_log + sync_history data.
* feat(ui): admin nav + dashboard widget point at /admin/activity
* feat(activity): recursive-audit suppression for AC read endpoints (60s window per actor+filter)
* feat(activity): emit PostHog events when integration enabled (no-op default)
* fix(audit): move v40 indices out of _SYSTEM_SCHEMA + update test_repositories to unpack query() tuple
_SYSTEM_SCHEMA CREATE INDEX on audit_log(timestamp) failed when migration
tests hand-roll a bare audit_log (id, action) without the timestamp column.
Fix: remove indices from _SYSTEM_SCHEMA; add ADD COLUMN IF NOT EXISTS guards
for timestamp and other pre-v40 columns in _v39_to_v40() so the upgrade path
is safe on any hand-rolled schema; call _v39_to_v40 explicitly in the
fresh-install (current==0) path to restore index creation there.
Also unpack the (rows, next_cursor) tuple from AuditRepository.query() in
the three TestAuditRepository tests that still treated it as a list.
* docs: CHANGELOG entry for Activity Center MVP
* chore: refresh stale module docstring in app/api/activity.py
* feat(cli): agnes admin activity — terminal access to Activity Center (timeline + health + sync)
* fix(db): _v39_to_v40 — add IF NOT EXISTS guard for 'action' column
The v39→v40 ladder step adds defensive ADD COLUMN IF NOT EXISTS for
every audit_log column so a hand-rolled bare audit_log (id only) is
safe through the ladder. 'action' was missing from the guard list,
causing CREATE INDEX idx_audit_action_time to fail on tests that
stub audit_log with only an id column (tests/test_e2e_extract.py::
TestSchemaMigration::test_migration_preserves_and_extends).
Local 6/6 schema tests + the previously-failing CI test pass.
* docs(spec): platform telemetry epic — Boss directive + Activity Monitoring plan rebased onto v40 (stacked on zs/spec-activity-center)
* feat(db): schema v41 — 7 usage_* tables for telemetry (events, summary, rollups, attribution)
* chore(db): tighten v41 — usage_session_summary.session_id NOT NULL + upgrade test asserts all 7 tables
* feat(usage): UsageAttributionRepository — replace/delete/lookup over usage_attribution_* tables
* refactor(marketplace): extract list_inner_skills/agents/commands to src/marketplace_listing.py for reuse
* feat(usage): explode plugin attribution on marketplace sync + store entity write; backfill script
* refactor(marketplace): finish src/marketplace_listing.py extraction — drop duplicate _list_inner_* + _parse_frontmatter from app/api/marketplace.py
* feat(usage): promote attribution helpers to src/usage_attribution_helpers.py; hook update_entity rename + bundle-swap; clarify best-effort semantics
* feat(usage): UsageProcessor real extraction + rollup rebuild + 10 fixture-driven tests
* fix(usage): include tool_id in event hash + executemany + rollup transaction (critical multi-tool-turn drop fix)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(marketplace): popularity stats — invocations_30d + trend + sort=most_used|trending + Most Popular section
* feat(admin): /admin/users/<id> Sessions section — list + single-file + bulk-zip downloads (audit-logged)
* feat(usage): admin export endpoint + CLI — csv/json/parquet streaming, filters, audit-logged
* feat(usage): agnes admin ask — LLM Text-to-SQL over usage_events with SELECT-only validator (audit-logged)
* feat(usage): reprocess + prune endpoints + scheduler daily prune job + CLI
* docs: PLATFORM_SETUP.md operator playbook + HOWTO/ cookbook (5 guides + index)
Adds docs/PLATFORM_SETUP.md as a consolidated operator playbook covering
bootstrap, TLS, marketplaces (curated + flea), scheduler env vars, telemetry
extraction/export/ask/prune, privacy posture, and daily routine.
Adds docs/HOWTO/ with 5 analyst cookbook guides: first query, snapshots for
remote tables, private sessions, feedback + admin ask, and customizing skills.
Existing setup docs (QUICKSTART, DEPLOYMENT, ONBOARDING, HEADLESS_USAGE)
get a one-line cross-reference at the top pointing to PLATFORM_SETUP.md.
* docs(changelog): platform telemetry epic — usage_* foundation + surfaces + admin access + docs
Comprehensive [Unreleased] entry covering: usage_events/session_summary/
tool_daily/plugin_daily tables (v41), attribution lookup tables, backfill
script, marketplace Most Popular + invocation chips + sort, admin Sessions
section, export/ask/reprocess/prune endpoints + CLI mirrors, Activity Center
(v40), PLATFORM_SETUP.md + HOWTO/ docs, and operations notes for v41 upgrade.
* fix(security): block DuckDB read_*/http_*/glob functions in usage_ask validator + symlink escape guard in session zip + clarify mark-private semantics
* fix(admin): parquet export tempfile cleanup on COPY failure + correct processed-first sort on /admin/users/<id>/sessions
* feat(audit): close 8 production audit gaps — query (local/remote/hybrid), catalog/schema/sample, snapshot estimate/create, check-access
* feat(ui): /admin/usage summary dashboard + per-user activity tab on /admin/users/<id>
* fix(audit): cap error messages at 200 chars + audit user_activity reads + recursion guard on usage.summary
* fix(audit): catalog.list audits on error path + clean up deferred json import
* fix(ux): client_kind=cli for PAT auth + timeline empty state + email-instead-of-uuid + nav reorder + help text + loading indicators + ask doc
* feat(observability): unify /admin/activity into single page with saved views
- KPI cards (events, users, error rate, p95) clickable as quick-filters
- Faceted filter dropdowns populated from audit_log in the current window
- Sortable audit table, cursor pagination, per-row JSON side panel
- Saved views (schema v43: user_observability_views) — per-user state
- Top bar: window selector + 30s Live toggle + saved views dropdown
- /admin/scheduler-runs → 308 redirect (source=scheduler filter)
- New endpoints: /api/admin/observability/{facets,kpis,views}
* test: update activity + scheduler-runs tests for unified page
- test_admin_activity_page_renders asserts new structural anchors
- test_admin_scheduler_runs_page_admin_only asserts 308 redirect
* fix(observability): respect [hidden] on modal + side panel
CSS `display: flex` on .obs-modal beat the [hidden] attribute's UA
display:none, so the save-view modal rendered on page load and Cancel
clicks couldn't dismiss it. Gate the modal's flex layout on
:not([hidden]); add the same display:none guard prophylactically to
.obs-panel and .obs-views-panel.
* feat(observability): user enrichment in audit + interactive /admin/usage
Activity:
- /api/admin/activity now joins users for user_email + user_name per row
- User column renders "name (id-prefix)" or "email (id-prefix)" instead
of an opaque truncated UUID; falls back to id when the user record is
missing
Usage:
- /admin/usage rewritten as the same filter/group-by/search pattern as
/admin/activity. Faceted dropdowns (User / Tool / Source / Event type)
populated from usage_events; debounced free-text search across
tool_name / skill_name / subagent_type / command_name
- New endpoints /api/admin/usage/{facets,kpis,query}; the query endpoint
supports group_by in {day, username, tool_name, source, ref_id} with
sort + offset pagination, plus an ungrouped raw-events mode
- 4 KPI cards (events, distinct users, distinct tools, error rate) are
clickable quick-filters; clicking a grouped row applies the bucket as
a filter
- Old static `?window=7d|30d|all` server preload removed; all state is
client-side via since_minutes + group_by + filters in the URL
* fix(observability): clearer labels, all-column sort, drop saved views UI
- Rename page titles: "Activity" → "Server activity", "Usage" → "Tool usage"
with a one-line subtitle on each explaining what the page covers and
linking the other one. The two pages source different data (audit_log
vs usage_events) and the previous labels conflated them.
- Drop the saved-views dropdown + save modal from /admin/activity. The
modal pop-open bug was the trigger; the value wasn't there yet. The
/api/admin/observability/views CRUD + DuckDB table stay in place.
- Rename "Live (30s)" to "Auto-refresh (30s)" with a tooltip clarifying
that it's the re-fetch rate, not the time range. Time range now
labeled "Time range" instead of "Window".
- All audit-table columns are sortable (User, Source, Action, Resource,
Result added); sort is page-local with a Jinja comment explaining the
trade-off. Same for raw usage rows.
- Fix duplicate sort-arrow bug — the literal "▼" in the Time th HTML was
rendering alongside the CSS ::before arrow. Removed the literal; CSS
is the single source of truth.
* feat(observability): global Sessions browser + transcript viewer + CLI
Web:
- /admin/sessions — list every collected session JSONL across all users
with time-range, user, model, errors-only and free-text filters. Default
sort surfaces error-heavy sessions first. KPI cards (sessions, distinct
users, sessions w/ errors, tool error rate) clickable as quick-filters.
- /admin/sessions/<username>/<file> — transcript viewer rendering the
JSONL chronologically: user prompts, assistant text, tool calls (with
JSON input) and tool results (with flattened output). Errors get a red
border + chip and a "Next error" navigation button at the top.
- Admin dropdown gains a "Sessions" link.
API:
- GET /api/admin/sessions/{list,kpis,facets} — filtered cross-user reads
off usage_session_summary
- GET /api/admin/sessions/{username}/{file}/transcript — parses JSONL via
the existing services.session_pipeline.lib, returns chronological events
- GET /api/admin/sessions/{username}/{file}/download — JSONL stream, same
path-safety guards as the per-user endpoint, audit-logged
CLI:
- `agnes admin sessions list [--user X] [--errors] [--since 7d]` — table
output with `!` prefix on rows that hit a tool error
- `agnes admin sessions show <username> <file>` — transcript dump, with
`--errors` to print only the failed tool_result blocks
- `agnes admin sessions download <username> <file> [-o path]`
- `agnes admin sessions kpis` — top-level numbers
* feat(internal): expose telemetry tables to agnes query with row-level RBAC
Three new registered tables backed by system.duckdb, queryable through
the same /api/query plumbing analysts use for Keboola / BigQuery /
local sources:
agnes_sessions → usage_session_summary (filter: username)
agnes_usage → usage_events (filter: username)
agnes_audit → audit_log (filter: user_id)
RBAC is per-row, not per-table: admins see every user's rows; non-admins
see only their own. The filter is built server-side from the auth user
dict; non-admin filter values are regex-validated before SQL interpolation.
Implementation:
- new connector connectors/internal/ with access (filter+exec) + registry
(idempotent table_registry seed at startup)
- /api/query detects internal table refs and short-circuits to a CTE
wrapper that prepends "WITH agnes_x AS (SELECT * FROM <src> WHERE …),
…" then "SELECT * FROM (<user_sql>) AS _q". DuckDB cursor on the
shared system.duckdb handle — opening parallel handles / ATTACH on the
same file is blocked process-wide.
- mixing internal + BQ / registered local tables in one SELECT is
rejected (v1 limitation)
- src.rbac.can_access_table waves internal tables through for all
authenticated users; row scoping is the actual security control
- /api/v2/schema and /api/v2/sample gained internal branches; sample
intentionally skips its cache because rows are RBAC-scoped per caller
- audit row written as action='query.internal' with is_admin flag
Tests: connectors/internal/access — RBAC, filter clause, schema, CTE
wrapper coexistence with user-supplied aggregations, unsafe-username
rejection. 16/16 passing.
Motivating queries this enables:
SELECT tool_name, COUNT(*) FROM agnes_usage
WHERE is_error GROUP BY 1 ORDER BY 2 DESC
-- analyst self-introspection: which tools fail for me?
SELECT user_id, COUNT(*) FROM agnes_audit
WHERE action = 'session.transcript_view' GROUP BY 1
-- admin: who's been looking at whose session transcripts?
* feat(admin): group dropdown into 5 named sections + internal tables in /catalog
Admin dropdown gains section headers so admins can land on the right
page without re-reading the full menu:
Activity Center Server activity / Tool usage / Sessions
Users & Access Users / Groups / Resource access / Tokens
Data Tables
Agent Experience Curated Marketplaces / Flea Submissions /
Agent Setup Prompt / Agent Workspace Prompt
Server Server config
"Agent Experience" frames the curated content + prompts as one cluster
— it's all admin-controlled material that shapes what an analyst's AI
agent encounters. "Configuration" → "Server" since only one item lives
there now.
Renamed the section's first two items:
"Activity" → "Server activity" (matches page H1)
"Usage" → "Tool usage"
Also fixes /catalog visibility of the internal tables (agnes_sessions /
_usage / _audit) for non-admin users: ``app.auth.access.can_access``
short-circuits to True for resource_type='table' + an internal-table id.
Without this, non-admins saw the tables in /api/v2/catalog (which uses
the same RBAC bypass) but not on the /catalog HTML page (which calls
can_access directly, requiring a resource_grants row internal tables
don't have).
CSS for `.app-nav-menu-section`: small caps, muted, non-clickable; first
section trims top padding so the panel doesn't open with an awkward gap.
* refactor(admin): move corporate memory into Admin > Agent Experience
Memory link was the only admin-only entry in the primary nav (gated by
session.user.is_admin). Moves it into the Admin dropdown under Agent
Experience, alongside Curated Marketplaces / Flea Submissions / Prompts
— all admin-curated content that shapes what an analyst's AI agent
encounters.
Renamed the nav label to "Shared Knowledge" to match what the page
actually is (admin-curated organisational knowledge from session
verification, surfaced to agents). URL stays at /corporate-memory; the
route still gates on require_admin per the existing comment.
Side effect: primary nav (Home / Marketplace / Data Packages) is now
uniform for every authenticated user — no conditional admin-only entry.
* ui: rename admin entries to Curated Knowledge / Init Prompt / Workspace Prompt
- "Shared Knowledge" → "Curated Knowledge" (parallel with "Curated
Marketplaces" in the same Agent Experience section; "curated" tells
the admin what they do there — review + approve)
- "Agent Setup Prompt" → "Init Prompt" (matches the `agnes init` flow
it actually drives)
- "Agent Workspace Prompt" → "Workspace Prompt" (the "Agent" prefix
was redundant — every item in the section is agent-facing)
Renames page titles + H1s on /admin/agent-prompt and
/admin/workspace-prompt to match.
* refactor: rename Usage → Telemetry across user-facing surfaces
External surfaces all switch; internal Python module / file names and the
physical DB tables (usage_events, usage_session_summary, usage_tool_daily,
usage_plugin_daily) stay — renaming them would force a schema migration
+ a redo of the LLM Text-to-SQL prompt for no analyst-visible win.
Changes:
- Admin dropdown: "Tool usage" → "Telemetry"
- Page H1 / <title>: same
- URL: /admin/usage → /admin/telemetry; old URL 308-redirects
- API prefix: /api/admin/usage/* → /api/admin/telemetry/*
- CLI: primary command `agnes admin telemetry …`; `agnes admin usage` kept
as a deprecated alias so existing operator scripts keep working
- Internal data-source table id: agnes_usage → agnes_telemetry. The
registry seed now evicts any stale internal-source row whose id no
longer matches INTERNAL_TABLES, so the old `agnes_usage` row is
removed from table_registry on next app boot
- All tests + JS endpoint paths updated
* test(rbac): include auto-appended internal tables in expectations
get_accessible_tables now appends agnes_sessions / agnes_telemetry /
agnes_audit to every authenticated user's accessible-tables list so the
internal data source shows up in /catalog. The two existing rbac tests
asserted hardcoded list shapes that pre-dated the change.
Rewritten to assert "granted tables + the canonical internal-table set"
instead of literal lists, so the test stays correct if the internal
table roster changes again later.
* ui: visual dividers between admin-dropdown sections
Adds a 1px top border + 6px top margin to every section header except
the first, so the five named groups (Activity Center, Users & Access,
Data, Agent Experience, Server) read as visually separated clusters.
The header itself stays small-caps + muted as before — the border is
additive.
* ui(memory): match obs-topbar visual on /corporate-memory
The Curated Knowledge page (linked from the admin dropdown's Agent
Experience section) opened straight into the stats bar — no title,
no subtitle, no shared chrome with the other admin pages. Adds an
obs-topbar-style header at the top of .container-memory:
- H1 "Curated Knowledge"
- subtitle explaining what the page is + how AI agents pull from it
The `.ck-*` class set duplicates the inline obs-* styles from
/admin/activity etc. for this one page; promoting the obs-* class set
to style-custom.css for shared reuse is the obvious next step (4 pages
already inline the same CSS), tracked as a follow-up.
Page <title> also renamed from "Corporate Memory" → "Curated Knowledge".
* ui(tables): list Agnes internal tables in /admin/tables + group in /catalog
/admin/tables previously rendered three per-source-type listings
(BQ / Keboola / Jira) and dropped any row whose source_type didn't
match — so the agnes_sessions / agnes_telemetry / agnes_audit rows
seeded into table_registry were invisible. Adds a fourth read-only
section "Agnes internal tables" that filters source_type === 'internal'
and renders the same registry-table layout the other sections use,
with two changes:
- no Register button (these rows are seeded on every app boot from
connectors/internal/registry.py)
- Edit + Delete actions hidden (any change would be reverted on the
next start). Manage access stays so admins can still inspect.
Mode badge picks up a new mode-internal CSS class (teal accent) so the
display doesn't lie and call it "local".
In /catalog, internal tables now group under an "agnes" accordion
section (bucket="agnes" on seed) instead of falling into the catch-all
"default". Single source of truth for which tables exist; admins find
them where they expect.
* ui(tables): Agnes internal as a 4th tab next to BQ/Keboola/Jira
Previous iteration mounted the internal-table listing as a separate
standalone card under the tab strip. Reshapes it to a proper
tab-content section so admins switch between data sources via one
consistent nav (BigQuery / Keboola / Jira / Agnes internal).
- New tab button "Agnes internal" in the tab-nav.
- The listing card becomes <section id="tab-content-internal"
class="tab-content">; switchTab() already routes by id so no JS
change beyond extending the hash allowlist for direct #internal
links.
- Tab content keeps the read-only treatment from the previous commit
(no Register button, no Edit / Delete in renderRegistryListing).
* ui: rename Curated Knowledge → Curated Memory
Settles the naming back on "Curated Memory" — parallel structure with
"Curated Marketplaces" in the same Agent Experience section, and zero
rename ripple: URL (/corporate-memory), API (/api/memory/*), CLI
(agnes admin memory), and Python modules all stay on "memory" so the
admin label finally lines up with the underlying surfaces.
The "Curated" prefix still tells admins what they do on the page
(review pending → approve / mandate / reject) and reads as a sibling
of "Curated Marketplaces" right next to it in the dropdown.
Touches: admin dropdown label, page <title>, page H1. DB tables stay
on knowledge_* (already the canonical naming for the data shape).
* ui: rename "Server activity" → "Audit log"
"Audit log" is what the page actually is — server-side audit_log table
rendered with KPI cards + filter bar + sortable table. The "Server
activity" label confused the term with Claude Code session telemetry
(Telemetry page) and didn't make the source/concept clear.
Touches:
- Admin dropdown nav label
- /admin/activity page H1 + subtitle
- /admin/telemetry subtitle cross-link
- test_activity_api page-renders assertion
URL (/admin/activity) and API (/api/admin/activity/*) stay — the
"activity" name has stuck at the route layer for a year; rerouting
those would churn dashboards/bookmarks for zero analyst-visible win.
* ui(admin-nav): gray band on each section header for clearer separation
Previous iteration used a 1px top border between section labels — the
labels still blended into the items above/below at a glance. Switches
to a light gray background band per section header, extended edge-to-
edge inside the panel via negative horizontal margins. Bolder
font-weight (700) reinforces the separation; bumping the font color
isn't needed because the band itself does the work.
First section's header tucks into the panel's top border-radius so the
band reaches the corners without a gap.
* ui(catalog): rename internal-table category to "Agnes Internal"
`bucket` is what /catalog renders as the accordion category header
verbatim — "agnes" lowercase didn't read as a real category name and
got confused with a system identifier. Bumps to "Agnes Internal".
Seed re-applies on every app boot so existing rows pick up the new
bucket value via `ON CONFLICT (id) DO UPDATE`.
* ui(catalog): split Agnes Internal into its own card on /catalog
Previously the three internal tables landed inside the "Core Business
Data" card under an "Agnes Internal" accordion alongside Keboola / BQ
buckets — readers conflated system telemetry with business datasets,
and the data_stats header counter ("3 tables · ~X rows total") only
ever counted synced rows so internal tables looked invisible.
Split the catalog page into two cards:
- Core Business Data: only non-internal source_types (Keboola, BQ,
Jira). Accordions group by bucket as before. Stats counter reflects
this card's tables.
- Agnes Internal: a dedicated card with its own visual treatment
(teal accent matching the mode-internal badge in /admin/tables).
Flat list (no accordion — only 3 rows, never grows here), each
row carries the canonical `agnes query` snippet. Read-only — no
profiler click, no In-stack toggle, no sync metadata.
Route adds `internal_card` context object; template renders the new
card only when it's non-None.
* fix(rbac): hide internal tables from /admin/access + drop "my" framing
Two related cleanups for the Agnes-internal tables:
1. /admin/access (resource grants) no longer lists them. The
`can_access` check has a hardcoded internal-table bypass — security
is row-level (per-request view filter), so a table-grain
`resource_grants` row would do nothing. Surfacing them in the UI
let admins set up grants that silently no-op. Filter at the
`_table_blocks` projection so the UI tree never sees them.
2. Display names drop the analyst-perspective "my" framing:
"Agnes — my sessions" → "Agnes sessions"
"Agnes — my telemetry events" → "Agnes telemetry events"
"Agnes — my audit log" → "Agnes audit log"
The "my" only makes sense from the querying analyst's seat
(`SELECT … FROM agnes_sessions` returns *their* rows); on /admin/*
pages where admin sees / configures them across users, the
pronoun was misleading. Description text now spells out the
row-level RBAC contract explicitly.
Display names update via TableRegistryRepository.register's ON CONFLICT
UPDATE on next app boot; no manual cleanup needed.
* ui: subtitle notes about agnes_* tables on each Activity Center page
The recursive observability story — Agnes serves its own audit /
telemetry / session data through the same `agnes query` plumbing
analysts use for business data — wasn't surfaced anywhere on the
admin pages that show that data. Three pages get a one-liner with
the canonical `agnes query` snippet + the RBAC contract (analysts
see their own rows, admin sees all):
- /admin/activity (Audit log) → agnes_audit
- /admin/telemetry (Tool usage) → agnes_telemetry
- /admin/sessions → agnes_sessions
Sets up the discovery moment for admins: they're reading the page,
they see "you can query this from Claude Code", they remember it
when an analyst asks "how do I find my own failed tool calls?".
* ui(tables): explain "Show log" empty-state on /admin/tables
Cache warmup log <pre> renders with a dark background and is only
populated by the SSE stream during a Re-warm all run. Opening the
page cold + clicking Show log just revealed a black bar with no
context — admins couldn't tell what they were looking at.
Adds an inline paragraph above the <pre> explaining what the log is,
the row format, when it fills in, and where to find the historical
audit trail (/admin/activity). The actual <pre> stays empty until
SSE events arrive, but the surrounding copy carries the meaning.
* ui(tables): auto-open cache-warmup log on Re-warm all click
A Re-warm all run takes ~24s per remote BQ row. With the <details>
collapsed by default, operators saw the button disable, watched a
quiet ~24s pass, and assumed nothing had happened — the streaming
log was hidden behind a closed disclosure.
Two small JS tweaks:
- cacheWarmupRun() opens the details on click, so streamed lines
appear without an extra interaction
- cacheWarmupOnStart() hides the inline hint paragraph the moment
real log content lands, so the dark log block isn't competing
with redundant context
Hint paragraph also clarifies that only `query_mode='remote'` BQ
rows are warmed — operators with only materialized/internal tables
would see total=0 and the page would "do nothing" by spec.
* ui: trim Agnes internal copy across surfaces
Descriptions had grown to explain the extraction pipeline ("parsed
out of session JSONLs"), the underlying table ("Backed by
usage_session_summary"), the RBAC mechanic ("row-level RBAC at query
time — analysts see their own; admin sees all"), and the SQL snippet.
Every implementation detail meant another rewrite on the next iter.
Strips to one stable line per surface: what the data is, plus
"Also available locally for analysis". Mechanics live in code +
docs; the page copy says what the user needs to know.
Touched:
- connectors/internal/access.py: INTERNAL_TABLES descriptions
- activity_center.html / admin_usage.html / admin_sessions.html
subtitles
- catalog.html Agnes Internal card description + row strip
- admin_tables.html "Agnes internal" tab hint
* fix(internal): is_user_admin arity bugs + + saved-view payload cap
Round-1 code review (PR #278) caught two blocking bugs and three nits.
Blocking — both `is_user_admin(user)` (single dict arg) calls raised
TypeError. is_user_admin signature is `(user_id, conn)`. Affected:
- app/api/query.py:_run_internal_query — every POST /api/query that
references agnes_sessions / agnes_telemetry / agnes_audit blew up
with a 500. The headline analyst-facing feature of this PR was
unusable through the API.
- app/api/v2_sample.py — same shape; `GET /api/v2/sample/agnes_*`
returned 500.
Both fixed to call `is_user_admin(user.get("id"), conn)`. Added two
FastAPI-level tests in test_internal_data_source.py that go through
the TestClient — the existing unit tests on `execute_internal_query`
and `build_filter_clause` skipped the request-handler layer where the
bugs lived, which is why this landed.
Nits also closed:
- connectors/internal/access.py: `+` allowed in _USERNAME_RE /
_USER_ID_RE so RFC 5321 email local-parts (alice+test@x) resolve
correctly without hitting InternalAccessError.
- app/api/observability.py: saved-view payload capped at 64 KiB to
prevent an admin from bloating system.duckdb with a malformed save.
* fix(security): close non-admin data-leak via underlying-table refs
PR #278 R2 review surfaced a non-admin-exploitable bypass: SQL whose
string literal contains 'agnes_sessions' routed into the privileged
internal-query path, then queried the underlying physical table
(usage_session_summary / usage_events / audit_log) directly, escaping
the CTE wrapper's row filter. Two reinforcing defenses:
1. find_internal_refs() now strips single-quoted string literals
before scanning for alias names — a literal alone no longer
routes the request into the privileged code path.
2. execute_internal_query() rejects non-admin SQL that references
the underlying physical tables (usage_*, audit_log). The CTE
wrapper only scopes the agnes_* aliases; a direct FROM on the
base table — or a shadowing inner WITH that still has to read
the base table — bypasses RBAC. Block before execution with an
actionable error pointing to the agnes_* alias. Admins are
unaffected (god-mode short-circuit on the filter clause).
3. tests/test_internal_data_source.py — three new negative tests
covering literal-only matches, direct-table refs, and CTE
shadow attempts.
Also tightens usage_ask.py's SELECT-only validator: pragma_table_info,
pragma_storage_info, pragma_database_*, and duckdb_tables / columns /
views / indexes / schemas are reflection functions that leak metadata
the analyst question shouldn't reach. \bPRAGMA\b in _FORBIDDEN never
matched the function-call form (word-boundary between `A` and `_`).
* fix(security): dynamic denylist for non-admin internal queries
R3 review (PR #278) caught a wider data-leak than R2: the underlying-
physical-table guard listed only the 7 usage_* + audit_log tables,
but system.duckdb has 30+ other sensitive tables — users (emails +
ids), personal_access_tokens, resource_grants, user_groups,
user_observability_views, store_*, marketplace_*, knowledge_*, etc.
A non-admin SQL like
SELECT * FROM agnes_sessions
UNION ALL SELECT email, id, … FROM users LIMIT 1
would leak every user's row.
Replaces the hardcoded denylist with a **dynamic allowlist** —
non-admin SQL may reference ONLY the registered agnes_* aliases.
Every other table in `information_schema.tables` (main schema) is
rejected. Future migrations that add a new sensitive table are
automatically covered without re-editing this module.
Also strips SQL comments (`/* */` and `--`) before the identifier
scan so a comment-wrapped table name (`/**/users/**/`) can't slip
past the regex.
Four new negative tests pin: `users`, `personal_access_tokens`,
block-comment wrap, line-comment wrap.
Plus: per-user view-count cap (100) on /api/admin/observability/views
so an admin can't fill system.duckdb with thousands of saved views.
* release: 0.54.0 — Activity Center + Telemetry + Sessions + internal datasource
Cuts the work shipped across this PR (Activity Center build, recursive
internal data source) into a versioned release. Bumps pyproject.toml
to 0.54.0; renames the top of CHANGELOG.md from [Unreleased] to
[0.54.0] — 2026-05-12 with a header summary; opens a fresh
[Unreleased] section for the next round.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2553 lines
92 KiB
HTML
2553 lines
92 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>Data Catalog - Data Analyst Portal</title>
|
|
{% if not config.THEME_FONT_URL %}
|
|
<link rel="preconnect" href="https://fonts.googleapis.com">
|
|
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
|
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
|
|
{% endif %}
|
|
<link rel="stylesheet" href="{{ url_for('static', filename='style-custom.css') }}">
|
|
<style>
|
|
:root {
|
|
/* Colors - Design System */
|
|
--primary: #0073D1;
|
|
--primary-light: rgba(0, 115, 209, 0.1);
|
|
--text-primary: #1A253C;
|
|
--text-secondary: #6B7280;
|
|
--background: #F5F7FA;
|
|
--surface: #FFFFFF;
|
|
--border: #E5E7EB;
|
|
--border-light: #F3F4F6;
|
|
--success: #10B77F;
|
|
--warning: #F59F0A;
|
|
--error: #EA580C;
|
|
|
|
/* Typography */
|
|
--font-primary: 'Inter', system-ui, sans-serif;
|
|
--font-mono: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
|
|
|
|
/* Shadows */
|
|
--shadow-sm: rgba(0, 0, 0, 0.05) 0px 1px 2px 0px;
|
|
--shadow-md: rgba(0, 0, 0, 0.1) 0px 4px 6px -1px;
|
|
}
|
|
|
|
* {
|
|
margin: 0;
|
|
padding: 0;
|
|
box-sizing: border-box;
|
|
}
|
|
|
|
body {
|
|
font-family: var(--font-primary);
|
|
font-size: 14px;
|
|
color: var(--text-primary);
|
|
background: var(--background);
|
|
line-height: 1.5;
|
|
}
|
|
|
|
/* ── Header (dashboard-style) ── */
|
|
.header {
|
|
background: var(--surface);
|
|
border-bottom: 1px solid var(--border);
|
|
padding: 0 32px;
|
|
height: 72px;
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: space-between;
|
|
position: sticky;
|
|
top: 0;
|
|
z-index: 100;
|
|
}
|
|
|
|
.header-left {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 16px;
|
|
}
|
|
|
|
.header-back {
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: center;
|
|
width: 32px;
|
|
height: 32px;
|
|
border-radius: 6px;
|
|
color: var(--text-secondary);
|
|
text-decoration: none;
|
|
transition: all 0.15s ease;
|
|
}
|
|
|
|
.header-back:hover {
|
|
background: var(--border-light);
|
|
color: var(--text-primary);
|
|
}
|
|
|
|
.header-logo-group {
|
|
display: flex;
|
|
flex-direction: column;
|
|
justify-content: center;
|
|
gap: 2px;
|
|
}
|
|
|
|
.header-logo svg {
|
|
display: block;
|
|
}
|
|
|
|
.header-subtitle {
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: var(--text-secondary);
|
|
letter-spacing: 0.4px;
|
|
text-transform: uppercase;
|
|
margin-top: 2px;
|
|
}
|
|
|
|
.header-right {
|
|
font-size: 12px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
/* ── Page Title ── */
|
|
.page-title {
|
|
max-width: 900px;
|
|
margin: 0 auto;
|
|
padding: 32px 24px 24px;
|
|
}
|
|
|
|
.page-title h1 {
|
|
font-size: 24px;
|
|
font-weight: 800;
|
|
color: var(--text-primary);
|
|
margin-bottom: 4px;
|
|
}
|
|
|
|
.page-title p {
|
|
font-size: 14px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
/* ── Source Cards ── */
|
|
.source-cards {
|
|
max-width: 900px;
|
|
margin: 0 auto;
|
|
padding: 0 24px 32px;
|
|
display: flex;
|
|
flex-direction: column;
|
|
gap: 20px;
|
|
}
|
|
|
|
.source-card {
|
|
background: var(--surface);
|
|
border: 1px solid var(--border);
|
|
border-radius: 12px;
|
|
overflow: hidden;
|
|
box-shadow: var(--shadow-sm);
|
|
}
|
|
|
|
.source-card-header {
|
|
display: flex;
|
|
align-items: flex-start;
|
|
justify-content: space-between;
|
|
padding: 20px 24px;
|
|
gap: 16px;
|
|
}
|
|
|
|
.source-card-left {
|
|
display: flex;
|
|
gap: 14px;
|
|
flex: 1;
|
|
min-width: 0;
|
|
}
|
|
|
|
.source-card-icon {
|
|
width: 42px;
|
|
height: 42px;
|
|
border-radius: 10px;
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: center;
|
|
flex-shrink: 0;
|
|
}
|
|
|
|
.source-card-icon.primary {
|
|
background: var(--primary-light);
|
|
}
|
|
|
|
.source-card-icon.jira {
|
|
background: rgba(107, 114, 128, 0.1);
|
|
}
|
|
|
|
.source-card-info {
|
|
flex: 1;
|
|
min-width: 0;
|
|
}
|
|
|
|
.source-card-name {
|
|
font-size: 16px;
|
|
font-weight: 700;
|
|
color: var(--text-primary);
|
|
margin-bottom: 2px;
|
|
}
|
|
|
|
.source-card-desc {
|
|
font-size: 13px;
|
|
color: var(--text-secondary);
|
|
margin-bottom: 6px;
|
|
}
|
|
|
|
.source-card-meta {
|
|
font-size: 12px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
.source-card-right {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 10px;
|
|
flex-shrink: 0;
|
|
}
|
|
|
|
/* ── Toggle Switch ── */
|
|
.toggle-switch {
|
|
position: relative;
|
|
width: 36px;
|
|
height: 20px;
|
|
}
|
|
|
|
.toggle-switch input {
|
|
opacity: 0;
|
|
width: 0;
|
|
height: 0;
|
|
}
|
|
|
|
.toggle-slider {
|
|
position: absolute;
|
|
cursor: pointer;
|
|
top: 0;
|
|
left: 0;
|
|
right: 0;
|
|
bottom: 0;
|
|
background-color: #E5E7EB;
|
|
transition: .2s;
|
|
border-radius: 20px;
|
|
}
|
|
|
|
.toggle-slider:before {
|
|
position: absolute;
|
|
content: "";
|
|
height: 14px;
|
|
width: 14px;
|
|
left: 3px;
|
|
bottom: 3px;
|
|
background-color: white;
|
|
transition: .2s;
|
|
border-radius: 50%;
|
|
}
|
|
|
|
input:checked + .toggle-slider {
|
|
background-color: var(--primary);
|
|
}
|
|
|
|
input:checked + .toggle-slider:before {
|
|
transform: translateX(16px);
|
|
}
|
|
|
|
.toggle-switch.locked .toggle-slider {
|
|
cursor: not-allowed;
|
|
background-color: var(--primary);
|
|
opacity: 0.6;
|
|
}
|
|
|
|
.toggle-switch.locked .toggle-slider:before {
|
|
transform: translateX(16px);
|
|
}
|
|
|
|
input:disabled + .toggle-slider {
|
|
cursor: not-allowed;
|
|
opacity: 0.6;
|
|
}
|
|
|
|
/* ── Badges ── */
|
|
.badge-included {
|
|
flex-shrink: 0;
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: var(--primary);
|
|
background: var(--primary-light);
|
|
border-radius: 6px;
|
|
padding: 4px 10px;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.badge-subscribed {
|
|
flex-shrink: 0;
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: var(--primary);
|
|
background: var(--primary-light);
|
|
border-radius: 6px;
|
|
padding: 4px 10px;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.badge-unsubscribed {
|
|
flex-shrink: 0;
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: var(--text-secondary);
|
|
background: var(--border-light);
|
|
border-radius: 6px;
|
|
padding: 4px 10px;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
/* ── Accordion ── */
|
|
.accordion-category {
|
|
border-top: 1px solid var(--border-light);
|
|
}
|
|
|
|
.accordion-trigger {
|
|
width: 100%;
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 10px;
|
|
padding: 12px 24px;
|
|
background: none;
|
|
border: none;
|
|
cursor: pointer;
|
|
font-family: var(--font-primary);
|
|
font-size: 14px;
|
|
font-weight: 500;
|
|
color: var(--text-primary);
|
|
text-align: left;
|
|
transition: background 0.1s ease;
|
|
}
|
|
|
|
.accordion-trigger:hover {
|
|
background: var(--border-light);
|
|
}
|
|
|
|
.accordion-chevron {
|
|
width: 16px;
|
|
height: 16px;
|
|
color: var(--text-secondary);
|
|
transition: transform 0.2s ease;
|
|
flex-shrink: 0;
|
|
}
|
|
|
|
.accordion-trigger.expanded .accordion-chevron {
|
|
transform: rotate(90deg);
|
|
}
|
|
|
|
.accordion-count {
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: var(--text-secondary);
|
|
background: var(--border-light);
|
|
padding: 1px 7px;
|
|
border-radius: 9999px;
|
|
margin-left: auto;
|
|
}
|
|
|
|
.accordion-content {
|
|
display: none;
|
|
}
|
|
|
|
.accordion-content.expanded {
|
|
display: block;
|
|
}
|
|
|
|
/* ── Table Rows inside Accordion ── */
|
|
.table-row {
|
|
display: flex;
|
|
align-items: center;
|
|
padding: 10px 24px 10px 50px;
|
|
border-top: 1px solid var(--border-light);
|
|
cursor: pointer;
|
|
transition: background 0.1s ease;
|
|
}
|
|
|
|
.table-row:hover {
|
|
background: rgba(243, 244, 246, 0.5);
|
|
}
|
|
|
|
/* Request-access modal CSS dropped in v0.25 — feature removed. */
|
|
|
|
.table-row-left {
|
|
flex: 1;
|
|
min-width: 0;
|
|
}
|
|
|
|
.table-row-name {
|
|
font-weight: 500;
|
|
font-family: var(--font-mono);
|
|
font-size: 13px;
|
|
color: var(--text-primary);
|
|
}
|
|
|
|
.table-row-desc {
|
|
font-size: 12px;
|
|
color: var(--text-secondary);
|
|
margin-top: 1px;
|
|
overflow: hidden;
|
|
text-overflow: ellipsis;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.table-row-right {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 12px;
|
|
flex-shrink: 0;
|
|
margin-left: 16px;
|
|
}
|
|
|
|
.rows-badge {
|
|
font-size: 12px;
|
|
font-weight: 500;
|
|
padding: 2px 8px;
|
|
border-radius: 4px;
|
|
background: var(--border-light);
|
|
color: var(--text-secondary);
|
|
font-variant-numeric: tabular-nums;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.rows-badge.large {
|
|
background: rgba(245, 159, 10, 0.1);
|
|
color: #B45309;
|
|
}
|
|
|
|
/* ── Query Mode Badges ── */
|
|
.query-mode-badge {
|
|
font-size: 10px;
|
|
font-weight: 600;
|
|
padding: 2px 7px;
|
|
border-radius: 4px;
|
|
text-transform: uppercase;
|
|
letter-spacing: 0.3px;
|
|
white-space: nowrap;
|
|
}
|
|
.query-mode-badge.local {
|
|
background: var(--primary-light);
|
|
color: var(--primary);
|
|
}
|
|
.query-mode-badge.live {
|
|
background: rgba(16, 183, 127, 0.1);
|
|
color: #047857;
|
|
}
|
|
|
|
.table-sync-info {
|
|
font-size: 11px;
|
|
color: var(--text-secondary);
|
|
margin-top: 2px;
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 5px;
|
|
}
|
|
.table-sync-info .live-dot {
|
|
width: 6px;
|
|
height: 6px;
|
|
border-radius: 50%;
|
|
background: #10B77F;
|
|
animation: pulse-live 2s ease-in-out infinite;
|
|
flex-shrink: 0;
|
|
}
|
|
@keyframes pulse-live {
|
|
0%, 100% { opacity: 1; box-shadow: 0 0 0 0 rgba(16, 183, 127, 0.4); }
|
|
50% { opacity: 0.7; box-shadow: 0 0 0 3px rgba(16, 183, 127, 0); }
|
|
}
|
|
|
|
.data-freshness-note {
|
|
padding: 8px 24px;
|
|
font-size: 12px;
|
|
color: var(--text-secondary);
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 6px;
|
|
border-top: 1px solid var(--border-light);
|
|
}
|
|
.data-freshness-note svg {
|
|
flex-shrink: 0;
|
|
opacity: 0.5;
|
|
}
|
|
|
|
.profile-link {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 4px;
|
|
font-size: 11px;
|
|
color: var(--primary);
|
|
opacity: 0.4;
|
|
transition: opacity 0.15s;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.table-row:hover .profile-link {
|
|
opacity: 1;
|
|
}
|
|
|
|
/* ── Jira Unsubscribed State ── */
|
|
.source-card-unsubscribed {
|
|
padding: 16px 24px;
|
|
border-top: 1px solid var(--border-light);
|
|
font-size: 13px;
|
|
color: var(--text-secondary);
|
|
background: var(--border-light);
|
|
}
|
|
|
|
/* ── Jira Attachments Option ── */
|
|
.jira-attachment-option {
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: space-between;
|
|
padding: 14px 24px;
|
|
border-top: 1px solid var(--border-light);
|
|
background: var(--background);
|
|
}
|
|
|
|
.jira-attachment-label {
|
|
display: flex;
|
|
flex-direction: column;
|
|
}
|
|
|
|
.jira-attachment-label span:first-child {
|
|
font-size: 13px;
|
|
font-weight: 500;
|
|
color: var(--text-primary);
|
|
}
|
|
|
|
.jira-attachment-label span:last-child {
|
|
font-size: 12px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
/* ── Footer ── */
|
|
.footer {
|
|
text-align: center;
|
|
padding: 24px;
|
|
color: var(--text-secondary);
|
|
font-size: 12px;
|
|
}
|
|
|
|
.footer a {
|
|
color: var(--primary);
|
|
text-decoration: none;
|
|
}
|
|
|
|
.footer a:hover {
|
|
text-decoration: underline;
|
|
}
|
|
|
|
/* ── Responsive ── */
|
|
@media (max-width: 640px) {
|
|
.source-card-header {
|
|
flex-direction: column;
|
|
}
|
|
|
|
.source-card-right {
|
|
align-self: flex-end;
|
|
}
|
|
|
|
.table-row {
|
|
padding-left: 24px;
|
|
}
|
|
|
|
.table-row-desc {
|
|
white-space: normal;
|
|
}
|
|
}
|
|
|
|
/* ═══════════════════════════════════════════════ */
|
|
/* Profiler Modal - preserved from original */
|
|
/* ═══════════════════════════════════════════════ */
|
|
.profiler-overlay {
|
|
display: none;
|
|
position: fixed;
|
|
top: 0;
|
|
left: 0;
|
|
right: 0;
|
|
bottom: 0;
|
|
background: rgba(0, 0, 0, 0.5);
|
|
z-index: 1000;
|
|
padding: 40px 24px;
|
|
overflow: hidden;
|
|
}
|
|
|
|
.profiler-overlay.active {
|
|
display: flex;
|
|
align-items: flex-start;
|
|
justify-content: center;
|
|
}
|
|
|
|
.profiler-modal {
|
|
max-width: 1100px;
|
|
width: 100%;
|
|
max-height: calc(100vh - 80px);
|
|
background: var(--surface);
|
|
border-radius: 12px;
|
|
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3);
|
|
overflow: hidden;
|
|
display: flex;
|
|
flex-direction: column;
|
|
}
|
|
|
|
.profiler-header {
|
|
display: flex;
|
|
justify-content: space-between;
|
|
align-items: center;
|
|
padding: 20px 24px;
|
|
border-bottom: 1px solid var(--border);
|
|
background: var(--background);
|
|
}
|
|
|
|
.profiler-header-left {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 12px;
|
|
}
|
|
|
|
.profiler-header h2 {
|
|
font-size: 20px;
|
|
font-weight: 600;
|
|
font-family: var(--font-mono);
|
|
}
|
|
|
|
.profiler-close {
|
|
width: 32px;
|
|
height: 32px;
|
|
border: none;
|
|
background: none;
|
|
cursor: pointer;
|
|
border-radius: 6px;
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: center;
|
|
color: var(--text-secondary);
|
|
transition: all 0.15s;
|
|
}
|
|
|
|
.profiler-close:hover {
|
|
background: var(--border-light);
|
|
color: var(--text-primary);
|
|
}
|
|
|
|
.profiler-body {
|
|
padding: 24px;
|
|
overflow-y: auto;
|
|
overflow-x: hidden;
|
|
flex: 1;
|
|
min-height: 0;
|
|
}
|
|
|
|
/* Alert badges */
|
|
.alert-badge {
|
|
display: inline-flex;
|
|
align-items: center;
|
|
font-size: 11px;
|
|
font-weight: 600;
|
|
padding: 2px 8px;
|
|
border-radius: 4px;
|
|
text-transform: uppercase;
|
|
letter-spacing: 0.02em;
|
|
}
|
|
|
|
.alert-badge.constant { background: #FEF3C7; color: #92400E; }
|
|
.alert-badge.unique { background: #EDE9FE; color: #5B21B6; }
|
|
.alert-badge.high_missing { background: #FEE2E2; color: #991B1B; }
|
|
.alert-badge.missing { background: #FEF3C7; color: #92400E; }
|
|
.alert-badge.imbalance { background: #DBEAFE; color: #1E40AF; }
|
|
.alert-badge.zeros { background: #DBEAFE; color: #1E40AF; }
|
|
.alert-badge.high_cardinality { background: #E0E7FF; color: #3730A3; }
|
|
|
|
/* Quality badge */
|
|
.quality-badge {
|
|
font-size: 12px;
|
|
font-weight: 600;
|
|
padding: 3px 10px;
|
|
border-radius: 9999px;
|
|
}
|
|
|
|
.quality-badge.good { background: #D1FAE5; color: #065F46; }
|
|
.quality-badge.warning { background: #FEF3C7; color: #92400E; }
|
|
.quality-badge.poor { background: #FEE2E2; color: #991B1B; }
|
|
|
|
/* Profiler navigation tabs */
|
|
.profiler-tabs {
|
|
display: flex;
|
|
gap: 0;
|
|
border-bottom: 1px solid var(--border);
|
|
padding: 0 24px;
|
|
flex-shrink: 0;
|
|
}
|
|
|
|
.profiler-tab {
|
|
padding: 10px 20px;
|
|
font-size: 13px;
|
|
font-weight: 500;
|
|
color: var(--text-secondary);
|
|
background: none;
|
|
border: none;
|
|
cursor: pointer;
|
|
border-bottom: 2px solid transparent;
|
|
transition: all 0.15s;
|
|
}
|
|
|
|
.profiler-tab:hover {
|
|
color: var(--text-primary);
|
|
background: var(--border-light);
|
|
}
|
|
|
|
.profiler-tab.active {
|
|
color: var(--primary);
|
|
border-bottom-color: var(--primary);
|
|
}
|
|
|
|
.profiler-tab .tab-count {
|
|
font-size: 11px;
|
|
font-weight: 600;
|
|
background: var(--border-light);
|
|
padding: 1px 6px;
|
|
border-radius: 9999px;
|
|
margin-left: 6px;
|
|
}
|
|
|
|
.profiler-tab.active .tab-count {
|
|
background: rgba(0, 115, 209, 0.1);
|
|
color: var(--primary);
|
|
}
|
|
|
|
.profiler-section {
|
|
display: none;
|
|
}
|
|
|
|
.profiler-section.active {
|
|
display: block;
|
|
}
|
|
|
|
/* Overview section */
|
|
.overview-grid {
|
|
display: grid;
|
|
grid-template-columns: 1fr 1fr;
|
|
gap: 24px;
|
|
margin-bottom: 24px;
|
|
}
|
|
|
|
.overview-stats-table {
|
|
width: 100%;
|
|
border-collapse: collapse;
|
|
}
|
|
|
|
.overview-stats-table td {
|
|
padding: 8px 12px;
|
|
font-size: 13px;
|
|
border-bottom: 1px solid var(--border-light);
|
|
}
|
|
|
|
.overview-stats-table td:first-child {
|
|
font-weight: 500;
|
|
color: var(--text-primary);
|
|
width: 55%;
|
|
}
|
|
|
|
.overview-stats-table td:last-child {
|
|
color: var(--text-secondary);
|
|
text-align: right;
|
|
font-variant-numeric: tabular-nums;
|
|
}
|
|
|
|
.overview-stats-table .highlight {
|
|
color: #DC2626;
|
|
font-weight: 500;
|
|
}
|
|
|
|
.overview-title {
|
|
font-size: 15px;
|
|
font-weight: 600;
|
|
margin-bottom: 12px;
|
|
color: var(--text-primary);
|
|
}
|
|
|
|
/* Variable cards */
|
|
.variable-card {
|
|
border: 1px solid var(--border);
|
|
border-radius: 8px;
|
|
margin-bottom: 16px;
|
|
overflow: hidden;
|
|
}
|
|
|
|
.variable-header {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 8px;
|
|
padding: 12px 16px;
|
|
background: var(--background);
|
|
border-bottom: 1px solid var(--border);
|
|
}
|
|
|
|
.variable-name {
|
|
font-weight: 600;
|
|
font-family: var(--font-mono);
|
|
font-size: 14px;
|
|
color: var(--primary);
|
|
}
|
|
|
|
.variable-type {
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: var(--text-secondary);
|
|
background: var(--border-light);
|
|
padding: 2px 8px;
|
|
border-radius: 4px;
|
|
}
|
|
|
|
.variable-body {
|
|
display: grid;
|
|
grid-template-columns: auto 1fr;
|
|
gap: 0;
|
|
}
|
|
|
|
.variable-stats {
|
|
padding: 12px 16px;
|
|
min-width: 0;
|
|
}
|
|
|
|
.variable-stats table {
|
|
width: 100%;
|
|
border-collapse: collapse;
|
|
}
|
|
|
|
.variable-stats td {
|
|
padding: 4px 0;
|
|
font-size: 13px;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.variable-stats td:first-child {
|
|
color: var(--text-secondary);
|
|
padding-right: 12px;
|
|
}
|
|
|
|
.variable-stats td:last-child {
|
|
text-align: right;
|
|
font-variant-numeric: tabular-nums;
|
|
}
|
|
|
|
.variable-chart {
|
|
padding: 12px 16px;
|
|
display: flex;
|
|
align-items: flex-start;
|
|
justify-content: center;
|
|
min-height: 120px;
|
|
min-width: 0;
|
|
}
|
|
|
|
.variable-chart canvas {
|
|
max-width: 100%;
|
|
max-height: 150px;
|
|
}
|
|
|
|
/* Numeric range bar visualization */
|
|
.num-viz {
|
|
width: 100%;
|
|
font-size: 12px;
|
|
}
|
|
|
|
.num-range {
|
|
margin-bottom: 10px;
|
|
}
|
|
|
|
.num-range-labels {
|
|
display: flex;
|
|
justify-content: space-between;
|
|
font-size: 11px;
|
|
color: var(--text-secondary);
|
|
margin-bottom: 4px;
|
|
font-variant-numeric: tabular-nums;
|
|
}
|
|
|
|
.num-range-track {
|
|
position: relative;
|
|
height: 14px;
|
|
background: var(--border-light);
|
|
border-radius: 7px;
|
|
margin-bottom: 6px;
|
|
}
|
|
|
|
.num-range-whisker {
|
|
position: absolute;
|
|
top: 0;
|
|
height: 100%;
|
|
background: rgba(0, 115, 209, 0.15);
|
|
border-radius: 7px;
|
|
}
|
|
|
|
.num-range-iqr {
|
|
position: absolute;
|
|
top: 0;
|
|
height: 100%;
|
|
background: rgba(0, 115, 209, 0.45);
|
|
border-radius: 5px;
|
|
}
|
|
|
|
.num-range-marker {
|
|
position: absolute;
|
|
top: -3px;
|
|
width: 2px;
|
|
height: 20px;
|
|
border-radius: 1px;
|
|
}
|
|
|
|
.num-range-legend {
|
|
display: flex;
|
|
gap: 10px;
|
|
font-size: 10px;
|
|
color: var(--text-secondary);
|
|
margin-top: 2px;
|
|
position: relative;
|
|
overflow: visible;
|
|
}
|
|
|
|
.num-range-legend span::before {
|
|
content: '';
|
|
display: inline-block;
|
|
width: 8px;
|
|
height: 8px;
|
|
border-radius: 2px;
|
|
margin-right: 3px;
|
|
vertical-align: middle;
|
|
}
|
|
|
|
.num-range-legend .legend-mean::before { background: #DC2626; }
|
|
.num-range-legend .legend-median::before { background: #059669; }
|
|
.num-range-legend .legend-iqr::before { background: rgba(0, 115, 209, 0.45); }
|
|
|
|
.num-constant {
|
|
text-align: center;
|
|
padding: 12px 0;
|
|
color: var(--text-secondary);
|
|
font-size: 13px;
|
|
}
|
|
|
|
.num-constant strong {
|
|
color: var(--text-primary);
|
|
font-variant-numeric: tabular-nums;
|
|
}
|
|
|
|
.num-stats-row {
|
|
display: flex;
|
|
flex-wrap: wrap;
|
|
gap: 6px;
|
|
margin-top: 10px;
|
|
}
|
|
|
|
.num-stat {
|
|
display: inline-flex;
|
|
align-items: center;
|
|
gap: 4px;
|
|
background: var(--border-light);
|
|
border-radius: 10px;
|
|
padding: 3px 8px;
|
|
font-size: 11px;
|
|
line-height: 1.3;
|
|
cursor: help;
|
|
position: relative;
|
|
}
|
|
|
|
/* Tooltip trigger styling */
|
|
[data-tip] {
|
|
cursor: help;
|
|
}
|
|
|
|
/* Floating tooltip element (appended to body via JS) */
|
|
.tip-box {
|
|
position: fixed;
|
|
background: var(--text-primary);
|
|
color: #fff;
|
|
font-size: 11px;
|
|
font-weight: 400;
|
|
font-family: var(--font-primary);
|
|
line-height: 1.4;
|
|
padding: 6px 10px;
|
|
border-radius: 6px;
|
|
max-width: 260px;
|
|
pointer-events: none;
|
|
opacity: 0;
|
|
transition: opacity 0.15s ease;
|
|
z-index: 9999;
|
|
}
|
|
|
|
.tip-box.visible {
|
|
opacity: 1;
|
|
}
|
|
|
|
.num-stat-label {
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
.num-stat-value {
|
|
font-weight: 500;
|
|
font-variant-numeric: tabular-nums;
|
|
}
|
|
|
|
.num-stat.alert-zeros .num-stat-value {
|
|
color: #DC2626;
|
|
}
|
|
|
|
/* Categorical bar chart */
|
|
.cat-bars {
|
|
width: 100%;
|
|
padding: 8px 0;
|
|
}
|
|
|
|
.cat-bar-row {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 8px;
|
|
margin-bottom: 4px;
|
|
font-size: 12px;
|
|
}
|
|
|
|
.cat-bar-label {
|
|
width: 100px;
|
|
text-align: right;
|
|
overflow: hidden;
|
|
text-overflow: ellipsis;
|
|
white-space: nowrap;
|
|
color: var(--text-secondary);
|
|
flex-shrink: 0;
|
|
}
|
|
|
|
.cat-bar-track {
|
|
flex: 1;
|
|
height: 18px;
|
|
background: var(--border-light);
|
|
border-radius: 2px;
|
|
overflow: hidden;
|
|
position: relative;
|
|
}
|
|
|
|
.cat-bar-fill {
|
|
height: 100%;
|
|
background: var(--primary);
|
|
border-radius: 2px;
|
|
display: flex;
|
|
align-items: center;
|
|
padding: 0 6px;
|
|
min-width: 30px;
|
|
}
|
|
|
|
.cat-bar-count {
|
|
font-size: 11px;
|
|
font-weight: 500;
|
|
color: white;
|
|
white-space: nowrap;
|
|
}
|
|
|
|
.cat-bar-count.outside {
|
|
color: var(--text-secondary);
|
|
margin-left: 6px;
|
|
}
|
|
|
|
/* Boolean bar */
|
|
.bool-bar {
|
|
display: flex;
|
|
height: 24px;
|
|
border-radius: 4px;
|
|
overflow: hidden;
|
|
width: 100%;
|
|
margin-top: 8px;
|
|
}
|
|
|
|
.bool-bar .true-part {
|
|
background: var(--primary);
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: center;
|
|
color: white;
|
|
font-size: 11px;
|
|
font-weight: 600;
|
|
}
|
|
|
|
.bool-bar .false-part {
|
|
background: #E5E7EB;
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: center;
|
|
color: var(--text-secondary);
|
|
font-size: 11px;
|
|
font-weight: 600;
|
|
}
|
|
|
|
/* Insights list */
|
|
.insight-group {
|
|
margin-bottom: 16px;
|
|
border-radius: 8px;
|
|
border: 1px solid var(--border-light);
|
|
overflow: hidden;
|
|
}
|
|
.insight-group.border-imbalance { border-left: 3px solid #3B82F6; }
|
|
.insight-group.border-unique { border-left: 3px solid #7C3AED; }
|
|
.insight-group.border-high_missing { border-left: 3px solid #DC2626; }
|
|
.insight-group.border-missing { border-left: 3px solid #D97706; }
|
|
.insight-group.border-zeros { border-left: 3px solid #3B82F6; }
|
|
.insight-group.border-high_cardinality { border-left: 3px solid #4338CA; }
|
|
.insight-group.border-constant { border-left: 3px solid #D97706; }
|
|
|
|
.insight-group-header {
|
|
display: flex;
|
|
align-items: baseline;
|
|
gap: 10px;
|
|
padding: 10px 16px;
|
|
background: var(--background);
|
|
}
|
|
|
|
.alert-explanation {
|
|
font-size: 12px;
|
|
color: var(--text-secondary);
|
|
line-height: 1.4;
|
|
}
|
|
|
|
.insight-items {
|
|
padding: 0;
|
|
}
|
|
|
|
.insight-item {
|
|
display: flex;
|
|
align-items: center;
|
|
justify-content: space-between;
|
|
padding: 7px 16px;
|
|
border-top: 1px solid var(--border-light);
|
|
font-size: 13px;
|
|
}
|
|
|
|
.insight-col {
|
|
font-weight: 500;
|
|
font-family: var(--font-mono);
|
|
color: var(--primary);
|
|
cursor: pointer;
|
|
text-decoration: none;
|
|
}
|
|
|
|
.insight-col:hover {
|
|
text-decoration: underline;
|
|
}
|
|
|
|
.insight-detail {
|
|
color: var(--text-secondary);
|
|
font-size: 12px;
|
|
}
|
|
|
|
.variable-highlight {
|
|
animation: highlightPulse 2s ease-out;
|
|
}
|
|
|
|
@keyframes highlightPulse {
|
|
0% { box-shadow: 0 0 0 3px rgba(0, 115, 209, 0.4); }
|
|
100% { box-shadow: none; }
|
|
}
|
|
|
|
/* Missing values chart */
|
|
.missing-chart-container {
|
|
height: 200px;
|
|
margin-bottom: 24px;
|
|
}
|
|
|
|
/* Relationships */
|
|
.relationship-card {
|
|
border: 1px solid var(--border);
|
|
border-radius: 8px;
|
|
padding: 16px;
|
|
margin-bottom: 12px;
|
|
}
|
|
|
|
.relationship-header {
|
|
font-size: 13px;
|
|
font-weight: 600;
|
|
margin-bottom: 8px;
|
|
}
|
|
|
|
.relationship-item {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 8px;
|
|
padding: 6px 0;
|
|
font-size: 13px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
.relationship-table-link {
|
|
font-family: var(--font-mono);
|
|
font-weight: 500;
|
|
color: var(--primary);
|
|
cursor: pointer;
|
|
text-decoration: none;
|
|
}
|
|
|
|
.relationship-table-link:hover {
|
|
text-decoration: underline;
|
|
}
|
|
|
|
.relationship-direction {
|
|
font-size: 11px;
|
|
padding: 1px 6px;
|
|
border-radius: 4px;
|
|
font-weight: 500;
|
|
}
|
|
|
|
.relationship-direction.belongs_to { background: #DBEAFE; color: #1E40AF; }
|
|
.relationship-direction.has_many { background: #D1FAE5; color: #065F46; }
|
|
|
|
/* Relationship Mermaid diagram */
|
|
.relationship-diagram-wrap {
|
|
background: var(--background);
|
|
border: 1px solid var(--border);
|
|
border-radius: 8px;
|
|
padding: 16px;
|
|
margin-bottom: 16px;
|
|
overflow-x: auto;
|
|
}
|
|
|
|
.relationship-diagram-wrap .mermaid {
|
|
display: flex;
|
|
justify-content: center;
|
|
}
|
|
|
|
.relationship-diagram-wrap svg {
|
|
max-width: 100%;
|
|
height: auto;
|
|
}
|
|
|
|
.relationship-diagram-wrap svg .node.clickable-node {
|
|
cursor: pointer;
|
|
}
|
|
|
|
.relationship-diagram-wrap svg .node.clickable-node:hover {
|
|
filter: brightness(0.92);
|
|
transition: filter 0.15s ease;
|
|
}
|
|
|
|
.relationship-legend {
|
|
display: flex;
|
|
gap: 16px;
|
|
justify-content: center;
|
|
margin-top: 10px;
|
|
font-size: 11px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
.relationship-legend-item {
|
|
display: flex;
|
|
align-items: center;
|
|
gap: 5px;
|
|
}
|
|
|
|
.relationship-legend-dot {
|
|
width: 10px;
|
|
height: 10px;
|
|
border-radius: 50%;
|
|
display: inline-block;
|
|
}
|
|
|
|
.relationship-legend-dot.current { background: #2563eb; }
|
|
.relationship-legend-dot.parent { background: #DBEAFE; border: 1px solid #93C5FD; }
|
|
.relationship-legend-dot.child { background: #D1FAE5; border: 1px solid #6EE7B7; }
|
|
.relationship-legend-dot.metric { background: #FEF3C7; border: 1px solid #FCD34D; }
|
|
|
|
/* Metrics badges */
|
|
.metric-badge {
|
|
display: inline-flex;
|
|
align-items: center;
|
|
gap: 4px;
|
|
font-size: 12px;
|
|
font-weight: 500;
|
|
padding: 4px 10px;
|
|
background: rgba(0, 115, 209, 0.08);
|
|
color: var(--primary);
|
|
border-radius: 6px;
|
|
margin: 4px 4px 4px 0;
|
|
}
|
|
|
|
/* Sample data table */
|
|
.sample-table {
|
|
width: 100%;
|
|
border-collapse: collapse;
|
|
font-size: 12px;
|
|
overflow-x: auto;
|
|
display: block;
|
|
}
|
|
|
|
.sample-table th {
|
|
font-weight: 600;
|
|
background: var(--background);
|
|
padding: 8px 12px;
|
|
border: 1px solid var(--border);
|
|
white-space: nowrap;
|
|
font-family: var(--font-mono);
|
|
font-size: 11px;
|
|
}
|
|
|
|
.sample-table td {
|
|
padding: 6px 12px;
|
|
border: 1px solid var(--border-light);
|
|
max-width: 200px;
|
|
overflow: hidden;
|
|
text-overflow: ellipsis;
|
|
white-space: nowrap;
|
|
font-variant-numeric: tabular-nums;
|
|
}
|
|
|
|
/* Loading spinner */
|
|
.profiler-loading {
|
|
display: flex;
|
|
flex-direction: column;
|
|
align-items: center;
|
|
justify-content: center;
|
|
padding: 60px;
|
|
color: var(--text-secondary);
|
|
gap: 12px;
|
|
}
|
|
|
|
.spinner {
|
|
width: 32px;
|
|
height: 32px;
|
|
border: 3px solid var(--border);
|
|
border-top-color: var(--primary);
|
|
border-radius: 50%;
|
|
animation: spin 0.8s linear infinite;
|
|
}
|
|
|
|
@keyframes spin {
|
|
to { transform: rotate(360deg); }
|
|
}
|
|
|
|
/* Empty state */
|
|
.profiler-empty {
|
|
text-align: center;
|
|
padding: 48px;
|
|
color: var(--text-secondary);
|
|
}
|
|
|
|
.profiler-empty p {
|
|
margin-top: 8px;
|
|
font-size: 13px;
|
|
}
|
|
</style>
|
|
<link rel="stylesheet" href="{{ url_for('static', filename='style-custom.css') }}">
|
|
<link rel="stylesheet" href="{{ url_for('static', filename='css/metric_modal.css', v=git_version) }}">
|
|
<!-- Prism.js for SQL syntax highlighting -->
|
|
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/prismjs@1.29.0/themes/prism.min.css">
|
|
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.7/dist/chart.umd.min.js"></script>
|
|
{% include '_theme.html' %}
|
|
</head>
|
|
<body>
|
|
|
|
<!-- ═══════════════ HEADER ═══════════════ -->
|
|
{% include '_app_header.html' %}
|
|
|
|
<!-- ═══════════════ PAGE TITLE ═══════════════ -->
|
|
<div class="page-title">
|
|
<h1>Data Catalog</h1>
|
|
<p>Browse available data sources and manage your subscriptions</p>
|
|
</div>
|
|
|
|
<!-- ═══════════════ SOURCE CARDS ═══════════════ -->
|
|
<div class="source-cards">
|
|
|
|
<!-- ── Card 1: Core Business Data ── -->
|
|
<div class="source-card">
|
|
<div class="source-card-header">
|
|
<div class="source-card-left">
|
|
<div class="source-card-icon primary">
|
|
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#0073D1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
|
|
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
|
|
<polyline points="14 2 14 8 20 8"/>
|
|
<line x1="16" y1="13" x2="8" y2="13"/>
|
|
<line x1="16" y1="17" x2="8" y2="17"/>
|
|
<polyline points="10 9 9 9 8 9"/>
|
|
</svg>
|
|
</div>
|
|
<div class="source-card-info">
|
|
<div class="source-card-name">Core Business Data</div>
|
|
<div class="source-card-desc">Core business data from internal systems</div>
|
|
<div class="source-card-meta">
|
|
{{ data_stats.total_tables or data_stats.tables }} tables · ~{{ data_stats.rows_display }} rows total
|
|
{% if data_stats.last_updated %}
|
|
· Synced {{ data_stats.last_updated }}
|
|
{% endif %}
|
|
</div>
|
|
</div>
|
|
</div>
|
|
<div class="source-card-right">
|
|
<span class="badge-included">Always included</span>
|
|
<label class="toggle-switch locked">
|
|
<input type="checkbox" checked disabled>
|
|
<span class="toggle-slider"></span>
|
|
</label>
|
|
</div>
|
|
</div>
|
|
|
|
{% for category in catalog_data %}
|
|
<div class="accordion-category">
|
|
<button class="accordion-trigger" onclick="toggleAccordion(this)">
|
|
<svg class="accordion-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
|
<polyline points="9 18 15 12 9 6"/>
|
|
</svg>
|
|
{{ category.name }}
|
|
<span class="accordion-count">{{ category.count }} tables</span>
|
|
</button>
|
|
<div class="accordion-content">
|
|
{% for table in category.tables %}
|
|
<div class="table-row" {% if table.query_mode != 'remote' %}onclick="openProfiler('{{ table.name }}')"{% endif %}>
|
|
<div class="table-row-left">
|
|
<div class="table-row-name">
|
|
{{ table.name }}
|
|
{% if table.query_mode == 'remote' %}
|
|
<span class="query-mode-badge live">Live</span>
|
|
{% else %}
|
|
<span class="query-mode-badge local">Local</span>
|
|
{% endif %}
|
|
</div>
|
|
<div class="table-row-desc">{{ table.description }}</div>
|
|
<div class="table-sync-info">
|
|
{% if table.query_mode == 'remote' %}
|
|
<span class="live-dot"></span> Queried directly from BigQuery
|
|
{% elif table.last_sync %}
|
|
Synced {{ table.last_sync }}
|
|
{% endif %}
|
|
</div>
|
|
</div>
|
|
<div class="table-row-right">
|
|
<span class="rows-badge{{ ' large' if table.rows_large }}">{{ table.rows_display }}</span>
|
|
{% if table.query_mode != 'remote' %}
|
|
<span class="profile-link">
|
|
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M2 12s3-7 10-7 10 7 10 7-3 7-10 7-10-7-10-7z"/><circle cx="12" cy="12" r="3"/></svg>
|
|
Profile
|
|
</span>
|
|
{% endif %}
|
|
</div>
|
|
</div>
|
|
{% endfor %}
|
|
</div>
|
|
</div>
|
|
{% endfor %}
|
|
</div>
|
|
|
|
{# ── Card: Agnes Internal ─────────────────────────────────────────
|
|
Renders the three system tables (agnes_sessions / agnes_telemetry /
|
|
agnes_audit) in their own card so they don't pollute the
|
|
Core Business Data header counters. Read-only — no profiler
|
|
click, no `In stack` toggle; analysts query them via
|
|
`agnes query`. #}
|
|
{% if internal_card %}
|
|
<div class="source-card">
|
|
<div class="source-card-header">
|
|
<div class="source-card-left">
|
|
<div class="source-card-icon" style="background: rgba(15, 118, 110, 0.1);">
|
|
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#0F766E" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
|
|
<ellipse cx="12" cy="5" rx="9" ry="3"/>
|
|
<path d="M3 5v14a9 3 0 0 0 18 0V5"/>
|
|
<path d="M3 12a9 3 0 0 0 18 0"/>
|
|
</svg>
|
|
</div>
|
|
<div class="source-card-info">
|
|
<div class="source-card-name">Agnes Internal</div>
|
|
<div class="source-card-desc">System data — sessions, telemetry, audit. Also available locally for analysis.</div>
|
|
<div class="source-card-meta">{{ internal_card.count }} table{{ 's' if internal_card.count != 1 else '' }}</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
{# Plain list — no accordion, no profile, no In stack toggle. #}
|
|
<div class="accordion-content expanded" style="display: block;">
|
|
{% for table in internal_card.tables %}
|
|
<div class="table-row" style="cursor: default;">
|
|
<div class="table-row-left">
|
|
<div class="table-row-name">
|
|
<code>{{ table.id }}</code>
|
|
</div>
|
|
<div class="table-row-desc">{{ table.description }}</div>
|
|
</div>
|
|
</div>
|
|
{% endfor %}
|
|
</div>
|
|
</div>
|
|
{% endif %}
|
|
|
|
<!-- ── Card: Business Metrics ── -->
|
|
{% if metrics_data %}
|
|
{% set metrics_total = namespace(n=0) %}
|
|
{% for c in metrics_data %}{% set metrics_total.n = metrics_total.n + c.metrics|length %}{% endfor %}
|
|
<div class="source-card">
|
|
<div class="source-card-header">
|
|
<div class="source-card-left">
|
|
<div class="source-card-icon primary">
|
|
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#0073D1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
|
|
<path d="M3 3v18h18"/>
|
|
<path d="M18.7 8l-5.1 5.2-2.8-2.7L7 14.3"/>
|
|
</svg>
|
|
</div>
|
|
<div class="source-card-info">
|
|
<div class="source-card-name">Business Metrics</div>
|
|
<div class="source-card-desc">Standardized metric definitions with SQL examples and documentation</div>
|
|
<div class="source-card-meta">{{ metrics_total.n }} metrics · {{ metrics_data|length }} categories</div>
|
|
</div>
|
|
</div>
|
|
<div class="source-card-right">
|
|
<span class="badge-included">Always included</span>
|
|
<label class="toggle-switch locked">
|
|
<input type="checkbox" checked disabled>
|
|
<span class="toggle-slider"></span>
|
|
</label>
|
|
</div>
|
|
</div>
|
|
|
|
{% if data_stats and data_stats.last_updated %}
|
|
<div class="data-freshness-note">
|
|
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><polyline points="12 6 12 12 16 14"/></svg>
|
|
Calculated from data synced {{ data_stats.last_updated }}
|
|
</div>
|
|
{% endif %}
|
|
|
|
{% for category in metrics_data %}
|
|
<div class="accordion-category">
|
|
<button class="accordion-trigger" onclick="toggleAccordion(this)">
|
|
<svg class="accordion-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
|
<polyline points="9 18 15 12 9 6"/>
|
|
</svg>
|
|
<span class="category-tag {{ category.css }}">{{ category.label }}</span>
|
|
<span class="accordion-count">{{ category.metrics|length }} metric{{ 's' if category.metrics|length != 1 }}</span>
|
|
</button>
|
|
<div class="accordion-content">
|
|
{% for metric in category.metrics %}
|
|
<div class="table-row" onclick="openMetricModal('{{ metric.path }}')">
|
|
<div class="table-row-left">
|
|
<div class="table-row-name">{{ metric.display_name }}</div>
|
|
<div class="table-row-desc">{{ metric.description }}</div>
|
|
</div>
|
|
<div class="table-row-right">
|
|
<span class="rows-badge">{{ metric.grain }}</span>
|
|
</div>
|
|
</div>
|
|
{% endfor %}
|
|
</div>
|
|
</div>
|
|
{% endfor %}
|
|
</div>
|
|
{% endif %}
|
|
|
|
</div>
|
|
|
|
|
|
</div>
|
|
|
|
<!-- ═══════════════ PROFILER MODAL (preserved 1:1) ═══════════════ -->
|
|
<div id="profilerOverlay" class="profiler-overlay" onclick="if(event.target===this)closeProfiler()">
|
|
<div class="profiler-modal">
|
|
<div class="profiler-header">
|
|
<div class="profiler-header-left">
|
|
<h2 id="profilerTitle">-</h2>
|
|
<span id="profilerQuality" class="quality-badge"></span>
|
|
<span id="profilerAlertSummary"></span>
|
|
</div>
|
|
<button class="profiler-close" onclick="closeProfiler()">
|
|
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
|
<path d="M18 6L6 18M6 6l12 12"/>
|
|
</svg>
|
|
</button>
|
|
</div>
|
|
<!-- Tabs (outside scrollable body so they stay fixed) -->
|
|
<div class="profiler-tabs" id="profilerTabs" style="display:none;">
|
|
<button class="profiler-tab active" onclick="switchTab('overview')">Overview</button>
|
|
<button class="profiler-tab" onclick="switchTab('variables')" id="tabVariables">Columns</button>
|
|
<button class="profiler-tab" onclick="switchTab('alerts')" id="tabAlerts">Insights</button>
|
|
<button class="profiler-tab" onclick="switchTab('missing')" id="tabMissing">Missing Values</button>
|
|
<button class="profiler-tab" onclick="switchTab('relationships')" id="tabRelationships">Relationships</button>
|
|
<button class="profiler-tab" onclick="switchTab('sample')" id="tabSample">Sample</button>
|
|
</div>
|
|
<div class="profiler-body">
|
|
<div id="profilerLoading" class="profiler-loading">
|
|
<div class="spinner"></div>
|
|
<span>Loading profile data...</span>
|
|
</div>
|
|
<div id="profilerContent" style="display:none;">
|
|
|
|
<!-- Overview Section -->
|
|
<div id="sectionOverview" class="profiler-section active"></div>
|
|
|
|
<!-- Variables Section -->
|
|
<div id="sectionVariables" class="profiler-section"></div>
|
|
|
|
<!-- Alerts Section -->
|
|
<div id="sectionAlerts" class="profiler-section"></div>
|
|
|
|
<!-- Missing Values Section -->
|
|
<div id="sectionMissing" class="profiler-section">
|
|
<div class="missing-chart-container">
|
|
<canvas id="missingChart"></canvas>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Relationships Section -->
|
|
<div id="sectionRelationships" class="profiler-section"></div>
|
|
|
|
<!-- Sample Section -->
|
|
<div id="sectionSample" class="profiler-section"></div>
|
|
</div>
|
|
<div id="profilerError" class="profiler-empty" style="display:none;">
|
|
<svg width="48" height="48" viewBox="0 0 24 24" fill="none" stroke="var(--text-secondary)" stroke-width="1.5" opacity="0.5"><circle cx="12" cy="12" r="10"/><path d="M12 8v4M12 16h.01"/></svg>
|
|
<p id="profilerErrorMsg">Profile data not available for this table.</p>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<footer class="footer">
|
|
<p>© {{ config.INSTANCE_COPYRIGHT or 'AI Data Analyst' }} · Updated daily</p>
|
|
</footer>
|
|
<script>
|
|
/* ═══════════════ ACCORDION ═══════════════ */
|
|
function toggleAccordion(trigger) {
|
|
const content = trigger.nextElementSibling;
|
|
const isExpanded = trigger.classList.contains('expanded');
|
|
|
|
if (isExpanded) {
|
|
trigger.classList.remove('expanded');
|
|
content.classList.remove('expanded');
|
|
} else {
|
|
trigger.classList.add('expanded');
|
|
content.classList.add('expanded');
|
|
}
|
|
}
|
|
|
|
/* ═══════════════ DATASET SUBSCRIPTIONS ═══════════════ */
|
|
/* ═══════════════ PROFILER (preserved 1:1) ═══════════════ */
|
|
let currentCharts = [];
|
|
let currentProfile = null;
|
|
let mermaidDiagramRendered = false;
|
|
let currentDiagramNodeMap = {};
|
|
|
|
function openProfiler(tableName, initialTab) {
|
|
const overlay = document.getElementById('profilerOverlay');
|
|
const loading = document.getElementById('profilerLoading');
|
|
const content = document.getElementById('profilerContent');
|
|
const error = document.getElementById('profilerError');
|
|
|
|
overlay.classList.add('active');
|
|
loading.style.display = 'flex';
|
|
content.style.display = 'none';
|
|
document.getElementById('profilerTabs').style.display = 'none';
|
|
error.style.display = 'none';
|
|
document.body.style.overflow = 'hidden';
|
|
|
|
document.getElementById('profilerTitle').textContent = tableName;
|
|
|
|
fetch(`/api/catalog/profile/${tableName}`)
|
|
.then(r => {
|
|
if (!r.ok) throw new Error('not found');
|
|
return r.json();
|
|
})
|
|
.then(data => {
|
|
currentProfile = data;
|
|
loading.style.display = 'none';
|
|
content.style.display = 'block';
|
|
document.getElementById('profilerTabs').style.display = 'flex';
|
|
renderProfile(data, initialTab);
|
|
})
|
|
.catch(err => {
|
|
loading.style.display = 'none';
|
|
document.getElementById('profilerTabs').style.display = 'none';
|
|
error.style.display = 'block';
|
|
});
|
|
}
|
|
|
|
function closeProfiler() {
|
|
document.getElementById('profilerOverlay').classList.remove('active');
|
|
document.getElementById('profilerTabs').style.display = 'none';
|
|
document.body.style.overflow = '';
|
|
destroyCharts();
|
|
currentProfile = null;
|
|
mermaidDiagramRendered = false;
|
|
currentDiagramNodeMap = {};
|
|
}
|
|
|
|
function destroyCharts() {
|
|
currentCharts.forEach(c => c.destroy());
|
|
currentCharts = [];
|
|
}
|
|
|
|
function switchTab(tabName) {
|
|
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
|
|
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
|
|
event.target.classList.add('active');
|
|
document.getElementById('section' + tabName.charAt(0).toUpperCase() + tabName.slice(1)).classList.add('active');
|
|
|
|
// Lazy-render Mermaid diagram when Relationships tab is first shown
|
|
if (tabName === 'relationships' && !mermaidDiagramRendered && typeof mermaid !== 'undefined') {
|
|
const mermaidEl = document.querySelector('#sectionRelationships .mermaid');
|
|
if (mermaidEl && !mermaidEl.getAttribute('data-processed')) {
|
|
mermaid.run({ nodes: [mermaidEl] }).then(() => {
|
|
attachDiagramClickHandlers();
|
|
});
|
|
mermaidDiagramRendered = true;
|
|
}
|
|
}
|
|
}
|
|
|
|
function navigateToVariable(columnName) {
|
|
// Switch to Columns tab
|
|
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
|
|
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
|
|
const varTab = document.querySelector('#tabVariables');
|
|
if (varTab) varTab.classList.add('active');
|
|
document.getElementById('sectionVariables').classList.add('active');
|
|
|
|
// Scroll to the variable card within the modal body
|
|
const card = document.getElementById('var_' + columnName);
|
|
if (card) {
|
|
// Use scrollIntoView within the scrollable .profiler-body container
|
|
setTimeout(() => {
|
|
card.scrollIntoView({ behavior: 'smooth', block: 'center' });
|
|
card.classList.add('variable-highlight');
|
|
setTimeout(() => card.classList.remove('variable-highlight'), 2000);
|
|
}, 50);
|
|
}
|
|
}
|
|
|
|
function navigateToInsight(insightType) {
|
|
// Switch to Insights tab
|
|
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
|
|
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
|
|
const tab = document.querySelector('#tabAlerts');
|
|
if (tab) tab.classList.add('active');
|
|
document.getElementById('sectionAlerts').classList.add('active');
|
|
|
|
const group = document.getElementById('insight_' + insightType);
|
|
if (group) {
|
|
setTimeout(() => {
|
|
group.scrollIntoView({ behavior: 'smooth', block: 'start' });
|
|
group.classList.add('variable-highlight');
|
|
setTimeout(() => group.classList.remove('variable-highlight'), 2000);
|
|
}, 50);
|
|
}
|
|
}
|
|
|
|
function formatNumber(n) {
|
|
if (n === null || n === undefined) return '-';
|
|
if (Math.abs(n) >= 1e9) return (n / 1e9).toFixed(1) + 'B';
|
|
if (Math.abs(n) >= 1e6) return (n / 1e6).toFixed(1) + 'M';
|
|
if (Math.abs(n) >= 1e3) return (n / 1e3).toFixed(1) + 'K';
|
|
if (Number.isInteger(n)) return n.toLocaleString();
|
|
return Number(n).toFixed(2);
|
|
}
|
|
|
|
const INSIGHT_TOOLTIPS = {
|
|
unique: "Every value in this column is different. This is typical for ID columns and usually means this column can uniquely identify each row.",
|
|
high_cardinality: "This column has a very large number of distinct values. It may be an ID or free-text field. Not ideal for grouping or filtering in reports.",
|
|
high_missing: "Most values in this column are empty. Consider whether this data is collected consistently or if this column is only relevant for specific records.",
|
|
missing: "Some values in this column are empty. Check if the gaps are expected (e.g. optional fields) or indicate a data collection issue.",
|
|
zeros: "A large portion of values are zero. This may be normal (e.g. no change periods) or could indicate missing data entered as zero.",
|
|
imbalance: "One value dominates this column (appears in 90%+ of rows). This is normal for status or flag columns but limits analytical usefulness.",
|
|
constant: "This column has the same value in every row. It provides no analytical value and can usually be ignored in analysis.",
|
|
};
|
|
|
|
function renderAlertBadge(alert, clickable) {
|
|
const tooltip = INSIGHT_TOOLTIPS[alert] || '';
|
|
const label = alert.replace('_', ' ');
|
|
if (clickable) {
|
|
return `<span class="alert-badge ${alert}" title="${tooltip}" style="cursor:pointer;" onclick="event.stopPropagation();navigateToInsight('${alert}')">${label}</span>`;
|
|
}
|
|
return `<span class="alert-badge ${alert}" title="${tooltip}">${label}</span>`;
|
|
}
|
|
|
|
function renderProfile(data, initialTab) {
|
|
destroyCharts();
|
|
|
|
// Header badges
|
|
const quality = data.avg_completeness || 0;
|
|
const qBadge = document.getElementById('profilerQuality');
|
|
qBadge.textContent = quality.toFixed(1) + '% complete';
|
|
qBadge.className = 'quality-badge ' + (quality >= 95 ? 'good' : quality >= 80 ? 'warning' : 'poor');
|
|
|
|
const alertCount = (data.alerts || []).length;
|
|
document.getElementById('profilerAlertSummary').innerHTML = alertCount > 0
|
|
? `<span class="alert-badge imbalance">${alertCount} insight${alertCount > 1 ? 's' : ''}</span>`
|
|
: '';
|
|
|
|
// Tab counts
|
|
const cols = data.columns || [];
|
|
document.getElementById('tabVariables').innerHTML = `Columns <span class="tab-count">${cols.length}</span>`;
|
|
document.getElementById('tabAlerts').innerHTML = `Insights <span class="tab-count">${alertCount}</span>`;
|
|
|
|
renderOverview(data);
|
|
renderVariables(data);
|
|
renderAlerts(data);
|
|
renderMissing(data);
|
|
renderRelationships(data);
|
|
renderSample(data);
|
|
|
|
// Set active tab (default: overview)
|
|
const targetTab = initialTab || 'overview';
|
|
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
|
|
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
|
|
|
|
const targetSection = document.getElementById('section' + targetTab.charAt(0).toUpperCase() + targetTab.slice(1));
|
|
if (targetSection) {
|
|
targetSection.classList.add('active');
|
|
// Activate matching tab button
|
|
document.querySelectorAll('.profiler-tab').forEach(t => {
|
|
if (t.getAttribute('onclick') && t.getAttribute('onclick').includes("'" + targetTab + "'")) {
|
|
t.classList.add('active');
|
|
}
|
|
});
|
|
// Trigger lazy Mermaid render if opening relationships tab directly
|
|
if (targetTab === 'relationships' && !mermaidDiagramRendered && typeof mermaid !== 'undefined') {
|
|
const mermaidEl = document.querySelector('#sectionRelationships .mermaid');
|
|
if (mermaidEl && !mermaidEl.getAttribute('data-processed')) {
|
|
mermaid.run({ nodes: [mermaidEl] }).then(() => {
|
|
attachDiagramClickHandlers();
|
|
});
|
|
mermaidDiagramRendered = true;
|
|
}
|
|
}
|
|
} else {
|
|
// Fallback to overview
|
|
document.querySelector('.profiler-tab').classList.add('active');
|
|
document.getElementById('sectionOverview').classList.add('active');
|
|
}
|
|
}
|
|
|
|
function renderOverview(data) {
|
|
const vt = data.variable_types || {};
|
|
const vtHtml = Object.entries(vt).map(([t, c]) =>
|
|
`<tr><td>${t}</td><td>${c}</td></tr>`
|
|
).join('');
|
|
|
|
const missingPct = data.missing_cells_pct || 0;
|
|
const missingClass = missingPct > 20 ? ' class="highlight"' : '';
|
|
|
|
const dr = data.date_range;
|
|
const dateRow = dr ? `<tr><td>Date Coverage</td><td>${dr.earliest} to ${dr.latest}</td></tr>` : '';
|
|
|
|
// Catalog enrichment section
|
|
let catalogHtml = '';
|
|
if (data.catalog) {
|
|
const cat = data.catalog;
|
|
const tagsHtml = (cat.tags || []).length > 0
|
|
? `<div style="margin-top:8px;"><span style="font-size:11px;color:var(--text-secondary);font-weight:500;">Tags:</span> ${cat.tags.map(t => `<span class="metric-badge" style="display:inline-block;margin:2px 4px 2px 0;">${t}</span>`).join('')}</div>`
|
|
: '';
|
|
const ownersHtml = (cat.owners || []).length > 0
|
|
? `<div style="margin-top:8px;"><span style="font-size:11px;color:var(--text-secondary);font-weight:500;">Owners:</span> ${cat.owners.join(', ')}</div>`
|
|
: '';
|
|
const tierHtml = cat.tier
|
|
? `<div style="margin-top:8px;"><span style="font-size:11px;color:var(--text-secondary);font-weight:500;">Tier:</span> <strong>${cat.tier}</strong></div>`
|
|
: '';
|
|
const urlHtml = cat.url
|
|
? `<div style="margin-top:8px;"><a href="${cat.url}" target="_blank" style="font-size:11px;color:var(--primary);text-decoration:none;">View in Data Catalog →</a></div>`
|
|
: '';
|
|
|
|
catalogHtml = `<div style="margin-top:16px;border-top:1px solid var(--border);padding-top:16px;"><div class="overview-title">Data Catalog</div>${tierHtml}${ownersHtml}${tagsHtml}${urlHtml}</div>`;
|
|
}
|
|
|
|
document.getElementById('sectionOverview').innerHTML = `
|
|
<div class="overview-grid">
|
|
<div>
|
|
<div class="overview-title">Dataset Statistics</div>
|
|
<table class="overview-stats-table">
|
|
<tr><td>Number of variables</td><td>${data.column_count || 0}</td></tr>
|
|
<tr><td>Number of observations</td><td>${formatNumber(data.row_count || 0)}</td></tr>
|
|
<tr><td>Missing cells</td><td${missingClass}>${formatNumber(data.missing_cells || 0)}</td></tr>
|
|
<tr><td>Missing cells (%)</td><td${missingClass}>${missingPct.toFixed(1)}%</td></tr>
|
|
<tr><td>Duplicate rows</td><td>${formatNumber(data.duplicate_rows || 0)}</td></tr>
|
|
<tr><td>File size</td><td>${(data.file_size_mb || 0).toFixed(2)} MB</td></tr>
|
|
${dateRow}
|
|
<tr><td>Last sync</td><td>${data.last_sync ? data.last_sync.substring(0, 16).replace('T', ' ') : '-'}</td></tr>
|
|
</table>
|
|
</div>
|
|
<div>
|
|
<div class="overview-title">Variable Types</div>
|
|
<table class="overview-stats-table">${vtHtml}</table>
|
|
${data.description ? `<div style="margin-top:16px;"><div class="overview-title">Description</div><p style="font-size:13px;color:var(--text-secondary);">${data.description}</p></div>` : ''}
|
|
${(data.used_by_metrics || []).length > 0 ? `<div style="margin-top:16px;"><div class="overview-title">Used by Metrics</div><div>${data.used_by_metrics.map(m => m.file ? `<a class="metric-badge" style="cursor:pointer;text-decoration:none;" onclick="openMetricModal('${m.file}')">${m.name}</a>` : `<span class="metric-badge">${typeof m === 'string' ? m : m.name}</span>`).join('')}</div></div>` : ''}
|
|
${catalogHtml}
|
|
</div>
|
|
</div>`;
|
|
}
|
|
|
|
function buildNumericViz(ns, idx) {
|
|
const hasRange = ns.min != null && ns.max != null;
|
|
const isConstant = hasRange && ns.min === ns.max;
|
|
|
|
let rangeHtml = '';
|
|
|
|
if (isConstant) {
|
|
rangeHtml = `<div class="num-constant">Constant value: <strong>${formatNumber(ns.min)}</strong></div>`;
|
|
} else if (hasRange) {
|
|
const range = ns.max - ns.min;
|
|
const pos = (val) => Math.max(0, Math.min(100, ((val - ns.min) / range) * 100));
|
|
|
|
// Whisker (p5 -> p95)
|
|
let whiskerHtml = '';
|
|
if (ns.p5 != null && ns.p95 != null) {
|
|
const wLeft = pos(ns.p5);
|
|
const wWidth = pos(ns.p95) - wLeft;
|
|
whiskerHtml = `<div class="num-range-whisker" style="left:${wLeft}%;width:${wWidth}%"></div>`;
|
|
}
|
|
|
|
// IQR box (p25 -> p75)
|
|
let iqrHtml = '';
|
|
if (ns.p25 != null && ns.p75 != null) {
|
|
const iLeft = pos(ns.p25);
|
|
const iWidth = pos(ns.p75) - iLeft;
|
|
iqrHtml = `<div class="num-range-iqr" style="left:${iLeft}%;width:${iWidth}%"></div>`;
|
|
}
|
|
|
|
// Mean marker (red)
|
|
let meanMarker = '';
|
|
if (ns.mean != null) {
|
|
meanMarker = `<div class="num-range-marker" style="left:${pos(ns.mean)}%;background:#DC2626;transform:translateX(-50%)"></div>`;
|
|
}
|
|
|
|
// Median marker (green)
|
|
let medianMarker = '';
|
|
if (ns.median != null) {
|
|
medianMarker = `<div class="num-range-marker" style="left:${pos(ns.median)}%;background:#059669;transform:translateX(-50%)"></div>`;
|
|
}
|
|
|
|
rangeHtml = `<div class="num-range">
|
|
<div class="num-range-labels">
|
|
<span>${formatNumber(ns.min)}</span>
|
|
<span>${formatNumber(ns.max)}</span>
|
|
</div>
|
|
<div class="num-range-track">
|
|
${whiskerHtml}${iqrHtml}${meanMarker}${medianMarker}
|
|
</div>
|
|
<div class="num-range-legend">
|
|
<span class="legend-mean" data-tip="Average value. If far from Median, data is skewed by extreme values.">Mean</span>
|
|
<span class="legend-median" data-tip="Middle value. Half of records are above, half below this point.">Median</span>
|
|
<span class="legend-iqr" data-tip="Where 50% of your data falls. Narrow = consistent values, wide = high variation.">Typical range</span>
|
|
</div>
|
|
</div>`;
|
|
}
|
|
|
|
// Stat pills with business-friendly tooltips
|
|
const PILL_TIPS = {
|
|
mean: 'Average value across all rows. Useful for understanding the typical size, but can be skewed by extreme values.',
|
|
median: 'The middle value when sorted. More reliable than Mean when data has outliers \u2014 if Mean and Median differ a lot, you have skewed data.',
|
|
std: 'Standard Deviation \u2014 how spread out the values are. Low = values cluster near the mean. High = wide variation across records.',
|
|
zeros: 'How many records have value zero. High percentage may indicate missing data entered as zero, or genuinely inactive records.',
|
|
negative: 'Records with negative values. Could indicate returns, corrections, or data entry errors depending on the business context.',
|
|
};
|
|
const pills = [];
|
|
if (ns.mean != null) pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.mean}"><span class="num-stat-label">Mean</span><span class="num-stat-value">${formatNumber(ns.mean)}</span></span>`);
|
|
if (ns.median != null) pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.median}"><span class="num-stat-label">Median</span><span class="num-stat-value">${formatNumber(ns.median)}</span></span>`);
|
|
if (ns.stddev != null) pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.std}"><span class="num-stat-label">Std</span><span class="num-stat-value">${formatNumber(ns.stddev)}</span></span>`);
|
|
if (ns.zeros) {
|
|
const alertClass = ns.zeros_pct > 50 ? ' alert-zeros' : '';
|
|
pills.push(`<span class="num-stat${alertClass}" data-tip="${PILL_TIPS.zeros}"><span class="num-stat-label">Zeros</span><span class="num-stat-value">${formatNumber(ns.zeros)} (${ns.zeros_pct.toFixed(1)}%)</span></span>`);
|
|
}
|
|
if (ns.negative) {
|
|
pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.negative}"><span class="num-stat-label">Negative</span><span class="num-stat-value">${formatNumber(ns.negative)} (${ns.negative_pct.toFixed(1)}%)</span></span>`);
|
|
}
|
|
|
|
return `<div class="num-viz">
|
|
${rangeHtml}
|
|
<div style="height:140px;"><canvas id="hist_${idx}"></canvas></div>
|
|
<div class="num-stats-row">${pills.join('')}</div>
|
|
</div>`;
|
|
}
|
|
|
|
function renderVariables(data) {
|
|
const cols = data.columns || [];
|
|
let html = '';
|
|
|
|
cols.forEach((col, idx) => {
|
|
const alerts = (col.alerts || []).map(a => renderAlertBadge(a, true)).join(' ');
|
|
const type = col.type || 'STRING';
|
|
const simplifiedType = type.includes('TIMESTAMP') ? 'DateTime' :
|
|
type === 'DATE' ? 'Date' :
|
|
type === 'BOOLEAN' ? 'Boolean' :
|
|
type === 'NUMERIC' || type.includes('INT') || type.includes('FLOAT') || type.includes('DOUBLE') ? 'Numeric' :
|
|
'Text';
|
|
|
|
const missingPct = 100 - (col.completeness_pct || 100);
|
|
const missingClass = missingPct > 30 ? ' class="highlight"' : missingPct > 5 ? ' style="color:#B45309;"' : '';
|
|
|
|
const distinctPct = col.unique_pct || 0;
|
|
const distinctClass = distinctPct >= 99.9 ? ' style="color:#DC2626;"' : '';
|
|
|
|
let chartHtml = '';
|
|
|
|
// Categorical bar chart
|
|
if (col.string_stats && col.string_stats.top_values && col.string_stats.top_values.length > 0) {
|
|
const topVals = col.string_stats.top_values.slice(0, 8);
|
|
const maxCount = topVals[0].count;
|
|
chartHtml = '<div class="cat-bars">' + topVals.map(v => {
|
|
const pct = (v.count / maxCount) * 100;
|
|
const label = v.value.length > 15 ? v.value.substring(0, 15) + '...' : v.value;
|
|
const countStr = formatNumber(v.count);
|
|
const showInside = pct > 25;
|
|
return `<div class="cat-bar-row">
|
|
<span class="cat-bar-label" title="${v.value}">${label}</span>
|
|
<div class="cat-bar-track">
|
|
<div class="cat-bar-fill" style="width:${Math.max(pct, 3)}%">
|
|
${showInside ? `<span class="cat-bar-count">${countStr}</span>` : ''}
|
|
</div>
|
|
</div>
|
|
${!showInside ? `<span class="cat-bar-count outside">${countStr}</span>` : ''}
|
|
</div>`;
|
|
}).join('') + '</div>';
|
|
}
|
|
|
|
// Numeric stats with range bar visualization
|
|
if (col.numeric_stats) {
|
|
const ns = col.numeric_stats;
|
|
chartHtml = buildNumericViz(ns, idx);
|
|
}
|
|
|
|
// Date stats - labels beside chart for maximum chart size
|
|
if (col.date_stats) {
|
|
const ds = col.date_stats;
|
|
chartHtml = `<div style="display:flex;gap:16px;align-items:stretch;width:100%;padding:8px 0;">
|
|
<div style="font-size:12px;min-width:130px;flex-shrink:0;display:flex;flex-direction:column;justify-content:center;gap:6px;">
|
|
<div><span style="color:var(--text-secondary);display:block;font-size:10px;">Earliest</span><span style="font-weight:500;">${ds.earliest}</span></div>
|
|
<div><span style="color:var(--text-secondary);display:block;font-size:10px;">Latest</span><span style="font-weight:500;">${ds.latest}</span></div>
|
|
<div><span style="color:var(--text-secondary);display:block;font-size:10px;">Span</span><span style="font-weight:500;">${formatNumber(ds.span_days)} days</span></div>
|
|
</div>
|
|
<div style="flex:1;min-height:140px;"><canvas id="datehist_${idx}"></canvas></div>
|
|
</div>`;
|
|
}
|
|
|
|
// Boolean bar
|
|
if (col.boolean_stats) {
|
|
const bs = col.boolean_stats;
|
|
const truePct = bs.true_pct || 0;
|
|
chartHtml = `<div style="padding:8px 0;width:100%;">
|
|
<div class="bool-bar">
|
|
<div class="true-part" style="width:${truePct}%">${truePct > 15 ? `True ${truePct.toFixed(1)}%` : ''}</div>
|
|
<div class="false-part" style="width:${100 - truePct}%">${(100 - truePct) > 15 ? `False ${(100 - truePct).toFixed(1)}%` : ''}</div>
|
|
</div>
|
|
<div style="display:flex;justify-content:space-between;font-size:11px;color:var(--text-secondary);margin-top:4px;">
|
|
<span>True: ${formatNumber(bs.true_count)}</span>
|
|
<span>False: ${formatNumber(bs.false_count)}</span>
|
|
</div>
|
|
</div>`;
|
|
}
|
|
|
|
// Fallback: sample values
|
|
if (!chartHtml && col.sample_values && col.sample_values.length > 0) {
|
|
chartHtml = `<div style="font-size:12px;padding:8px 0;color:var(--text-secondary);overflow:hidden;width:100%;">
|
|
<div style="font-weight:500;margin-bottom:4px;">Sample values</div>
|
|
${col.sample_values.slice(0, 5).map(v => `<div style="font-family:var(--font-mono);font-size:11px;padding:2px 0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;" title="${String(v).replace(/"/g, '"')}">${v}</div>`).join('')}
|
|
</div>`;
|
|
}
|
|
|
|
html += `
|
|
<div class="variable-card" id="var_${col.name}">
|
|
<div class="variable-header">
|
|
<span class="variable-name">${col.name}</span>
|
|
<span class="variable-type">${simplifiedType}</span>
|
|
${col.is_primary_key ? '<span class="alert-badge unique">PK</span>' : ''}
|
|
${alerts}
|
|
</div>
|
|
<div class="variable-body">
|
|
<div class="variable-stats">
|
|
<table>
|
|
<tr><td>Distinct</td><td${distinctClass}>${formatNumber(col.unique_count)}</td></tr>
|
|
<tr><td>Distinct (%)</td><td${distinctClass}>${distinctPct.toFixed(1)}%</td></tr>
|
|
<tr><td>Missing</td><td${missingClass}>${formatNumber(col.null_count || 0)}</td></tr>
|
|
<tr><td>Missing (%)</td><td${missingClass}>${missingPct.toFixed(1)}%</td></tr>
|
|
</table>
|
|
</div>
|
|
<div class="variable-chart">${chartHtml}</div>
|
|
</div>
|
|
</div>`;
|
|
});
|
|
|
|
document.getElementById('sectionVariables').innerHTML = html;
|
|
|
|
// Render Chart.js histograms for numeric columns
|
|
cols.forEach((col, idx) => {
|
|
if (col.numeric_stats && col.numeric_stats.histogram) {
|
|
const canvas = document.getElementById(`hist_${idx}`);
|
|
if (canvas) {
|
|
const h = col.numeric_stats.histogram;
|
|
const chart = new Chart(canvas, {
|
|
type: 'bar',
|
|
data: {
|
|
labels: h.bins || h.counts.map((_, i) => i),
|
|
datasets: [{
|
|
data: h.counts,
|
|
backgroundColor: 'rgba(0, 115, 209, 0.6)',
|
|
borderRadius: 2,
|
|
}]
|
|
},
|
|
options: {
|
|
responsive: true,
|
|
maintainAspectRatio: false,
|
|
plugins: { legend: { display: false } },
|
|
scales: {
|
|
x: { display: false },
|
|
y: { display: false }
|
|
}
|
|
}
|
|
});
|
|
currentCharts.push(chart);
|
|
}
|
|
}
|
|
|
|
if (col.date_stats && col.date_stats.histogram) {
|
|
const canvas = document.getElementById(`datehist_${idx}`);
|
|
if (canvas) {
|
|
const h = col.date_stats.histogram;
|
|
const chart = new Chart(canvas, {
|
|
type: 'bar',
|
|
data: {
|
|
labels: h.bins || h.counts.map((_, i) => i),
|
|
datasets: [{
|
|
data: h.counts,
|
|
backgroundColor: 'rgba(0, 115, 209, 0.6)',
|
|
borderRadius: 2,
|
|
}]
|
|
},
|
|
options: {
|
|
responsive: true,
|
|
maintainAspectRatio: false,
|
|
plugins: { legend: { display: false } },
|
|
scales: {
|
|
x: { ticks: { maxTicksLimit: 6, font: { size: 9 } } },
|
|
y: { display: false }
|
|
}
|
|
}
|
|
});
|
|
currentCharts.push(chart);
|
|
}
|
|
}
|
|
});
|
|
}
|
|
|
|
function extractInsightDetail(alert) {
|
|
// Strip column name prefix from message to avoid redundancy
|
|
const msg = alert.message || '';
|
|
const col = alert.column || '';
|
|
const rest = msg.startsWith(col) ? msg.substring(col.length).trim() : msg;
|
|
// Clean up leading connectors like "is", "has"
|
|
return rest.replace(/^(is|has)\s+/, '');
|
|
}
|
|
|
|
function renderAlerts(data) {
|
|
const alerts = data.alerts || [];
|
|
if (alerts.length === 0) {
|
|
document.getElementById('sectionAlerts').innerHTML = '<div class="profiler-empty"><p>No data quality insights detected.</p></div>';
|
|
return;
|
|
}
|
|
|
|
// Group alerts by type for cleaner display
|
|
const byType = {};
|
|
alerts.forEach(a => {
|
|
if (!byType[a.type]) byType[a.type] = [];
|
|
byType[a.type].push(a);
|
|
});
|
|
|
|
let html = '';
|
|
Object.entries(byType).forEach(([type, items]) => {
|
|
const tip = INSIGHT_TOOLTIPS[type] || '';
|
|
html += `<div class="insight-group border-${type}" id="insight_${type}">
|
|
<div class="insight-group-header">
|
|
${renderAlertBadge(type)}
|
|
${tip ? `<span class="alert-explanation">${tip}</span>` : ''}
|
|
</div>
|
|
<div class="insight-items">`;
|
|
items.forEach(a => {
|
|
const detail = extractInsightDetail(a);
|
|
html += `<div class="insight-item">
|
|
<a class="insight-col" onclick="navigateToVariable('${a.column}')">${a.column}</a>
|
|
<span class="insight-detail">${detail}</span>
|
|
</div>`;
|
|
});
|
|
html += '</div></div>';
|
|
});
|
|
document.getElementById('sectionAlerts').innerHTML = html;
|
|
}
|
|
|
|
function renderMissing(data) {
|
|
const cols = (data.columns || []).filter(c => c.completeness_pct !== undefined);
|
|
if (cols.length === 0) return;
|
|
|
|
const canvas = document.getElementById('missingChart');
|
|
if (!canvas) return;
|
|
|
|
const sorted = [...cols].sort((a, b) => (a.completeness_pct || 0) - (b.completeness_pct || 0));
|
|
|
|
const chart = new Chart(canvas, {
|
|
type: 'bar',
|
|
data: {
|
|
labels: sorted.map(c => c.name),
|
|
datasets: [{
|
|
label: 'Completeness %',
|
|
data: sorted.map(c => c.completeness_pct || 0),
|
|
backgroundColor: sorted.map(c => {
|
|
const pct = c.completeness_pct || 0;
|
|
return pct >= 95 ? 'rgba(16, 183, 127, 0.6)' :
|
|
pct >= 80 ? 'rgba(245, 159, 10, 0.6)' :
|
|
'rgba(220, 38, 38, 0.6)';
|
|
}),
|
|
borderRadius: 2,
|
|
}]
|
|
},
|
|
options: {
|
|
indexAxis: 'y',
|
|
responsive: true,
|
|
maintainAspectRatio: false,
|
|
plugins: { legend: { display: false } },
|
|
scales: {
|
|
x: { min: 0, max: 100, ticks: { callback: v => v + '%' } },
|
|
y: { ticks: { font: { size: 10, family: "'Inter', sans-serif" } } }
|
|
}
|
|
}
|
|
});
|
|
currentCharts.push(chart);
|
|
|
|
// Adjust canvas container height based on column count
|
|
canvas.parentElement.style.height = Math.max(200, cols.length * 22 + 40) + 'px';
|
|
}
|
|
|
|
function sanitizeMermaidId(name) {
|
|
return 'n_' + name.replace(/[^a-zA-Z0-9_]/g, '_');
|
|
}
|
|
|
|
function truncateLabel(text, max) {
|
|
return text.length > max ? text.substring(0, max) + '...' : text;
|
|
}
|
|
|
|
function buildMermaidDiagram(tableName, relations, metrics) {
|
|
const MAX_NODES = 15;
|
|
const parents = relations.filter(r => r.direction === 'belongs_to');
|
|
const children = relations.filter(r => r.direction === 'has_many');
|
|
const metricsList = metrics || [];
|
|
|
|
const currentId = sanitizeMermaidId(tableName);
|
|
const nodeMap = {}; // sanitizedId -> {type, name, file?}
|
|
let lines = ['flowchart LR'];
|
|
|
|
// Current table node (bold box)
|
|
lines.push(` ${currentId}["<b>${tableName}</b>"]`);
|
|
nodeMap[currentId] = { type: 'current', name: tableName };
|
|
|
|
let nodeCount = 1;
|
|
let overflowParents = 0;
|
|
let overflowChildren = 0;
|
|
const renderedParentIds = [];
|
|
const renderedChildIds = [];
|
|
|
|
// Parent nodes (belongs_to) - current --> parent
|
|
parents.forEach(r => {
|
|
if (nodeCount >= MAX_NODES) { overflowParents++; return; }
|
|
const pid = sanitizeMermaidId(r.table);
|
|
const label = truncateLabel(r.join_column, 20);
|
|
lines.push(` ${currentId} -->|"${label}"| ${pid}["${r.table}"]`);
|
|
renderedParentIds.push(pid);
|
|
nodeMap[pid] = { type: 'table', name: r.table };
|
|
nodeCount++;
|
|
});
|
|
|
|
// Child nodes (has_many) - child --> current
|
|
children.forEach(r => {
|
|
if (nodeCount >= MAX_NODES) { overflowChildren++; return; }
|
|
const cid = sanitizeMermaidId(r.table);
|
|
const label = truncateLabel(r.foreign_column || r.join_column, 20);
|
|
lines.push(` ${cid}["${r.table}"] -->|"${label}"| ${currentId}`);
|
|
renderedChildIds.push(cid);
|
|
nodeMap[cid] = { type: 'table', name: r.table };
|
|
nodeCount++;
|
|
});
|
|
|
|
// Metric nodes - current table -.-> metric (dashed)
|
|
const renderedMetricIds = [];
|
|
let overflowMetrics = 0;
|
|
metricsList.forEach(m => {
|
|
if (nodeCount >= MAX_NODES) { overflowMetrics++; return; }
|
|
const mid = sanitizeMermaidId('metric_' + m.name);
|
|
lines.push(` ${currentId} -.->|"metric"| ${mid}("${truncateLabel(m.name, 20)}")`);
|
|
renderedMetricIds.push(mid);
|
|
nodeMap[mid] = { type: 'metric', name: m.name, file: m.file };
|
|
nodeCount++;
|
|
});
|
|
|
|
// Overflow nodes
|
|
if (overflowParents > 0) {
|
|
lines.push(` ${currentId} -.->|" "| n_more_parents("+${overflowParents} more")`);
|
|
lines.push(` style n_more_parents stroke-dasharray: 5 5,fill:#f8fafc,color:#94a3b8`);
|
|
}
|
|
if (overflowChildren > 0) {
|
|
lines.push(` n_more_children("+${overflowChildren} more") -.->|" "| ${currentId}`);
|
|
lines.push(` style n_more_children stroke-dasharray: 5 5,fill:#f8fafc,color:#94a3b8`);
|
|
}
|
|
if (overflowMetrics > 0) {
|
|
lines.push(` ${currentId} -.->|" "| n_more_metrics("+${overflowMetrics} more")`);
|
|
lines.push(` style n_more_metrics stroke-dasharray: 5 5,fill:#f8fafc,color:#94a3b8`);
|
|
}
|
|
|
|
// Styling - only for actually rendered nodes
|
|
lines.push(` style ${currentId} fill:#2563eb,stroke:#1d4ed8,color:#fff,font-weight:bold`);
|
|
renderedParentIds.forEach(pid => {
|
|
lines.push(` style ${pid} fill:#dbeafe,stroke:#93c5fd,color:#1e40af`);
|
|
});
|
|
renderedChildIds.forEach(cid => {
|
|
lines.push(` style ${cid} fill:#d1fae5,stroke:#6ee7b7,color:#065f46`);
|
|
});
|
|
renderedMetricIds.forEach(mid => {
|
|
lines.push(` style ${mid} fill:#fef3c7,stroke:#fcd34d,color:#92400e`);
|
|
});
|
|
|
|
return { diagram: lines.join('\n'), nodeMap };
|
|
}
|
|
|
|
function attachDiagramClickHandlers() {
|
|
const wrap = document.querySelector('.relationship-diagram-wrap');
|
|
if (!wrap) return;
|
|
|
|
const svg = wrap.querySelector('svg');
|
|
if (!svg) return;
|
|
|
|
svg.querySelectorAll('.node').forEach(node => {
|
|
// Mermaid node IDs: "flowchart-n_contract_line-0"
|
|
const match = node.id.match(/^flowchart-(.+?)-\d+$/);
|
|
if (!match) return;
|
|
|
|
const sanitizedId = match[1];
|
|
const entry = currentDiagramNodeMap[sanitizedId];
|
|
if (!entry || entry.type === 'current') return;
|
|
|
|
node.classList.add('clickable-node');
|
|
if (entry.type === 'metric' && entry.file) {
|
|
node.addEventListener('click', () => {
|
|
openMetricModal(entry.file);
|
|
});
|
|
} else if (entry.type === 'table') {
|
|
node.addEventListener('click', () => {
|
|
closeProfiler();
|
|
setTimeout(() => openProfiler(entry.name, 'relationships'), 300);
|
|
});
|
|
}
|
|
});
|
|
}
|
|
|
|
function renderRelationships(data) {
|
|
const rels = data.related_tables || [];
|
|
const metrics = data.used_by_metrics || [];
|
|
|
|
if (rels.length === 0 && metrics.length === 0) {
|
|
document.getElementById('sectionRelationships').innerHTML = '<div class="profiler-empty"><p>No relationships or metric references found.</p></div>';
|
|
return;
|
|
}
|
|
|
|
let html = '';
|
|
|
|
// Mermaid diagram (relationships + metrics)
|
|
if (rels.length > 0 || metrics.length > 0) {
|
|
const tableName = document.getElementById('profilerTitle').textContent;
|
|
const result = buildMermaidDiagram(tableName, rels, metrics);
|
|
currentDiagramNodeMap = result.nodeMap;
|
|
|
|
let legendHtml = '<span class="relationship-legend-item"><span class="relationship-legend-dot current"></span> Current table</span>';
|
|
if (rels.some(r => r.direction === 'belongs_to'))
|
|
legendHtml += '<span class="relationship-legend-item"><span class="relationship-legend-dot parent"></span> References (belongs_to)</span>';
|
|
if (rels.some(r => r.direction === 'has_many'))
|
|
legendHtml += '<span class="relationship-legend-item"><span class="relationship-legend-dot child"></span> Referenced by (has_many)</span>';
|
|
if (metrics.length > 0)
|
|
legendHtml += '<span class="relationship-legend-item"><span class="relationship-legend-dot metric"></span> Business Metric</span>';
|
|
|
|
html += `<div class="relationship-diagram-wrap">
|
|
<pre class="mermaid">${result.diagram}</pre>
|
|
<div class="relationship-legend">${legendHtml}</div>
|
|
</div>`;
|
|
|
|
const outgoing = rels.filter(r => r.direction === 'belongs_to');
|
|
const incoming = rels.filter(r => r.direction === 'has_many');
|
|
|
|
if (outgoing.length > 0) {
|
|
html += '<div class="relationship-card"><div class="relationship-header">References (belongs to)</div>';
|
|
outgoing.forEach(r => {
|
|
html += `<div class="relationship-item">
|
|
<span class="relationship-direction belongs_to">belongs_to</span>
|
|
<a class="relationship-table-link" onclick="closeProfiler();setTimeout(()=>openProfiler('${r.table}'),300)">${r.table}</a>
|
|
<span>via ${r.join_column} → ${r.foreign_column || r.join_column}</span>
|
|
</div>`;
|
|
});
|
|
html += '</div>';
|
|
}
|
|
|
|
if (incoming.length > 0) {
|
|
html += '<div class="relationship-card"><div class="relationship-header">Referenced by (has many)</div>';
|
|
incoming.forEach(r => {
|
|
html += `<div class="relationship-item">
|
|
<span class="relationship-direction has_many">has_many</span>
|
|
<a class="relationship-table-link" onclick="closeProfiler();setTimeout(()=>openProfiler('${r.table}'),300)">${r.table}</a>
|
|
<span>via ${r.foreign_column || r.join_column}</span>
|
|
</div>`;
|
|
});
|
|
html += '</div>';
|
|
}
|
|
}
|
|
|
|
if (metrics.length > 0) {
|
|
html += '<div class="relationship-card"><div class="relationship-header">Used by Business Metrics</div>';
|
|
html += '<div>' + metrics.map(m => {
|
|
if (m.file) {
|
|
return `<a class="metric-badge" style="cursor:pointer;text-decoration:none;" onclick="openMetricModal('${m.file}')">${m.name}</a>`;
|
|
}
|
|
return `<span class="metric-badge">${m.name}</span>`;
|
|
}).join('') + '</div>';
|
|
html += '</div>';
|
|
}
|
|
|
|
document.getElementById('sectionRelationships').innerHTML = html;
|
|
mermaidDiagramRendered = false;
|
|
}
|
|
|
|
function renderSample(data) {
|
|
const rows = data.sample_rows || [];
|
|
if (rows.length === 0) {
|
|
document.getElementById('sectionSample').innerHTML = '<div class="profiler-empty"><p>No sample data available.</p></div>';
|
|
return;
|
|
}
|
|
|
|
const cols = Object.keys(rows[0]);
|
|
let html = '<div style="overflow-x:auto;"><table class="sample-table"><thead><tr>';
|
|
html += cols.map(c => `<th>${c}</th>`).join('');
|
|
html += '</tr></thead><tbody>';
|
|
rows.forEach(row => {
|
|
html += '<tr>' + cols.map(c => {
|
|
const v = row[c];
|
|
const display = v === null ? '<span style="color:#D1D5DB;">null</span>' :
|
|
String(v).length > 50 ? String(v).substring(0, 50) + '...' : String(v);
|
|
return `<td title="${String(v || '')}">${display}</td>`;
|
|
}).join('') + '</tr>';
|
|
});
|
|
html += '</tbody></table></div>';
|
|
document.getElementById('sectionSample').innerHTML = html;
|
|
}
|
|
|
|
// Close on Escape
|
|
document.addEventListener('keydown', e => {
|
|
if (e.key === 'Escape') closeProfiler();
|
|
});
|
|
|
|
// Floating tooltip for [data-tip] elements
|
|
(function() {
|
|
const box = document.createElement('div');
|
|
box.className = 'tip-box';
|
|
document.body.appendChild(box);
|
|
let timer = null;
|
|
|
|
document.addEventListener('mouseover', e => {
|
|
const el = e.target.closest('[data-tip]');
|
|
if (!el) return;
|
|
clearTimeout(timer);
|
|
timer = setTimeout(() => {
|
|
box.textContent = el.getAttribute('data-tip');
|
|
box.classList.add('visible');
|
|
const r = el.getBoundingClientRect();
|
|
let top = r.bottom + 6;
|
|
let left = r.left + r.width / 2;
|
|
// Measure box to avoid right edge overflow
|
|
box.style.left = '0px';
|
|
box.style.top = '0px';
|
|
const bw = box.offsetWidth;
|
|
left = Math.max(8, Math.min(left - bw / 2, window.innerWidth - bw - 8));
|
|
// If below viewport, flip above
|
|
if (top + box.offsetHeight > window.innerHeight - 8) {
|
|
top = r.top - box.offsetHeight - 6;
|
|
}
|
|
box.style.left = left + 'px';
|
|
box.style.top = top + 'px';
|
|
}, 400);
|
|
});
|
|
|
|
document.addEventListener('mouseout', e => {
|
|
const el = e.target.closest('[data-tip]');
|
|
if (!el) return;
|
|
clearTimeout(timer);
|
|
box.classList.remove('visible');
|
|
});
|
|
})();
|
|
</script>
|
|
|
|
<!-- ═══════════════ METRIC MODAL ═══════════════ -->
|
|
<div id="metricModalOverlay" class="metric-modal-overlay">
|
|
<div id="metricModal" class="metric-modal" onclick="event.stopPropagation()">
|
|
<!-- Header -->
|
|
<div class="metric-modal-header">
|
|
<div class="metric-modal-title-section">
|
|
<h2 id="metricModalTitle" class="metric-modal-title"></h2>
|
|
<div id="metricModalMetadata" class="metric-metadata-chips"></div>
|
|
</div>
|
|
<button class="metric-modal-close" onclick="closeMetricModal()" aria-label="Close modal">
|
|
<svg fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
|
|
<path stroke-linecap="round" stroke-linejoin="round" d="M6 18L18 6M6 6l12 12"/>
|
|
</svg>
|
|
</button>
|
|
</div>
|
|
|
|
<!-- Tabs -->
|
|
<nav class="metric-tabs">
|
|
<button class="metric-tab active" data-tab="tabOverview" onclick="switchMetricTab('tabOverview')">
|
|
Overview
|
|
</button>
|
|
<button class="metric-tab" data-tab="tabHowToUse" onclick="switchMetricTab('tabHowToUse')">
|
|
How to Use
|
|
</button>
|
|
<button class="metric-tab" data-tab="tabSQLExamples" onclick="switchMetricTab('tabSQLExamples')">
|
|
SQL Examples
|
|
</button>
|
|
<button class="metric-tab" data-tab="tabTechnical" onclick="switchMetricTab('tabTechnical')">
|
|
Metric Details
|
|
</button>
|
|
</nav>
|
|
|
|
<!-- Body -->
|
|
<div id="metricModalBody" class="metric-modal-body">
|
|
<!-- Content loaded dynamically -->
|
|
</div>
|
|
</div>
|
|
</div>
|
|
|
|
<!-- Prism.js for SQL syntax highlighting -->
|
|
<script src="https://cdn.jsdelivr.net/npm/prismjs@1.29.0/prism.min.js"></script>
|
|
<script src="https://cdn.jsdelivr.net/npm/prismjs@1.29.0/components/prism-sql.min.js"></script>
|
|
<script src="{{ url_for('static', filename='js/metric_modal.js', v=git_version) }}"></script>
|
|
|
|
<!-- Mermaid.js for relationship diagrams -->
|
|
<script type="module">
|
|
import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';
|
|
mermaid.initialize({
|
|
startOnLoad: false,
|
|
theme: 'base',
|
|
themeVariables: {
|
|
fontFamily: 'Inter, system-ui, sans-serif',
|
|
fontSize: '13px',
|
|
primaryColor: '#dbeafe',
|
|
primaryBorderColor: '#2563eb',
|
|
primaryTextColor: '#1e40af',
|
|
lineColor: '#94a3b8',
|
|
secondaryColor: '#f1f5f9',
|
|
tertiaryColor: '#f5f7fa',
|
|
edgeLabelBackground: '#f1f5f9',
|
|
},
|
|
flowchart: {
|
|
curve: 'basis',
|
|
padding: 12,
|
|
htmlLabels: true,
|
|
useMaxWidth: true,
|
|
},
|
|
});
|
|
// Expose mermaid globally for lazy rendering
|
|
window.mermaid = mermaid;
|
|
</script>
|
|
{% include "_version_badge.html" %}
|
|
</body>
|
|
</html>
|