AI-Cognitive-Leap/agnes-the-ai-analyst
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
agnes-the-ai-analyst/app/web/templates/catalog.html
ZdenekSrotyr d2c76cb221
User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 
* fix: redirect unauthenticated HTML routes to /login (#10)

* docs(plan): user mgmt + PAT + CLI distribution implementation plan (#9 #10 #11 #12)

* build(docker): produce wheel artifact for /cli/download (#9)

* feat(db): schema v5 — users.active + deactivated_at/by (#11)

* feat(api): /cli/download wheel + /cli/install.sh with baked server URL (#9)

* feat(users): repository supports active flag + count_admins (#11)

* feat(ui): /install page with per-deployment install instructions (#9)

* feat(api): user PATCH/reset-password/set-password/activate/deactivate (#11)

* fix(cli): da login prompts for password and sends it in body (#9)

* test(api): safeguard tests for self-deactivate and last admin (#11)

* feat(auth): reject requests from deactivated users (#11)

* fixup(#10): propagate next through /login buttons + lock down sanitizer tests

* feat(cli): da admin set-role/activate/deactivate/reset-password/set-password (#11)

* feat(ui): /admin/users management page (#11)

* feat(db): schema v6 — personal_access_tokens (#12)

* feat(users): access_tokens repository (#12)

* feat(auth): JWT carries typ (session|pat) and explicit jti (#12)

* feat(auth): reject revoked/expired PATs; update last_used_at (#12)

* feat(api): /auth/tokens CRUD + admin revoke; session-only guard (#12)

* feat(cli): da auth token create/list/revoke (#12)

* feat(ui): /profile page with PAT create/list/revoke (#12)

* docs: PAT usage and session/PAT TTL clarification (#12)

* feat(auth): PAT first-use-from-new-IP audit + last_used_ip (schema v7) (#12)

Closes remaining acceptance gap from issue #12: audit_log entry on first use
of a PAT from an IP that differs from the recorded last_used_ip.

- schema v7: personal_access_tokens.last_used_ip column
- AccessTokenRepository.mark_used now stores the client IP
- get_current_user extracts client IP (X-Forwarded-For first hop, fallback
  to request.client.host) and emits a token.first_use_new_ip audit when the
  IP changes on a subsequent use (not the very first use)
- tests: new-ip audit, same-ip no-op, first-ever-use no-op, schema v7 column

* fix: address Devin review findings on PR #28

- app/main.py: exclude /auth/* from HTML redirect handler so JSON
  endpoints under /auth/ (PAT CRUD used by `da auth token` CLI) keep
  their 401 JSON contract (Devin #1, bug)
- app/api/tokens.py: reject expires_in_days <= 0 explicitly; use
  `is not None` so 0 no longer silently creates a non-expiring token
  (Devin #2)
- app/api/users.py: validate role against Role enum in create_user
  to match update_user and prevent 500 on role-protected requests
  later (Devin #3)
- app/web/templates/admin_users.html: escape user-supplied strings
  before innerHTML; move onclick handlers to addEventListener via
  data attributes so emails with quotes / HTML no longer break the UI
  or enable stored XSS (Devin #4)
- app/auth/router.py, app/auth/providers/{password,google}.py:
  reject deactivated users at login instead of issuing a JWT that
  would then fail on the next request — removes the confusing
  redirect loop (Devin #5)
- CLAUDE.md: document schema v7 instead of stale v4 (Devin #6)
- tests/test_web_ui.py: regression test for the /auth/* JSON 401

* feat(web): add /profile and /admin/users links to dashboard nav

* feat(web): point setup banner at /install page

* chore(web): drop unused setup_instructions context

* fix: address Devin review round 2 on PR #28

- app/api/tokens.py: when expires_in_days is None (the "never" option),
  use a ~100-year JWT expiry so the token doesn't silently die in 24h
  via the session-default fallback in create_access_token. The real
  expiry enforcement stays in verify_token's DB-level check (Devin 🔴)
- app/web/templates/profile.html: escape t.name and other user-supplied
  strings via esc() helper before innerHTML, same pattern as
  admin_users.html. Move revoke onclick to data-attribute +
  addEventListener (Devin 🟡)
- app/api/cli_artifacts.py: use `mktemp -d` with X's at end of template
  for GNU/BSD portability, place wheel inside the temp dir and
  clean up with rm -rf (Devin 🚩)

* feat(web): redesign /install page; make curl one-liner primary, collapse manual

Rebuild the public /install page using the dashboard visual language
(shared header, card layout, gradient hero, design tokens from
style-custom.css). The page is now anchored on the one-liner install
path: curl -fsSL <server>/cli/install.sh | bash is rendered as the
primary, prominent step 1, while the old manual wheel-download flow
is tucked behind a closed-by-default <details> block for users in
restricted/offline environments.

Information architecture:
  hero (server URL + version)
  -> step 1: quick install (one-liner, big Copy button)
  -> step 2: create PAT on /profile + export DA_TOKEN / da auth whoami
  -> step 3: Claude Code / MCP via ~/.config/da/token.json
  -> collapsed "Manual install" details for download-wheel flow
  -> footer link to docs/HEADLESS_USAGE.md

Every shell snippet has a vanilla-JS "Copy" button that confirms
visually ("Copied!" for 1.5s) and falls back to textarea+execCommand
on non-secure contexts. No new dependencies, no bundler.

The route now also pulls an optional user so the header shows the
same nav (Dashboard / Profile / Logout) as dashboard.html when a
session exists, while staying fully public when signed out.

* fix(cli): use real wheel filename in install.sh (broken pip/uv install)

The installer wrote the downloaded wheel as agnes_cli.whl, which lacks a
PEP-427 version component — both pip and uv tool install reject it and
abort the one-liner.

Use curl -OJ so Content-Disposition determines the on-disk filename, then
resolve it via glob. Install an EXIT trap to remove the tmpdir even when
install fails.

* fix(web): correct manual install wheel glob and add PEP 668 / PATH hints

- Wheel glob is agnes_the_ai_analyst-*.whl (not agnes-*.whl) — the old
  pattern never matched the real artefact name from the build.
- Add — or — separator between uv tool install and pip install.
- Warn that pip install --user is blocked on macOS Homebrew / modern
  Debian (PEP 668) and recommend uv tool install as the default path.
- Both flows now show the ~/.local/bin PATH hint so a fresh shell can
  find the da binary after install.

* fix(web): consistent session.user reference in install header

The avatar-letter fallback inside {% if session.user %} was reading
user.name / user.email directly, but the route dependency can pass
user=None — those references resolved to an empty FlexDict and produced
an empty avatar circle. Read everything through session.user to match
the guard and the dashboard pattern.

* fix(web): point headless usage link at GitHub source

/docs/HEADLESS_USAGE.md 404s — no static route serves repo docs. Point
the footer link at the rendered markdown on GitHub instead of adding a
dedicated docs serving route just for one file.

* feat(web): /install hero size, anon sign-in banner, step 2 copy polish

- Bump hero h1 from 26px to 30px to match dashboard primary scale.
- Anonymous visitors see a small sign-in banner above Step 2 (creating
  a token requires auth; without the banner the flow appears stuck).
- Add an 'After generating your token' section label inside Step 2 so
  the /profile CTA button no longer looks wedged mid-sentence between
  adjacent paragraphs.

* chore(web): /install a11y + version pill polish

- aria-live='polite' on copy buttons so screen readers announce the
  'Copied!' state change.
- Replace redundant INSTANCE_NAME eyebrow (already in the header logo)
  with 'Getting started'.
- Hide the version pill when AGNES_VERSION is unset/'dev' — avoids the
  misleading 'vdev' label in local/unbuilt runs.
- Manual summary focus-visible outline-offset +2px (was -2px which
  clipped inside the card), and mark the chevron as decorative.

* fix(web): use session.user in dashboard avatar fallback

Inside {% if session.user %} guard, the avatar fallback referenced
(user.name or user.email). If user is None the block crashes when
the profile picture is absent. Align with the guard variable.

* fix: address Devin review round 3 on PR #28

- app/api/users.py: stop auto-sending email from reset_password. The
  magic-link sender would deliver a "Login Link" that — when clicked —
  consumes the reset_token via verify_magic_link and logs the user in
  WITHOUT prompting for a new password. Admins now share the raw
  reset_token from the API response manually, or use set-password
  directly. email_sent is always False. Documented inline. (Devin 🟡)
- app/api/cli_artifacts.py: harden /cli/install.sh generation against
  shell injection via Host header or AGNES_VERSION. base_url is
  validated against a strict scheme+host+port regex; version against
  an alnum + dot/dash/underscore allowlist. Both values are also
  piped through shlex.quote() as defense in depth. (Devin 🟡)

The shared users.reset_token column between magic-link and password-
reset flows (Devin 🚩) remains an architectural gap; splitting into
separate columns needs schema v8 and is tracked for a follow-up PR.

* docs, chore(grpn): manual-deploy helpers + hackathon deploy learnings

Adds scripts/grpn/ — Makefile + agnes-auto-upgrade.sh + README for
operating Agnes on GRPN's existing foundryai-development VM when the
full Terraform flow is blocked by org policies:

- iam.disableServiceAccountKeyCreation (org constraint) forbids SA
  JSON keys, so GCP_SA_KEY-based CI is unavailable
- No projectIamAdmin delegation → bootstrap-gcp.sh can't grant roles
- Secret Manager IAM bindings require setIamPolicy which editor lacks

Helper targets: deploy, deploy-tag, recreate, restart, stop, start,
status, version, logs, ps, env, ssh, tunnel, open, bootstrap-admin,
set-data-source, install-cron, uninstall-cron.

docs/superpowers/plans/2026-04-22-grpn-deploy-learnings.md — running
log of all org-policy constraints hit during the hackathon deploy,
with workarounds and derived follow-ups (WIF support, external_ip
variable, customer onboarding IAM checklist).

Not a replacement for the TF flow — stopgap until WIF lands.

* fix(web): make header logos clickable links to home

* feat(web): one-click "Setup a new Claude Code" button

Adds a single-button flow on the dashboard and /install page that
generates a fresh personal access token via POST /auth/tokens and
copies a complete, paste-ready setup script (server URL, token,
install/verify commands) to the clipboard. Falls back to a modal
textarea when the clipboard is blocked; redirects to /login on 401;
surfaces backend errors inline.

- dashboard.html: replaces the top "Set up your local environment"
  anchor with a real button wired to setupNewClaude(). Removes the
  duplicate bottom setup banner to keep a single entry point.
- install.html: for signed-in users, Step 1 leads with the one-click
  button and demotes the curl one-liner into a collapsible "Or run
  manually" aside. Anonymous visitors still see the curl flow plus a
  sign-in hint.
- No new deps. Vanilla JS. Token lives in memory/clipboard only —
  never rendered into persistent DOM.

* feat(cli): add "da auth import-token" for non-interactive PAT login

Writes a provided JWT into ~/.config/da/token.json using the canonical
{access_token, email, role} shape expected by save_token(). Decodes the
token locally to pull email/role claims, verifies it against the server
via GET /api/catalog/tables, and refuses to overwrite an existing token
file if the server returns 401. --email / --role overrides exist for
tokens missing those claims; --skip-verify bypasses the server round-trip
for offline / CI scenarios.

* test(cli): cover da auth import-token success + 401 + claim-fallback paths

Three new tests in TestAuthImportToken:
- valid JWT + 200 -> canonical token.json written
- 401 from /api/catalog/tables -> exit 1, existing token file untouched
- JWT without email/role claims -> refused without overrides, accepted
  with --email / --role flags

* feat(web): update one-click Claude setup instructions — explicit uv install, import-token, skills question

Replaces the fragile `cat > token.json <<EOF` clipboard payload with an
explicit, auditable sequence:

  1. `curl -fsSL /cli/download` + `uv tool install --force` (no opaque
     `curl | bash`).
  2. `da auth import-token --token ...` instead of hand-written JSON.
  3. Explicit PATH persistence for zsh/bash.
  4. A required question to the user about whether to copy the bundled
     skills into ~/.claude/skills/agnes/ or pull them on-demand via
     `da skills show`.
  5. A final confirmation step with whoami + version output.

Factored both pages to include a shared partial
(app/web/templates/_claude_setup_instructions.jinja) so dashboard.html
and install.html can never drift apart again. {server_url} and {token}
stay as runtime placeholders substituted by renderSetupInstructions().

* feat(ui): modernize /admin/users + unify header nav across pages

- New shared partial app/web/templates/_app_header.html — single source
  of truth for the top navigation. Used by base.html and dashboard.html
  (which doesn't extend base.html). Active page highlighted via
  request.url.path. Admin "Users" link gated by session.user.role.
- style-custom.css: add .app-header / .app-nav-link / .app-btn-logout /
  .app-avatar styles (mirrors dashboard's previous inline copy under
  app-* prefix). Mobile-friendly fallback at <720px.
- base.html: include the new partial so every page extending base
  (admin_users, profile, login_email, error, …) gets the same chrome
  the dashboard has.
- dashboard.html: replace its inline <header class="header"> markup
  with the shared partial. Inline .header CSS left in place as
  harmless dead code (separate cleanup PR).
- admin_users.html: rewritten with avatars, role pills (color-coded
  per role), toggle switch for active, search/filter input, toast
  notifications, modal dialogs replacing alert/confirm/prompt,
  one-click copy for the reset token, empty / loading states.
  All XSS-safe via the existing esc() helper + data-attribute
  event delegation.
- tests/test_web_ui.py: smoke test that /admin/users renders the new
  shared header chrome and the modernized markup.

* feat(api): serve CLI wheel at /cli/agnes.whl for direct uv install

uv tool install inspects the URL path suffix to recognise a wheel, so
/cli/download (which has no .whl suffix) cannot be installed directly.
Expose a stable /cli/agnes.whl alias over the same wheel lookup so users
can run: uv tool install --force https://<server>/cli/agnes.whl

* test(cli): cover da auth import-token --server persisting to config.yaml

The server persistence was already implemented in the import-token command
(save_config({server}) call) but not covered by tests. Add an explicit test
so the one-step setup contract — single import-token call writes both token
and server — cannot regress.

* feat(web): simpler Claude setup — single uv install URL, single import-token call

User feedback: the prior clipboard payload repeated the server URL and
token across multiple steps (curl + tmpfile + install + rm + separate
seed-config + import-token). Collapse to:

 1. uv tool install --force {server_url}/cli/agnes.whl  (single URL, direct)
 2. da auth import-token --token ... --server ...        (one call, persists both)
 3. da auth whoami
 4. skills (ask user first)
 5. confirm

uv accepts HTTPS URLs that end in .whl and installs them directly, so
the tmpfile dance is unnecessary. import-token --server already persists
the server to config.yaml, so no separate printf > config.yaml step.

* fix(tests): update admin users heading assertion after template rename

The admin_users.html template now uses <h2 class="users-title">Users</h2>
instead of <h2>User management</h2>. Update the assertion to match.

* feat(ui): unify header across remaining 7 standalone pages

These 7 pages render their own full <html> and don't extend base.html,
so the previous unification commit only covered base + dashboard. Each
had its own ad-hoc <header> markup with inconsistent classes
(.top-header / .header / .page-header), inconsistent nav-link sets,
and inconsistent avatar/email styling.

Replace each inline <header>...</header> block with the shared
{% include '_app_header.html' %} so /activity-center, /admin/permissions,
/admin/tables, /catalog, /corporate-memory, /corporate-memory/admin,
and /install all show the same chrome (Dashboard / Install CLI /
Profile / Users / email + avatar / Logout) with the active page
highlighted via request.url.path.

Old inline header CSS (.header, .top-header, .page-header, .nav-link,
etc.) is left in place as harmless dead code; it can be cleaned up in
a follow-up sweep.

* feat(web): add readable preview of Claude setup payload on dashboard + /install

Move the line-by-line setup instructions into app/web/setup_instructions.py
as the single source of truth, then render them in two modes from the
existing _claude_setup_instructions.jinja partial:

- preview_mode=True  → visible, read-only <pre><code> block with the real
  server URL and a clearly-styled placeholder token (never a real one).
- preview_mode=False → the JS SETUP_INSTRUCTIONS_TEMPLATE used by the
  one-click flow (unchanged behaviour).

Both /dashboard (env-setup-cta card) and /install (Step 1 card) now show
the preview directly under the 'Setup a new Claude Code' button so users
can see exactly what will land in their clipboard before they click.

* feat(web): update setup instructions — `da diagnose` step, explicit section titles

Rework the Claude Code setup payload to:

- Give every numbered step an unambiguous verb header ("1) Install the CLI",
  "2) Log in", "3) Verify the login", "4) Run diagnostics", "5) Skills (ask
  the user first)", "6) Confirm").
- Add step 4 `da diagnose` as the post-login health check. The CLI already
  ships this command (cli/commands/diagnose.py); it prints "Overall:
  healthy" and a list of green checks that map cleanly to next actions.
- Ask the skills copy-vs-on-demand question verbatim so Claude Code always
  prompts the user the same way.
- Replace the terse "Confirm" line with a 4-bullet summary (version,
  whoami, skills choice, diagnose status) so the return message is
  structured and comparable across setups.

* chore(web): remove stale MCP card from /install (no MCP server today)

The 'Use with Claude Code / MCP' card (Step 3 on /install) referenced an
MCP integration Agnes does not ship. Remove the whole card. The one-click
'Setup a new Claude Code' flow in Step 1 already covers the long-lived
client use case and is less confusing than dangling persistence tips for
a non-existent integration.

* feat(api): include user_email + last_used_ip + user_id in admin tokens list response

Adds AdminTokenItem response model (superset of TokenListItem) and
AccessTokenRepository.list_all_with_user() joining personal_access_tokens
with users to denormalize user_email. Needed for /admin/tokens UI where
admins triage tokens across all users.

* feat(web): /admin/tokens page — list, filter, search, revoke across all users

Adds a new admin-only page with client-side filtering (status, user email,
last-used window), column sorting, counts bar (active/revoked/expired),
and an inline revoke action. Mirrors the /admin/users visual language.

* feat(web): add Tokens nav link for admins + deep-link from admin/users row

Admin-only nav entry to /admin/tokens, and a per-row Tokens button on
/admin/users that prefills the token page's user filter via ?user=<email>.

* test(admin): cover /admin/tokens rendering, filter state, non-admin denial, revoke

Verifies admin can render the page (title + JS hooks present), a non-admin
is blocked, unauthenticated users are redirected, the admin list response
includes user_email / user_id / last_used_ip, and admin can revoke another
user's token.

* feat(web): modern redesign of /admin/tokens — hero, stat strip, refined table, responsive cards, a11y

* feat(web): ditch the table — /admin/tokens as a card stack, modern GitHub-style list

Replaces the table-based layout with a stack of self-contained token cards
inside a <ul role=list>. Each card is a flex row: avatar + name/meta on the
left, last-used block in the middle, status pill + outlined 'Revoke' button
on the right. Status and sort controls are pill-shaped toggle chips; user
email search has an inline search icon. No <table>/<tr>/<th>/<td> anywhere.
Responsive below 720px (card stacks vertically) and 480px (stat chips 2x2).
Preserves filter IDs (flt-status, flt-user, flt-last-used) and data-revoke
for existing tests.

* feat(web): add /tokens (role-aware) — single page for both user PAT CRUD and admin overview

- Rename admin_tokens.html -> tokens.html with a new is_admin context flag.
- New route GET /tokens: renders the same card-stack UI for everyone.
  * Admins: loads /auth/admin/tokens, shows owner column + stat strip, keeps
    the owner-email search box and sort-by-owner chip.
  * Non-admins: loads /auth/tokens (own tokens only), hides owner column +
    stat chips, adds a 'New token' CTA in the hero that opens a modal
    (name + expires_in_days) calling POST /auth/tokens. The raw token is
    revealed once in a dismissable banner and cleared from the DOM on Hide.
- GET /admin/tokens now 302-redirects to /tokens, preserving query string
  (so the /admin/users deep-link ?user=foo still works).

* feat(web): /tokens full-bleed layout to match dashboard width

The hero, toolbar, and card list used to sit inside base.html's .container
(max-width 800px). Break out with negative horizontal margins so the page
spans the viewport like /dashboard does, capped at 1440px for readability
on very wide screens with a 24px gutter on each side.

- No change to base.html itself. The override is scoped to .tokens-page.
- body { overflow-x: hidden; } guards against rare horizontal scrollbars.
- < 808px viewport: reset to natural flow (mobile already narrower).
- ≥ 1488px viewport: cap to 1440px and re-center.

* chore(web): remove /profile template + nav link (redirect /profile -> /tokens)

The old /profile PAT CRUD page is now redundant — the modern /tokens page
covers both user and admin flows. Delete the template; the router's
/profile handler already 302-redirects to /tokens.

Nav cleanup:
- Remove the 'Profile' link.
- Show a single 'Tokens' link to every signed-in user (previously only
  admins saw it).
- Active-state matches /tokens, /admin/tokens, and /profile so the
  highlight survives the redirect chain.

/install CTA now points at /tokens instead of /profile.

* test: cover /tokens for admin + non-admin flows, /profile redirect, nav update

tests/test_admin_tokens_ui.py
- Point admin rendering test at /tokens directly and tighten assertions
  (admin-only stat strip + owner search, non-admin CTA absent).
- Add test_non_admin_can_render_tokens_page: personal body, New-token CTA,
  create-modal, reveal banner; stat strip + owner search absent.
- Add test_admin_tokens_redirects_to_tokens: 302 to /tokens, query string
  (?user=...) preserved for the /admin/users deep-link.
- Add test_profile_redirects_to_tokens: 302 to /tokens.
- Add test_non_admin_can_create_pat_via_tokens_page_api: exercises the
  POST /auth/tokens call that the non-admin create-modal submits.

tests/test_pat.py
- test_profile_page_renders -> test_profile_page_redirects_to_tokens:
  assert the 302 + that /tokens lands on the unified non-admin body.

tests/test_web_ui.py
- admin_users nav assertion: 'Tokens' link present, 'Profile' link absent.
- Add test_nav_shows_tokens_link_for_non_admin: non-admins see the same
  'Tokens' link (previously only admins did).
- Add test_profile_redirects_to_tokens back-compat check.

* feat(web): collapse 'What Claude Code will receive' by default

The preview block on /dashboard and /install now uses <details>/<summary>
so it is hidden by default. Click the chevron/title to expand and review
the clipboard payload. Markup stays in the DOM so existing tests that
assert on content continue to pass.

* fix(web): /tokens width — override .container to 1280px like dashboard

The negative-margin full-bleed trick was fragile and pushed content past
the right edge on deployed viewports. Replace with a simple max-width
override of base.html's .container on this page only, matching
/dashboard's 1280px center-column layout.

* feat(web): split role-aware /tokens into my_tokens.html + admin_tokens.html

* feat(web): router — separate handlers for /tokens (own) and /admin/tokens (all)

* feat(web): nav — show Tokens for all, add All tokens for admins

* test: cover split token pages (own vs all) + admin access gating

* feat(web): move 'My tokens' into a user dropdown menu

Replaces the separate Tokens/email/Logout nav trio with a rounded
avatar trigger that opens a dropdown containing the user's email,
role, a 'My tokens' link, and Logout. Admin-only 'All tokens' stays
as a top-level nav item since it's an admin function, not a personal
one. Click-outside and Escape close the panel; chevron rotates on
open.

* fix(api): allow PATs to list/get/revoke their own tokens (CLI flow)

The documented 'da auth token list/revoke' CLI flow in
docs/HEADLESS_USAGE.md uses a PAT, but the previous dependency
(require_session_token) returned 403. Only create_token must be
session-only to prevent PAT-spawning-PAT chains; listing and
revoking your own tokens is safe with a PAT.

* fix(api): cap expires_in_days at 3650 to avoid datetime overflow (500 to 400)

Values above ~11 million days overflowed datetime.max in
datetime.now(utc) + timedelta(days=...) and surfaced as an
unhandled OverflowError → 500. Cap at 10 years with a clear
400 instead; the no-expiry code path is unaffected.

* fix(api): relax _SAFE_URL_RE to allow path prefixes, underscores, and IPv6

The previous regex rejected legitimate reverse-proxy base_url values
(https://host/agnes/), underscores in Docker Compose hostnames, and
IPv6 literals (http://[::1]:8000). Widen the charset and allow an
optional trailing path. shlex.quote continues to provide
defense-in-depth against any metacharacter that slips through.

* fix(web): /login/email and Google OAuth propagate next_path

Previously, /login/email silently dropped the ?next=<path> query
param so the hidden form field rendered empty and login always
landed on /dashboard. Google's button was hard-coded to
/auth/google/login, ignoring next entirely.

- /login page now appends ?next to the Google button URL
- /login/email reads + sanitizes next, passes as template context
- google_login stashes sanitized next_path in session['login_next']
- google_callback pops + re-sanitizes and redirects there

Sanitization factored into app/auth/_common.safe_next_path.

* fix(auth): differentiate argon2 VerifyMismatchError from internal errors in web login

The previous except (VerifyMismatchError, Exception) collapsed both
cases into the generic 'invalid credentials' redirect, silently
hiding corrupted-hash / library errors from ops. Split the two:
bad password still gets ?error=invalid; anything else logs via
logger.exception and redirects with ?err=auth_internal so ops have
a visible signal and users don't retry forever against a broken
password_hash column.

* docs: correct CLAUDE.md table name (personal_access_tokens)

v7 note referenced 'access_tokens.last_used_ip' but the real table
is personal_access_tokens (as mentioned two tokens earlier in the
same bullet). Same-file consistency fix.

* chore(web): clarify admin user-reset UI — encourage Set password over the unused reset_token

POST /api/users/{id}/reset-password stores and returns a token
but no endpoint consumes it — the magic-link sender would log the
user in without prompting for a new password, defeating the reset.
- Drop the 'Reset' row action from admin_users so admins aren't
  pointed at a dead end.
- Rewrite the reveal-modal copy to tell admins to use Set password
  and explicitly note that the magic-link flow isn't available
  for reset tokens in this build.
The API endpoint stays for API-level future use.

* test: cover PAT CLI flow, expires_in_days overflow, proxy base_url, next propagation

- tests/test_pat.py: PAT can list own tokens (200, was 403);
  PAT can revoke own tokens (204); create_token returns 400 for
  expires_in_days > 3650 (was 500 via datetime overflow).
- tests/test_cli_artifacts.py: _SAFE_URL_RE accepts reverse-proxy
  path prefixes, underscores, and IPv6 literals; end-to-end check
  of cli_install_script with a stubbed base_url that includes
  a path prefix (Agnes behind /agnes/).
- tests/test_web_ui.py: /login propagates ?next to the Google
  button URL; /login/email renders next in the hidden form field
  and strips hostile values; unit coverage of safe_next_path.

* fix(security): use \Z instead of $ in URL/version allowlists (trailing-\n bypass)

Python regex `$` also matches just before a trailing newline, so a Host
header or AGNES_VERSION value like "good.example.com\n$(rm -rf /)"
would slip past the allowlist. `\Z` anchors to strict end-of-string.

shlex.quote downstream remains as defense-in-depth, but the allowlist
is now the tight gate it claims to be.

* fix(auth): PAT with null expiry omits JWT exp claim (DB is the source of truth)

Previously a PAT created with `expires_in_days=null` (user-requested
"never expires") set the DB `expires_at` to NULL (correct) but still
baked a ~100y `exp` claim into the JWT. That is misleading: the PAT
silently did expire eventually, despite the UI and API promising
"no expiry".

`create_access_token` now accepts `omit_exp=True` to skip the `exp`
claim entirely. `app/api/tokens.py` passes that when `expires_in_days
is None`. The authoritative expiry check lives in
`app/auth/dependencies.py`, which reads `expires_at` from the DB row —
unchanged. PyJWT accepts claim-less JWTs indefinitely.

* test: cover trailing-newline regex bypass + no-exp JWT for unbounded PAT

- test_safe_url_re_rejects_trailing_newline_bypass: asserts both
  `_SAFE_URL_RE` and `_SAFE_VERSION_RE` reject values with a trailing
  `\n` (previously accepted because Python `$` matches before `\n`).
- test_pat_null_expiry_jwt_has_no_exp_claim: POST /auth/tokens with
  `expires_in_days=null`, decode the returned JWT, assert `exp` is
  absent while `typ=pat`, `sub`, and `jti` are still present.
- test_pat_with_null_expiry_is_accepted_by_verify_token: verify_token
  round-trips a claim-less JWT without ExpiredSignatureError.
- test_pat_null_expiry_end_to_end_allows_authenticated_request: use
  the null-expiry PAT against /auth/tokens and confirm it authenticates.

* docs(auth): document X-Forwarded-For trust model in _client_ip

Deployment runs behind Caddy which strips incoming X-Forwarded-For
and sets its own, so the leftmost hop is trustworthy. Clarify that
the stored last_used_ip is audit-only and never used for access
control — if the app is ever exposed directly, this value becomes
client-settable.

* docs: /profile → /tokens in install.sh next-steps, CLI error, HEADLESS_USAGE, security skill

After splitting PAT management to /tokens (with /profile as a back-compat
302), stale references remained in user-facing text. Update them to the
canonical /tokens URL so shell scripts, CLI error hints, docs, and the
bundled security skill are all consistent.
2026-04-22 14:24:28 +02:00

2748 lines
98 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Data Catalog - Data Analyst Portal</title>
{% if not config.THEME_FONT_URL %}
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
{% endif %}
<style>
:root {
/* Colors - Design System */
--primary: #0073D1;
--primary-light: rgba(0, 115, 209, 0.1);
--text-primary: #1A253C;
--text-secondary: #6B7280;
--background: #F5F7FA;
--surface: #FFFFFF;
--border: #E5E7EB;
--border-light: #F3F4F6;
--success: #10B77F;
--warning: #F59F0A;
--error: #EA580C;
/* Typography */
--font-primary: 'Inter', system-ui, sans-serif;
--font-mono: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
/* Shadows */
--shadow-sm: rgba(0, 0, 0, 0.05) 0px 1px 2px 0px;
--shadow-md: rgba(0, 0, 0, 0.1) 0px 4px 6px -1px;
}
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: var(--font-primary);
font-size: 14px;
color: var(--text-primary);
background: var(--background);
line-height: 1.5;
}
/* ── Header (dashboard-style) ── */
.header {
background: var(--surface);
border-bottom: 1px solid var(--border);
padding: 0 32px;
height: 72px;
display: flex;
align-items: center;
justify-content: space-between;
position: sticky;
top: 0;
z-index: 100;
}
.header-left {
display: flex;
align-items: center;
gap: 16px;
}
.header-back {
display: flex;
align-items: center;
justify-content: center;
width: 32px;
height: 32px;
border-radius: 6px;
color: var(--text-secondary);
text-decoration: none;
transition: all 0.15s ease;
}
.header-back:hover {
background: var(--border-light);
color: var(--text-primary);
}
.header-logo-group {
display: flex;
flex-direction: column;
justify-content: center;
gap: 2px;
}
.header-logo svg {
display: block;
}
.header-subtitle {
font-size: 11px;
font-weight: 500;
color: var(--text-secondary);
letter-spacing: 0.4px;
text-transform: uppercase;
margin-top: 2px;
}
.header-right {
font-size: 12px;
color: var(--text-secondary);
}
/* ── Page Title ── */
.page-title {
max-width: 900px;
margin: 0 auto;
padding: 32px 24px 24px;
}
.page-title h1 {
font-size: 24px;
font-weight: 800;
color: var(--text-primary);
margin-bottom: 4px;
}
.page-title p {
font-size: 14px;
color: var(--text-secondary);
}
/* ── Source Cards ── */
.source-cards {
max-width: 900px;
margin: 0 auto;
padding: 0 24px 32px;
display: flex;
flex-direction: column;
gap: 20px;
}
.source-card {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 12px;
overflow: hidden;
box-shadow: var(--shadow-sm);
}
.source-card-header {
display: flex;
align-items: flex-start;
justify-content: space-between;
padding: 20px 24px;
gap: 16px;
}
.source-card-left {
display: flex;
gap: 14px;
flex: 1;
min-width: 0;
}
.source-card-icon {
width: 42px;
height: 42px;
border-radius: 10px;
display: flex;
align-items: center;
justify-content: center;
flex-shrink: 0;
}
.source-card-icon.primary {
background: var(--primary-light);
}
.source-card-icon.jira {
background: rgba(107, 114, 128, 0.1);
}
.source-card-info {
flex: 1;
min-width: 0;
}
.source-card-name {
font-size: 16px;
font-weight: 700;
color: var(--text-primary);
margin-bottom: 2px;
}
.source-card-desc {
font-size: 13px;
color: var(--text-secondary);
margin-bottom: 6px;
}
.source-card-meta {
font-size: 12px;
color: var(--text-secondary);
}
.source-card-right {
display: flex;
align-items: center;
gap: 10px;
flex-shrink: 0;
}
/* ── Toggle Switch ── */
.toggle-switch {
position: relative;
width: 36px;
height: 20px;
}
.toggle-switch input {
opacity: 0;
width: 0;
height: 0;
}
.toggle-slider {
position: absolute;
cursor: pointer;
top: 0;
left: 0;
right: 0;
bottom: 0;
background-color: #E5E7EB;
transition: .2s;
border-radius: 20px;
}
.toggle-slider:before {
position: absolute;
content: "";
height: 14px;
width: 14px;
left: 3px;
bottom: 3px;
background-color: white;
transition: .2s;
border-radius: 50%;
}
input:checked + .toggle-slider {
background-color: var(--primary);
}
input:checked + .toggle-slider:before {
transform: translateX(16px);
}
.toggle-switch.locked .toggle-slider {
cursor: not-allowed;
background-color: var(--primary);
opacity: 0.6;
}
.toggle-switch.locked .toggle-slider:before {
transform: translateX(16px);
}
input:disabled + .toggle-slider {
cursor: not-allowed;
opacity: 0.6;
}
/* ── Badges ── */
.badge-included {
flex-shrink: 0;
font-size: 11px;
font-weight: 500;
color: var(--primary);
background: var(--primary-light);
border-radius: 6px;
padding: 4px 10px;
white-space: nowrap;
}
.badge-subscribed {
flex-shrink: 0;
font-size: 11px;
font-weight: 500;
color: var(--primary);
background: var(--primary-light);
border-radius: 6px;
padding: 4px 10px;
white-space: nowrap;
}
.badge-unsubscribed {
flex-shrink: 0;
font-size: 11px;
font-weight: 500;
color: var(--text-secondary);
background: var(--border-light);
border-radius: 6px;
padding: 4px 10px;
white-space: nowrap;
}
/* ── Accordion ── */
.accordion-category {
border-top: 1px solid var(--border-light);
}
.accordion-trigger {
width: 100%;
display: flex;
align-items: center;
gap: 10px;
padding: 12px 24px;
background: none;
border: none;
cursor: pointer;
font-family: var(--font-primary);
font-size: 14px;
font-weight: 500;
color: var(--text-primary);
text-align: left;
transition: background 0.1s ease;
}
.accordion-trigger:hover {
background: var(--border-light);
}
.accordion-chevron {
width: 16px;
height: 16px;
color: var(--text-secondary);
transition: transform 0.2s ease;
flex-shrink: 0;
}
.accordion-trigger.expanded .accordion-chevron {
transform: rotate(90deg);
}
.accordion-count {
font-size: 11px;
font-weight: 500;
color: var(--text-secondary);
background: var(--border-light);
padding: 1px 7px;
border-radius: 9999px;
margin-left: auto;
}
.accordion-content {
display: none;
}
.accordion-content.expanded {
display: block;
}
/* ── Table Rows inside Accordion ── */
.table-row {
display: flex;
align-items: center;
padding: 10px 24px 10px 50px;
border-top: 1px solid var(--border-light);
cursor: pointer;
transition: background 0.1s ease;
}
.table-row:hover {
background: rgba(243, 244, 246, 0.5);
}
.table-row-locked {
opacity: 0.75;
}
.table-row-locked:hover {
background: rgba(234, 88, 12, 0.04);
}
.access-badge {
display: inline-block;
font-size: 11px;
font-weight: 600;
padding: 1px 6px;
border-radius: 4px;
margin-left: 6px;
vertical-align: middle;
font-family: var(--font-primary);
}
.access-badge.locked {
font-size: 13px;
padding: 0 2px;
}
.access-badge.pending {
background: #FFF7ED;
color: #EA580C;
border: 1px solid #FDBA74;
}
.btn-request-access {
font-size: 12px;
font-weight: 500;
padding: 4px 12px;
border-radius: 6px;
background: var(--primary-light);
color: var(--primary);
cursor: pointer;
transition: all 0.15s ease;
white-space: nowrap;
}
.btn-request-access:hover {
background: var(--primary);
color: #fff;
}
/* Request Access Modal */
.request-access-overlay {
display: none;
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
z-index: 1100;
padding: 40px 24px;
}
.request-access-overlay.active {
display: flex;
align-items: center;
justify-content: center;
}
.request-access-modal {
max-width: 500px;
width: 100%;
background: var(--surface);
border-radius: 12px;
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3);
overflow: hidden;
}
.request-access-modal .modal-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 20px 24px 16px;
border-bottom: 1px solid var(--border);
}
.request-access-modal .modal-header h3 {
font-size: 16px;
font-weight: 600;
}
.request-access-modal .modal-body {
padding: 20px 24px;
}
.request-access-modal .modal-body label {
display: block;
font-size: 13px;
font-weight: 500;
color: var(--text-secondary);
margin-top: 16px;
margin-bottom: 6px;
}
.request-access-modal .modal-body textarea {
width: 100%;
border: 1px solid var(--border);
border-radius: 8px;
padding: 10px 12px;
font-family: var(--font-primary);
font-size: 13px;
resize: vertical;
}
.request-access-modal .modal-body textarea:focus {
outline: none;
border-color: var(--primary);
box-shadow: 0 0 0 3px var(--primary-light);
}
.request-access-modal .modal-footer {
display: flex;
justify-content: flex-end;
gap: 8px;
padding: 16px 24px 20px;
border-top: 1px solid var(--border);
}
.request-access-modal .btn-secondary {
padding: 8px 16px;
border-radius: 8px;
border: 1px solid var(--border);
background: var(--surface);
font-size: 13px;
font-weight: 500;
cursor: pointer;
color: var(--text-primary);
}
.request-access-modal .btn-primary {
padding: 8px 16px;
border-radius: 8px;
border: none;
background: var(--primary);
color: #fff;
font-size: 13px;
font-weight: 500;
cursor: pointer;
}
.request-access-modal .btn-primary:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.table-row-left {
flex: 1;
min-width: 0;
}
.table-row-name {
font-weight: 500;
font-family: var(--font-mono);
font-size: 13px;
color: var(--text-primary);
}
.table-row-desc {
font-size: 12px;
color: var(--text-secondary);
margin-top: 1px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.table-row-right {
display: flex;
align-items: center;
gap: 12px;
flex-shrink: 0;
margin-left: 16px;
}
.rows-badge {
font-size: 12px;
font-weight: 500;
padding: 2px 8px;
border-radius: 4px;
background: var(--border-light);
color: var(--text-secondary);
font-variant-numeric: tabular-nums;
white-space: nowrap;
}
.rows-badge.large {
background: rgba(245, 159, 10, 0.1);
color: #B45309;
}
/* ── Query Mode Badges ── */
.query-mode-badge {
font-size: 10px;
font-weight: 600;
padding: 2px 7px;
border-radius: 4px;
text-transform: uppercase;
letter-spacing: 0.3px;
white-space: nowrap;
}
.query-mode-badge.local {
background: var(--primary-light);
color: var(--primary);
}
.query-mode-badge.live {
background: rgba(16, 183, 127, 0.1);
color: #047857;
}
.table-sync-info {
font-size: 11px;
color: var(--text-secondary);
margin-top: 2px;
display: flex;
align-items: center;
gap: 5px;
}
.table-sync-info .live-dot {
width: 6px;
height: 6px;
border-radius: 50%;
background: #10B77F;
animation: pulse-live 2s ease-in-out infinite;
flex-shrink: 0;
}
@keyframes pulse-live {
0%, 100% { opacity: 1; box-shadow: 0 0 0 0 rgba(16, 183, 127, 0.4); }
50% { opacity: 0.7; box-shadow: 0 0 0 3px rgba(16, 183, 127, 0); }
}
.data-freshness-note {
padding: 8px 24px;
font-size: 12px;
color: var(--text-secondary);
display: flex;
align-items: center;
gap: 6px;
border-top: 1px solid var(--border-light);
}
.data-freshness-note svg {
flex-shrink: 0;
opacity: 0.5;
}
.profile-link {
display: flex;
align-items: center;
gap: 4px;
font-size: 11px;
color: var(--primary);
opacity: 0.4;
transition: opacity 0.15s;
white-space: nowrap;
}
.table-row:hover .profile-link {
opacity: 1;
}
/* ── Jira Unsubscribed State ── */
.source-card-unsubscribed {
padding: 16px 24px;
border-top: 1px solid var(--border-light);
font-size: 13px;
color: var(--text-secondary);
background: var(--border-light);
}
/* ── Jira Attachments Option ── */
.jira-attachment-option {
display: flex;
align-items: center;
justify-content: space-between;
padding: 14px 24px;
border-top: 1px solid var(--border-light);
background: var(--background);
}
.jira-attachment-label {
display: flex;
flex-direction: column;
}
.jira-attachment-label span:first-child {
font-size: 13px;
font-weight: 500;
color: var(--text-primary);
}
.jira-attachment-label span:last-child {
font-size: 12px;
color: var(--text-secondary);
}
/* ── Footer ── */
.footer {
text-align: center;
padding: 24px;
color: var(--text-secondary);
font-size: 12px;
}
.footer a {
color: var(--primary);
text-decoration: none;
}
.footer a:hover {
text-decoration: underline;
}
/* ── Responsive ── */
@media (max-width: 640px) {
.source-card-header {
flex-direction: column;
}
.source-card-right {
align-self: flex-end;
}
.table-row {
padding-left: 24px;
}
.table-row-desc {
white-space: normal;
}
}
/* ═══════════════════════════════════════════════ */
/* Profiler Modal - preserved from original */
/* ═══════════════════════════════════════════════ */
.profiler-overlay {
display: none;
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
z-index: 1000;
padding: 40px 24px;
overflow: hidden;
}
.profiler-overlay.active {
display: flex;
align-items: flex-start;
justify-content: center;
}
.profiler-modal {
max-width: 1100px;
width: 100%;
max-height: calc(100vh - 80px);
background: var(--surface);
border-radius: 12px;
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3);
overflow: hidden;
display: flex;
flex-direction: column;
}
.profiler-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 20px 24px;
border-bottom: 1px solid var(--border);
background: var(--background);
}
.profiler-header-left {
display: flex;
align-items: center;
gap: 12px;
}
.profiler-header h2 {
font-size: 20px;
font-weight: 600;
font-family: var(--font-mono);
}
.profiler-close {
width: 32px;
height: 32px;
border: none;
background: none;
cursor: pointer;
border-radius: 6px;
display: flex;
align-items: center;
justify-content: center;
color: var(--text-secondary);
transition: all 0.15s;
}
.profiler-close:hover {
background: var(--border-light);
color: var(--text-primary);
}
.profiler-body {
padding: 24px;
overflow-y: auto;
overflow-x: hidden;
flex: 1;
min-height: 0;
}
/* Alert badges */
.alert-badge {
display: inline-flex;
align-items: center;
font-size: 11px;
font-weight: 600;
padding: 2px 8px;
border-radius: 4px;
text-transform: uppercase;
letter-spacing: 0.02em;
}
.alert-badge.constant { background: #FEF3C7; color: #92400E; }
.alert-badge.unique { background: #EDE9FE; color: #5B21B6; }
.alert-badge.high_missing { background: #FEE2E2; color: #991B1B; }
.alert-badge.missing { background: #FEF3C7; color: #92400E; }
.alert-badge.imbalance { background: #DBEAFE; color: #1E40AF; }
.alert-badge.zeros { background: #DBEAFE; color: #1E40AF; }
.alert-badge.high_cardinality { background: #E0E7FF; color: #3730A3; }
/* Quality badge */
.quality-badge {
font-size: 12px;
font-weight: 600;
padding: 3px 10px;
border-radius: 9999px;
}
.quality-badge.good { background: #D1FAE5; color: #065F46; }
.quality-badge.warning { background: #FEF3C7; color: #92400E; }
.quality-badge.poor { background: #FEE2E2; color: #991B1B; }
/* Profiler navigation tabs */
.profiler-tabs {
display: flex;
gap: 0;
border-bottom: 1px solid var(--border);
padding: 0 24px;
flex-shrink: 0;
}
.profiler-tab {
padding: 10px 20px;
font-size: 13px;
font-weight: 500;
color: var(--text-secondary);
background: none;
border: none;
cursor: pointer;
border-bottom: 2px solid transparent;
transition: all 0.15s;
}
.profiler-tab:hover {
color: var(--text-primary);
background: var(--border-light);
}
.profiler-tab.active {
color: var(--primary);
border-bottom-color: var(--primary);
}
.profiler-tab .tab-count {
font-size: 11px;
font-weight: 600;
background: var(--border-light);
padding: 1px 6px;
border-radius: 9999px;
margin-left: 6px;
}
.profiler-tab.active .tab-count {
background: rgba(0, 115, 209, 0.1);
color: var(--primary);
}
.profiler-section {
display: none;
}
.profiler-section.active {
display: block;
}
/* Overview section */
.overview-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 24px;
margin-bottom: 24px;
}
.overview-stats-table {
width: 100%;
border-collapse: collapse;
}
.overview-stats-table td {
padding: 8px 12px;
font-size: 13px;
border-bottom: 1px solid var(--border-light);
}
.overview-stats-table td:first-child {
font-weight: 500;
color: var(--text-primary);
width: 55%;
}
.overview-stats-table td:last-child {
color: var(--text-secondary);
text-align: right;
font-variant-numeric: tabular-nums;
}
.overview-stats-table .highlight {
color: #DC2626;
font-weight: 500;
}
.overview-title {
font-size: 15px;
font-weight: 600;
margin-bottom: 12px;
color: var(--text-primary);
}
/* Variable cards */
.variable-card {
border: 1px solid var(--border);
border-radius: 8px;
margin-bottom: 16px;
overflow: hidden;
}
.variable-header {
display: flex;
align-items: center;
gap: 8px;
padding: 12px 16px;
background: var(--background);
border-bottom: 1px solid var(--border);
}
.variable-name {
font-weight: 600;
font-family: var(--font-mono);
font-size: 14px;
color: var(--primary);
}
.variable-type {
font-size: 11px;
font-weight: 500;
color: var(--text-secondary);
background: var(--border-light);
padding: 2px 8px;
border-radius: 4px;
}
.variable-body {
display: grid;
grid-template-columns: auto 1fr;
gap: 0;
}
.variable-stats {
padding: 12px 16px;
min-width: 0;
}
.variable-stats table {
width: 100%;
border-collapse: collapse;
}
.variable-stats td {
padding: 4px 0;
font-size: 13px;
white-space: nowrap;
}
.variable-stats td:first-child {
color: var(--text-secondary);
padding-right: 12px;
}
.variable-stats td:last-child {
text-align: right;
font-variant-numeric: tabular-nums;
}
.variable-chart {
padding: 12px 16px;
display: flex;
align-items: flex-start;
justify-content: center;
min-height: 120px;
min-width: 0;
}
.variable-chart canvas {
max-width: 100%;
max-height: 150px;
}
/* Numeric range bar visualization */
.num-viz {
width: 100%;
font-size: 12px;
}
.num-range {
margin-bottom: 10px;
}
.num-range-labels {
display: flex;
justify-content: space-between;
font-size: 11px;
color: var(--text-secondary);
margin-bottom: 4px;
font-variant-numeric: tabular-nums;
}
.num-range-track {
position: relative;
height: 14px;
background: var(--border-light);
border-radius: 7px;
margin-bottom: 6px;
}
.num-range-whisker {
position: absolute;
top: 0;
height: 100%;
background: rgba(0, 115, 209, 0.15);
border-radius: 7px;
}
.num-range-iqr {
position: absolute;
top: 0;
height: 100%;
background: rgba(0, 115, 209, 0.45);
border-radius: 5px;
}
.num-range-marker {
position: absolute;
top: -3px;
width: 2px;
height: 20px;
border-radius: 1px;
}
.num-range-legend {
display: flex;
gap: 10px;
font-size: 10px;
color: var(--text-secondary);
margin-top: 2px;
position: relative;
overflow: visible;
}
.num-range-legend span::before {
content: '';
display: inline-block;
width: 8px;
height: 8px;
border-radius: 2px;
margin-right: 3px;
vertical-align: middle;
}
.num-range-legend .legend-mean::before { background: #DC2626; }
.num-range-legend .legend-median::before { background: #059669; }
.num-range-legend .legend-iqr::before { background: rgba(0, 115, 209, 0.45); }
.num-constant {
text-align: center;
padding: 12px 0;
color: var(--text-secondary);
font-size: 13px;
}
.num-constant strong {
color: var(--text-primary);
font-variant-numeric: tabular-nums;
}
.num-stats-row {
display: flex;
flex-wrap: wrap;
gap: 6px;
margin-top: 10px;
}
.num-stat {
display: inline-flex;
align-items: center;
gap: 4px;
background: var(--border-light);
border-radius: 10px;
padding: 3px 8px;
font-size: 11px;
line-height: 1.3;
cursor: help;
position: relative;
}
/* Tooltip trigger styling */
[data-tip] {
cursor: help;
}
/* Floating tooltip element (appended to body via JS) */
.tip-box {
position: fixed;
background: var(--text-primary);
color: #fff;
font-size: 11px;
font-weight: 400;
font-family: var(--font-primary);
line-height: 1.4;
padding: 6px 10px;
border-radius: 6px;
max-width: 260px;
pointer-events: none;
opacity: 0;
transition: opacity 0.15s ease;
z-index: 9999;
}
.tip-box.visible {
opacity: 1;
}
.num-stat-label {
color: var(--text-secondary);
}
.num-stat-value {
font-weight: 500;
font-variant-numeric: tabular-nums;
}
.num-stat.alert-zeros .num-stat-value {
color: #DC2626;
}
/* Categorical bar chart */
.cat-bars {
width: 100%;
padding: 8px 0;
}
.cat-bar-row {
display: flex;
align-items: center;
gap: 8px;
margin-bottom: 4px;
font-size: 12px;
}
.cat-bar-label {
width: 100px;
text-align: right;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--text-secondary);
flex-shrink: 0;
}
.cat-bar-track {
flex: 1;
height: 18px;
background: var(--border-light);
border-radius: 2px;
overflow: hidden;
position: relative;
}
.cat-bar-fill {
height: 100%;
background: var(--primary);
border-radius: 2px;
display: flex;
align-items: center;
padding: 0 6px;
min-width: 30px;
}
.cat-bar-count {
font-size: 11px;
font-weight: 500;
color: white;
white-space: nowrap;
}
.cat-bar-count.outside {
color: var(--text-secondary);
margin-left: 6px;
}
/* Boolean bar */
.bool-bar {
display: flex;
height: 24px;
border-radius: 4px;
overflow: hidden;
width: 100%;
margin-top: 8px;
}
.bool-bar .true-part {
background: var(--primary);
display: flex;
align-items: center;
justify-content: center;
color: white;
font-size: 11px;
font-weight: 600;
}
.bool-bar .false-part {
background: #E5E7EB;
display: flex;
align-items: center;
justify-content: center;
color: var(--text-secondary);
font-size: 11px;
font-weight: 600;
}
/* Insights list */
.insight-group {
margin-bottom: 16px;
border-radius: 8px;
border: 1px solid var(--border-light);
overflow: hidden;
}
.insight-group.border-imbalance { border-left: 3px solid #3B82F6; }
.insight-group.border-unique { border-left: 3px solid #7C3AED; }
.insight-group.border-high_missing { border-left: 3px solid #DC2626; }
.insight-group.border-missing { border-left: 3px solid #D97706; }
.insight-group.border-zeros { border-left: 3px solid #3B82F6; }
.insight-group.border-high_cardinality { border-left: 3px solid #4338CA; }
.insight-group.border-constant { border-left: 3px solid #D97706; }
.insight-group-header {
display: flex;
align-items: baseline;
gap: 10px;
padding: 10px 16px;
background: var(--background);
}
.alert-explanation {
font-size: 12px;
color: var(--text-secondary);
line-height: 1.4;
}
.insight-items {
padding: 0;
}
.insight-item {
display: flex;
align-items: center;
justify-content: space-between;
padding: 7px 16px;
border-top: 1px solid var(--border-light);
font-size: 13px;
}
.insight-col {
font-weight: 500;
font-family: var(--font-mono);
color: var(--primary);
cursor: pointer;
text-decoration: none;
}
.insight-col:hover {
text-decoration: underline;
}
.insight-detail {
color: var(--text-secondary);
font-size: 12px;
}
.variable-highlight {
animation: highlightPulse 2s ease-out;
}
@keyframes highlightPulse {
0% { box-shadow: 0 0 0 3px rgba(0, 115, 209, 0.4); }
100% { box-shadow: none; }
}
/* Missing values chart */
.missing-chart-container {
height: 200px;
margin-bottom: 24px;
}
/* Relationships */
.relationship-card {
border: 1px solid var(--border);
border-radius: 8px;
padding: 16px;
margin-bottom: 12px;
}
.relationship-header {
font-size: 13px;
font-weight: 600;
margin-bottom: 8px;
}
.relationship-item {
display: flex;
align-items: center;
gap: 8px;
padding: 6px 0;
font-size: 13px;
color: var(--text-secondary);
}
.relationship-table-link {
font-family: var(--font-mono);
font-weight: 500;
color: var(--primary);
cursor: pointer;
text-decoration: none;
}
.relationship-table-link:hover {
text-decoration: underline;
}
.relationship-direction {
font-size: 11px;
padding: 1px 6px;
border-radius: 4px;
font-weight: 500;
}
.relationship-direction.belongs_to { background: #DBEAFE; color: #1E40AF; }
.relationship-direction.has_many { background: #D1FAE5; color: #065F46; }
/* Relationship Mermaid diagram */
.relationship-diagram-wrap {
background: var(--background);
border: 1px solid var(--border);
border-radius: 8px;
padding: 16px;
margin-bottom: 16px;
overflow-x: auto;
}
.relationship-diagram-wrap .mermaid {
display: flex;
justify-content: center;
}
.relationship-diagram-wrap svg {
max-width: 100%;
height: auto;
}
.relationship-diagram-wrap svg .node.clickable-node {
cursor: pointer;
}
.relationship-diagram-wrap svg .node.clickable-node:hover {
filter: brightness(0.92);
transition: filter 0.15s ease;
}
.relationship-legend {
display: flex;
gap: 16px;
justify-content: center;
margin-top: 10px;
font-size: 11px;
color: var(--text-secondary);
}
.relationship-legend-item {
display: flex;
align-items: center;
gap: 5px;
}
.relationship-legend-dot {
width: 10px;
height: 10px;
border-radius: 50%;
display: inline-block;
}
.relationship-legend-dot.current { background: #2563eb; }
.relationship-legend-dot.parent { background: #DBEAFE; border: 1px solid #93C5FD; }
.relationship-legend-dot.child { background: #D1FAE5; border: 1px solid #6EE7B7; }
.relationship-legend-dot.metric { background: #FEF3C7; border: 1px solid #FCD34D; }
/* Metrics badges */
.metric-badge {
display: inline-flex;
align-items: center;
gap: 4px;
font-size: 12px;
font-weight: 500;
padding: 4px 10px;
background: rgba(0, 115, 209, 0.08);
color: var(--primary);
border-radius: 6px;
margin: 4px 4px 4px 0;
}
/* Sample data table */
.sample-table {
width: 100%;
border-collapse: collapse;
font-size: 12px;
overflow-x: auto;
display: block;
}
.sample-table th {
font-weight: 600;
background: var(--background);
padding: 8px 12px;
border: 1px solid var(--border);
white-space: nowrap;
font-family: var(--font-mono);
font-size: 11px;
}
.sample-table td {
padding: 6px 12px;
border: 1px solid var(--border-light);
max-width: 200px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-variant-numeric: tabular-nums;
}
/* Loading spinner */
.profiler-loading {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
padding: 60px;
color: var(--text-secondary);
gap: 12px;
}
.spinner {
width: 32px;
height: 32px;
border: 3px solid var(--border);
border-top-color: var(--primary);
border-radius: 50%;
animation: spin 0.8s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
/* Empty state */
.profiler-empty {
text-align: center;
padding: 48px;
color: var(--text-secondary);
}
.profiler-empty p {
margin-top: 8px;
font-size: 13px;
}
</style>
<link rel="stylesheet" href="{{ url_for('static', filename='css/metric_modal.css', v=git_version) }}">
<!-- Prism.js for SQL syntax highlighting -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/prismjs@1.29.0/themes/prism.min.css">
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.7/dist/chart.umd.min.js"></script>
{% include '_theme.html' %}
</head>
<body>
<!-- ═══════════════ HEADER ═══════════════ -->
{% include '_app_header.html' %}
<!-- ═══════════════ PAGE TITLE ═══════════════ -->
<div class="page-title">
<h1>Data Catalog</h1>
<p>Browse available data sources and manage your subscriptions</p>
</div>
<!-- ═══════════════ SOURCE CARDS ═══════════════ -->
<div class="source-cards">
<!-- ── Card 1: Core Business Data ── -->
<div class="source-card">
<div class="source-card-header">
<div class="source-card-left">
<div class="source-card-icon primary">
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#0073D1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
<path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z"/>
<polyline points="14 2 14 8 20 8"/>
<line x1="16" y1="13" x2="8" y2="13"/>
<line x1="16" y1="17" x2="8" y2="17"/>
<polyline points="10 9 9 9 8 9"/>
</svg>
</div>
<div class="source-card-info">
<div class="source-card-name">Core Business Data</div>
<div class="source-card-desc">Core business data from internal systems</div>
<div class="source-card-meta">
{{ data_stats.total_tables or data_stats.tables }} tables &middot; ~{{ data_stats.rows_display }} rows total
{% if data_stats.last_updated %}
&middot; Synced {{ data_stats.last_updated }}
{% endif %}
</div>
</div>
</div>
<div class="source-card-right">
<span class="badge-included">Always included</span>
<label class="toggle-switch locked">
<input type="checkbox" checked disabled>
<span class="toggle-slider"></span>
</label>
</div>
</div>
{% for category in catalog_data %}
<div class="accordion-category">
<button class="accordion-trigger" onclick="toggleAccordion(this)">
<svg class="accordion-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<polyline points="9 18 15 12 9 6"/>
</svg>
{{ category.name }}
<span class="accordion-count">{{ category.count }} tables</span>
</button>
<div class="accordion-content">
{% for table in category.tables %}
<div class="table-row{% if not table.is_public and not table.has_access %} table-row-locked{% endif %}" {% if not table.is_public and not table.has_access %}{% if table.pending_request %}title="Access request pending"{% else %}onclick="openRequestModal('{{ table.id }}', '{{ table.name }}')"{% endif %}{% elif table.query_mode != 'remote' %}onclick="openProfiler('{{ table.name }}')"{% endif %}>
<div class="table-row-left">
<div class="table-row-name">
{{ table.name }}
{% if not table.is_public and not table.has_access %}
{% if table.pending_request %}
<span class="access-badge pending">Pending</span>
{% else %}
<span class="access-badge locked" title="Private table - request access">&#128274;</span>
{% endif %}
{% endif %}
{% if table.query_mode == 'remote' %}
<span class="query-mode-badge live">Live</span>
{% else %}
<span class="query-mode-badge local">Local</span>
{% endif %}
</div>
<div class="table-row-desc">{{ table.description }}</div>
<div class="table-sync-info">
{% if not table.is_public and not table.has_access and not table.pending_request %}
<span style="color: var(--text-secondary);">Click to request access</span>
{% elif table.query_mode == 'remote' %}
<span class="live-dot"></span> Queried directly from BigQuery
{% elif table.last_sync %}
Synced {{ table.last_sync }}
{% endif %}
</div>
</div>
<div class="table-row-right">
{% if not table.is_public and not table.has_access %}
{% if table.pending_request %}
<span class="rows-badge" style="background: #FFF7ED; color: #EA580C;">Awaiting review</span>
{% else %}
<span class="btn-request-access" onclick="event.stopPropagation(); openRequestModal('{{ table.id }}', '{{ table.name }}')">Request Access</span>
{% endif %}
{% else %}
<span class="rows-badge{{ ' large' if table.rows_large }}">{{ table.rows_display }}</span>
{% if table.query_mode != 'remote' %}
<span class="profile-link">
<svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><path d="M2 12s3-7 10-7 10 7 10 7-3 7-10 7-10-7-10-7z"/><circle cx="12" cy="12" r="3"/></svg>
Profile
</span>
{% endif %}
{% endif %}
</div>
</div>
{% endfor %}
</div>
</div>
{% endfor %}
</div>
<!-- ── Card: Business Metrics ── -->
{% if metrics_data %}
{% set metrics_total = namespace(n=0) %}
{% for c in metrics_data %}{% set metrics_total.n = metrics_total.n + c.metrics|length %}{% endfor %}
<div class="source-card">
<div class="source-card-header">
<div class="source-card-left">
<div class="source-card-icon primary">
<svg width="22" height="22" viewBox="0 0 24 24" fill="none" stroke="#0073D1" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round">
<path d="M3 3v18h18"/>
<path d="M18.7 8l-5.1 5.2-2.8-2.7L7 14.3"/>
</svg>
</div>
<div class="source-card-info">
<div class="source-card-name">Business Metrics</div>
<div class="source-card-desc">Standardized metric definitions with SQL examples and documentation</div>
<div class="source-card-meta">{{ metrics_total.n }} metrics &middot; {{ metrics_data|length }} categories</div>
</div>
</div>
<div class="source-card-right">
<span class="badge-included">Always included</span>
<label class="toggle-switch locked">
<input type="checkbox" checked disabled>
<span class="toggle-slider"></span>
</label>
</div>
</div>
{% if data_stats and data_stats.last_updated %}
<div class="data-freshness-note">
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><polyline points="12 6 12 12 16 14"/></svg>
Calculated from data synced {{ data_stats.last_updated }}
</div>
{% endif %}
{% for category in metrics_data %}
<div class="accordion-category">
<button class="accordion-trigger" onclick="toggleAccordion(this)">
<svg class="accordion-chevron" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<polyline points="9 18 15 12 9 6"/>
</svg>
<span class="category-tag {{ category.css }}">{{ category.label }}</span>
<span class="accordion-count">{{ category.metrics|length }} metric{{ 's' if category.metrics|length != 1 }}</span>
</button>
<div class="accordion-content">
{% for metric in category.metrics %}
<div class="table-row" onclick="openMetricModal('{{ metric.path }}')">
<div class="table-row-left">
<div class="table-row-name">{{ metric.display_name }}</div>
<div class="table-row-desc">{{ metric.description }}</div>
</div>
<div class="table-row-right">
<span class="rows-badge">{{ metric.grain }}</span>
</div>
</div>
{% endfor %}
</div>
</div>
{% endfor %}
</div>
{% endif %}
</div>
</div>
<!-- ═══════════════ PROFILER MODAL (preserved 1:1) ═══════════════ -->
<div id="profilerOverlay" class="profiler-overlay" onclick="if(event.target===this)closeProfiler()">
<div class="profiler-modal">
<div class="profiler-header">
<div class="profiler-header-left">
<h2 id="profilerTitle">-</h2>
<span id="profilerQuality" class="quality-badge"></span>
<span id="profilerAlertSummary"></span>
</div>
<button class="profiler-close" onclick="closeProfiler()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
<path d="M18 6L6 18M6 6l12 12"/>
</svg>
</button>
</div>
<!-- Tabs (outside scrollable body so they stay fixed) -->
<div class="profiler-tabs" id="profilerTabs" style="display:none;">
<button class="profiler-tab active" onclick="switchTab('overview')">Overview</button>
<button class="profiler-tab" onclick="switchTab('variables')" id="tabVariables">Columns</button>
<button class="profiler-tab" onclick="switchTab('alerts')" id="tabAlerts">Insights</button>
<button class="profiler-tab" onclick="switchTab('missing')" id="tabMissing">Missing Values</button>
<button class="profiler-tab" onclick="switchTab('relationships')" id="tabRelationships">Relationships</button>
<button class="profiler-tab" onclick="switchTab('sample')" id="tabSample">Sample</button>
</div>
<div class="profiler-body">
<div id="profilerLoading" class="profiler-loading">
<div class="spinner"></div>
<span>Loading profile data...</span>
</div>
<div id="profilerContent" style="display:none;">
<!-- Overview Section -->
<div id="sectionOverview" class="profiler-section active"></div>
<!-- Variables Section -->
<div id="sectionVariables" class="profiler-section"></div>
<!-- Alerts Section -->
<div id="sectionAlerts" class="profiler-section"></div>
<!-- Missing Values Section -->
<div id="sectionMissing" class="profiler-section">
<div class="missing-chart-container">
<canvas id="missingChart"></canvas>
</div>
</div>
<!-- Relationships Section -->
<div id="sectionRelationships" class="profiler-section"></div>
<!-- Sample Section -->
<div id="sectionSample" class="profiler-section"></div>
</div>
<div id="profilerError" class="profiler-empty" style="display:none;">
<svg width="48" height="48" viewBox="0 0 24 24" fill="none" stroke="var(--text-secondary)" stroke-width="1.5" opacity="0.5"><circle cx="12" cy="12" r="10"/><path d="M12 8v4M12 16h.01"/></svg>
<p id="profilerErrorMsg">Profile data not available for this table.</p>
</div>
</div>
</div>
</div>
<footer class="footer">
<p>&copy; {{ config.INSTANCE_COPYRIGHT or 'AI Data Analyst' }} &middot; Updated daily</p>
</footer>
<script>
/* ═══════════════ ACCORDION ═══════════════ */
function toggleAccordion(trigger) {
const content = trigger.nextElementSibling;
const isExpanded = trigger.classList.contains('expanded');
if (isExpanded) {
trigger.classList.remove('expanded');
content.classList.remove('expanded');
} else {
trigger.classList.add('expanded');
content.classList.add('expanded');
}
}
/* ═══════════════ DATASET SUBSCRIPTIONS ═══════════════ */
/* ═══════════════ PROFILER (preserved 1:1) ═══════════════ */
let currentCharts = [];
let currentProfile = null;
let mermaidDiagramRendered = false;
let currentDiagramNodeMap = {};
function openProfiler(tableName, initialTab) {
const overlay = document.getElementById('profilerOverlay');
const loading = document.getElementById('profilerLoading');
const content = document.getElementById('profilerContent');
const error = document.getElementById('profilerError');
overlay.classList.add('active');
loading.style.display = 'flex';
content.style.display = 'none';
document.getElementById('profilerTabs').style.display = 'none';
error.style.display = 'none';
document.body.style.overflow = 'hidden';
document.getElementById('profilerTitle').textContent = tableName;
fetch(`/api/catalog/profile/${tableName}`)
.then(r => {
if (!r.ok) throw new Error('not found');
return r.json();
})
.then(data => {
currentProfile = data;
loading.style.display = 'none';
content.style.display = 'block';
document.getElementById('profilerTabs').style.display = 'flex';
renderProfile(data, initialTab);
})
.catch(err => {
loading.style.display = 'none';
document.getElementById('profilerTabs').style.display = 'none';
error.style.display = 'block';
});
}
function closeProfiler() {
document.getElementById('profilerOverlay').classList.remove('active');
document.getElementById('profilerTabs').style.display = 'none';
document.body.style.overflow = '';
destroyCharts();
currentProfile = null;
mermaidDiagramRendered = false;
currentDiagramNodeMap = {};
}
function destroyCharts() {
currentCharts.forEach(c => c.destroy());
currentCharts = [];
}
function switchTab(tabName) {
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
event.target.classList.add('active');
document.getElementById('section' + tabName.charAt(0).toUpperCase() + tabName.slice(1)).classList.add('active');
// Lazy-render Mermaid diagram when Relationships tab is first shown
if (tabName === 'relationships' && !mermaidDiagramRendered && typeof mermaid !== 'undefined') {
const mermaidEl = document.querySelector('#sectionRelationships .mermaid');
if (mermaidEl && !mermaidEl.getAttribute('data-processed')) {
mermaid.run({ nodes: [mermaidEl] }).then(() => {
attachDiagramClickHandlers();
});
mermaidDiagramRendered = true;
}
}
}
function navigateToVariable(columnName) {
// Switch to Columns tab
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
const varTab = document.querySelector('#tabVariables');
if (varTab) varTab.classList.add('active');
document.getElementById('sectionVariables').classList.add('active');
// Scroll to the variable card within the modal body
const card = document.getElementById('var_' + columnName);
if (card) {
// Use scrollIntoView within the scrollable .profiler-body container
setTimeout(() => {
card.scrollIntoView({ behavior: 'smooth', block: 'center' });
card.classList.add('variable-highlight');
setTimeout(() => card.classList.remove('variable-highlight'), 2000);
}, 50);
}
}
function navigateToInsight(insightType) {
// Switch to Insights tab
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
const tab = document.querySelector('#tabAlerts');
if (tab) tab.classList.add('active');
document.getElementById('sectionAlerts').classList.add('active');
const group = document.getElementById('insight_' + insightType);
if (group) {
setTimeout(() => {
group.scrollIntoView({ behavior: 'smooth', block: 'start' });
group.classList.add('variable-highlight');
setTimeout(() => group.classList.remove('variable-highlight'), 2000);
}, 50);
}
}
function formatNumber(n) {
if (n === null || n === undefined) return '-';
if (Math.abs(n) >= 1e9) return (n / 1e9).toFixed(1) + 'B';
if (Math.abs(n) >= 1e6) return (n / 1e6).toFixed(1) + 'M';
if (Math.abs(n) >= 1e3) return (n / 1e3).toFixed(1) + 'K';
if (Number.isInteger(n)) return n.toLocaleString();
return Number(n).toFixed(2);
}
const INSIGHT_TOOLTIPS = {
unique: "Every value in this column is different. This is typical for ID columns and usually means this column can uniquely identify each row.",
high_cardinality: "This column has a very large number of distinct values. It may be an ID or free-text field. Not ideal for grouping or filtering in reports.",
high_missing: "Most values in this column are empty. Consider whether this data is collected consistently or if this column is only relevant for specific records.",
missing: "Some values in this column are empty. Check if the gaps are expected (e.g. optional fields) or indicate a data collection issue.",
zeros: "A large portion of values are zero. This may be normal (e.g. no change periods) or could indicate missing data entered as zero.",
imbalance: "One value dominates this column (appears in 90%+ of rows). This is normal for status or flag columns but limits analytical usefulness.",
constant: "This column has the same value in every row. It provides no analytical value and can usually be ignored in analysis.",
};
function renderAlertBadge(alert, clickable) {
const tooltip = INSIGHT_TOOLTIPS[alert] || '';
const label = alert.replace('_', ' ');
if (clickable) {
return `<span class="alert-badge ${alert}" title="${tooltip}" style="cursor:pointer;" onclick="event.stopPropagation();navigateToInsight('${alert}')">${label}</span>`;
}
return `<span class="alert-badge ${alert}" title="${tooltip}">${label}</span>`;
}
function renderProfile(data, initialTab) {
destroyCharts();
// Header badges
const quality = data.avg_completeness || 0;
const qBadge = document.getElementById('profilerQuality');
qBadge.textContent = quality.toFixed(1) + '% complete';
qBadge.className = 'quality-badge ' + (quality >= 95 ? 'good' : quality >= 80 ? 'warning' : 'poor');
const alertCount = (data.alerts || []).length;
document.getElementById('profilerAlertSummary').innerHTML = alertCount > 0
? `<span class="alert-badge imbalance">${alertCount} insight${alertCount > 1 ? 's' : ''}</span>`
: '';
// Tab counts
const cols = data.columns || [];
document.getElementById('tabVariables').innerHTML = `Columns <span class="tab-count">${cols.length}</span>`;
document.getElementById('tabAlerts').innerHTML = `Insights <span class="tab-count">${alertCount}</span>`;
renderOverview(data);
renderVariables(data);
renderAlerts(data);
renderMissing(data);
renderRelationships(data);
renderSample(data);
// Set active tab (default: overview)
const targetTab = initialTab || 'overview';
document.querySelectorAll('.profiler-tab').forEach(t => t.classList.remove('active'));
document.querySelectorAll('.profiler-section').forEach(s => s.classList.remove('active'));
const targetSection = document.getElementById('section' + targetTab.charAt(0).toUpperCase() + targetTab.slice(1));
if (targetSection) {
targetSection.classList.add('active');
// Activate matching tab button
document.querySelectorAll('.profiler-tab').forEach(t => {
if (t.getAttribute('onclick') && t.getAttribute('onclick').includes("'" + targetTab + "'")) {
t.classList.add('active');
}
});
// Trigger lazy Mermaid render if opening relationships tab directly
if (targetTab === 'relationships' && !mermaidDiagramRendered && typeof mermaid !== 'undefined') {
const mermaidEl = document.querySelector('#sectionRelationships .mermaid');
if (mermaidEl && !mermaidEl.getAttribute('data-processed')) {
mermaid.run({ nodes: [mermaidEl] }).then(() => {
attachDiagramClickHandlers();
});
mermaidDiagramRendered = true;
}
}
} else {
// Fallback to overview
document.querySelector('.profiler-tab').classList.add('active');
document.getElementById('sectionOverview').classList.add('active');
}
}
function renderOverview(data) {
const vt = data.variable_types || {};
const vtHtml = Object.entries(vt).map(([t, c]) =>
`<tr><td>${t}</td><td>${c}</td></tr>`
).join('');
const missingPct = data.missing_cells_pct || 0;
const missingClass = missingPct > 20 ? ' class="highlight"' : '';
const dr = data.date_range;
const dateRow = dr ? `<tr><td>Date Coverage</td><td>${dr.earliest} to ${dr.latest}</td></tr>` : '';
// Catalog enrichment section
let catalogHtml = '';
if (data.catalog) {
const cat = data.catalog;
const tagsHtml = (cat.tags || []).length > 0
? `<div style="margin-top:8px;"><span style="font-size:11px;color:var(--text-secondary);font-weight:500;">Tags:</span> ${cat.tags.map(t => `<span class="metric-badge" style="display:inline-block;margin:2px 4px 2px 0;">${t}</span>`).join('')}</div>`
: '';
const ownersHtml = (cat.owners || []).length > 0
? `<div style="margin-top:8px;"><span style="font-size:11px;color:var(--text-secondary);font-weight:500;">Owners:</span> ${cat.owners.join(', ')}</div>`
: '';
const tierHtml = cat.tier
? `<div style="margin-top:8px;"><span style="font-size:11px;color:var(--text-secondary);font-weight:500;">Tier:</span> <strong>${cat.tier}</strong></div>`
: '';
const urlHtml = cat.url
? `<div style="margin-top:8px;"><a href="${cat.url}" target="_blank" style="font-size:11px;color:var(--primary);text-decoration:none;">View in Data Catalog →</a></div>`
: '';
catalogHtml = `<div style="margin-top:16px;border-top:1px solid var(--border);padding-top:16px;"><div class="overview-title">Data Catalog</div>${tierHtml}${ownersHtml}${tagsHtml}${urlHtml}</div>`;
}
document.getElementById('sectionOverview').innerHTML = `
<div class="overview-grid">
<div>
<div class="overview-title">Dataset Statistics</div>
<table class="overview-stats-table">
<tr><td>Number of variables</td><td>${data.column_count || 0}</td></tr>
<tr><td>Number of observations</td><td>${formatNumber(data.row_count || 0)}</td></tr>
<tr><td>Missing cells</td><td${missingClass}>${formatNumber(data.missing_cells || 0)}</td></tr>
<tr><td>Missing cells (%)</td><td${missingClass}>${missingPct.toFixed(1)}%</td></tr>
<tr><td>Duplicate rows</td><td>${formatNumber(data.duplicate_rows || 0)}</td></tr>
<tr><td>File size</td><td>${(data.file_size_mb || 0).toFixed(2)} MB</td></tr>
${dateRow}
<tr><td>Last sync</td><td>${data.last_sync ? data.last_sync.substring(0, 16).replace('T', ' ') : '-'}</td></tr>
</table>
</div>
<div>
<div class="overview-title">Variable Types</div>
<table class="overview-stats-table">${vtHtml}</table>
${data.description ? `<div style="margin-top:16px;"><div class="overview-title">Description</div><p style="font-size:13px;color:var(--text-secondary);">${data.description}</p></div>` : ''}
${(data.used_by_metrics || []).length > 0 ? `<div style="margin-top:16px;"><div class="overview-title">Used by Metrics</div><div>${data.used_by_metrics.map(m => m.file ? `<a class="metric-badge" style="cursor:pointer;text-decoration:none;" onclick="openMetricModal('${m.file}')">${m.name}</a>` : `<span class="metric-badge">${typeof m === 'string' ? m : m.name}</span>`).join('')}</div></div>` : ''}
${catalogHtml}
</div>
</div>`;
}
function buildNumericViz(ns, idx) {
const hasRange = ns.min != null && ns.max != null;
const isConstant = hasRange && ns.min === ns.max;
let rangeHtml = '';
if (isConstant) {
rangeHtml = `<div class="num-constant">Constant value: <strong>${formatNumber(ns.min)}</strong></div>`;
} else if (hasRange) {
const range = ns.max - ns.min;
const pos = (val) => Math.max(0, Math.min(100, ((val - ns.min) / range) * 100));
// Whisker (p5 -> p95)
let whiskerHtml = '';
if (ns.p5 != null && ns.p95 != null) {
const wLeft = pos(ns.p5);
const wWidth = pos(ns.p95) - wLeft;
whiskerHtml = `<div class="num-range-whisker" style="left:${wLeft}%;width:${wWidth}%"></div>`;
}
// IQR box (p25 -> p75)
let iqrHtml = '';
if (ns.p25 != null && ns.p75 != null) {
const iLeft = pos(ns.p25);
const iWidth = pos(ns.p75) - iLeft;
iqrHtml = `<div class="num-range-iqr" style="left:${iLeft}%;width:${iWidth}%"></div>`;
}
// Mean marker (red)
let meanMarker = '';
if (ns.mean != null) {
meanMarker = `<div class="num-range-marker" style="left:${pos(ns.mean)}%;background:#DC2626;transform:translateX(-50%)"></div>`;
}
// Median marker (green)
let medianMarker = '';
if (ns.median != null) {
medianMarker = `<div class="num-range-marker" style="left:${pos(ns.median)}%;background:#059669;transform:translateX(-50%)"></div>`;
}
rangeHtml = `<div class="num-range">
<div class="num-range-labels">
<span>${formatNumber(ns.min)}</span>
<span>${formatNumber(ns.max)}</span>
</div>
<div class="num-range-track">
${whiskerHtml}${iqrHtml}${meanMarker}${medianMarker}
</div>
<div class="num-range-legend">
<span class="legend-mean" data-tip="Average value. If far from Median, data is skewed by extreme values.">Mean</span>
<span class="legend-median" data-tip="Middle value. Half of records are above, half below this point.">Median</span>
<span class="legend-iqr" data-tip="Where 50% of your data falls. Narrow = consistent values, wide = high variation.">Typical range</span>
</div>
</div>`;
}
// Stat pills with business-friendly tooltips
const PILL_TIPS = {
mean: 'Average value across all rows. Useful for understanding the typical size, but can be skewed by extreme values.',
median: 'The middle value when sorted. More reliable than Mean when data has outliers \u2014 if Mean and Median differ a lot, you have skewed data.',
std: 'Standard Deviation \u2014 how spread out the values are. Low = values cluster near the mean. High = wide variation across records.',
zeros: 'How many records have value zero. High percentage may indicate missing data entered as zero, or genuinely inactive records.',
negative: 'Records with negative values. Could indicate returns, corrections, or data entry errors depending on the business context.',
};
const pills = [];
if (ns.mean != null) pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.mean}"><span class="num-stat-label">Mean</span><span class="num-stat-value">${formatNumber(ns.mean)}</span></span>`);
if (ns.median != null) pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.median}"><span class="num-stat-label">Median</span><span class="num-stat-value">${formatNumber(ns.median)}</span></span>`);
if (ns.stddev != null) pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.std}"><span class="num-stat-label">Std</span><span class="num-stat-value">${formatNumber(ns.stddev)}</span></span>`);
if (ns.zeros) {
const alertClass = ns.zeros_pct > 50 ? ' alert-zeros' : '';
pills.push(`<span class="num-stat${alertClass}" data-tip="${PILL_TIPS.zeros}"><span class="num-stat-label">Zeros</span><span class="num-stat-value">${formatNumber(ns.zeros)} (${ns.zeros_pct.toFixed(1)}%)</span></span>`);
}
if (ns.negative) {
pills.push(`<span class="num-stat" data-tip="${PILL_TIPS.negative}"><span class="num-stat-label">Negative</span><span class="num-stat-value">${formatNumber(ns.negative)} (${ns.negative_pct.toFixed(1)}%)</span></span>`);
}
return `<div class="num-viz">
${rangeHtml}
<div style="height:140px;"><canvas id="hist_${idx}"></canvas></div>
<div class="num-stats-row">${pills.join('')}</div>
</div>`;
}
function renderVariables(data) {
const cols = data.columns || [];
let html = '';
cols.forEach((col, idx) => {
const alerts = (col.alerts || []).map(a => renderAlertBadge(a, true)).join(' ');
const type = col.type || 'STRING';
const simplifiedType = type.includes('TIMESTAMP') ? 'DateTime' :
type === 'DATE' ? 'Date' :
type === 'BOOLEAN' ? 'Boolean' :
type === 'NUMERIC' || type.includes('INT') || type.includes('FLOAT') || type.includes('DOUBLE') ? 'Numeric' :
'Text';
const missingPct = 100 - (col.completeness_pct || 100);
const missingClass = missingPct > 30 ? ' class="highlight"' : missingPct > 5 ? ' style="color:#B45309;"' : '';
const distinctPct = col.unique_pct || 0;
const distinctClass = distinctPct >= 99.9 ? ' style="color:#DC2626;"' : '';
let chartHtml = '';
// Categorical bar chart
if (col.string_stats && col.string_stats.top_values && col.string_stats.top_values.length > 0) {
const topVals = col.string_stats.top_values.slice(0, 8);
const maxCount = topVals[0].count;
chartHtml = '<div class="cat-bars">' + topVals.map(v => {
const pct = (v.count / maxCount) * 100;
const label = v.value.length > 15 ? v.value.substring(0, 15) + '...' : v.value;
const countStr = formatNumber(v.count);
const showInside = pct > 25;
return `<div class="cat-bar-row">
<span class="cat-bar-label" title="${v.value}">${label}</span>
<div class="cat-bar-track">
<div class="cat-bar-fill" style="width:${Math.max(pct, 3)}%">
${showInside ? `<span class="cat-bar-count">${countStr}</span>` : ''}
</div>
</div>
${!showInside ? `<span class="cat-bar-count outside">${countStr}</span>` : ''}
</div>`;
}).join('') + '</div>';
}
// Numeric stats with range bar visualization
if (col.numeric_stats) {
const ns = col.numeric_stats;
chartHtml = buildNumericViz(ns, idx);
}
// Date stats - labels beside chart for maximum chart size
if (col.date_stats) {
const ds = col.date_stats;
chartHtml = `<div style="display:flex;gap:16px;align-items:stretch;width:100%;padding:8px 0;">
<div style="font-size:12px;min-width:130px;flex-shrink:0;display:flex;flex-direction:column;justify-content:center;gap:6px;">
<div><span style="color:var(--text-secondary);display:block;font-size:10px;">Earliest</span><span style="font-weight:500;">${ds.earliest}</span></div>
<div><span style="color:var(--text-secondary);display:block;font-size:10px;">Latest</span><span style="font-weight:500;">${ds.latest}</span></div>
<div><span style="color:var(--text-secondary);display:block;font-size:10px;">Span</span><span style="font-weight:500;">${formatNumber(ds.span_days)} days</span></div>
</div>
<div style="flex:1;min-height:140px;"><canvas id="datehist_${idx}"></canvas></div>
</div>`;
}
// Boolean bar
if (col.boolean_stats) {
const bs = col.boolean_stats;
const truePct = bs.true_pct || 0;
chartHtml = `<div style="padding:8px 0;width:100%;">
<div class="bool-bar">
<div class="true-part" style="width:${truePct}%">${truePct > 15 ? `True ${truePct.toFixed(1)}%` : ''}</div>
<div class="false-part" style="width:${100 - truePct}%">${(100 - truePct) > 15 ? `False ${(100 - truePct).toFixed(1)}%` : ''}</div>
</div>
<div style="display:flex;justify-content:space-between;font-size:11px;color:var(--text-secondary);margin-top:4px;">
<span>True: ${formatNumber(bs.true_count)}</span>
<span>False: ${formatNumber(bs.false_count)}</span>
</div>
</div>`;
}
// Fallback: sample values
if (!chartHtml && col.sample_values && col.sample_values.length > 0) {
chartHtml = `<div style="font-size:12px;padding:8px 0;color:var(--text-secondary);overflow:hidden;width:100%;">
<div style="font-weight:500;margin-bottom:4px;">Sample values</div>
${col.sample_values.slice(0, 5).map(v => `<div style="font-family:var(--font-mono);font-size:11px;padding:2px 0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;" title="${String(v).replace(/"/g, '&quot;')}">${v}</div>`).join('')}
</div>`;
}
html += `
<div class="variable-card" id="var_${col.name}">
<div class="variable-header">
<span class="variable-name">${col.name}</span>
<span class="variable-type">${simplifiedType}</span>
${col.is_primary_key ? '<span class="alert-badge unique">PK</span>' : ''}
${alerts}
</div>
<div class="variable-body">
<div class="variable-stats">
<table>
<tr><td>Distinct</td><td${distinctClass}>${formatNumber(col.unique_count)}</td></tr>
<tr><td>Distinct (%)</td><td${distinctClass}>${distinctPct.toFixed(1)}%</td></tr>
<tr><td>Missing</td><td${missingClass}>${formatNumber(col.null_count || 0)}</td></tr>
<tr><td>Missing (%)</td><td${missingClass}>${missingPct.toFixed(1)}%</td></tr>
</table>
</div>
<div class="variable-chart">${chartHtml}</div>
</div>
</div>`;
});
document.getElementById('sectionVariables').innerHTML = html;
// Render Chart.js histograms for numeric columns
cols.forEach((col, idx) => {
if (col.numeric_stats && col.numeric_stats.histogram) {
const canvas = document.getElementById(`hist_${idx}`);
if (canvas) {
const h = col.numeric_stats.histogram;
const chart = new Chart(canvas, {
type: 'bar',
data: {
labels: h.bins || h.counts.map((_, i) => i),
datasets: [{
data: h.counts,
backgroundColor: 'rgba(0, 115, 209, 0.6)',
borderRadius: 2,
}]
},
options: {
responsive: true,
maintainAspectRatio: false,
plugins: { legend: { display: false } },
scales: {
x: { display: false },
y: { display: false }
}
}
});
currentCharts.push(chart);
}
}
if (col.date_stats && col.date_stats.histogram) {
const canvas = document.getElementById(`datehist_${idx}`);
if (canvas) {
const h = col.date_stats.histogram;
const chart = new Chart(canvas, {
type: 'bar',
data: {
labels: h.bins || h.counts.map((_, i) => i),
datasets: [{
data: h.counts,
backgroundColor: 'rgba(0, 115, 209, 0.6)',
borderRadius: 2,
}]
},
options: {
responsive: true,
maintainAspectRatio: false,
plugins: { legend: { display: false } },
scales: {
x: { ticks: { maxTicksLimit: 6, font: { size: 9 } } },
y: { display: false }
}
}
});
currentCharts.push(chart);
}
}
});
}
function extractInsightDetail(alert) {
// Strip column name prefix from message to avoid redundancy
const msg = alert.message || '';
const col = alert.column || '';
const rest = msg.startsWith(col) ? msg.substring(col.length).trim() : msg;
// Clean up leading connectors like "is", "has"
return rest.replace(/^(is|has)\s+/, '');
}
function renderAlerts(data) {
const alerts = data.alerts || [];
if (alerts.length === 0) {
document.getElementById('sectionAlerts').innerHTML = '<div class="profiler-empty"><p>No data quality insights detected.</p></div>';
return;
}
// Group alerts by type for cleaner display
const byType = {};
alerts.forEach(a => {
if (!byType[a.type]) byType[a.type] = [];
byType[a.type].push(a);
});
let html = '';
Object.entries(byType).forEach(([type, items]) => {
const tip = INSIGHT_TOOLTIPS[type] || '';
html += `<div class="insight-group border-${type}" id="insight_${type}">
<div class="insight-group-header">
${renderAlertBadge(type)}
${tip ? `<span class="alert-explanation">${tip}</span>` : ''}
</div>
<div class="insight-items">`;
items.forEach(a => {
const detail = extractInsightDetail(a);
html += `<div class="insight-item">
<a class="insight-col" onclick="navigateToVariable('${a.column}')">${a.column}</a>
<span class="insight-detail">${detail}</span>
</div>`;
});
html += '</div></div>';
});
document.getElementById('sectionAlerts').innerHTML = html;
}
function renderMissing(data) {
const cols = (data.columns || []).filter(c => c.completeness_pct !== undefined);
if (cols.length === 0) return;
const canvas = document.getElementById('missingChart');
if (!canvas) return;
const sorted = [...cols].sort((a, b) => (a.completeness_pct || 0) - (b.completeness_pct || 0));
const chart = new Chart(canvas, {
type: 'bar',
data: {
labels: sorted.map(c => c.name),
datasets: [{
label: 'Completeness %',
data: sorted.map(c => c.completeness_pct || 0),
backgroundColor: sorted.map(c => {
const pct = c.completeness_pct || 0;
return pct >= 95 ? 'rgba(16, 183, 127, 0.6)' :
pct >= 80 ? 'rgba(245, 159, 10, 0.6)' :
'rgba(220, 38, 38, 0.6)';
}),
borderRadius: 2,
}]
},
options: {
indexAxis: 'y',
responsive: true,
maintainAspectRatio: false,
plugins: { legend: { display: false } },
scales: {
x: { min: 0, max: 100, ticks: { callback: v => v + '%' } },
y: { ticks: { font: { size: 10, family: "'Inter', sans-serif" } } }
}
}
});
currentCharts.push(chart);
// Adjust canvas container height based on column count
canvas.parentElement.style.height = Math.max(200, cols.length * 22 + 40) + 'px';
}
function sanitizeMermaidId(name) {
return 'n_' + name.replace(/[^a-zA-Z0-9_]/g, '_');
}
function truncateLabel(text, max) {
return text.length > max ? text.substring(0, max) + '...' : text;
}
function buildMermaidDiagram(tableName, relations, metrics) {
const MAX_NODES = 15;
const parents = relations.filter(r => r.direction === 'belongs_to');
const children = relations.filter(r => r.direction === 'has_many');
const metricsList = metrics || [];
const currentId = sanitizeMermaidId(tableName);
const nodeMap = {}; // sanitizedId -> {type, name, file?}
let lines = ['flowchart LR'];
// Current table node (bold box)
lines.push(` ${currentId}["<b>${tableName}</b>"]`);
nodeMap[currentId] = { type: 'current', name: tableName };
let nodeCount = 1;
let overflowParents = 0;
let overflowChildren = 0;
const renderedParentIds = [];
const renderedChildIds = [];
// Parent nodes (belongs_to) - current --> parent
parents.forEach(r => {
if (nodeCount >= MAX_NODES) { overflowParents++; return; }
const pid = sanitizeMermaidId(r.table);
const label = truncateLabel(r.join_column, 20);
lines.push(` ${currentId} -->|"${label}"| ${pid}["${r.table}"]`);
renderedParentIds.push(pid);
nodeMap[pid] = { type: 'table', name: r.table };
nodeCount++;
});
// Child nodes (has_many) - child --> current
children.forEach(r => {
if (nodeCount >= MAX_NODES) { overflowChildren++; return; }
const cid = sanitizeMermaidId(r.table);
const label = truncateLabel(r.foreign_column || r.join_column, 20);
lines.push(` ${cid}["${r.table}"] -->|"${label}"| ${currentId}`);
renderedChildIds.push(cid);
nodeMap[cid] = { type: 'table', name: r.table };
nodeCount++;
});
// Metric nodes - current table -.-> metric (dashed)
const renderedMetricIds = [];
let overflowMetrics = 0;
metricsList.forEach(m => {
if (nodeCount >= MAX_NODES) { overflowMetrics++; return; }
const mid = sanitizeMermaidId('metric_' + m.name);
lines.push(` ${currentId} -.->|"metric"| ${mid}("${truncateLabel(m.name, 20)}")`);
renderedMetricIds.push(mid);
nodeMap[mid] = { type: 'metric', name: m.name, file: m.file };
nodeCount++;
});
// Overflow nodes
if (overflowParents > 0) {
lines.push(` ${currentId} -.->|" "| n_more_parents("+${overflowParents} more")`);
lines.push(` style n_more_parents stroke-dasharray: 5 5,fill:#f8fafc,color:#94a3b8`);
}
if (overflowChildren > 0) {
lines.push(` n_more_children("+${overflowChildren} more") -.->|" "| ${currentId}`);
lines.push(` style n_more_children stroke-dasharray: 5 5,fill:#f8fafc,color:#94a3b8`);
}
if (overflowMetrics > 0) {
lines.push(` ${currentId} -.->|" "| n_more_metrics("+${overflowMetrics} more")`);
lines.push(` style n_more_metrics stroke-dasharray: 5 5,fill:#f8fafc,color:#94a3b8`);
}
// Styling - only for actually rendered nodes
lines.push(` style ${currentId} fill:#2563eb,stroke:#1d4ed8,color:#fff,font-weight:bold`);
renderedParentIds.forEach(pid => {
lines.push(` style ${pid} fill:#dbeafe,stroke:#93c5fd,color:#1e40af`);
});
renderedChildIds.forEach(cid => {
lines.push(` style ${cid} fill:#d1fae5,stroke:#6ee7b7,color:#065f46`);
});
renderedMetricIds.forEach(mid => {
lines.push(` style ${mid} fill:#fef3c7,stroke:#fcd34d,color:#92400e`);
});
return { diagram: lines.join('\n'), nodeMap };
}
function attachDiagramClickHandlers() {
const wrap = document.querySelector('.relationship-diagram-wrap');
if (!wrap) return;
const svg = wrap.querySelector('svg');
if (!svg) return;
svg.querySelectorAll('.node').forEach(node => {
// Mermaid node IDs: "flowchart-n_contract_line-0"
const match = node.id.match(/^flowchart-(.+?)-\d+$/);
if (!match) return;
const sanitizedId = match[1];
const entry = currentDiagramNodeMap[sanitizedId];
if (!entry || entry.type === 'current') return;
node.classList.add('clickable-node');
if (entry.type === 'metric' && entry.file) {
node.addEventListener('click', () => {
openMetricModal(entry.file);
});
} else if (entry.type === 'table') {
node.addEventListener('click', () => {
closeProfiler();
setTimeout(() => openProfiler(entry.name, 'relationships'), 300);
});
}
});
}
function renderRelationships(data) {
const rels = data.related_tables || [];
const metrics = data.used_by_metrics || [];
if (rels.length === 0 && metrics.length === 0) {
document.getElementById('sectionRelationships').innerHTML = '<div class="profiler-empty"><p>No relationships or metric references found.</p></div>';
return;
}
let html = '';
// Mermaid diagram (relationships + metrics)
if (rels.length > 0 || metrics.length > 0) {
const tableName = document.getElementById('profilerTitle').textContent;
const result = buildMermaidDiagram(tableName, rels, metrics);
currentDiagramNodeMap = result.nodeMap;
let legendHtml = '<span class="relationship-legend-item"><span class="relationship-legend-dot current"></span> Current table</span>';
if (rels.some(r => r.direction === 'belongs_to'))
legendHtml += '<span class="relationship-legend-item"><span class="relationship-legend-dot parent"></span> References (belongs_to)</span>';
if (rels.some(r => r.direction === 'has_many'))
legendHtml += '<span class="relationship-legend-item"><span class="relationship-legend-dot child"></span> Referenced by (has_many)</span>';
if (metrics.length > 0)
legendHtml += '<span class="relationship-legend-item"><span class="relationship-legend-dot metric"></span> Business Metric</span>';
html += `<div class="relationship-diagram-wrap">
<pre class="mermaid">${result.diagram}</pre>
<div class="relationship-legend">${legendHtml}</div>
</div>`;
const outgoing = rels.filter(r => r.direction === 'belongs_to');
const incoming = rels.filter(r => r.direction === 'has_many');
if (outgoing.length > 0) {
html += '<div class="relationship-card"><div class="relationship-header">References (belongs to)</div>';
outgoing.forEach(r => {
html += `<div class="relationship-item">
<span class="relationship-direction belongs_to">belongs_to</span>
<a class="relationship-table-link" onclick="closeProfiler();setTimeout(()=>openProfiler('${r.table}'),300)">${r.table}</a>
<span>via ${r.join_column} → ${r.foreign_column || r.join_column}</span>
</div>`;
});
html += '</div>';
}
if (incoming.length > 0) {
html += '<div class="relationship-card"><div class="relationship-header">Referenced by (has many)</div>';
incoming.forEach(r => {
html += `<div class="relationship-item">
<span class="relationship-direction has_many">has_many</span>
<a class="relationship-table-link" onclick="closeProfiler();setTimeout(()=>openProfiler('${r.table}'),300)">${r.table}</a>
<span>via ${r.foreign_column || r.join_column}</span>
</div>`;
});
html += '</div>';
}
}
if (metrics.length > 0) {
html += '<div class="relationship-card"><div class="relationship-header">Used by Business Metrics</div>';
html += '<div>' + metrics.map(m => {
if (m.file) {
return `<a class="metric-badge" style="cursor:pointer;text-decoration:none;" onclick="openMetricModal('${m.file}')">${m.name}</a>`;
}
return `<span class="metric-badge">${m.name}</span>`;
}).join('') + '</div>';
html += '</div>';
}
document.getElementById('sectionRelationships').innerHTML = html;
mermaidDiagramRendered = false;
}
function renderSample(data) {
const rows = data.sample_rows || [];
if (rows.length === 0) {
document.getElementById('sectionSample').innerHTML = '<div class="profiler-empty"><p>No sample data available.</p></div>';
return;
}
const cols = Object.keys(rows[0]);
let html = '<div style="overflow-x:auto;"><table class="sample-table"><thead><tr>';
html += cols.map(c => `<th>${c}</th>`).join('');
html += '</tr></thead><tbody>';
rows.forEach(row => {
html += '<tr>' + cols.map(c => {
const v = row[c];
const display = v === null ? '<span style="color:#D1D5DB;">null</span>' :
String(v).length > 50 ? String(v).substring(0, 50) + '...' : String(v);
return `<td title="${String(v || '')}">${display}</td>`;
}).join('') + '</tr>';
});
html += '</tbody></table></div>';
document.getElementById('sectionSample').innerHTML = html;
}
// Close on Escape
document.addEventListener('keydown', e => {
if (e.key === 'Escape') closeProfiler();
});
// Floating tooltip for [data-tip] elements
(function() {
const box = document.createElement('div');
box.className = 'tip-box';
document.body.appendChild(box);
let timer = null;
document.addEventListener('mouseover', e => {
const el = e.target.closest('[data-tip]');
if (!el) return;
clearTimeout(timer);
timer = setTimeout(() => {
box.textContent = el.getAttribute('data-tip');
box.classList.add('visible');
const r = el.getBoundingClientRect();
let top = r.bottom + 6;
let left = r.left + r.width / 2;
// Measure box to avoid right edge overflow
box.style.left = '0px';
box.style.top = '0px';
const bw = box.offsetWidth;
left = Math.max(8, Math.min(left - bw / 2, window.innerWidth - bw - 8));
// If below viewport, flip above
if (top + box.offsetHeight > window.innerHeight - 8) {
top = r.top - box.offsetHeight - 6;
}
box.style.left = left + 'px';
box.style.top = top + 'px';
}, 400);
});
document.addEventListener('mouseout', e => {
const el = e.target.closest('[data-tip]');
if (!el) return;
clearTimeout(timer);
box.classList.remove('visible');
});
})();
</script>
<!-- ═══════════════ REQUEST ACCESS MODAL ═══════════════ -->
<div id="requestAccessOverlay" class="request-access-overlay" onclick="if(event.target===this)closeRequestModal()">
<div class="request-access-modal">
<div class="modal-header">
<h3>Request Access</h3>
<button class="btn-secondary" style="border:none;font-size:18px;padding:4px 8px;line-height:1;" onclick="closeRequestModal()">&times;</button>
</div>
<div class="modal-body">
<p>Table: <strong id="requestTableName"></strong></p>
<label for="requestReason">Reason (optional)</label>
<textarea id="requestReason" rows="3" placeholder="Why do you need access to this table?"></textarea>
<p id="requestStatus" style="margin-top:12px;display:none;font-size:13px;"></p>
</div>
<div class="modal-footer">
<button class="btn-secondary" onclick="closeRequestModal()">Cancel</button>
<button class="btn-primary" id="submitRequestBtn" onclick="submitAccessRequest()">Request Access</button>
</div>
</div>
</div>
<script>
let requestTableId = '';
function openRequestModal(tableId, tableName) {
requestTableId = tableId;
document.getElementById('requestTableName').textContent = tableName;
document.getElementById('requestReason').value = '';
document.getElementById('requestStatus').style.display = 'none';
document.getElementById('submitRequestBtn').disabled = false;
document.getElementById('submitRequestBtn').textContent = 'Request Access';
document.getElementById('requestAccessOverlay').classList.add('active');
}
function closeRequestModal() {
document.getElementById('requestAccessOverlay').classList.remove('active');
}
async function submitAccessRequest() {
const reason = document.getElementById('requestReason').value;
const btn = document.getElementById('submitRequestBtn');
const status = document.getElementById('requestStatus');
btn.disabled = true;
try {
const resp = await fetch('/api/access-requests', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({table_id: requestTableId, reason: reason})
});
if (resp.ok) {
status.textContent = 'Request submitted! An admin will review it.';
status.style.color = 'var(--success)';
status.style.display = 'block';
btn.textContent = 'Submitted';
} else if (resp.status === 409) {
status.textContent = 'You already have a pending request for this table.';
status.style.color = 'var(--warning)';
status.style.display = 'block';
} else {
throw new Error('Request failed');
}
} catch(e) {
status.textContent = 'Error submitting request. Please try again.';
status.style.color = 'var(--error)';
status.style.display = 'block';
btn.disabled = false;
}
}
</script>
<!-- ═══════════════ METRIC MODAL ═══════════════ -->
<div id="metricModalOverlay" class="metric-modal-overlay">
<div id="metricModal" class="metric-modal" onclick="event.stopPropagation()">
<!-- Header -->
<div class="metric-modal-header">
<div class="metric-modal-title-section">
<h2 id="metricModalTitle" class="metric-modal-title"></h2>
<div id="metricModalMetadata" class="metric-metadata-chips"></div>
</div>
<button class="metric-modal-close" onclick="closeMetricModal()" aria-label="Close modal">
<svg fill="none" stroke="currentColor" stroke-width="2" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" d="M6 18L18 6M6 6l12 12"/>
</svg>
</button>
</div>
<!-- Tabs -->
<nav class="metric-tabs">
<button class="metric-tab active" data-tab="tabOverview" onclick="switchMetricTab('tabOverview')">
Overview
</button>
<button class="metric-tab" data-tab="tabHowToUse" onclick="switchMetricTab('tabHowToUse')">
How to Use
</button>
<button class="metric-tab" data-tab="tabSQLExamples" onclick="switchMetricTab('tabSQLExamples')">
SQL Examples
</button>
<button class="metric-tab" data-tab="tabTechnical" onclick="switchMetricTab('tabTechnical')">
Metric Details
</button>
</nav>
<!-- Body -->
<div id="metricModalBody" class="metric-modal-body">
<!-- Content loaded dynamically -->
</div>
</div>
</div>
<!-- Prism.js for SQL syntax highlighting -->
<script src="https://cdn.jsdelivr.net/npm/prismjs@1.29.0/prism.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/prismjs@1.29.0/components/prism-sql.min.js"></script>
<script src="{{ url_for('static', filename='js/metric_modal.js', v=git_version) }}"></script>
<!-- Mermaid.js for relationship diagrams -->
<script type="module">
import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@11/dist/mermaid.esm.min.mjs';
mermaid.initialize({
startOnLoad: false,
theme: 'base',
themeVariables: {
fontFamily: 'Inter, system-ui, sans-serif',
fontSize: '13px',
primaryColor: '#dbeafe',
primaryBorderColor: '#2563eb',
primaryTextColor: '#1e40af',
lineColor: '#94a3b8',
secondaryColor: '#f1f5f9',
tertiaryColor: '#f5f7fa',
edgeLabelBackground: '#f1f5f9',
},
flowchart: {
curve: 'basis',
padding: 12,
htmlLabels: true,
useMaxWidth: true,
},
});
// Expose mermaid globally for lazy rendering
window.mermaid = mermaid;
</script>
{% include "_version_badge.html" %}
</body>
</html>
Reference in a new issue View git blame Copy permalink