agnes-the-ai-analyst/app/web/templates/admin_tables.html
Vojtech 001e5ce40e
feat(web): /home value-first redesign + unified page-shell across app (#366)
* feat(web): value-first /home reskin (CEO mock palette + pillars + first-session)

Restructures `/home` to lead with product value instead of install steps,
matching the CEO mock proposed for the homepage:

- New intro hero on top — eyebrow `Welcome, {{ display_name }}`, H1
  `{{ instance_brand }} is your team's AI workspace`, lede framing the
  product as an "AI Chief of Staff", two CTAs (`Set up in ~15 min →`
  jumps to the wizard, `Just browse — no install needed` jumps to
  `#look-around`), and a four-pillar row (Data packages · Plugins ·
  Skills · Memory). Renders for both onboarded and not-onboarded users
  so the value framing is consistent across visits.
- New `first-session` narrative — five-beat walkthrough (launch → pick
  project → memory loads → ask → close) with mock terminal frames
  carrying traffic-light dots, prompts, and dimmed system output.
- Setup wizard chrome — progress chip (`Step 1 of N · ~15 min ·
  One-time · Reversible`), thin progress bar, and per-step number
  badges on each `.install-block` so the wizard reads as bounded
  instead of an open-ended scroll.
- Palette shift from blue to green/navy: `--hp-primary` aliases
  `#2ea877` (mint), `--hp-hero-bg` is navy `#0f1b3a`, code panels stay
  near-black `#0c1224` with warm-yellow `#ffd866` accents. The token
  alias is reused so downstream rules pick up the new accent
  automatically; instance theme overrides via
  `config.theme_overrides()` still win.
- VS Code surface tile carries a `Recommended` pill; the existing
  "Want to look around first?" section is renamed to `Explore your
  workspace` and gets the `#look-around` anchor.

All test-pinned class names and IDs (`install-hero`, `install-block`,
`home-mock`, `self-mark-btn`, `setupClaudeBtn`, `offboard-strip`,
`home-getting-started`, `home-gs-item`, `home-overview`,
`home-usage`) preserved as structural anchors; new visual language
overlays via additional classes. Existing onboarded/not-onboarded
branching, `/api/me/onboarded` POST, status frame gating, post-CTA
modal, and OS-tab switching JS unchanged. Stray `~/FoundryAI`
comment swapped for `~/{{ workspace_dir }}` to honor the
vendor-agnostic OSS rule.

51 home tests pass without modification.

* fix(web): /home palette inversion — dark intro hero on top, light setup card below

Previous reskin commit kept the install-hero as a dark navy gradient and
rendered the new intro hero as a light surface — opposite of what the CEO
mock specifies. Playwright comparison vs `data/ceo_home.html` confirmed:

- CEO mock: dark navy hero at TOP (with white pillars on navy), LIGHT
  white setup card BELOW with light step rows and dark code panels
  inset.
- Previous: light intro hero on top, dark setup card below. Inverted.

This patch flips both:

- `.home-hero-intro` now: dark navy gradient `#0f1b3a → #1a2a5f`, green
  radial glow in the corner, green eyebrow, white H1 (`accent` span
  green), rgba-white lede, green pill primary CTA, translucent-white
  secondary CTA, pillars row separated by hairline border-top with
  green square-dot bullets in front of each pillar header.
- `.install-hero` and `.install-block` now: white surface card with
  thin green accent strip across the top, light step rows split by
  hairline borders, green-tinted step-number circles (`#e6f9f0` bg,
  `#1f8a5e` ink), green progress chip + bar. Code panels
  (`.install-cmd`) and terminal frames stay dark — they're the "type
  this" surfaces.
- All previously-rgba-white descendants of `.install-hero`
  (close button, eyebrow, h1, lead, links, code chips, OS tabs,
  install notes, setup-CTA button, self-mark fallback, auto-detect
  badge, terminal-howto disclosure) re-skinned for light surface.

All 12 home page tests still pass (no markup changes, only CSS).

* fix(web): /home parity polish — system font + mock sizes + blue info hint + gray step-num

After v2 palette flip, user comparison vs CEO mock surfaced three
remaining gaps in the wizard area:

- Font stack mismatch: Agnes inherits Inter via `style-custom.css`,
  but the CEO mock uses the platform system stack (San Francisco on
  macOS, Segoe UI on Windows). The rendered weight/letterforms read
  noticeably different. `.home-mock` now declares
  `-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif`
  for itself and all descendants, with the monospace stack reserved
  for `code`/`kbd`/`pre`, `.install-cmd`, and `.terminal-body`.
- Step number badges were green-tinted; mock uses neutral gray
  (`#f0f2f6` bg, `#4a5168` ink) — green is reserved for the "done"
  state. Switched to `--hp-surface-dim` + `--hp-text-secondary`.
- "Don't have a terminal open?" disclosure was an amber/yellow
  variant left over from the old dark-hero palette. Mock uses a
  blue info-hint vocabulary (`--info-bg: #eef3ff`,
  `--info-line: #4f7cf2`, `--info-ink: #1c3994`) with white kbd
  chips. Added the info-* tokens to the `:root` block and re-skinned
  `details.terminal-howto` (incl. summary, body, kbd) to match.

Step-body type sizes also brought in line with the mock spec —
`.install-block .label` (step h3 equivalent) is now 17px / 700 with
6px gap; `.install-note` body type is 14px / 1.55.

`--hp-info-bg / --hp-info-ink / --hp-info-line / --hp-warn-bg /
--hp-warn-ink / --hp-warn-line / --hp-surface-dim` added as
first-class tokens so future hint/warn callouts pick the same colors
without a duplicate vocabulary.

12/12 home tests pass.

* feat(web): centralize design tokens + reword /home wizard to 6 steps (CEO mock parity)

Two intertwined changes that touch both global design + /home structure:

GLOBAL TOKEN SHIFT (app/web/static/style-custom.css)
- `--primary` flipped from blue `#0073D1` to green `#2ea877` — same brand
  alias the rest of the app referenced, so every page picks up the new
  accent automatically. Old `--primary-dark` / `--primary-light` recolored
  to match.
- New tokens added: `--brand-accent`, `--hero-bg`, `--hero-ink`,
  `--surface-dim`, `--info-bg/ink/line`, `--warn-bg/ink/line`. Brings
  the global vocabulary in line with the CEO mock's `:root` block so
  callouts and hero surfaces don't have to invent local tokens.
- `--font-primary` switched from Inter-led stack to the system stack
  (`-apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Inter",
  system-ui, sans-serif`) so weight/letterforms render identically on
  macOS (San Francisco) and Windows (Segoe UI) — matches the mock and
  avoids a font-loading flash for analysts without Inter installed.
- Shadow tints re-cast in navy `rgba(15,27,58,...)`; focus ring uses
  the new green `rgba(46,168,119,0.25)`.
- `.app-nav-link` font-size 13px → 14px, padding 6px 12px → 8px 14px,
  hover bg → `--primary-light` (mint), color → `--primary-dark`.
  `.app-nav-menu-item.is-active` re-tinted to the same green system.
- Sweep across 26 templates (style-custom.css + 25 template files)
  replacing every hardcoded `#0073D1` / `#005BA3` / `#E6F3FC` /
  `rgba(0,115,209,…)` / `rgba(0,86,163,…)` with token references or
  the new green hexes — 175 occurrences total. Pages that styled their
  own buttons / borders / shadows pick up the new brand color without
  per-page overrides.

/HOME WIZARD: 6 STEPS PER MOCK (app/web/templates/home_not_onboarded.html)
- Step 1 reworded `Install Claude Code on your computer` + `~3 min`
  subhead (mock copy).
- Step 2 renamed `Pick a folder for {{ instance_brand }}` (was
  `create your workspace folder`) — same `mkdir` command, mock-aligned
  framing.
- NEW Step 3 `Open a terminal inside that folder` — no shell command,
  just the "you are standing in the right directory" reassurance with
  a Finder/PowerShell/file-manager howto disclosure. Mirrors the CEO
  mock's Step 3.
- Step 4 (was Step 3, gated by `home_automode.show`) renamed
  `Launch Claude with auto-approve on`. Body copy lightly updated so
  it references "the next step" instead of "Step 4".
- Step 5 (was Step 4) renamed `Get the install script and paste it
  into Claude`. The setup-cta-lead now explicitly says
  "pasting the script into Claude Code will install {{ instance_brand
  }}…" so existing test assertions pinning the `install Agnes`
  substring still match.
- NEW Step 6 `Optional: create a one-word shortcut for next time` —
  prints an `echo 'alias {{workspace_dir|lower}}=…' >> ~/.zshrc`
  one-liner for Unix and an `Add-Content $PROFILE …` equivalent for
  Windows. OS tabs + copy buttons reuse the existing wizard chrome.
- Progress chip dynamic: `Step 1 of 6` when home_automode is on,
  `Step 1 of 5` when off. Progress bar fill `100 // total_steps` so
  the bar sits at 16-20 % on first paint.
- `.step-lede` token added for the new short body copy beneath each
  step label (14.5px / ink-soft).
- `macOS / Linux / WSL` tab labels changed to `macOS / Linux` per
  user instruction. Terminal-howto `WSL:` paragraph dropped; the
  paste-shortcut hint now reads `(Linux)` instead of `(Linux/WSL)`.
  Functional WSL handling in `connector_prompts.py` (it's a Linux
  detection fallback, not user-facing label) preserved.
- `setup_instructions.py` Claude Code install hint:
  `npm (Linux / WSL)` → `npm (Linux)`.

SURFACES — 4 CARDS PER MOCK
- Replaced the 3-tile `.home-usage-grid` with a 4-card grid:
  - VS Code (Recommended) — `.surface-card.feature`, green ring,
    DAILY USE eyebrow + 5-step numbered list + `Open VS Code setup
    guide →` link to `/setup-advanced#vscode`.
  - Terminal — QUICK ACCESS eyebrow + 4-step list.
  - Claude Code (Desktop app) — CONNECT IT eyebrow + 4-step list.
  - Cowork (claude.ai) — `.surface-card.incomplete`, warn-tinted
    border + `Instructions needed` badge + a TODO callout describing
    the missing content. The card is intentionally honest about the
    gap rather than hiding it.

TEST UPDATES
- `test_web_home_page.py` negative onboarded-state assertions
  rebased on the new step labels (6 entries instead of 4).
- `test_home_route_resolution.py` `test_home_renders_automode_block_by_default`
  + its `_when_env_off` counterpart now check the new
  `Step 4 — Launch Claude with auto-approve on` label.

* fix(web): /home section content + layout — verbatim mock match

User comparison flagged several remaining gaps; this patch rewrites
the three lower sections of /home to match the CEO mock spec exactly:

FIRST-SESSION (5 beats)
- h2 28px / 700 / -.5px tracking (was 19px / 600).
- lede 18px ink-soft (was 13.5px secondary).
- `.session-walk` wrapper, 36px gap between beats (mock spec).
- `.session-step` grid 48px / 1fr, gap 22px — number circle on
  the left, content on the right.
- `.session-num` 40 × 40 circle with SOLID GREEN bg (`--primary`)
  and WHITE text + soft green shadow (was 28px mint pill w/
  dark-green text).
- `.session-content h3` 18px / 600 (was 14.5px / 600).
- `.session-content > p` 15px.
- `.session-content .annotation` 13.5px ink-muted body type with
  `strong` for highlighting (replaces the upper-case "WHAT'S
  HAPPENING" eyebrow pattern that didn't match the mock).
- `.session-intro` callout card (white surface + mint icon block)
  framing the "five beats" tagline.
- `.session-tldr` summary box (brand-light bg + brand-dark left
  border) wrapping up the loop.
- Terminal frames re-skinned: `#0c1224` body / `#182241` bar /
  real macOS traffic-light colors `#ff5f57` / `#febc2e` / `#28c840`.
- Terminal body 13px / 1.65 line-height with mock-spec class
  vocabulary: `.you` (yellow input), `.ai-name` (brand bold),
  `.path` (light blue), `.dim` (translucent code-ink), `.caret`
  (blinking cursor).
- Five beats rewritten with mock's exact narrative flow (launch →
  menu → pick → ask → close), vendor-agnostic project names
  (`RevenueAnalysis`, `Onboarding`, etc.) replacing the customer-
  specific `GRPN_*` examples in the mock. Templated `{{
  instance_brand }}` / `{{ workspace_dir }}` / `{{ workspace_dir |
  lower }}` (the shortcut alias) everywhere.

SURFACES (4 cards)
- The section is no longer wrapped in a white rectangle; the
  `.home-usage` class loses its bg + border + padding (mock has the
  cards directly on the page bg).
- h2 28px (was 22px). Eyebrow 12px / 1.5px tracking / brand-dark.
- `.surface-card.feature` (VS Code) now uses 2px green border +
  vertical brand-light → white gradient (was 1px ring).
- `.surface-card.incomplete` (Cowork) uses 2px red border (`#e35e5e`)
  + vertical red-tint → white gradient (was yellow flat bg).
- `.surface-card .steps` panel: inner surface-dim bg + 8px radius
  + 13px font.
- `.surface-foot` top-border + ink-muted (mock spec).
- `.badge-warn` now a solid red box (`#e35e5e` bg + white ink + 4px
  radius) instead of a yellow pill, matching the mock.
- Header layout fixed: the global absorbed `header { display: flex;
  justify-content: space-between }` rule was making the h2 sit on
  the right of the eyebrow; explicit `display: block` override on
  `.home-mock section > header` puts the title on the LEFT under
  the eyebrow as the mock has.

BROWSE — Explore your workspace
- Wrapped in `<section class="browse-section">` with proper
  eyebrow + h2 + lede (was a bare `.section-label` div).
- `.browse-grid` 5-col grid (was responsive auto-fill, 4-card
  layout). Skills tile added as a 5th card linking to
  `/marketplace?type=skills`.
- `.browse-card` mock-spec: 22 20 padding, 28px icon, 15px title,
  12.5px ink-muted desc, hover lifts -2px with brand border +
  shadow-md.

Section wrappers (`.home-usage`, `.first-session`) no longer carry
the white card chrome — they sit directly on the page bg, matching
the mock. Only Getting Started + Overview keep their white cards.

GLOBAL eyebrow vocabulary (`.home-hero-intro .eyebrow`,
`.first-session > .eyebrow`, `.surfaces > header .eyebrow`,
`.browse-section .eyebrow`) all aligned to mock spec: 12px / 700 /
1.5px tracking / brand-dark color / 14px bottom margin.

Hero h1 bumped to 44px / 800 / -1px tracking (was 32px / 600).

51/51 home tests pass.

* fix(web): /home session-intro card + terminal-body verbatim mock match

User comparison flagged three remaining /home gaps; this patch
addresses each:

- `.session-intro` rule was missing — the "five beats" tagline
  rendered as a bare line with no card chrome. Added the mock-
  spec card: white surface, 14px radius, 20×24 padding, 1px
  border + shadow-sm, with a 44×44 brand-light icon block on the
  left.

- Beat 1 terminal-title was `~/{{ workspace_dir }} — zsh` (mock-
  style shell-pwd format), but the user wants every terminal
  frame across all 5 beats to read `claude — {{ instance_brand }}`.
  Updated.

- Terminal-body line structure for beats 2-5 rewritten verbatim
  from the CEO mock:
  - `<span class="prompt">&gt;</span><span class="you">…</span>`
    now has no space between the prompt and user input (mock
    pattern: zero gap, the .prompt's `margin-right: 8px` provides
    the visual separation).
  - Beat 2 menu items use `<strong>[N]</strong>` numbering with
    project entries on indented lines, each project name followed
    by a `<span class="dim">(N ago)</span>` timestamp at a fixed
    column — instead of my prior single-line concatenation.
  - Beat 3 narrative split into 4 stanzas separated by blank lines
    (matches mock): the "Switched to <strong>X</strong>" status,
    then dim Loaded/Last-session lines, then a stand-alone "One
    unprocessed input detected:" pair, then the "Want me to
    process …" question. My prior version dim-wrapped the entire
    block, which looked off.
  - Beat 4 narrative split into headline summary + risks section
    with <strong> heads + bullet lists separated by blank lines,
    matching the mock's "Q1 close summary" / "Open risks" rhythm.
    The Q1 question carries the mock's manual line-break + 2-
    space continuation indent inside the `.you` span — without
    that, terminal-body's `white-space: pre-wrap` would auto-wrap
    awkwardly at a different column than the mock.
  - Beat 5 exit narrative uses two separate dim lines + a
    standalone `.ai-name` "See you next time." line, then prompt
    + caret. My prior version collapsed everything into one dim
    block.
  - Project names changed from customer-specific (`GRPN_*`) to
    generic (RevenueAnalysis, WeeklyReview, Onboarding, OpsDb,
    HRHandShake) so the OSS distribution stays vendor-agnostic
    per CLAUDE.md.
  - `Marketing plan` examples replaced with `Q1 close` so the
    narrative stays plausible for an analyst audience.

12/12 home tests pass.

* fix(web): /home surfaces verbatim mock — VS Code thumb, Terminal expected-output, NEW badge

User comparison flagged three remaining surface-section gaps:

- VS Code surface card was rendering a generic "Screenshot pending"
  placeholder; the mock has a labeled inline mockup
  (`<a class="vscode-thumb">` w/ `.thumb-fallback`) showing the
  recommended 4-pane layout (EXPLORER yellow, TERMINAL 1 purple,
  TERMINAL 2 green, TERMINAL 3 orange) on a dark navy bg + a
  "Recommended layout" caption pill. CSS `.vscode-thumb` block
  added — uses gradient-strip backgrounds to draw the colored
  panel bars without needing a base64 image.

- "Recommended" badge was a pill (999px radius) with
  `--brand-accent` bg + navy text. Mock uses `.badge` instead of
  `.recommend-pill` — solid `--primary` (brand-dark green) bg
  with WHITE text and 4px radius. Replaced the class + CSS rule
  so the badge reads as a tag, not a pill.

- Terminal surface card was missing the "What you should see"
  subsection — mock has an `.expected-output` block showing a
  sample of the welcome menu inside a dim dashed panel. Added the
  block with the mock's exact rendered output (templated to
  `{{ instance_brand }}` + generic project names instead of
  customer-specific GRPN entries) plus the `.expected-output`
  CSS (surface-dim bg + dashed border + `::before` "WHAT YOU
  SHOULD SEE" eyebrow per mock spec).

Also addressed the explore-section feedback:

- Skills browse-card now carries the `new` class so it picks up
  the `.browse-card.new::after` corner badge ("NEW", green bg,
  white text, 10px / 700 / 0.5px tracking) per mock.
- Browse cards align same height via `align-self: stretch` (grid
  default) + `flex-grow: 1` on `.browse-desc` so descriptions
  fill remaining vertical space; previously the Skills tile sat
  shorter because its desc text was longer than others'.

Structural HTML changes to all four surface cards: dropped the
inner `<div class="surface-card-head">` wrapper + `<p
class="surface-pitch">` class in favor of mock's flat layout
(`.what` + `.steps` + `.when-to-use`). `<ol class="surface-steps">`
replaced with `<div class="steps"><strong
class="steps-eyebrow">DAILY USE / QUICK ACCESS / CONNECT IT</strong>
<ol>...</ol></div>` so the eyebrow + numbered list share the
mock's tinted surface-dim panel.

12/12 home tests pass.

* fix(web): align /home setup walkthrough to design spec

- Setup-section header (eyebrow + heading + lede) floats above the
  install hero; install card has no accent strip; step labels drop
  `Step N —` prefix; closing strip is single flex row.
- VS Code surface card renders recommended-layout screenshot from
  `/static/img/vscode-layout.png` with click-to-enlarge lightbox.
- Workspace install path cascades to `~/Desktop/{workspace_dir}` in
  every step, surface card, first-session annotation, and shortcut.
- Step 1 verify text restores Enterprise — Finance and Legal option.
- Step 6 shortcut installs a shell function with arg forwarding
  (`"$@"` unix / `@args` windows) and a user-facing Auto / YOLO
  permission-mode toggle.
- Step 5 manual-fallback details inline on the CTA row; description
  reads at step-lede size, not 13px chip.
- Setup-section heading no longer right-aligns (was inheriting
  `header { display: flex; justify-content: space-between }` from
  the legacy stylesheet; wrapper changed to `<div>`).
- Getting Started `<details>` block removed (duplicated links).

* test(web): align /home tests with restructured setup wizard

- Replace test_getting_started_card_renders_on_home with
  test_setup_section_renders_for_not_onboarded — asserts the new
  setup-section-header floats above the install hero and Getting
  Started markup is absent (block removed in the prior commit).
- Update automode-block test to match labels without the
  `Step N —` prefix.
- Update setup-CTA partial test to match the relabeled
  "Copy install script to clipboard" button.

Drop orphaned CSS for `.home-getting-started`, `.home-gs-summary*`,
and `.home-gs-item` — selectors had no matching markup after the
Getting Started block was removed.

Also: Step 3 `pwd` expected-output uses an absolute path
(`/Users/yourname/Desktop/{workspace_dir}`) instead of the
tilde-prefixed form, matching what the command actually prints.

* fix(web): repaint home_onboarded + setup_advanced; align CTA label

- home_onboarded + setup_advanced still carried the retired blue
  `#0056A3` as both `--hp-primary-dark` and the hero gradient
  endpoint. Both reference `var(--primary-dark)` now so the green
  palette cascades.
- setup_advanced YOLO snippet was the old `alias` form (no cd, no
  arg forwarding). Replaced with the shell function variant from
  /home Step 6 — drops into ~/Desktop/{workspace_dir} and forwards
  "\$@" (unix) / @args (Windows).
- setup_advanced ~/{workspace_dir} path references cascaded to
  ~/Desktop/{workspace_dir} so install story matches /home.
- Dashboard's "Setup a new Claude Code" button label aligned to the
  canonical "Copy install script to clipboard" — matches /home and
  the new docstring in _claude_setup_cta.jinja, which now mandates
  this label across consumers.

* fix(web): keep base brand blue; scope green palette to /home redesign

User noticed login + dashboard had turned green when the /home
redesign flipped --primary from blue (#0073D1) to green (#2ea877)
in commit 278f202e. The brand-wide flip went further than the
redesign needed — only /home, /home (onboarded), and /setup-advanced
intentionally use the green/navy spec; every other page (login,
dashboard, catalog, marketplace, admin, profile) was just inheriting
the green because --primary cascaded everywhere.

Revert the global brand colour to blue and lock the green into the
two outstanding redesign scopes:

- style-custom.css: --primary back to #0073D1, --primary-light back
  to rgba(0,115,209,0.1), --primary-dark back to #005BA3,
  --brand-accent back to a lighter blue.
- home_onboarded.html: .home-mock now sets --hp-primary,
  --hp-primary-dark, --hp-primary-light to explicit green hex
  (matching home_not_onboarded), so the hero stays green regardless
  of the global brand.
- setup_advanced.html: same lock — .advanced-mock pins the green
  palette in-scope.

Hero gradients on both pages now reference the local --hp-primary
chain (not the global --primary), so any future palette tweak inside
either scope cascades correctly without disturbing the rest of the app.

* refactor(web): hoist --hp-* into shared design-tokens.css (--ds-*)

PR 2 of the design-system extraction ladder. Pure mechanical rename
+ dedup; no visual diff on any rendered page (verified on /home,
/dashboard).

- New app/web/static/css/design-tokens.css declares the full token
  set on :root: brand surface (green primary, primary-dark, mint
  light, brand-accent), hero (navy bg + ink), code-panel (near-black
  bg + cool ink + warm-yellow), light surfaces (bg/surface/border),
  text (primary/secondary/muted), orange accent, info + warn
  callout vocabularies, navy-tinted elevation shadows, system font
  stack + mono.
- base.html loads it alongside style-custom.css so the tokens are
  globally available.
- Rename --hp-* -> --ds-* in home_not_onboarded (313 refs),
  home_onboarded (15), setup_advanced (39). 367 token references
  pointed at one of three local blocks; now all point at the
  global :root.
- Drop the three local token blocks. Each scope class
  (.home-mock / .advanced-mock) only keeps its base ink + font-size
  + line-height rules.

The legacy --primary family stays canonical for the blue base
brand — login, dashboard, catalog, marketplace, admin still read
blue. The design system is opt-in via the scope class.

* refactor(web): extract shared components.css; migrate /home markup

PR 3 of the design-system extraction ladder. First batch of
reusable components lifted out of home_not_onboarded.html into a
new shared stylesheet; markup migrated to consume them.

- New app/web/static/css/components.css with five components, all
  reusable on any page that loads design-tokens.css:
    .callout-rec        — amber lightbulb recommendation box
    .callout-hint       — blue info hint box
    .code-output        — "WHAT YOU SHOULD SEE" terminal output block
    .lightbox           — full-bleed image enlarge overlay
    .setup-section-header — wizard header (eyebrow + h2 + lede)
- base.html loads components.css after design-tokens.css.
- home_not_onboarded.html markup renamed:
    class="rec"             -> class="callout-rec"
    class="hint"            -> class="callout-hint"
    class="expected-output" -> class="code-output"
- Local CSS rules removed from home_not_onboarded.html for each of
  the extracted components — ~150 lines down to 5-line "extracted to
  components.css" comments. The bespoke wizard-specific styles
  (.install-cmd, .os-tabs, .mode-tabs, .terminal-frame) stay
  template-local for now since they only have one consumer.

Visual regression check: /home install hero renders the amber rec
callout, blue hint callout, dashed code-output block, green section
header, and click-to-enlarge VS Code thumb identically to the
pre-extraction render. 43 home tests pass.

* fix(web): unify page-headers — activity-center full-width, marketplace shares box

- /activity-center audit-log hero rendered as half-width because the
  _page_hero include was inside <header class="obs-topbar">, a flex
  row that pinned the time-range + auto-refresh controls next to it.
  The hero is now a sibling rendered before the <header>, so it
  spans the full container width like every other admin page; the
  controls keep their flex row underneath.
- Marketplace hero unified with .page-header--hero. Markup is now
  <section class="page-header page-header--hero mp-hero"> so the
  shared box drives padding/radius/gradient/max-width/shadow; the
  .mp-hero override block only carries the right-anchored cover
  image and the rules for the search row + scope checkboxes (which
  the canonical hero doesn't have). Inner text uses the canonical
  .page-header__eyebrow / __title / __subtitle classes.
- .page-header--hero shadow tint now follows the brand blue
  (rgba(0, 115, 209, 0.2)) instead of the leftover green from the
  prior palette flip; same depth highlight everywhere the gradient
  is blue.

* fix(web): unify remaining page heroes — admin, profile, install, store, stack

Sweep across pages that carried bespoke gradient hero markup so
every page-hero shares the canonical `.page-header--hero`
dimensions (padding 28/32/24, border-radius 14, max-width
var(--width-app), navy-tinted shadow, gradient with --primary →
--primary-dark). Inner text uses the .page-header__eyebrow /
__title / __subtitle classes so typography matches across the app.

- admin_tables: migrated to _page_hero.html include.
- admin_tokens: kept .tokens-hero wrapper for the counts-chip row
  but added the canonical class on the same element; stripped
  duplicate gradient + padding + typography rules.
- install: same pattern (kept hero-meta pill row).
- profile: migrated to _page_hero.html include.
- store_upload: kept .upload-hero wrapper for the .meta chip row;
  composite class with the canonical hero.
- setup_advanced: .advanced-mock .ad-hero now matches canonical
  dimensions; green palette retained via --ds-primary/dark.
- stack_card.css: .stack-hero (catalog + corporate-memory search
  hero) uses canonical gradient + padding + max-width.

The detail-page heroes (marketplace_plugin_detail,
marketplace_item_detail, catalog_*_detail, store_edit,
admin_group_detail, admin_store_submission_detail) stay bespoke
for now — they're rich detail headers with photos, badges, install
actions; converting them would lose contract context. Same applies
to dashboard.html env-setup-cta (it's a CTA card, not a page hero).

* fix(web): canonicalise .container — single page shell every page inherits

Previously each admin page set its own `.container:has(.<page>)
{max-width: none}` + `.<page>-page {max-width: 1400px}` override,
and per-page hero markup either nested inside flex toolbars (which
pinned the hero next to filter controls and squeezed it half-width)
or self-constrained with a different max-width than the page. /home,
/dashboard, /marketplace, and /admin/* ended up at different widths
with different nav-to-hero gaps.

- style-custom.css `.container` now carries the canonical 1280px
  max-width + `16px 32px 48px` padding so every page inherits the
  same nav-to-hero gap and side gutters. `.container > main` is
  margin/padding 0 so the container is the sole owner of gutters.
- `.page-header--hero` drops its self-constraining max-width and
  auto-centering margin — the container provides the width, so the
  hero sits flush with the table/toolbar below it.
- `.stack-hero` (catalog + corporate-memory) and `.advanced-mock
  .ad-hero` (/setup-advanced) follow the same pattern: container
  owns the width.
- Per-page max-width overrides stripped from admin_users,
  admin_access, admin_groups, admin_marketplaces, admin_welcome,
  admin_workspace_prompt.
- _page_hero include extracted from inside flex toolbars on
  admin_users, admin_access, admin_groups, admin_marketplaces,
  admin_server_config, admin_welcome, admin_workspace_prompt,
  admin_sessions, admin_session_detail, admin_usage,
  activity_center. The toolbar (`.users-toolbar`, `.gp-toolbar`,
  etc.) keeps only the filter + action controls; hero renders
  before it as a sibling.
- _page_chrome.html trimmed to just the page-background tint for
  the redesign scopes; the duplicate `.container` rules it carried
  are now redundant.

Verified: /home, /admin/marketplaces, /admin/users all render
container width 1280px with hero top at 88px (16px below the
72px-tall sticky nav). Same spacing as /home design spec.

* fix(web): admin_tables + admin_corporate_memory inherit canonical .container

Both pages were overriding `{% block layout %}` from base.html,
which bypasses the canonical `.container` wrapper. Result: hero
span the full viewport (1596px on a wide screen) while the inner
content sat at a narrower max-width — hero and content didn't
align, and the nav-to-hero gap differed from every other admin
page.

Switched both templates to `{% block content %}` so they render
inside the canonical `.container` from base.html — same path as
admin_groups, admin_users, admin_marketplaces, etc.

- admin_tables: dropped local `.page-title { max-width: 1600px }`
  + `.content { max-width: 1600px }` overrides (kept typography +
  inner gutter rules) and the mobile padding overrides that paired
  with them. Container now owns the gutters.
- admin_corporate_memory: only the block keyword needed changing;
  the template already had a clean inner structure (no max-width
  override on `.container-memory`).

Verified on /admin/tables and /admin/corporate-memory:
- .container width 1280, padding 16/32/48
- Hero top 88 (nav 72 + container padding-top 16)
- Hero + content both 1216px wide, both at left 190 — perfect
  alignment with /admin/groups.

* fix(web): drop .page-shell padding override + admin_tables stale :root

Two regressions discovered after the canonical-container unification:

1. `.container:has(.page-shell)` still set `padding: 28px 32px 48px`
   while the canonical `.container` had moved to `16px 32px 48px`.
   Every page-shell consumer (/admin/sessions, /admin/sessions/<id>,
   /admin/usage, /marketplace, /dashboard, marketplace detail pages,
   /me/activity, /store/*, /admin/store-submissions) was rendering
   with a 28px nav-to-hero gap while /admin/users + /admin/groups
   rendered with 16px. Same width, mismatched vertical rhythm.
   The opt-in rule is now a no-op marker: canonical container
   already provides 1280px + 16/32/48 + main margin/padding 0.

2. admin_tables.html had a stale `<style>` block that re-declared
   `:root { --primary: var(--primary); ... }`. The self-referential
   token resolved to empty, collapsing the page-header hero's
   `linear-gradient(135deg, var(--primary), var(--primary-dark))`
   to no background — the hero appeared as a pale ghost without
   colour. The entire shadow `:root` block was a stale copy of the
   design tokens that style-custom.css already provides. Dropped
   it; tokens now resolve from the global `:root`.

After both fixes /admin/sessions, /admin/tables, and every other
page-shell consumer match /admin/groups exactly: container 1280px,
container padding-top 16px, hero at top 88px / left 190px / width
1216px.

* fix(web): drop /admin/tokens .tokens-page width + padding override

`.tokens-page` carried its own `max-width: 1280px; margin: 0 auto;
padding: 28px 8px 48px` block — the canonical `.container` already
provides width + 16/32/48 padding, so the nested wrapper was
adding 28px on top of the container's 16px (= 44px nav-to-hero
gap, vs 16px on every other admin page) and shrinking the hero
sideways by 8px on each side (1200px vs the canonical 1216px).

After: container owns the layout; `.tokens-page` is just a
font-family scope. /admin/tokens hero now sits at top 88, left 190,
width 1216 — same numbers as /admin/groups / /admin/users.

* fix(web): hero links readable on blue; /admin/access Groups link href

- New `.page-header--hero a` rule in style-custom.css forces any
  anchor inside a gradient hero to render white + underlined so
  links stay readable on the blue background. Previously links
  inherited the global `var(--primary)` blue, which disappeared
  on top of the matching blue gradient. No per-page class needed —
  drop a plain `<a>` in any hero subtitle and it just works.
- /admin/access hero subtitle was Jinja-passing the inline link
  with HTML-entity-encoded quotes (`href=&quot;...&quot;`). The
  entities decoded to literal `"` characters inside the rendered
  href, producing `/admin/%22/admin/groups%22` — a 404. Switched
  the `set` to a block-set (`{% set page_hero_subtitle %}...{% endset %}`)
  so the inline `<a href="/admin/groups">Groups</a>` survives
  unescaped through `_page_hero.html`. Also stripped the now-redundant
  inline `style="color:#fff;text-decoration:underline;"` — the new
  shared rule handles it.

* fix(web): /dashboard top padding matches every other page

`.main` on /dashboard had `padding: 28px 32px 48px` while every
other page now uses `16px 32px 48px` via the canonical
`.container`. Dashboard bypasses `.container` (overrides
base.html's `layout` block to render a full-width `<main>`
directly), so the padding lives on `.main` itself — bumped the
top to 16px to match.

After: first child top = 88, left = 190, width = 1216 — same
numbers as /admin/groups / /admin/users / /admin/marketplaces.

* fix(web): green eyebrow + white title on .page-header--hero (matches /home)

`.page-header--hero .page-header__eyebrow` was faint white
(rgba(255,255,255,0.75)) — readable but unbranded against the blue
gradient. Changed to `var(--ds-brand-accent)` (mint green #54d3a0)
so every page hero pairs a green eyebrow with white title +
subtitle, echoing /home's setup-section header (green eyebrow,
dark heading combo). One CSS rule applies everywhere — no
per-page styling needed.

Also bumped the eyebrow to font-weight 700 / letter-spacing 1.2px
so the green stands out cleanly against the gradient.

* fix(web): page-header--hero + stack-hero use /home navy gradient

`.page-header--hero` and `.stack-hero` were on the brand-blue
gradient (`var(--primary)` → `var(--primary-dark)`) while
/home's hero (`.home-hero-intro`) sits on the deeper navy
gradient (`#0f1b3a` → `#1a2a5f`). Every other page-hero now
uses that same navy gradient so /home, /marketplace, /catalog,
/corporate-memory, /admin/*, /profile, /install, /dashboard,
/setup-advanced share one brand surface. Shadow tint adjusted
to the navy depth (rgba(15, 27, 58, 0.22)).

Brand blue stays the link/CTA colour everywhere else; only the
hero box itself is navy.

* fix(web): primary buttons green; marketplace tabs navy translucent

Two parity tweaks pulling the rest of the app toward /home's
visual language.

- `.btn-primary` (both rules in style-custom.css) now uses
  `var(--ds-primary)` / `var(--ds-primary-dark)` green fill,
  matching the "Copy install script to clipboard" button on
  /home. Brand-blue `--primary` still drives link colour and the
  accent surface; only the filled button background flipped to
  green. Every page with a `.btn-primary` (admin "+Add user",
  "+Add marketplace", catalog, marketplace actions, dashboard,
  modals) now reads as the same "do it" affordance.
- `.mp-tabs` (Curated Marketplace / Flea Market / My Stack tab
  group) now sits on the navy `--ds-hero-bg` with translucent
  white pills (rgba(255,255,255,0.10) inactive, 0.18 active) —
  same translucent-white-on-navy treatment as the "Just browse —
  no install needed" pill on /home. Icons render as soft white;
  per-tab colour-coding dropped in favour of the unified surface.

* fix(web): catalog/memory tabs + empty-state CTA + admin action buttons

Bring /catalog and /memory in line with /home + /marketplace:

- `.stack-tabs` (Browse / My Stack / Recipes on /catalog,
  Browse / My Stack on /memory) now uses the navy `--ds-hero-bg`
  container with translucent-white-on-navy pills, mirroring the
  `.mp-tabs` treatment and /home's "Just browse — no install
  needed" CTA pill. Per-tab icon colour-coding dropped — icons
  render as soft white on the navy fill.
- `.stack-tabs-row__actions .btn` (right-slot "+New Recipe",
  "+New Data Package" admin CTAs) now uses green primary fill
  (`--ds-primary`), matching `.btn-primary` and /home's
  "Copy install script to clipboard" button.
- `.stack-empty .cta a` (empty-state action button — the
  "Open /admin/tables →" CTA on /catalog and equivalent on
  /memory) flipped from blue `--primary` to green `--ds-primary`
  so the colour aligns with every other primary button in the app.

* fix(web): marketplace Search button green (--ds-primary) matching other CTAs

* fix(web): unify Search button + admin-action button across browse pages

- Added Search button (`<button class="stack-hero__search-btn">`)
  to /catalog and /memory heroes — same green pill as /marketplace.
  Wired to the existing live-filter pipeline (button click runs
  `applyFilters()` and refocuses the input). All three browse pages
  now wear the identical search bar UI.
- `.stack-hero__search-btn` shares `--ds-primary` fill with
  `.mp-hero .search-btn`.
- `.mp-actions .btn` ("Submit a skill or plugin" CTA on /marketplace)
  flipped from the legacy blue-outline to the same green primary
  fill + dimensions (`display: inline-flex; line-height: 1;
  padding: 9px 16px; gap: 6px`) as `.stack-tabs-row__actions .btn`
  on /catalog and /memory. All three right-slot action buttons
  render at identical height now.
- `.stack-tabs-row__actions .btn` got `inline-flex` + `line-height: 1`
  + `gap: 6px` so a `<button class="btn">` and a `<a class="btn">`
  both render at exactly 33px high — the embedded
  `.admin-only-hint` chip no longer pushes one variant taller
  than the other.

* fix(web): marketplace guide CTAs green (fastpath + primary); drop flea purple

* fix(web): dashboard CTA hero on navy; readable <code> chips in hero

- `.env-setup-cta` on /dashboard ("Set up a new Claude Code"
  card) flipped from the brand-blue gradient + green-tinted shadow
  to the canonical navy gradient (`--ds-hero-bg` → `#1a2a5f`) with
  navy-tinted shadow + 14px radius + 28/32/24 padding, matching
  `.page-header--hero` and /home's `.home-hero-intro`. Dashboard's
  top CTA now sits on the same brand surface as every other hero.
- Added `.page-header--hero code` rule — translucent white pill +
  warm-yellow ink (#ffd866) so `<code>` chips embedded in hero
  subtitles read as code samples against the navy gradient. The
  global `code` rule sets `color: var(--text-primary)` (dark),
  which turned in-hero chips into invisible dark-on-white-on-navy
  ghosts (e.g. the `-by-dev` suffix on /store/new).
- /store/new's `.page-header__subtitle code` dropped its inline
  style override — the shared rule handles it now.

* feat(web): two-theme switching via data-theme + admin toggle

Introduces a theme system that flips the entire UI palette between
"navy" (current design, default) and "blue" (pre-redesign palette)
via a single `<html data-theme="...">` attribute. Page markup, class
names, and component styles don't change — only the `--ds-*` token
values flip.

Backend
- New `app/instance_config.py::get_instance_theme()` resolves the
  active theme from `AGNES_INSTANCE_THEME` env > `instance.theme`
  in instance.yaml > default "navy". Unrecognised values clamp to
  "navy" so a typo doesn't break the page.
- `app/web/router.py::_build_context` injects `instance_theme`
  alongside `instance_brand` etc. so every template inherits it.
- `app/web/templates/base.html` renders
  `<html lang="en" data-theme="{{ instance_theme | default('navy') }}">`.

CSS
- `app/web/static/css/design-tokens.css` adds two new tokens to
  the default `:root` set: `--ds-hero-shadow` (drop-shadow tint
  on hero boxes) and `--ds-hero-eyebrow` (eyebrow accent colour).
  Plus a `:root[data-theme="blue"]` override block that flips
  seven tokens: `--ds-primary`, `--ds-primary-dark`,
  `--ds-primary-light`, `--ds-brand-accent`, `--ds-hero-bg`,
  `--ds-hero-bg-deep`, `--ds-hero-shadow`, `--ds-hero-eyebrow`.
  The blue theme aliases the brand surface tokens back to the
  legacy `--primary` family.
- `.page-header--hero`, `.stack-hero`, `.env-setup-cta`,
  `.home-mock .home-hero-intro` now reference the new
  `--ds-hero-shadow` and `--ds-hero-bg-deep` tokens instead of
  hard-coding `rgba(15, 27, 58, 0.22)` and `#1a2a5f` — gradient +
  shadow now flip with the theme.
- `.page-header--hero .page-header__eyebrow` uses
  `var(--ds-hero-eyebrow)` so the eyebrow goes mint-green on
  navy and translucent-white on blue (mint on blue reads poorly).

Admin
- `app/api/admin.py::_KNOWN_FIELDS["instance"]` now registers a
  `theme` field of kind `select` with options `["navy", "blue"]`
  and a `hint` explaining the trade-off. The existing
  /admin/server-config UI auto-renders a select for this — no
  template changes needed.

Defaults
- Default value is "navy" so existing instances see no visual
  change. Admins flip to "blue" via /admin/server-config to
  restore the pre-redesign look.

Restart note: uvicorn must reload to pick up the Python changes
(new getter, new template-context key, new known-field). CSS
changes hot-reload via browser refresh.

* fix(web): blue theme — home hero eyebrow + CTA contrast

`.home-hero-intro .eyebrow` and `.btn-intro-primary` referenced
`--ds-brand-accent` directly, which on the blue theme resolves to
the lighter brand-accent blue (#4F9DEB). Result: light-blue eyebrow
on the blue gradient ("WELCOME, ADMIN" barely readable) and a
light-blue button with darker-blue text ("Set up in ~15 min")
that all sat in the same hue range.

Introduces three new theme-aware tokens:
- `--ds-hero-eyebrow` already existed; blue theme bumped opacity
  to 0.92 so the eyebrow reads as full white.
- `--ds-hero-cta-bg` + `--ds-hero-cta-fg` + `--ds-hero-cta-bg-hover`
  flip the primary hero CTA: mint-green on navy (default), white-
  on-blue under `data-theme="blue"`.

`.home-hero-intro .eyebrow` now uses `--ds-hero-eyebrow` (mint on
navy / white on blue) and `.btn-intro-primary` uses the CTA token
trio.

Recommended palette on blue theme:
- Eyebrow: white at 92% opacity (clear on the blue gradient).
- Primary CTA pill: white background, brand-blue dark text
  (`--primary-dark` = #005BA3) for AAA-level contrast.
- Secondary CTA: translucent white pill (unchanged).

* fix(web): blue theme — callout-hint info bg/border/ink re-tinted to brand blue (was indigo, clashed with brand-blue hero)
2026-05-21 06:19:16 +00:00

5748 lines
282 KiB
HTML
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{% extends "base.html" %}
{% block title %}Table Management - {{ config.INSTANCE_NAME }}{% endblock %}
{% block body_attrs %}data-source-type="{{ data_source_type }}"{% endblock %}
{% block head_extra %}
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<style>
/* Design tokens come from the global `:root` block in
style-custom.css. The local override that used to live
here re-declared `--primary` as `var(--primary)` (a
circular reference that resolved to empty), which
collapsed the gradient on the page-header hero. */
body {
font-family: var(--font-primary);
font-size: 14px;
color: var(--text-primary);
background: var(--background);
line-height: 1.5;
}
/* ── Header (dashboard-style) ── */
.header {
background: var(--surface);
border-bottom: 1px solid var(--border);
padding: 0 32px;
height: 72px;
display: flex;
align-items: center;
justify-content: space-between;
position: sticky;
top: 0;
z-index: 100;
}
.header-left {
display: flex;
align-items: center;
gap: 16px;
}
.header-back {
display: flex;
align-items: center;
justify-content: center;
width: 32px;
height: 32px;
border-radius: 6px;
color: var(--text-secondary);
text-decoration: none;
transition: all 0.15s ease;
}
.header-back:hover {
background: var(--border-light);
color: var(--text-primary);
}
.header-logo-group {
display: flex;
flex-direction: column;
justify-content: center;
gap: 2px;
}
.header-logo svg {
display: block;
}
.header-subtitle {
font-size: 11px;
font-weight: 500;
color: var(--text-secondary);
letter-spacing: 0.4px;
text-transform: uppercase;
margin-top: 2px;
}
.header-right {
font-size: 12px;
color: var(--text-secondary);
}
/* ── Page Title ── */
.page-title h1 {
font-size: 24px;
font-weight: 600;
color: var(--text-primary);
margin-bottom: 4px;
}
.page-title p {
font-size: 14px;
color: var(--text-secondary);
}
/* ── Content Layout ── */
.content {
display: flex;
flex-direction: column;
gap: 24px;
}
/* ── Panel (shared card style) ── */
.panel {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 12px;
overflow: hidden;
box-shadow: var(--shadow-sm);
}
.panel-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 20px 24px;
border-bottom: 1px solid var(--border-light);
}
.panel-header-left {
display: flex;
align-items: center;
gap: 12px;
}
.panel-header-icon {
width: 36px;
height: 36px;
border-radius: 8px;
display: flex;
align-items: center;
justify-content: center;
flex-shrink: 0;
background: var(--primary-light);
}
.panel-title {
font-size: 16px;
font-weight: 600;
color: var(--text-primary);
}
.panel-subtitle {
font-size: 13px;
color: var(--text-secondary);
margin-top: 1px;
}
.panel-body {
padding: 20px 24px;
}
.panel-body-empty {
padding: 40px 24px;
text-align: center;
color: var(--text-secondary);
font-size: 13px;
}
/* ── Buttons ── */
.btn {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 8px 16px;
border: none;
border-radius: 8px;
font-family: var(--font-primary);
font-size: 13px;
font-weight: 500;
cursor: pointer;
transition: all 0.15s ease;
white-space: nowrap;
}
.btn-primary {
background: var(--primary);
color: white;
}
.btn-primary:hover {
background: #005FA8;
}
.btn-primary:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.btn-secondary {
background: var(--border-light);
color: var(--text-primary);
}
.btn-secondary:hover {
background: var(--border);
}
.btn-danger {
background: var(--error-light);
color: var(--error);
}
.btn-danger:hover {
background: rgba(234, 88, 12, 0.2);
}
.btn-sm {
padding: 5px 10px;
font-size: 12px;
border-radius: 6px;
}
.btn-icon {
width: 28px;
height: 28px;
padding: 0;
display: flex;
align-items: center;
justify-content: center;
border-radius: 6px;
background: none;
border: none;
cursor: pointer;
color: var(--text-secondary);
transition: all 0.15s ease;
}
.btn-icon:hover {
background: var(--border-light);
color: var(--text-primary);
}
.btn-icon.danger:hover {
background: var(--error-light);
color: var(--error);
}
/* ── Badges ── */
.badge {
flex-shrink: 0;
font-size: 11px;
font-weight: 500;
border-radius: 6px;
padding: 3px 8px;
white-space: nowrap;
}
.badge-registered {
color: #065F46;
background: #D1FAE5;
}
.badge-available {
color: var(--primary);
background: var(--primary-light);
}
/* ── Spinner ── */
.spinner {
display: inline-block;
width: 18px;
height: 18px;
border: 2px solid var(--border);
border-top-color: var(--primary);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to { transform: rotate(360deg); }
}
.spinner-lg {
width: 32px;
height: 32px;
border-width: 3px;
}
/* ── Loading state ── */
.loading-state {
display: flex;
flex-direction: column;
align-items: center;
gap: 12px;
padding: 40px 24px;
color: var(--text-secondary);
font-size: 13px;
}
/* ── Notification toast ── */
.toast {
position: fixed;
top: 84px;
right: 24px;
z-index: 200;
background: var(--surface);
border: 1px solid var(--border);
border-radius: 10px;
box-shadow: var(--shadow-md);
padding: 12px 16px;
display: flex;
align-items: center;
gap: 10px;
font-size: 13px;
transform: translateX(120%);
transition: transform 0.3s ease;
max-width: 360px;
}
.toast.visible {
transform: translateX(0);
}
.toast-success {
border-left: 3px solid var(--success);
}
.toast-error {
border-left: 3px solid var(--error);
}
.toast-icon {
width: 20px;
height: 20px;
flex-shrink: 0;
}
/* ── Bucket accordion ── */
.bucket-group {
border-top: 1px solid var(--border-light);
}
.bucket-group:first-child {
border-top: none;
}
.bucket-trigger {
width: 100%;
display: flex;
align-items: center;
gap: 10px;
padding: 12px 24px;
background: none;
border: none;
cursor: pointer;
font-family: var(--font-primary);
font-size: 14px;
font-weight: 500;
color: var(--text-primary);
text-align: left;
transition: background 0.1s ease;
}
.bucket-trigger:hover {
background: var(--border-light);
}
.bucket-chevron {
width: 16px;
height: 16px;
color: var(--text-secondary);
transition: transform 0.2s ease;
flex-shrink: 0;
}
.bucket-trigger.expanded .bucket-chevron {
transform: rotate(90deg);
}
.bucket-count {
font-size: 11px;
font-weight: 500;
color: var(--text-secondary);
background: var(--border-light);
padding: 1px 7px;
border-radius: 9999px;
margin-left: auto;
}
.bucket-content {
display: none;
}
.bucket-content.expanded {
display: block;
}
/* ── Table row (discovery results) ── */
.table-item {
display: flex;
align-items: center;
padding: 10px 24px 10px 50px;
border-top: 1px solid var(--border-light);
gap: 12px;
transition: background 0.1s ease;
}
.table-item:hover {
background: rgba(243, 244, 246, 0.5);
}
.table-item-info {
flex: 1;
min-width: 0;
}
.table-item-name {
font-weight: 500;
font-family: var(--font-mono);
font-size: 13px;
color: var(--text-primary);
}
.table-item-meta {
font-size: 12px;
color: var(--text-secondary);
margin-top: 1px;
display: flex;
gap: 12px;
flex-wrap: wrap;
}
.table-item-actions {
display: flex;
align-items: center;
gap: 8px;
flex-shrink: 0;
}
/* ── Registry table ── */
.registry-table {
width: 100%;
border-collapse: collapse;
table-layout: fixed;
}
.registry-table .col-description {
overflow: hidden;
display: -webkit-box;
-webkit-line-clamp: 2;
-webkit-box-orient: vertical;
overflow-wrap: anywhere;
line-height: 1.4;
color: var(--text-secondary);
}
.registry-table th {
text-align: left;
font-size: 11px;
font-weight: 600;
color: var(--text-secondary);
text-transform: uppercase;
letter-spacing: 0.4px;
padding: 10px 16px;
border-bottom: 1px solid var(--border);
background: var(--background);
}
.registry-table td {
padding: 12px 16px;
font-size: 13px;
border-bottom: 1px solid var(--border-light);
vertical-align: middle;
}
.registry-table tr:last-child td {
border-bottom: none;
}
.registry-table tr:hover td {
background: rgba(243, 244, 246, 0.5);
}
.registry-table .col-id {
font-family: var(--font-mono);
font-size: 12px;
max-width: 220px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.registry-table .col-actions {
width: 120px;
min-width: 120px;
white-space: nowrap;
vertical-align: top;
}
/* ── Registry table — wide layout ── */
.registry-table .col-mode {
width: 100px;
}
.registry-table .col-source {
width: 200px;
font-family: var(--font-mono);
font-size: 12px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: var(--text-secondary);
}
.registry-table .col-pk {
width: 100px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.registry-table .col-schedule {
width: 100px;
font-size: 12px;
color: var(--text-secondary);
}
.registry-table .col-folder {
width: 120px;
}
.registry-table .col-registered {
width: 160px;
font-size: 11px;
line-height: 1.4;
overflow: hidden;
}
.registry-table .col-registered .registered-by {
color: var(--text-primary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.registry-table .col-registered .registered-at {
color: var(--text-secondary);
}
.registry-table .col-status {
width: 40px;
text-align: center;
}
.mode-badge {
display: inline-block;
padding: 2px 8px;
border-radius: 4px;
font-size: 11px;
font-weight: 600;
}
.mode-badge.mode-local {
background: var(--success-light);
color: var(--success);
}
.mode-badge.mode-remote {
background: var(--primary-light);
color: var(--primary);
}
.mode-badge.mode-materialized {
background: rgba(139, 92, 246, 0.1);
color: #8B5CF6;
}
.mode-badge.mode-internal {
background: rgba(15, 118, 110, 0.1);
color: #0F766E;
}
.folder-badge {
display: inline-block;
padding: 1px 6px;
border-radius: 3px;
background: var(--background);
border: 1px solid var(--border);
font-size: 11px;
color: var(--text-secondary);
}
/* ── Modal overlay ── */
.modal-overlay {
display: none;
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.5);
z-index: 1000;
padding: 40px 24px;
overflow-y: auto;
}
.modal-overlay.active {
display: flex;
align-items: flex-start;
justify-content: center;
}
.modal {
max-width: 560px;
width: 100%;
background: var(--surface);
border-radius: 12px;
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.3);
/* BLOCKER fix (UX review #1): cap height at 90vh + scroll
* the body so Create + Edit Data Package modals don't push
* the Submit footer off-screen on 720px viewports. Flex
* layout keeps header + footer pinned and grows the body. */
max-height: 90vh;
display: flex;
flex-direction: column;
overflow: hidden;
}
.modal .modal-body {
overflow-y: auto;
flex: 1 1 auto;
}
.modal .modal-header,
.modal .modal-footer {
flex: 0 0 auto;
}
.modal-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 20px 24px;
border-bottom: 1px solid var(--border);
background: var(--background);
}
.modal-header h2 {
font-size: 18px;
font-weight: 600;
}
.modal-close {
width: 32px;
height: 32px;
border: none;
background: none;
cursor: pointer;
border-radius: 6px;
display: flex;
align-items: center;
justify-content: center;
color: var(--text-secondary);
transition: all 0.15s;
}
.modal-close:hover {
background: var(--border-light);
color: var(--text-primary);
}
.modal-body {
padding: 24px;
}
.modal-footer {
display: flex;
justify-content: flex-end;
gap: 10px;
padding: 16px 24px;
border-top: 1px solid var(--border);
background: var(--background);
}
/* ── Form ── */
.form-group {
margin-bottom: 18px;
}
.form-group:last-child {
margin-bottom: 0;
}
.form-label {
display: block;
font-size: 13px;
font-weight: 500;
color: var(--text-primary);
margin-bottom: 6px;
}
.form-input,
.form-select {
width: 100%;
padding: 8px 12px;
border: 1px solid var(--border);
border-radius: 8px;
font-family: var(--font-primary);
font-size: 13px;
color: var(--text-primary);
background: var(--surface);
transition: border-color 0.15s;
}
.form-input:focus,
.form-select:focus {
outline: none;
border-color: var(--primary);
box-shadow: 0 0 0 3px rgba(46, 168, 119, 0.1);
}
.form-input[readonly] {
background: var(--border-light);
color: var(--text-secondary);
cursor: not-allowed;
}
.form-select {
cursor: pointer;
appearance: none;
background-image: url("data:image/svg+xml,%3Csvg width='12' height='12' viewBox='0 0 24 24' fill='none' stroke='%236B7280' stroke-width='2' stroke-linecap='round' stroke-linejoin='round' xmlns='http://www.w3.org/2000/svg'%3E%3Cpolyline points='6 9 12 15 18 9'/%3E%3C/svg%3E");
background-repeat: no-repeat;
background-position: right 12px center;
padding-right: 32px;
}
.form-hint {
font-size: 12px;
color: var(--text-secondary);
margin-top: 4px;
}
/* ── Footer ── */
.footer {
text-align: center;
padding: 24px;
color: var(--text-secondary);
font-size: 12px;
}
.footer a {
color: var(--primary);
text-decoration: none;
}
.footer a:hover {
text-decoration: underline;
}
/* ── Responsive ── */
@media (max-width: 640px) {
.header {
padding: 0 16px;
}
.panel-header {
flex-direction: column;
align-items: flex-start;
gap: 12px;
}
.table-item {
padding-left: 24px;
flex-wrap: wrap;
}
.table-item-meta {
flex-direction: column;
gap: 4px;
}
.registry-table .col-id {
max-width: 160px;
}
.modal {
margin: 16px;
}
}
/* ── Tab nav (Phase D) ── */
.tab-nav {
display: flex;
gap: 4px;
border-bottom: 1px solid var(--border);
margin-bottom: 16px;
}
.tab {
padding: 8px 16px;
background: transparent;
border: 0;
cursor: pointer;
font-family: inherit;
font-size: inherit;
color: var(--text-secondary);
}
.tab[aria-selected="true"] {
border-bottom: 2px solid var(--primary);
color: var(--text-primary);
font-weight: 600;
}
.tab-content {
padding: 16px 0;
}
.tab-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 16px;
}
.tab-actions {
display: flex;
justify-content: flex-end;
margin-bottom: 16px;
}
</style>
{% endblock %}
{% block content %}
<!-- ═══════════════ PAGE TITLE ═══════════════ -->
{% set page_hero_eyebrow = "Administration" %}
{% set page_hero_title = "Tables & Data Packages" %}
{% set page_hero_subtitle = "Group registered tables into Data Packages so analysts can opt-in to bundles." %}
{% include "_page_hero.html" %}
<!-- ═══════════════ CONTENT ═══════════════ -->
<div class="content">
{# ─── Top action bar (package-centric, replaces connector-tab nav) ───
Single row of high-level actions. The "+ Register new table"
dropdown picks a connector → opens the matching modal (BQ /
Keboola / Jira) instead of letting the connector drive the
page layout. #}
<section id="adminTablesActionBar" class="card" style="margin-bottom: 16px;">
<div class="card-body" style="display:flex; gap:8px; flex-wrap:wrap; align-items:center; padding:12px 16px;">
<div style="position:relative;">
<button id="registerNewTableBtn" class="btn btn-primary" type="button"
onclick="toggleRegisterNewTableMenu(event)">
+ Register new table ▾
</button>
<div id="registerNewTableMenu"
style="display:none; position:absolute; top:100%; left:0; margin-top:4px;
background:var(--surface); border:1px solid var(--border); border-radius:8px;
box-shadow:var(--shadow-md); min-width:220px; z-index:20; padding:6px;">
<button class="btn btn-secondary" type="button" data-register-source="bigquery"
style="display:block; width:100%; text-align:left; margin-bottom:4px;"
onclick="closeRegisterNewTableMenu(); openRegisterModal('bigquery')">
BigQuery
</button>
<button class="btn btn-secondary" type="button" data-register-source="keboola"
style="display:block; width:100%; text-align:left; margin-bottom:4px;"
onclick="closeRegisterNewTableMenu(); openRegisterModal('keboola')">
Keboola
</button>
<a class="btn btn-secondary" type="button"
href="docs/connectors/jira.md"
style="display:block; width:100%; text-align:left; text-decoration:none;"
onclick="closeRegisterNewTableMenu()">
Jira (webhook-driven — see docs)
</a>
</div>
</div>
<button class="btn btn-secondary" type="button" onclick="groupTablesByBucket()"
title="One-click: create one Data Package per distinct bucket and assign its tables">
Group tables by bucket
</button>
<button class="btn btn-secondary" type="button" onclick="openBulkAssignModal('')">
Bulk assign tables
</button>
<button class="btn btn-secondary" type="button" onclick="openCreateDataPackageModal('', null)">
+ New Data Package
</button>
<span style="flex:1;"></span>
<details id="cacheWarmupCard" style="margin:0;">
<summary style="cursor:pointer; user-select:none; font-size:13px; color:var(--text-secondary);">
<span id="cacheWarmupSummary">Cache freshness — loading…</span>
</summary>
<div style="margin-top:8px; padding:12px; background:var(--background); border-radius:8px;">
<div id="cacheWarmupProgress" style="margin-bottom:8px;"></div>
<progress id="cacheWarmupBar" max="100" value="0" style="width:100%; display:none;"></progress>
<button class="btn btn-secondary btn-sm" id="cacheWarmupRunBtn" onclick="cacheWarmupRun()"
style="margin-top:8px;">
Re-warm all
</button>
<details id="cacheWarmupDetails" style="margin-top:8px;">
<summary style="cursor:pointer; user-select:none; font-size:12px;">Show log</summary>
<p id="cacheWarmupHint" style="font-size:11px; color:var(--text-secondary); margin:6px 0 0;">
Live log from <em>Re-warm all</em>. Historical runs aren't persisted; check
<a href="/admin/activity">/admin/activity</a> for the audit trail.
</p>
<pre id="cacheWarmupLog" style="background:#0a0a0a; color:#dcdcdc; font-family:ui-monospace, Menlo, monospace; font-size:11px; padding:6px; max-height:200px; overflow-y:auto; margin-top:6px; border-radius:4px;"></pre>
</details>
</div>
</details>
</div>
</section>
{# ─── Package-centric layout (replaces the connector-tab nav) ───
Hydrated client-side by loadAdminTablesLayout(): pulls both
/api/admin/data-packages and /api/admin/registry, renders one
collapsible <details> per package containing its member tables,
then an "Unpackaged tables" section for everything else.
"On the side" data-packages was the wrong framing — packages ARE
the org structure now, so every table appears under either a
package or the explicit Unpackaged bucket. Bucket / source_type
survive as inline tags but no longer drive the layout. #}
<section id="adminTablesLayout" class="card" style="margin-bottom: 16px;">
<div class="card-body" id="adminTablesLayoutBody" style="padding:16px;">
{# Search bar — UX parity with /catalog + /memory. Filters the
package details + the table rows inside them; an unpackaged
row matches by table name / source_type / bucket. Hides any
package whose name doesn't match AND has zero matching rows. #}
<div style="margin-bottom: 12px; display:flex; align-items:center; gap:8px;">
<input id="adminTablesSearch" type="search"
class="form-input"
placeholder="Filter packages and tables by name, source, or bucket…"
oninput="filterAdminTablesLayout()"
style="flex:1; max-width:520px;">
<span id="adminTablesSearchHint"
style="font-size:12px; color:var(--text-secondary);"></span>
</div>
<div id="adminTablesLayoutPackages">
<div style="padding:16px; color:var(--text-secondary); font-size:13px; text-align:center;">
Loading packages…
</div>
</div>
<div id="adminTablesLayoutUnpackaged" style="margin-top:20px;"></div>
</div>
</section>
{# Register modals (BigQuery / Keboola) live as top-level overlays,
reachable via the "+ Register new table" dropdown in the action
bar above. The connector tabs that used to drive the page layout
were dropped — every table now appears in the package-centric
layout regardless of source_type. #}
<!-- ── BigQuery Register Modal ── -->
<div class="modal-overlay" id="registerBqModal">
<div class="modal">
<div class="modal-header">
<h2>Register BigQuery Table</h2>
<button class="modal-close" onclick="closeRegisterBqModal()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="modal-body">
{# Two orthogonal questions: (1) live vs synced, (2) when synced,
whole table vs custom SQL. Visibility classes:
bq-access-live — only when accessMode='live'
bq-access-synced — only when accessMode='synced'
bq-source-table — only when accessMode='live' OR
(accessMode='synced' AND syncMode='whole')
bq-source-custom — only when accessMode='synced' AND syncMode='custom'
Backend payload: live → query_mode='remote'; synced/whole →
query_mode='materialized' with auto-built SELECT *; synced/custom
→ query_mode='materialized' with admin SQL. Server auto-detects
BASE TABLE vs VIEW at register time, so the UI doesn't ask. #}
<div class="form-group">
<label class="form-label">How should analysts access this data?</label>
<div class="bq-access-radio-group" style="display:flex; gap:12px; margin-top:6px;">
<label class="sync-option-card" style="flex:1; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="bqAccessMode" value="live" checked onchange="onBqAccessModeChange()">
<strong>Live from BigQuery</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Each analyst query goes straight to BQ. Always current.
Latency ≈ seconds; 0 disk on the analyst machine; cost =
bytes scanned per query. Best for huge tables or when
freshness matters.
</div>
</label>
<label class="sync-option-card" style="flex:1; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="bqAccessMode" value="synced" onchange="onBqAccessModeChange()">
<strong>Synced locally</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Agnes runs a SELECT on a schedule and ships a parquet
to analysts. Analyst-side latency &lt;100&nbsp;ms; disk =
snapshot size. Best when analysts hit the same data
often and speed beats freshness.
</div>
</label>
</div>
</div>
<div class="form-group bq-access-synced" style="display:none;">
<label class="form-label">What to sync?</label>
<div class="bq-sync-radio-group" style="display:flex; gap:12px; margin-top:6px;">
<label class="sync-option-card" style="flex:1; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="bqSyncMode" value="whole" checked onchange="onBqSyncModeChange()">
<strong>Whole table</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Agnes runs <code>SELECT *</code> automatically. No SQL
required. Disk + sync cost = full table size.
</div>
</label>
<label class="sync-option-card" style="flex:1; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="bqSyncMode" value="custom" onchange="onBqSyncModeChange()">
<strong>Custom query</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
You write the SELECT — filter, project, or aggregate
before the sync. Cuts disk + cost; cap via
<code>max_bytes_per_materialize</code> guardrail.
</div>
</label>
</div>
</div>
<div class="form-group bq-source-table">
<label class="form-label" for="bqDataset">
Dataset
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverBqDatasets()" style="float:right; margin-top:-3px;">
Discover
</button>
</label>
<input type="text" class="form-input" id="bqDataset" list="bqDatasetList" placeholder="e.g. analytics">
<datalist id="bqDatasetList"></datalist>
<div class="form-hint">BigQuery dataset name (no project prefix — read from instance.yaml).
Click <strong>Discover</strong> to populate the autocomplete from the BQ project's dataset list.</div>
</div>
<div class="form-group bq-source-table">
<label class="form-label" for="bqSourceTable">
Source Table / View
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverBqTables()" style="float:right; margin-top:-3px;">
List tables
</button>
</label>
<input type="text" class="form-input" id="bqSourceTable" list="bqTableList" placeholder="e.g. orders">
<datalist id="bqTableList"></datalist>
<div class="form-hint">Table or view name within the dataset. Click
<strong>List tables</strong> after filling Dataset to populate autocomplete.
<br><strong>Live access:</strong> BASE TABLEs query via
<code>bq."dataset"."table"</code> (Storage Read API; predicate pushdown).
VIEWs and MATERIALIZED_VIEWs query via the BQ jobs API (full-scan estimate;
cost-guarded by <code>bq_max_scan_bytes</code>).
<code>agnes query --remote</code> works for both.
<br><strong>Synced access:</strong> handles both table and view transparently
— the scheduler runs <code>SELECT *</code> through the jobs API and writes a
parquet.</div>
</div>
<div class="form-group bq-source-custom" style="display:none;">
<label class="form-label" for="bqSourceQuery">
SQL
<button type="button" class="btn btn-secondary btn-sm"
onclick="prefillFromTable()" style="float:right; margin-top:-3px;"
title="Prefill SELECT * FROM `project.dataset.table` so you only edit the WHERE / projection">
Use table as base
</button>
</label>
<textarea class="form-textarea" id="bqSourceQuery" rows="8"
placeholder="SELECT date, SUM(revenue) AS revenue&#10;FROM `project.dataset.orders`&#10;WHERE date >= DATE_SUB(CURRENT_DATE(), INTERVAL 90 DAY)&#10;GROUP BY 1"></textarea>
<div class="form-hint">
SELECT statement, no trailing semicolon. Native BQ identifiers
(<code>`project.dataset.table`</code>) recommended — DuckDB three-part
names like <code>bq."ds"."t"</code> work for the COPY but disable the
cost guardrail's BQ dry-run.
</div>
</div>
<div class="form-group">
<label class="form-label" for="bqViewName">View Name</label>
<input type="text" class="form-input" id="bqViewName" placeholder="orders_90d">
<div class="form-hint">Name analysts use to query the data (e.g.
<code>SELECT * FROM orders_90d</code>). Required for Custom query; defaults
to the source table for the other modes.</div>
</div>
<div class="form-group">
<label class="form-label" for="bqDescription">Description <span class="optional">(optional)</span></label>
<textarea class="form-textarea" id="bqDescription" placeholder="Brief description of the table contents..."></textarea>
</div>
<div class="form-group">
<label class="form-label" for="bqFolder">Folder <span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="bqFolder" placeholder="e.g. crm, finance, marketing">
<div class="form-hint">Logical grouping for catalog organization</div>
</div>
<div class="form-group bq-access-synced" style="display:none;">
<label class="form-label" for="bqSyncSchedule">Sync Schedule <span class="optional">(optional, default <code>every 1h</code>)</span></label>
<input type="text" class="form-input" id="bqSyncSchedule" placeholder="every 6h">
<div class="form-hint">
How often Agnes refreshes the local copy. Examples:
<code>every 15m</code>, <code>every 6h</code>,
<code>daily 03:00</code>, <code>daily 07:00,13:00,18:00</code> (UTC).
</div>
</div>
<div class="form-group" id="bqPrecheckSummary" style="display:none;">
<div class="form-label">Source check</div>
<div class="form-hint" id="bqPrecheckSummaryText"></div>
</div>
{# v49 (Task 8.8): Data Packages chip-input field.
The submit handler (registerBqTable) is intentionally
NOT wired to forward `package_ids` to /api/admin/tables
in this pass — backend extension lives in a focused
follow-up. For now the chip-input persists chosen
packages locally and the admin can attach via
`POST /api/admin/data-packages/<id>/tables` from the
CLI / package admin UI. #}
<div class="form-group">
<label class="form-label">Data Packages <span class="optional">(optional)</span></label>
<div class="chip-input"
data-source-url="/api/admin/data-packages"
data-allow-create="true"
data-name="bq_package_ids"
data-placeholder="Type to search or create…"
data-chip-input="data_package"></div>
<div class="form-hint">Bundle this table into one or more Data Packages so analysts can opt-in to it as a group.</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeRegisterBqModal()">Cancel</button>
<button class="btn btn-primary" id="registerBqSubmitBtn" onclick="registerBqTable()">
Register Table
</button>
</div>
</div>
</div>
<!-- ── BigQuery Edit Modal (C2 — physically inside the BQ tab,
mirror of #registerBqModal placement) ── -->
<div class="modal-overlay" id="editBqModal">
<div class="modal">
<div class="modal-header">
<h2>Edit BigQuery Table</h2>
<button class="modal-close" onclick="closeEditBqModal()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="modal-body">
<div class="form-group">
<label class="form-label" for="editBqTableId">Table ID</label>
<input type="text" class="form-input" id="editBqTableId" readonly>
<div class="form-hint">Slugified id, immutable. Source type:
<strong id="editBqSourceTypeBadge">bigquery</strong></div>
</div>
<div class="form-group">
<label class="form-label">How should analysts access this data?
<a href="docs/admin/query-modes.md" target="_blank" title="When to use which mode" style="margin-left: 6px; text-decoration: none; cursor: help;">?</a>
</label>
<div style="display:flex; gap:12px; margin-top:6px;">
<label class="sync-option-card" style="flex:1; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editBqAccessMode" value="live" onchange="onEditBqAccessModeChange()">
<strong>Live from BigQuery</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Each query goes to BQ. No local copy.
</div>
</label>
<label class="sync-option-card" style="flex:1; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editBqAccessMode" value="synced" onchange="onEditBqAccessModeChange()">
<strong>Synced locally</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Scheduled SELECT → parquet, queried locally.
</div>
</label>
</div>
<div class="form-hint" id="editBqModeWarning" style="display:none;
color:#EA580C;background:rgba(234,88,12,.08);padding:8px;border-radius:6px;margin-top:8px;">
<!-- Filled by onEditBqAccessModeChange() when switching
modes on an existing row — warns about parquet
drop / scheduling impact. -->
</div>
</div>
<div class="form-group bq-edit-access-synced" style="display:none;">
<label class="form-label">What to sync?</label>
<div style="display:flex; gap:12px; margin-top:6px;">
<label class="sync-option-card" style="flex:1; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editBqSyncMode" value="whole" onchange="onEditBqSyncModeChange()">
<strong>Whole table</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
<code>SELECT *</code> on a schedule.
</div>
</label>
<label class="sync-option-card" style="flex:1; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editBqSyncMode" value="custom" onchange="onEditBqSyncModeChange()">
<strong>Custom query</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Filter / aggregate before sync.
</div>
</label>
</div>
</div>
<div class="form-group bq-edit-source-table" style="display:none;">
<label class="form-label" for="editBqDataset">
Dataset
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverBqDatasets('editBqDatasetList')"
style="float:right; margin-top:-3px;">
Discover
</button>
</label>
<input type="text" class="form-input" id="editBqDataset"
list="editBqDatasetList" placeholder="e.g. analytics">
<datalist id="editBqDatasetList"></datalist>
</div>
<div class="form-group bq-edit-source-table" style="display:none;">
<label class="form-label" for="editBqSourceTable">
Source Table / View
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverBqTables('editBqDataset', 'editBqTableList')"
style="float:right; margin-top:-3px;">
List tables
</button>
</label>
<input type="text" class="form-input" id="editBqSourceTable"
list="editBqTableList" placeholder="e.g. orders">
<datalist id="editBqTableList"></datalist>
<div class="form-hint">Table or view name within the dataset.
<br><strong>Live access:</strong> BASE TABLEs query via
<code>bq."dataset"."table"</code> (Storage Read API; predicate pushdown).
VIEWs and MATERIALIZED_VIEWs query via the BQ jobs API (full-scan estimate;
cost-guarded by <code>bq_max_scan_bytes</code>).
<code>agnes query --remote</code> works for both.
<br><strong>Synced access:</strong> handles both transparently — the
scheduler runs <code>SELECT *</code> through the jobs API and writes a
parquet.</div>
</div>
<div class="form-group bq-edit-source-custom" style="display:none;">
<label class="form-label" for="editBqSourceQuery">
SQL
<button type="button" class="btn btn-secondary btn-sm"
onclick="prefillFromTable('editBqSourceQuery')"
style="float:right; margin-top:-3px;"
title="Prefill SELECT * FROM `project.dataset.table` so you only edit the WHERE / projection">
Use table as base
</button>
</label>
<textarea class="form-textarea" id="editBqSourceQuery" rows="8"></textarea>
<div class="form-hint">SELECT statement, no trailing semicolon. Native BQ
identifiers recommended for the cost guardrail to engage.</div>
</div>
<div class="form-group bq-edit-access-synced" style="display:none;">
<label class="form-label" for="editBqSyncSchedule">Sync Schedule
<span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="editBqSyncSchedule" placeholder="every 6h">
<div class="form-hint">How often Agnes refreshes the local copy.
<code>every 15m</code>, <code>every 6h</code>,
<code>daily 03:00</code> (UTC).</div>
</div>
<div class="form-group">
<label class="form-label" for="editBqDescription">Description <span class="optional">(optional)</span></label>
<textarea class="form-textarea" id="editBqDescription" placeholder="Brief description of the table contents..."></textarea>
</div>
<div class="form-group">
<label class="form-label" for="editBqFolder">Folder <span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="editBqFolder" placeholder="e.g. crm, finance, marketing">
<div class="form-hint">Logical grouping for catalog organization (does not affect storage).</div>
</div>
{# Data Packages chip-input — parity with the legacy
and Keboola edit modals. Hydrated on open with
the table's current memberships; saveBqTabEdit
diffs against `_editBqOriginalPackageIds` and
emits the minimal POST/DELETE delta. #}
<div class="form-group">
<label class="form-label">Data Packages <span class="optional">(optional)</span></label>
<div class="chip-input"
id="editBqPackagesChips"
data-source-url="/api/admin/data-packages"
data-allow-create="true"
data-name="bq_edit_package_ids"
data-placeholder="Type to search or create…"
data-chip-input="data_package"></div>
<div class="form-hint">Bundle this table into one or more Data Packages so analysts can opt-in to it as a group.</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeEditBqModal()">Cancel</button>
<button class="btn btn-primary" id="editBqSubmitBtn" onclick="saveBqTabEdit()">
Save Changes
</button>
</div>
</div>
</div>
<!-- ── Keboola Register Modal ── -->
<div class="modal-overlay" id="registerKeboolaModal">
<div class="modal">
<div class="modal-header">
<h2>Register Keboola Table</h2>
<button class="modal-close" onclick="closeRegisterKeboolaModal()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="modal-body">
{# Three sync-mode radios:
- whole / custom → query_mode='materialized' (DuckDB Keboola
extension; whole synthesizes SELECT *, custom uses admin SQL)
- direct → query_mode='local' (Storage API SDK, supports
v26 sync strategies: incremental/partitioned + where_filters) #}
<div class="form-group">
<label class="form-label">What to sync?</label>
<div class="bq-sync-radio-group" style="display:flex; gap:12px; margin-top:6px; flex-wrap:wrap;">
<label class="sync-option-card" style="flex:1; min-width:200px; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="kbSyncMode" value="whole" checked onchange="onKbSyncModeChange()">
<strong>Whole table (extension)</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
DuckDB Keboola extension pulls the full table on
each tick. Fastest path; full overwrite each run.
</div>
</label>
<label class="sync-option-card" style="flex:1; min-width:200px; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="kbSyncMode" value="direct" onchange="onKbSyncModeChange()">
<strong>Direct extract (Storage API)</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Storage API SDK. Supports incremental sync
(changedSince + PK merge), partitioned files,
and server-side <code>where_filters</code>.
</div>
</label>
<label class="sync-option-card" style="flex:1; min-width:200px; padding:12px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="kbSyncMode" value="custom" onchange="onKbSyncModeChange()">
<strong>Custom SQL</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Pre-aggregate or filter with your own SELECT
(e.g. last 30 days only, per-day rollup).
</div>
</label>
</div>
</div>
<div class="form-group">
<label class="form-label" for="kbViewName">View name (analyst-visible)</label>
<input type="text" class="form-input" id="kbViewName"
placeholder="e.g. orders_recent">
</div>
{# Discover/List tables backend currently routes by instance's data_source.type
ignoring the `source` query param. Hiding the buttons on non-Keboola instances
prevents wrong-shape responses; inputs stay for manual entry. Future fix: make
/api/admin/discover-tables accept ?source=keboola and remove this guard. #}
<div class="form-group kb-source-table">
<label class="form-label" for="kbBucket">
Bucket
{% if data_source_type == 'keboola' %}
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverKeboolaBuckets('kbBucketList')"
style="float:right; margin-top:-3px;">Discover</button>
{% endif %}
</label>
<input type="text" class="form-input" id="kbBucket"
list="kbBucketList" placeholder="e.g. in.c-sales">
<datalist id="kbBucketList"></datalist>
</div>
<div class="form-group kb-source-table">
<label class="form-label" for="kbSourceTable">
Source Table
{% if data_source_type == 'keboola' %}
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverKeboolaTables('kbBucket', 'kbTableList')"
style="float:right; margin-top:-3px;">List tables</button>
{% endif %}
</label>
<input type="text" class="form-input" id="kbSourceTable"
list="kbTableList" placeholder="e.g. orders">
<datalist id="kbTableList"></datalist>
</div>
<div class="form-group kb-source-custom" style="display:none;">
<label class="form-label" for="kbSourceQuery">
SQL
{% if data_source_type == 'keboola' %}
<button type="button" class="btn btn-secondary btn-sm"
onclick="prefillFromKeboolaTable('kbSourceQuery')"
style="float:right; margin-top:-3px;"
title="Prefill SELECT * FROM kbc.bucket.table so you only edit the WHERE / projection">
Use table as base
</button>
{% endif %}
</label>
<textarea class="form-textarea" id="kbSourceQuery" rows="8"></textarea>
<div class="form-hint">SELECT against <code>kbc."bucket"."table"</code>.
Result is materialized to parquet and distributed via
<code>agnes pull</code>.</div>
</div>
<div class="form-group">
<label class="form-label" for="kbSyncSchedule">Sync Schedule
<span class="optional">(optional, default <code>every 1h</code>)</span></label>
<input type="text" class="form-input" id="kbSyncSchedule" placeholder="every 6h">
<div class="form-hint">
How often Agnes refreshes the local copy. Examples:
<code>every 15m</code>, <code>every 6h</code>,
<code>daily 03:00</code>, <code>daily 07:00,13:00,18:00</code> (UTC).
</div>
</div>
<div class="form-group">
<label class="form-label" for="kbDescription">Description
<span class="optional">(optional)</span></label>
<textarea class="form-textarea" id="kbDescription"
placeholder="Brief description of the table contents..."></textarea>
</div>
<div class="form-group">
<label class="form-label" for="kbFolder">Folder
<span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="kbFolder"
placeholder="e.g. crm, finance, marketing">
</div>
<details class="form-group">
<summary>Advanced (optional)</summary>
<div class="form-group" style="margin-top:8px;">
<label class="form-label" for="kbPrimaryKey">Primary Key</label>
<input type="text" class="form-input" id="kbPrimaryKey"
placeholder="e.g. id">
<div class="form-hint">Comma-separated list. Required for
Direct extract → Incremental (used as the dedup key on
delta merge). Auto-filled from the Keboola source when
available.</div>
</div>
</details>
<!-- v26 Direct-extract sync-strategy panel — visible only when
"Direct extract (Storage API)" is selected. Field visibility
within the panel further branches on the sync_strategy
dropdown (incremental / partitioned). -->
<div class="form-group kb-direct-only" style="display:none; padding:12px; border:1px solid var(--border); border-radius:8px; background:var(--background);">
<h3 style="margin:0 0 12px 0; font-size:14px;">Direct extract — sync strategy</h3>
<div class="form-group">
<label class="form-label" for="kbStrategy">Sync strategy</label>
<select class="form-select" id="kbStrategy" onchange="onKbStrategyChange()">
<option value="full_refresh">Full refresh — pull entire table each tick</option>
<option value="incremental">Incremental — pull rows changed since last sync</option>
<option value="partitioned">Partitioned — per-partition files, per-month/day/year</option>
</select>
</div>
<div class="form-group kb-strategy-incremental kb-strategy-partitioned" style="display:none;">
<label class="form-label" for="kbIncrementalWindowDays">Incremental window (days)
<span class="optional">(default 7)</span></label>
<input type="number" class="form-input" id="kbIncrementalWindowDays" min="0" placeholder="7">
<div class="form-hint">Backtrack window applied to last_sync timestamp on each tick.
Higher = more reliable on late-arriving rows; lower = less data per tick.</div>
</div>
<div class="form-group kb-strategy-incremental kb-strategy-partitioned" style="display:none;">
<label class="form-label" for="kbMaxHistoryDays">Max history (days)
<span class="optional">(first sync only, default unbounded)</span></label>
<input type="number" class="form-input" id="kbMaxHistoryDays" min="1" placeholder="365">
<div class="form-hint">Cap on how far back the first-ever sync goes. Multi-year tables
without this can OOM at write — set 90/180/365 for safety.</div>
</div>
<div class="form-group kb-strategy-partitioned" style="display:none;">
<label class="form-label" for="kbPartitionBy">Partition by column <strong>(required)</strong></label>
<input type="text" class="form-input" id="kbPartitionBy" placeholder="e.g. event_date">
<div class="form-hint">Date / timestamp column whose value drives the partition key.
Rows with NULL or unparseable values are dropped (logged warning).</div>
</div>
<div class="form-group kb-strategy-partitioned" style="display:none;">
<label class="form-label" for="kbPartitionGranularity">Granularity</label>
<select class="form-select" id="kbPartitionGranularity">
<option value="month">Month — YYYY_MM.parquet (default)</option>
<option value="day">Day — YYYY_MM_DD.parquet</option>
<option value="year">Year — YYYY.parquet</option>
</select>
</div>
<div class="form-group kb-strategy-partitioned" style="display:none;">
<label class="form-label" for="kbInitialLoadChunkDays">Initial-load chunk size (days)
<span class="optional">(default 30)</span></label>
<input type="number" class="form-input" id="kbInitialLoadChunkDays" min="1" placeholder="30">
<div class="form-hint">First-sync chunked load step. Smaller = more API calls, less
memory per chunk. Larger = fewer calls, more memory.</div>
</div>
<div class="form-group kb-strategy-not-incremental" style="display:none;">
<label class="form-label" for="kbWhereFilters">Where filters
<span class="optional">(JSON array, optional)</span></label>
<textarea class="form-textarea" id="kbWhereFilters" rows="6"
placeholder='[{"column": "date", "operator": "ge", "values": ["{{last_3_months}}"]}]'></textarea>
<div class="form-hint">
Server-side row filter. Operators: <code>eq, ne, gt, ge, lt, le</code>.
Date placeholders resolved at sync time:
<code>{{ '{{today}}' }}</code>,
<code>{{ '{{last_week}}' }}</code>,
<code>{{ '{{last_month}}' }}</code>,
<code>{{ '{{last_2_months}}' }}</code>,
<code>{{ '{{last_3_months}}' }}</code>,
<code>{{ '{{last_6_months}}' }}</code>,
<code>{{ '{{last_year}}' }}</code>,
<code>{{ '{{last_2_years}}' }}</code>,
<code>{{ '{{start_of_3_months_ago}}' }}</code>.
Not compatible with Incremental strategy.
</div>
</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeRegisterKeboolaModal()">Cancel</button>
<button class="btn btn-primary" id="registerKeboolaSubmitBtn"
onclick="registerKeboolaTable()">Register</button>
</div>
</div>
</div>
<!-- ── Keboola Edit Modal (Phase F2) ── -->
<div class="modal-overlay" id="editKeboolaModal">
<div class="modal">
<div class="modal-header">
<h2>Edit Keboola Table</h2>
<button class="modal-close" onclick="closeEditKeboolaModal()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="modal-body">
<div class="form-group">
<label class="form-label" for="editKbTableId">Table ID</label>
<input type="text" class="form-input" id="editKbTableId" readonly>
<div class="form-hint">Slugified id, immutable.</div>
</div>
{# Three sync-mode radios (mirror of Register). #}
<div class="form-group">
<label class="form-label">What to sync?</label>
<div class="bq-sync-radio-group" style="display:flex; gap:12px; margin-top:6px; flex-wrap:wrap;">
<label class="sync-option-card" style="flex:1; min-width:200px; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editKbSyncMode" value="whole"
onchange="onEditKbSyncModeChange()">
<strong>Whole table (extension)</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
DuckDB Keboola extension; full overwrite each tick.
</div>
</label>
<label class="sync-option-card" style="flex:1; min-width:200px; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editKbSyncMode" value="direct"
onchange="onEditKbSyncModeChange()">
<strong>Direct extract (Storage API)</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Storage API SDK. Supports incremental, partitioned,
<code>where_filters</code>.
</div>
</label>
<label class="sync-option-card" style="flex:1; min-width:200px; padding:10px; border:1px solid var(--border); border-radius:8px; cursor:pointer;">
<input type="radio" name="editKbSyncMode" value="custom"
onchange="onEditKbSyncModeChange()">
<strong>Custom SQL</strong>
<div style="font-size:12px; color:var(--text-secondary); margin-top:4px;">
Pre-aggregate or filter with your own SELECT.
</div>
</label>
</div>
</div>
{# Discover/List tables backend currently routes by instance's data_source.type
ignoring the `source` query param. Hiding the buttons on non-Keboola instances
prevents wrong-shape responses; inputs stay for manual entry. Future fix: make
/api/admin/discover-tables accept ?source=keboola and remove this guard. #}
<div class="form-group editkb-source-table">
<label class="form-label" for="editKbBucket">
Bucket
{% if data_source_type == 'keboola' %}
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverKeboolaBuckets('editKbBucketList')"
style="float:right; margin-top:-3px;">Discover</button>
{% endif %}
</label>
<input type="text" class="form-input" id="editKbBucket"
list="editKbBucketList" placeholder="e.g. in.c-sales">
<datalist id="editKbBucketList"></datalist>
</div>
<div class="form-group editkb-source-table">
<label class="form-label" for="editKbSourceTable">
Source Table
{% if data_source_type == 'keboola' %}
<button type="button" class="btn btn-secondary btn-sm"
onclick="discoverKeboolaTables('editKbBucket', 'editKbTableList')"
style="float:right; margin-top:-3px;">List tables</button>
{% endif %}
</label>
<input type="text" class="form-input" id="editKbSourceTable"
list="editKbTableList" placeholder="e.g. orders">
<datalist id="editKbTableList"></datalist>
</div>
<div class="form-group editkb-source-custom" style="display:none;">
<label class="form-label" for="editKbSourceQuery">
SQL
{% if data_source_type == 'keboola' %}
<button type="button" class="btn btn-secondary btn-sm"
onclick="prefillFromKeboolaTable('editKbSourceQuery')"
style="float:right; margin-top:-3px;"
title="Prefill SELECT * FROM kbc.bucket.table so you only edit the WHERE / projection">
Use table as base
</button>
{% endif %}
</label>
<textarea class="form-textarea" id="editKbSourceQuery" rows="8"></textarea>
</div>
<div class="form-group">
<label class="form-label" for="editKbSyncSchedule">Sync Schedule
<span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="editKbSyncSchedule" placeholder="every 6h">
</div>
<div class="form-group">
<label class="form-label" for="editKbDescription">Description
<span class="optional">(optional)</span></label>
<textarea class="form-textarea" id="editKbDescription"></textarea>
</div>
<div class="form-group">
<label class="form-label" for="editKbFolder">Folder
<span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="editKbFolder">
</div>
<details class="form-group">
<summary>Advanced (optional)</summary>
<div class="form-group" style="margin-top:8px;">
<label class="form-label" for="editKbPrimaryKey">Primary Key</label>
<input type="text" class="form-input" id="editKbPrimaryKey"
placeholder="e.g. id">
<div class="form-hint">Comma-separated list. Required for
Direct extract → Incremental.</div>
</div>
</details>
<!-- v26 Direct-extract sync-strategy panel — mirror of Register. -->
<div class="form-group editkb-direct-only" style="display:none; padding:12px; border:1px solid var(--border); border-radius:8px; background:var(--background);">
<h3 style="margin:0 0 12px 0; font-size:14px;">Direct extract — sync strategy</h3>
<div class="form-group">
<label class="form-label" for="editKbStrategy">Sync strategy</label>
<select class="form-select" id="editKbStrategy" onchange="onEditKbStrategyChange()">
<option value="full_refresh">Full refresh — pull entire table each tick</option>
<option value="incremental">Incremental — pull rows changed since last sync</option>
<option value="partitioned">Partitioned — per-partition files, per-month/day/year</option>
</select>
</div>
<div class="form-group editkb-strategy-incremental editkb-strategy-partitioned" style="display:none;">
<label class="form-label" for="editKbIncrementalWindowDays">Incremental window (days)
<span class="optional">(default 7)</span></label>
<input type="number" class="form-input" id="editKbIncrementalWindowDays" min="0" placeholder="7">
</div>
<div class="form-group editkb-strategy-incremental editkb-strategy-partitioned" style="display:none;">
<label class="form-label" for="editKbMaxHistoryDays">Max history (days)
<span class="optional">(first sync only)</span></label>
<input type="number" class="form-input" id="editKbMaxHistoryDays" min="1" placeholder="365">
</div>
<div class="form-group editkb-strategy-partitioned" style="display:none;">
<label class="form-label" for="editKbPartitionBy">Partition by column <strong>(required)</strong></label>
<input type="text" class="form-input" id="editKbPartitionBy" placeholder="e.g. event_date">
</div>
<div class="form-group editkb-strategy-partitioned" style="display:none;">
<label class="form-label" for="editKbPartitionGranularity">Granularity</label>
<select class="form-select" id="editKbPartitionGranularity">
<option value="month">Month (default)</option>
<option value="day">Day</option>
<option value="year">Year</option>
</select>
</div>
<div class="form-group editkb-strategy-partitioned" style="display:none;">
<label class="form-label" for="editKbInitialLoadChunkDays">Initial-load chunk size (days)
<span class="optional">(default 30)</span></label>
<input type="number" class="form-input" id="editKbInitialLoadChunkDays" min="1" placeholder="30">
</div>
<div class="form-group editkb-strategy-not-incremental" style="display:none;">
<label class="form-label" for="editKbWhereFilters">Where filters
<span class="optional">(JSON array, optional)</span></label>
<textarea class="form-textarea" id="editKbWhereFilters" rows="6"
placeholder='[{"column": "date", "operator": "ge", "values": ["{{last_3_months}}"]}]'></textarea>
<div class="form-hint">
Operators: <code>eq, ne, gt, ge, lt, le</code>. Date placeholders
(<code>{{ '{{today}}' }}</code>, <code>{{ '{{last_3_months}}' }}</code>, etc.) resolved at
sync time. Not compatible with Incremental.
</div>
</div>
</div>
{# Data Packages chip-input — parity with the BQ
and legacy edit modals. Hydrated on open;
saveKeboolaTabEdit diffs vs
`_editKbOriginalPackageIds` on save. #}
<div class="form-group">
<label class="form-label">Data Packages <span class="optional">(optional)</span></label>
<div class="chip-input"
id="editKbPackagesChips"
data-source-url="/api/admin/data-packages"
data-allow-create="true"
data-name="kb_edit_package_ids"
data-placeholder="Type to search or create…"
data-chip-input="data_package"></div>
<div class="form-hint">Bundle this table into one or more Data Packages so analysts can opt-in to it as a group.</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeEditKeboolaModal()">Cancel</button>
<button class="btn btn-primary" id="editKeboolaSubmitBtn"
onclick="saveKeboolaTabEdit()">Save Changes</button>
</div>
</div>
</div>
</div>
{# Connector-based listings (Jira read-only hint, internal tables
section, per-tab Register buttons) lived inside the now-deleted
tab-content sections. Their content is folded into the
package-centric layout above: every table — including jira/* and
internal — appears under a Data Package or in Unpackaged tables.
Register modals (registerBqModal / registerKeboolaModal) remain in
DOM as top-level overlays and are opened from the
"+ Register new table" dropdown in the action bar. The
data-source-type marker lives on <body> so DATA_SOURCE_TYPE still
has somewhere to read from. #}
<!-- ═══════════════ EDIT MODAL (legacy fallback — Keboola-only fields
remaining; the BQ Edit modal moved into #tab-content-bigquery as
#editBqModal in C2; the Keboola Edit modal is #editKeboolaModal
in #tab-content-keboola) ═══════════════ -->
<div class="modal-overlay" id="editModal">
<div class="modal">
<div class="modal-header">
<h2>Edit Table</h2>
<button class="modal-close" onclick="closeEditModal()">
<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<line x1="18" y1="6" x2="6" y2="18"/>
<line x1="6" y1="6" x2="18" y2="18"/>
</svg>
</button>
</div>
<div class="modal-body">
<div class="form-group">
<label class="form-label" for="editTableId">Table ID</label>
<input type="text" class="form-input" id="editTableId" readonly>
<div class="form-hint">Slugified id, immutable. Source type:
<strong id="editSourceTypeBadge"></strong></div>
</div>
<!-- Keboola/Jira fallback fields. The richer Keboola modal
lives at #editKeboolaModal; this remains the catch-all
for any source_type that's neither bigquery nor keboola
(e.g. jira). -->
<div class="form-group keboola-edit-only">
<label class="form-label" for="editStrategy">Sync Strategy</label>
<select class="form-select" id="editStrategy">
<option value="full_refresh">Full Refresh</option>
<option value="incremental">Incremental</option>
<option value="partitioned">Partitioned</option>
</select>
</div>
<div class="form-group keboola-edit-only">
<label class="form-label" for="editPrimaryKey">Primary Key</label>
<input type="text" class="form-input" id="editPrimaryKey" placeholder="e.g. id">
</div>
<div class="form-group">
<label class="form-label" for="editDescription">Description <span class="optional">(optional)</span></label>
<textarea class="form-textarea" id="editDescription" placeholder="Brief description of the table contents..."></textarea>
</div>
<div class="form-group">
<label class="form-label" for="editFolder">Folder <span class="optional">(optional)</span></label>
<input type="text" class="form-input" id="editFolder" placeholder="e.g. crm, finance, marketing">
<div class="form-hint">Logical grouping for catalog organization (does not affect storage).</div>
</div>
{# Data Packages chip-input — parity with the BigQuery
edit modal. saveTableEdit() diffs the chip selection
against the current membership and emits add/remove
junction calls so admins can manage package membership
without leaving the edit modal. #}
<div class="form-group">
<label class="form-label">Data Packages <span class="optional">(optional)</span></label>
<div class="chip-input"
id="editGenericPackagesChips"
data-source-url="/api/admin/data-packages"
data-allow-create="true"
data-name="legacy_package_ids"
data-placeholder="Type to search or create…"
data-chip-input="data_package"></div>
<div class="form-hint">Bundle this table into one or more Data Packages so analysts can opt-in to it as a group.</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeEditModal()">Cancel</button>
<button class="btn btn-primary" id="editSubmitBtn" onclick="saveTableEdit()">
Save Changes
</button>
</div>
</div>
</div>
<!-- ═══════════════ TOAST ═══════════════ -->
<div class="toast" id="toast">
<div class="toast-icon" id="toastIcon"></div>
<span id="toastMessage"></span>
</div>
<!-- ═══════════════ FOOTER ═══════════════ -->
<footer class="footer">
<a href="{{ url_for('dashboard') }}">Back to Dashboard</a>
</footer>
<!-- ═══════════════ Create Data Package modal (Task 8.10) ═══════════════
Fired from a chip-input `chip-create` event — the chip-input emits
a CustomEvent with the typed name; the handler below opens this
modal pre-filled, POSTs to /api/admin/data-packages, and adds the
freshly-created chip back into the chip-input host via .addChip(). -->
<div class="modal-overlay" id="createDataPackageModal" style="display:none;">
<div class="modal" style="max-width:520px;">
<div class="modal-header">
<h2>Create Data Package</h2>
<button class="modal-close" onclick="closeCreateDataPackageModal()">×</button>
</div>
<div class="modal-body">
<div class="form-group">
<label class="form-label">Name</label>
<input id="cdp-name" type="text" class="form-input" placeholder="Sales bundle">
</div>
<div class="form-group">
<label class="form-label">Slug</label>
<input id="cdp-slug" type="text" class="form-input" placeholder="sales-bundle">
<div class="form-hint">URL-safe identifier; auto-derived from Name.</div>
</div>
<div class="form-group">
<label class="form-label">Description <span class="optional">(optional)</span></label>
<textarea id="cdp-desc" class="form-textarea" rows="2"></textarea>
</div>
{# v51 lifecycle + classification — drive the hero filter
checkboxes on /catalog and the eyebrow line above the
card title. Both fields are admin-facing only and have
safe defaults (status=prod, category=null). #}
<div class="form-group" style="display:flex; gap:12px;">
<div style="flex:1;">
<label class="form-label">Status</label>
<select id="cdp-status" class="form-input" style="height:38px;">
<option value="prod" selected>Prod — ready for analyst use</option>
<option value="poc">POC — try-before-you-buy</option>
<option value="coming-soon">Coming soon — visible but not usable yet</option>
<option value="draft">Draft — admin-only, hidden from analysts</option>
</select>
</div>
<div style="flex:1;">
<label class="form-label">Category <span class="optional">(optional)</span></label>
<input id="cdp-category" type="text" class="form-input"
placeholder="e.g. Sessions &amp; Traffic">
<div class="form-hint">Eyebrow line above the card title. Keep it short and consistent.</div>
</div>
</div>
<div class="form-group" style="display:flex; gap:12px;">
<div style="flex:1;">
<label class="form-label">Icon</label>
<input id="cdp-icon" type="text" class="form-input" value="📦" maxlength="4">
</div>
<div style="flex:1;">
<label class="form-label">Color</label>
{# v51 palette row — admin clicks a swatch to set #cdp-color.
Free-form picker stays below as the escape hatch. Palette
is a vendor-neutral design-system set (teal/blue/violet/
pink/red/amber/emerald/slate) — no brand colors. #}
<div class="cf-palette-row" data-target="cdp-color"></div>
{# type=color forces the native swatch picker so admins can't
type/append a malformed hex into the field. #}
<input id="cdp-color" type="color" class="form-input" value="#e0f2fe"
style="height:32px; padding:2px; cursor:pointer;">
</div>
</div>
{# Cover image upload (v50). When set, the /catalog card renders
<img src=cover_image_url> instead of letter initials — closes
the visual gap with /marketplace cards. PNG/JPEG/GIF/WebP, 5
MiB cap server-side. #}
<div class="form-group">
<label class="form-label">Cover image <span class="optional">(optional)</span></label>
<div style="display:flex; gap:12px; align-items:center;">
<div id="cdp-cover-preview"
style="width:120px; height:72px; border-radius:8px;
border:1px solid var(--border); background:#f1f5f9;
display:flex; align-items:center; justify-content:center;
font-size:12px; color:var(--text-secondary); overflow:hidden;">
No image
</div>
<div style="flex:1;">
<input id="cdp-cover-file" type="file"
accept="image/png,image/jpeg,image/gif,image/webp"
onchange="onCoverFilePicked(this, 'cdp')">
<input id="cdp-cover-url" type="hidden" value="">
<div class="form-hint">PNG / JPEG / GIF / WebP, max 5 MiB. Uploads on Save.</div>
</div>
</div>
</div>
{# Inline RBAC matrix — replaces the modal-on-modal step-2 that
popped up after Create succeeded. <details> is collapsed by
default so the modal stays compact for admins who'll set
access later via the Resource Access page; opening it loads
the group list lazily. submitCreateDataPackage() reads any
chosen requirements and POSTs grants in parallel. #}
<details class="form-group" id="cdp-rbac-details"
style="border:1px solid var(--border); border-radius:8px; padding:8px 12px;">
<summary style="cursor:pointer; font-weight:600; font-size:13px; user-select:none;">
Group access
<span style="color:var(--text-secondary); font-weight:normal; font-size:12px;">(optional)</span>
</summary>
<p class="form-hint" style="margin:8px 0;">
Grant per-group access tiers — <em>available</em> shows the
package in the group's Browse; <em>required</em>
auto-installs it into the group's stack on next pull. Leave
everything blank to manage access later from
<a href="/admin/access">Resource access</a>.
</p>
<div id="cdp-rbac-rows"
style="display:flex; flex-direction:column; gap:6px; max-height:30vh; overflow-y:auto;">
<div class="loading" style="padding:6px; color:var(--text-secondary); font-size:12px;">
Click to load groups…
</div>
</div>
</details>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeCreateDataPackageModal()">Cancel</button>
<button class="btn btn-primary" id="cdp-submit-btn" onclick="submitCreateDataPackage()">
Create &amp; assign
</button>
</div>
</div>
</div>
<script>
// ── Cover image upload helpers (v50) ─────────────────────────────
// Shared between Create / Edit Data Package modals. The `prefix`
// arg is the modal's id prefix (`cdp` or `edp`); same DOM-id
// convention `<prefix>-cover-file`, `<prefix>-cover-url`,
// `<prefix>-cover-preview`. Uploads on file pick (not on Save) so
// the admin gets an immediate preview + failure-feedback loop;
// the URL stashed in the hidden input rides along on the eventual
// Save POST/PUT body.
async function onCoverFilePicked(input, prefix) {
const file = input.files && input.files[0];
if (!file) return;
const previewEl = document.getElementById(prefix + '-cover-preview');
const urlEl = document.getElementById(prefix + '-cover-url');
if (file.size > 5 * 1024 * 1024) {
alert('Image is too large (max 5 MiB).');
input.value = '';
return;
}
previewEl.innerHTML = '<span style="font-size:11px;">Uploading…</span>';
try {
const fd = new FormData();
fd.append('file', file);
const r = await fetch('/api/admin/uploads/cover-image', {
method: 'POST',
credentials: 'same-origin',
body: fd,
});
if (!r.ok) {
const detail = await r.json().catch(function() { return {}; });
alert('Upload failed: ' + (detail.detail || r.statusText));
previewEl.textContent = 'No image';
input.value = '';
return;
}
const body = await r.json();
urlEl.value = body.url;
previewEl.innerHTML = '';
var img = document.createElement('img');
img.src = body.url;
img.alt = '';
img.style.cssText = 'width:100%;height:100%;object-fit:cover;display:block;';
previewEl.appendChild(img);
} catch (e) {
alert('Network error: ' + e.message);
previewEl.textContent = 'No image';
input.value = '';
}
}
function clearCoverImage(prefix) {
// Empty string is the API contract for "remove the existing cover".
// The Save handler reads the hidden input and ships it as-is.
document.getElementById(prefix + '-cover-url').value = '';
document.getElementById(prefix + '-cover-file').value = '';
var p = document.getElementById(prefix + '-cover-preview');
p.innerHTML = 'No image';
p.dataset.cleared = '1';
}
function _renderCoverPreview(prefix, url) {
var p = document.getElementById(prefix + '-cover-preview');
if (!p) return;
if (url) {
p.innerHTML = '';
var img = document.createElement('img');
img.src = url;
img.alt = '';
img.style.cssText = 'width:100%;height:100%;object-fit:cover;display:block;';
p.appendChild(img);
} else {
p.innerHTML = 'No image';
}
}
// Track which chip-input host triggered the create flow so we can
// append the new chip back into it on success.
let _cdpHost = null;
function openCreateDataPackageModal(typed, host) {
_cdpHost = host;
document.getElementById('cdp-name').value = typed || '';
document.getElementById('cdp-slug').value =
(typed || '').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '');
document.getElementById('cdp-desc').value = '';
document.getElementById('cdp-icon').value = '📦';
document.getElementById('cdp-color').value = '#e0f2fe';
// Reset cover image inputs on every open so a previous typed-and-
// cancelled value doesn't leak into the new package.
var coverUrl = document.getElementById('cdp-cover-url');
var coverFile = document.getElementById('cdp-cover-file');
if (coverUrl) coverUrl.value = '';
if (coverFile) coverFile.value = '';
_renderCoverPreview('cdp', '');
document.getElementById('createDataPackageModal').style.display = 'flex';
}
function closeCreateDataPackageModal() {
_cdpHost = null;
// Re-collapse + invalidate the lazy-loaded RBAC matrix so the
// next "+ New Data Package" starts with a clean, unloaded state.
const det = document.getElementById('cdp-rbac-details');
if (det) det.open = false;
_cdpRbacLoaded = false;
document.getElementById('createDataPackageModal').style.display = 'none';
}
// ── Edit Data Package modal ──────────────────────────────────────
// Opened from the Edit button on each /admin/tables package card.
// Loads pkg metadata + member tables; lets admin rename / change
// description / icon / color / delete; remove member tables inline.
async function openEditDataPackageModal(pkgId) {
document.getElementById('editDataPackageModal').style.display = 'flex';
document.getElementById('edp-id').value = pkgId;
document.getElementById('edp-tables-list').innerHTML =
'<div style="padding:12px; color:#5f6368;">Loading…</div>';
try {
const r = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId),
{ credentials: 'same-origin' });
if (!r.ok) throw new Error('HTTP ' + r.status);
const pkg = await r.json();
document.getElementById('edp-name').value = pkg.name || '';
document.getElementById('edp-slug').value = pkg.slug || '';
document.getElementById('edp-desc').value = pkg.description || '';
document.getElementById('edp-icon').value = pkg.icon || '';
document.getElementById('edp-color').value = pkg.color || '#e0f2fe';
// v51: pre-fill status + category from the loaded package.
document.getElementById('edp-status').value = pkg.status || 'prod';
document.getElementById('edp-category').value = pkg.category || '';
// v50: hydrate cover image (URL + preview); clear the file
// input so a previous typed-and-cancelled file doesn't leak.
var coverUrl = document.getElementById('edp-cover-url');
var coverFile = document.getElementById('edp-cover-file');
var coverPreview = document.getElementById('edp-cover-preview');
if (coverPreview) delete coverPreview.dataset.cleared;
if (coverFile) coverFile.value = '';
if (coverUrl) coverUrl.value = pkg.cover_image_url || '';
_renderCoverPreview('edp', pkg.cover_image_url || '');
renderEdpTablesList(pkg.tables || []);
// Reset inline-add panel state (hidden + empty list) so it
// starts clean each open.
var addPanel = document.getElementById('edp-inline-add');
if (addPanel) addPanel.style.display = 'none';
} catch (e) {
alert('Failed to load package: ' + e.message);
closeEditDataPackageModal();
}
}
function closeEditDataPackageModal() {
// Reset the RBAC matrix the same way Create does — collapse the
// <details>, invalidate the loaded flag so the next open
// re-fetches against the (possibly different) package's grants.
const det = document.getElementById('edp-rbac-details');
if (det) det.open = false;
_edpRbacLoaded = false;
_edpRbacOriginal = {};
document.getElementById('editDataPackageModal').style.display = 'none';
}
// ── Edit DP RBAC matrix — lazy hydrate + diff-on-save ──────────
// Mirrors the Create flow's _cdpHydrate / _submit helpers but
// pre-fills the per-group dropdowns from existing grants so the
// matrix shows reality, not a blank slate. _edpRbacOriginal keeps
// a {groupId: requirement|""} snapshot taken at load time so the
// save handler can diff and only emit the minimum POST/DELETE.
let _edpRbacLoaded = false;
let _edpRbacOriginal = {};
async function _edpHydrateRbacMatrix() {
if (_edpRbacLoaded) return;
const pkgId = document.getElementById('edp-id').value;
const rowsEl = document.getElementById('edp-rbac-rows');
rowsEl.innerHTML = '<div class="loading" style="padding:6px; color:var(--text-secondary); font-size:12px;">Loading groups + grants…</div>';
try {
const [gResp, grResp] = await Promise.all([
fetch('/api/admin/groups', { credentials: 'same-origin' }),
fetch('/api/admin/grants?resource_type=data_package',
{ credentials: 'same-origin' }),
]);
if (!gResp.ok) throw new Error('groups HTTP ' + gResp.status);
if (!grResp.ok) throw new Error('grants HTTP ' + grResp.status);
const gBody = await gResp.json();
const groups = Array.isArray(gBody) ? gBody : (gBody.groups || []);
const grants = await grResp.json();
// Build group_id → requirement map for THIS package only.
// Multiple grants per (group, resource) shouldn't happen (UNIQUE
// constraint on the table) but be defensive — last write wins.
const current = {};
(grants || []).forEach(g => {
if (g.resource_id !== pkgId) return;
current[g.group_id] = g.requirement;
});
_edpRbacOriginal = Object.assign({}, current);
if (!groups.length) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--text-secondary); font-size:12px;">No groups defined yet. Create groups in <a href="/admin/access">Resource access</a> first.</div>';
return;
}
rowsEl.innerHTML = groups.map(g => {
const gid = String(g.id || g.name || '');
const gname = String(g.name || gid);
const cur = current[gid] || '';
const opt = (v, label) =>
'<option value="' + v + '"' + (cur === v ? ' selected' : '') + '>' + label + '</option>';
return (
'<div data-group-id="' + gid + '" '
+ 'style="display:flex; gap:8px; align-items:center; '
+ 'padding:6px; border:1px solid var(--border); border-radius:6px;">'
+ '<span style="flex:1;">' + gname + '</span>'
+ '<select class="edp-rbac-req" style="padding:4px; border:1px solid var(--border); border-radius:4px;">'
+ opt('', '(no grant)')
+ opt('available', 'available')
+ opt('required', 'required')
+ '</select>'
+ '</div>'
);
}).join('');
_edpRbacLoaded = true;
} catch (e) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--error); font-size:12px;">Failed to load: ' + e.message + '</div>';
}
}
document.addEventListener('toggle', (e) => {
if (e.target && e.target.id === 'edp-rbac-details' && e.target.open) {
_edpHydrateRbacMatrix();
}
}, true);
// Diff current dropdown state vs _edpRbacOriginal; emit grant
// POST / DELETE calls in parallel. Returns failure count so the
// save handler can surface it in the toast.
async function _edpDiffApplyGrants(pkgId) {
// No-op when the admin never opened the <details>; original
// snapshot is empty and dropdowns aren't in the DOM.
if (!_edpRbacLoaded) return 0;
const rows = document.querySelectorAll('#edp-rbac-rows [data-group-id]');
const calls = [];
// Track which group_ids the admin touched so deletes target the
// right grant_id. For DELETE we need to look up the grant by
// (group_id, resource_id, resource_type) — fetch fresh list to
// get ids. Cheaper to re-fetch once than store ids client-side.
const wantDelete = []; // [group_id]
const wantWrite = []; // [{group_id, requirement}]
rows.forEach(row => {
const gid = row.getAttribute('data-group-id');
const sel = row.querySelector('.edp-rbac-req');
const cur = (sel && sel.value) || '';
const orig = _edpRbacOriginal[gid] || '';
if (cur === orig) return;
if (!cur) {
// Cleared — DELETE the existing grant.
wantDelete.push(gid);
} else {
// New or changed requirement — DELETE-then-POST (resource_
// grants UNIQUE constraint forbids two rows on the same
// (group, resource, type) so we can't just upsert).
if (orig) wantDelete.push(gid);
wantWrite.push({ group_id: gid, requirement: cur });
}
});
if (!wantDelete.length && !wantWrite.length) return 0;
// Resolve grant_ids for deletes via a fresh list. /admin/grants
// doesn't take resource_id as a query filter so we fetch all
// data_package grants and filter client-side; the set is small
// enough for any realistic instance.
let grantIdByGroup = {};
if (wantDelete.length) {
const lr = await fetch('/api/admin/grants?resource_type=data_package',
{ credentials: 'same-origin' });
if (lr.ok) {
const arr = await lr.json();
arr.forEach(g => {
if (g.resource_id !== pkgId) return;
grantIdByGroup[g.group_id] = g.id;
});
}
}
// Sequential DELETE → POST. ``resource_grants`` has a UNIQUE
// constraint on (group_id, resource_type, resource_id); running
// both in parallel via ``Promise.allSettled`` raced — if the
// POST landed first, the unique check rejected it and the old
// ``available`` row stayed. Awaiting all DELETEs before any
// POST fires keeps the (group, resource) slot empty for the
// re-insert.
const deleteResults = await Promise.allSettled(
wantDelete
.map(gid => grantIdByGroup[gid])
.filter(Boolean)
.map(gid => fetch('/api/admin/grants/' + encodeURIComponent(gid),
{ method: 'DELETE', credentials: 'same-origin' }))
);
const writeResults = await Promise.allSettled(
wantWrite.map(w => fetch('/api/admin/grants', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({
group_id: w.group_id,
resource_type: 'data_package',
resource_id: pkgId,
requirement: w.requirement,
}),
}))
);
const results = deleteResults.concat(writeResults);
return results.filter(
r => r.status === 'rejected' || (r.value && !r.value.ok)
).length;
}
function renderEdpTablesList(tables) {
const el = document.getElementById('edp-tables-list');
if (!tables.length) {
el.innerHTML = '<div style="padding:12px; color:#5f6368; font-size:13px;">'
+ 'No tables in this package yet. Click <strong>+ Add tables</strong> above.</div>';
return;
}
el.innerHTML = tables.map(function(t) {
const id = String(t.id || '');
const name = String(t.name || id);
return ''
+ '<div style="display:flex; align-items:center; justify-content:space-between; '
+ 'padding:8px 12px; border-bottom:1px solid var(--border-light, #f0f0f0);">'
+ ' <span style="font-size:13px; color:#202124;">' + escapeHtml(name) + '</span>'
+ ' <button class="btn" type="button" style="padding:3px 10px; font-size:11px; color:#b91c1c;" '
+ 'onclick="removeTableFromPackage(\'' + escapeHtmlAttr(id) + '\')">Remove</button>'
+ '</div>';
}).join('');
}
async function removeTableFromPackage(tableId) {
const pkgId = document.getElementById('edp-id').value;
if (!confirm('Remove this table from the package?')) return;
const r = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId)
+ '/tables/' + encodeURIComponent(tableId),
{ method: 'DELETE', credentials: 'same-origin' });
if (!r.ok) { alert('Remove failed: HTTP ' + r.status); return; }
// Refresh the table list inline.
const r2 = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId),
{ credentials: 'same-origin' });
if (r2.ok) renderEdpTablesList((await r2.json()).tables || []);
}
async function submitEditDataPackage() {
const btn = document.getElementById('edp-save-btn');
btn.disabled = true;
try {
const pkgId = document.getElementById('edp-id').value;
// v50 cover image semantics:
// - cleared via Remove button → preview has dataset.cleared, send ""
// - new image uploaded → hidden input has the new URL
// - left unchanged → hidden input still holds the
// previous URL, send it unchanged
// (no-op on the server because the
// value matches)
var coverPreview = document.getElementById('edp-cover-preview');
var coverUrl = document.getElementById('edp-cover-url').value;
var coverField = (coverPreview && coverPreview.dataset.cleared === '1')
? '' : (coverUrl || null);
const payload = {
name: document.getElementById('edp-name').value.trim(),
description: document.getElementById('edp-desc').value.trim() || null,
icon: document.getElementById('edp-icon').value.trim() || null,
color: document.getElementById('edp-color').value.trim() || null,
cover_image_url: coverField,
// v51: pass status + category through. Empty-string category
// hits the clear_category branch server-side; status uses the
// dropdown's current value (never blank).
status: document.getElementById('edp-status').value || 'prod',
category: document.getElementById('edp-category').value.trim(),
};
const r = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId), {
method: 'PUT',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
});
if (!r.ok) {
const detail = await r.json().catch(function() { return {}; });
alert('Save failed: ' + (detail.detail || r.statusText));
btn.disabled = false;
return;
}
// Apply any RBAC matrix changes — diff vs snapshot taken at
// load time, emits only the minimum POST/DELETE pair per
// changed row. No-op when admin never opened the <details>.
const rbacFails = await _edpDiffApplyGrants(pkgId);
if (rbacFails > 0) {
if (typeof showToast === 'function') {
showToast(rbacFails + ' group access change(s) failed', 'error');
} else {
alert(rbacFails + ' group access change(s) failed.');
}
}
closeEditDataPackageModal();
if (typeof loadDataPackagesSection === 'function') loadDataPackagesSection();
} catch (e) {
alert('Network error: ' + e.message);
btn.disabled = false;
}
}
// ── Inline +Add-tables picker (v50) ─────────────────────────────
// User feedback: the legacy "+ Add tables" button closed Edit and
// opened Bulk Assign, losing the admin's mental context. The
// picker below stays inline: pulls /api/admin/registry, filters
// out tables already in this package + internal rows, lets the
// admin tick what they want, then POSTs each to the package
// without closing the modal.
let _edpInlineAddTables = [];
function toggleEdpInlineAdd() {
var panel = document.getElementById('edp-inline-add');
if (!panel) return;
if (panel.style.display === 'block') {
panel.style.display = 'none';
return;
}
panel.style.display = 'block';
loadEdpInlineAddTables();
}
async function loadEdpInlineAddTables() {
var listEl = document.getElementById('edp-inline-add-list');
var pkgId = document.getElementById('edp-id').value;
listEl.innerHTML = '<div style="padding:8px; color:var(--text-secondary); font-size:12px;">'
+ 'Loading unassigned tables…</div>';
try {
// Pull all tables and the current package member set in parallel.
var [allResp, pkgResp] = await Promise.all([
fetch('/api/admin/registry', { credentials: 'same-origin' }),
fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId),
{ credentials: 'same-origin' }),
]);
if (!allResp.ok || !pkgResp.ok) {
listEl.innerHTML = '<div style="padding:8px; color:#b91c1c;">'
+ 'Failed to load tables (' + allResp.status + ' / ' + pkgResp.status + ').</div>';
return;
}
var allBody = await allResp.json();
var allTables = Array.isArray(allBody)
? allBody
: (Array.isArray(allBody.tables) ? allBody.tables : []);
var pkg = await pkgResp.json();
var memberIds = new Set((pkg.tables || []).map(function(t) { return t.id; }));
// Show only non-internal tables that aren't already in the package.
_edpInlineAddTables = allTables.filter(function(t) {
return (t.source_type || '') !== 'internal' && !memberIds.has(t.id);
});
renderEdpInlineAddList();
} catch (e) {
listEl.innerHTML = '<div style="padding:8px; color:#b91c1c;">'
+ 'Network error: ' + escapeHtml(e.message) + '</div>';
}
}
function renderEdpInlineAddList() {
var listEl = document.getElementById('edp-inline-add-list');
if (!_edpInlineAddTables.length) {
listEl.innerHTML = '<div style="padding:8px; color:var(--text-secondary); font-size:12px;">'
+ 'All non-internal tables are already in this package.</div>';
return;
}
listEl.innerHTML = _edpInlineAddTables.map(function(t) {
var id = String(t.id || '');
var name = String(t.name || id);
var st = String(t.source_type || '');
var bucket = String(t.bucket || '');
var meta = [st, bucket].filter(Boolean).join(' · ');
return ''
+ '<label data-edp-add-id="' + escapeHtmlAttr(id) + '" '
+ ' data-edp-add-name="' + escapeHtmlAttr(name.toLowerCase()) + '" '
+ ' style="display:flex; gap:8px; align-items:center; padding:4px 6px; '
+ 'border-radius:4px; cursor:pointer; font-size:13px;">'
+ ' <input type="checkbox" class="edp-add-cb" value="' + escapeHtmlAttr(id) + '">'
+ ' <span style="flex:1; font-family:var(--font-mono);">' + escapeHtml(name) + '</span>'
+ (meta ? ' <span style="font-size:11px; color:var(--text-secondary);">'
+ escapeHtml(meta) + '</span>' : '')
+ '</label>';
}).join('');
}
function filterEdpInlineAdd() {
var q = (document.getElementById('edp-inline-add-search').value || '').toLowerCase();
document.querySelectorAll('#edp-inline-add-list label[data-edp-add-id]').forEach(function(row) {
var name = row.getAttribute('data-edp-add-name') || '';
row.style.display = (!q || name.indexOf(q) !== -1) ? '' : 'none';
});
}
async function submitEdpInlineAdd() {
var pkgId = document.getElementById('edp-id').value;
var btn = document.getElementById('edp-inline-add-submit-btn');
var selected = Array.from(document.querySelectorAll('.edp-add-cb:checked'))
.map(function(cb) { return cb.value; });
if (!selected.length) {
alert('Pick at least one table to add.');
return;
}
btn.disabled = true;
try {
// POST each table_id; idempotent on the server so retrying after
// partial failure is safe. Parallel — the user-visible latency
// is the slowest individual call, not the sum.
var calls = selected.map(function(tid) {
return fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId) + '/tables', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ table_id: tid }),
}).then(function(r) { return { tid: tid, ok: r.ok, status: r.status }; });
});
var results = await Promise.all(calls);
var fails = results.filter(function(r) { return !r.ok; });
if (fails.length) {
alert(fails.length + ' of ' + results.length + ' add(s) failed.');
}
// Refresh the package's table list inline (no modal close).
var r2 = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId),
{ credentials: 'same-origin' });
if (r2.ok) renderEdpTablesList((await r2.json()).tables || []);
// Reload the inline-add candidate list so the just-added rows
// disappear (member set has changed).
await loadEdpInlineAddTables();
// Background-refresh the top-level Data Packages section so
// the new table counts show up after the user closes the modal.
if (typeof loadDataPackagesSection === 'function') loadDataPackagesSection();
} catch (e) {
alert('Network error: ' + e.message);
} finally {
btn.disabled = false;
}
}
async function deleteEditDataPackage() {
const pkgId = document.getElementById('edp-id').value;
const name = document.getElementById('edp-name').value.trim();
// v54: soft delete with Undo toast. confirm() removed — the
// 10s Undo on the toast is the recovery affordance; double
// safety (confirm + undo) is friction without payoff. Junction
// rows (data_package_tables) + resource_grants survive intact
// so restore brings the package back whole.
const r = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId),
{ method: 'DELETE', credentials: 'same-origin' });
if (!r.ok) { alert('Delete failed: HTTP ' + r.status); return; }
closeEditDataPackageModal();
if (typeof loadDataPackagesSection === 'function') loadDataPackagesSection();
if (typeof window.showUndoToast === 'function') {
window.showUndoToast(
'Data Package "' + name + '" deleted.',
'/api/admin/data-packages/' + encodeURIComponent(pkgId) + '/restore',
() => {
if (typeof loadDataPackagesSection === 'function') loadDataPackagesSection();
},
);
}
}
// Auto-derive a url-safe slug from a Name string. Mirrors the
// server-side normalisation used by the seed step in _v48_to_v49.
function _deriveSlug(name) {
return (name || '').toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-|-$/g, '');
}
async function submitCreateDataPackage() {
const btn = document.getElementById('cdp-submit-btn');
btn.disabled = true;
try {
const nameVal = document.getElementById('cdp-name').value.trim();
// Auto-derive slug from name when the slug field is blank —
// fixes the silent-fail-on-empty-slug bug where the form just
// stayed open with no error if the user typed Name and clicked
// Create without focusing-out of the Name input.
let slugVal = document.getElementById('cdp-slug').value.trim();
if (!slugVal && nameVal) {
slugVal = _deriveSlug(nameVal);
document.getElementById('cdp-slug').value = slugVal;
}
// v50: cover_image_url is set by onCoverFilePicked() when the
// admin selects + uploads a file. If they didn't, it stays
// empty — send null so the server doesn't see "".
var cdpCoverUrl = document.getElementById('cdp-cover-url').value;
const payload = {
name: nameVal,
slug: slugVal,
description: document.getElementById('cdp-desc').value.trim() || null,
icon: document.getElementById('cdp-icon').value.trim() || null,
color: document.getElementById('cdp-color').value.trim() || null,
cover_image_url: cdpCoverUrl || null,
// v51: lifecycle status + classification category.
status: document.getElementById('cdp-status').value || 'prod',
category: document.getElementById('cdp-category').value.trim() || null,
};
if (!payload.name || !payload.slug) {
alert('Name is required (slug auto-derives from it).');
btn.disabled = false;
return;
}
const r = await fetch('/api/admin/data-packages', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify(payload),
});
if (!r.ok) {
const detail = await r.json().catch(() => ({}));
alert('Failed: ' + (detail.detail || r.statusText));
btn.disabled = false;
return;
}
const body = await r.json();
const newPkgId = body.id;
if (_cdpHost && _cdpHost.addChip) {
_cdpHost.addChip({ id: newPkgId, name: payload.name });
}
// Inline grants — read any per-group requirement the admin set
// in the collapsible "Group access" section and POST in parallel.
// Per-grant failures surface in the success toast; they don't
// roll back the package creation (admin can retry from
// /admin/access).
const grantFailures = await _submitCdpGrantsInline(newPkgId);
closeCreateDataPackageModal();
if (typeof loadDataPackagesSection === 'function') {
try { loadDataPackagesSection(); } catch (_) {}
}
if (typeof showToast === 'function') {
const baseMsg = 'Data Package "' + payload.name + '" created';
if (grantFailures > 0) {
showToast(baseMsg + ' — ' + grantFailures + ' grant(s) failed', 'error');
} else {
showToast(baseMsg, 'success');
}
}
} catch (e) {
alert('Network error: ' + e.message);
btn.disabled = false;
} finally {
btn.disabled = false;
}
}
// Lazy-load groups into the inline RBAC matrix the first time the
// <details> is opened. Subsequent opens are no-ops. Groups are
// small + admin-only data — cache for the lifetime of the modal.
let _cdpRbacLoaded = false;
async function _cdpHydrateRbacMatrix() {
if (_cdpRbacLoaded) return;
const rowsEl = document.getElementById('cdp-rbac-rows');
rowsEl.innerHTML = '<div class="loading" style="padding:6px; color:var(--text-secondary); font-size:12px;">Loading groups…</div>';
try {
const r = await fetch('/api/admin/groups', { credentials: 'same-origin' });
if (!r.ok) throw new Error('HTTP ' + r.status);
const body = await r.json();
const groups = Array.isArray(body) ? body : (body.groups || []);
if (!groups.length) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--text-secondary); font-size:12px;">No groups defined yet. Create groups in <a href="/admin/access">Resource access</a> first.</div>';
return;
}
rowsEl.innerHTML = groups.map(g => {
const gid = String(g.id || g.name || '');
const gname = String(g.name || gid);
return (
'<div data-group-id="' + gid + '" '
+ 'style="display:flex; gap:8px; align-items:center; '
+ 'padding:6px; border:1px solid var(--border); border-radius:6px;">'
+ '<span style="flex:1;">' + gname + '</span>'
+ '<select class="cdp-rbac-req" style="padding:4px; border:1px solid var(--border); border-radius:4px;">'
+ '<option value="">(no grant)</option>'
+ '<option value="available">available</option>'
+ '<option value="required">required</option>'
+ '</select>'
+ '</div>'
);
}).join('');
_cdpRbacLoaded = true;
} catch (e) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--error); font-size:12px;">Failed to load groups: ' + e.message + '</div>';
}
}
// Wire the lazy load on first <details> open. Re-fetched if the
// admin re-opens a fresh modal (closeCreateDataPackageModal resets
// the flag).
document.addEventListener('toggle', (e) => {
if (e.target && e.target.id === 'cdp-rbac-details' && e.target.open) {
_cdpHydrateRbacMatrix();
}
}, true);
async function _submitCdpGrantsInline(pkgId) {
const rows = document.querySelectorAll('#cdp-rbac-rows [data-group-id]');
const calls = [];
rows.forEach(row => {
const gid = row.getAttribute('data-group-id');
const sel = row.querySelector('.cdp-rbac-req');
const req = sel && sel.value;
if (!req) return;
calls.push(fetch('/api/admin/grants', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({
group_id: gid,
resource_type: 'data_package',
resource_id: pkgId,
requirement: req,
}),
}));
});
if (!calls.length) return 0;
const results = await Promise.allSettled(calls);
return results.filter(
r => r.status === 'rejected' || (r.value && !r.value.ok)
).length;
}
// ── (removed) Step-2 RBAC modal-on-modal — folded into the
// create modal's collapsible "Group access" section. Stubs kept
// for callers that may still reference these names; they're no-ops
// now. Safe to delete in a follow-up once the audit confirms no
// other call sites. ──
let _cdpNewId = null;
let _cdpNewName = null;
function openCreateDataPackageRbacModal() { /* deprecated — inlined */ }
function skipCreateDataPackageRbac() {
if (typeof loadDataPackagesSection === 'function') {
loadDataPackagesSection();
}
}
function submitCreateDataPackageRbac() { /* deprecated — inlined */ }
// Hook: chip-input on this page fires `chip-create` when the user
// selects the "+ Create new" tail row in the dropdown.
document.addEventListener('chip-create', (e) => {
const host = e.detail && e.detail.host;
if (host && host.dataset.chipInput === 'data_package') {
openCreateDataPackageModal(e.detail.typed, host);
}
});
// ── Bulk-assign tables → package ─────────────────────────
// Single round-trip per table on submit (the existing
// POST /api/admin/data-packages/{id}/tables endpoint takes one
// table_id per call) but parallel via Promise.allSettled so the
// user-visible latency is the slowest individual call rather
// than the sum. Idempotent on the server, so re-submitting after
// a partial failure only retries the ones that didn't land.
let _bulkAssignPackages = []; // [{id, name, member_table_ids:Set}]
let _bulkAssignTables = []; // [{id, name, source_type, bucket}]
/* One-click factory: read table_registry → group by `bucket` →
create one Data Package per distinct bucket (skipping any whose
slug already exists) → bulk-assign its tables. Answers the
"why do I have to package by hand what's already grouped?"
friction for admins with many registered tables.
Sequential rather than parallel because the assign-table POSTs
depend on the package_id from the create POST. ~100ms per
bucket which is fine for the handful most instances have. */
// Bucket plan (loaded once when the preview modal opens; consumed
// by submitGroupByBucket). { bucket, slug, tables[], slugExists }
var _gbbPlan = [];
function closeGroupByBucketModal() {
document.getElementById('groupByBucketPreviewModal').style.display = 'none';
}
function _gbbSlugify(s) {
return (s || '').toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-|-$/g, '');
}
async function groupTablesByBucket() {
// Build the plan + render the preview modal. Replaces the old
// confirm() that just said "yes/no to ALL buckets" — admins now
// see which buckets would become packages, the row count per
// bucket, the resulting slug, and which ones would be skipped
// because a package with that slug already exists.
const rowsEl = document.getElementById('gbb-rows');
rowsEl.innerHTML = '<div class="loading" style="padding:8px; color:var(--text-secondary); font-size:13px;">Loading buckets…</div>';
document.getElementById('groupByBucketPreviewModal').style.display = 'flex';
try {
const [tableResp, pkgResp] = await Promise.all([
fetch('/api/admin/registry', { credentials: 'same-origin' }),
fetch('/api/admin/data-packages', { credentials: 'same-origin' }),
]);
if (!tableResp.ok || !pkgResp.ok) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--error); font-size:13px;">Failed to load tables or packages.</div>';
return;
}
const tables = await tableResp.json();
const existingPackages = await pkgResp.json();
const tableList = Array.isArray(tables) ? tables : (tables.items || tables.tables || []);
const existingSlugs = new Set((existingPackages || []).map(p => p.slug));
const buckets = {};
for (const t of tableList) {
const b = (t.bucket || '').trim();
if (!b || b.startsWith('agnes_')) continue;
buckets[b] = buckets[b] || [];
buckets[b].push(t);
}
const bucketNames = Object.keys(buckets).sort();
if (!bucketNames.length) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--text-secondary); font-size:13px;">No buckets to group. Register some tables first.</div>';
return;
}
_gbbPlan = bucketNames.map(function (bucket) {
const slug = _gbbSlugify(bucket);
return {
bucket: bucket,
slug: slug,
tables: buckets[bucket],
slugExists: existingSlugs.has(slug),
};
});
rowsEl.innerHTML = _gbbPlan.map(function (p, i) {
const disabled = p.slugExists;
const note = disabled
? '<span style="color:var(--warning); font-size:11px; margin-left:6px;">(slug exists — will skip)</span>'
: '';
return ''
+ '<label style="display:flex; align-items:center; gap:10px; padding:6px 8px; '
+ 'border:1px solid var(--border); border-radius:6px; '
+ 'background:' + (disabled ? 'var(--background)' : 'var(--surface)') + ';">'
+ ' <input type="checkbox" class="gbb-row" data-idx="' + i + '" '
+ (disabled ? 'disabled' : 'checked') + '>'
+ ' <span style="flex:1; font-weight:600; font-size:13px;">'
+ escapeHtml(p.bucket)
+ ' <code style="font-weight:normal; font-size:11px; color:var(--text-secondary); margin-left:6px;">' + escapeHtml(p.slug) + '</code>'
+ note
+ ' </span>'
+ ' <span style="color:var(--text-secondary); font-size:12px;">'
+ p.tables.length + ' table' + (p.tables.length === 1 ? '' : 's')
+ ' </span>'
+ '</label>';
}).join('');
} catch (e) {
rowsEl.innerHTML = '<div style="padding:8px; color:var(--error); font-size:13px;">Failed: ' + escapeHtml(e.message) + '</div>';
}
}
async function submitGroupByBucket() {
const btn = document.getElementById('gbb-submit-btn');
const checks = document.querySelectorAll('#gbb-rows input.gbb-row:checked');
if (!checks.length) {
showToast('No buckets selected.', 'error');
return;
}
btn.disabled = true;
const picks = Array.from(checks).map(function (el) {
return _gbbPlan[parseInt(el.dataset.idx, 10)];
}).filter(Boolean);
const created = [], failed = [];
for (const p of picks) {
const cResp = await fetch('/api/admin/data-packages', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
name: p.bucket,
slug: p.slug,
description: 'Auto-grouped from bucket `' + p.bucket + '` — ' + p.tables.length + ' tables.',
icon: null,
color: '#e0f2fe',
}),
});
if (!cResp.ok) { failed.push(p.bucket); continue; }
const pkg = await cResp.json();
await Promise.all(p.tables.map(function (t) {
return fetch('/api/admin/data-packages/' + encodeURIComponent(pkg.id) + '/tables', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ table_id: t.id }),
}).catch(function () { /* swallowed; count via created total */ });
}));
created.push(p.bucket);
}
btn.disabled = false;
closeGroupByBucketModal();
if (failed.length) {
showToast('Created ' + created.length + ', failed ' + failed.length + ': ' + failed.join(', '), 'error');
} else {
showToast('Created ' + created.length + ' package' + (created.length === 1 ? '' : 's'), 'success');
}
if (typeof loadDataPackagesSection === 'function') loadDataPackagesSection();
}
async function openBulkAssignModal(preselectPkgId) {
document.getElementById('bulkAssignTablesModal').style.display = 'flex';
document.getElementById('bulk-assign-search').value = '';
// Load packages + tables in parallel.
const [pkgResp, tableResp] = await Promise.all([
fetch('/api/admin/data-packages', { credentials: 'same-origin' }),
fetch('/api/admin/registry', { credentials: 'same-origin' }),
]);
const sel = document.getElementById('bulk-assign-package');
const list = document.getElementById('bulk-assign-list');
if (!pkgResp.ok) {
sel.innerHTML = '<option value="">Failed to load packages</option>';
list.innerHTML = '<div style="padding:12px; color:#b91c1c;">HTTP '
+ pkgResp.status + ' on /api/admin/data-packages</div>';
return;
}
if (!tableResp.ok) {
sel.innerHTML = '<option value="">Failed to load tables</option>';
list.innerHTML = '<div style="padding:12px; color:#b91c1c;">HTTP '
+ tableResp.status + ' on /api/admin/registry</div>';
return;
}
const pkgs = await pkgResp.json();
const tableBody = await tableResp.json();
// /api/admin/registry returns {tables, count}; defend against shape drift.
_bulkAssignTables = Array.isArray(tableBody)
? tableBody
: (Array.isArray(tableBody.tables) ? tableBody.tables : []);
// Drop internal rows — same filter the /catalog empty-state uses.
_bulkAssignTables = _bulkAssignTables.filter(function(t) {
return (t.source_type || '') !== 'internal';
});
// Fetch each package's member tables in parallel so we can pre-check
// already-assigned rows (and gray them out — server is idempotent
// but skipping the no-op call keeps the audit log clean).
const memberFetches = pkgs.map(function(p) {
return fetch('/api/admin/data-packages/' + encodeURIComponent(p.id),
{ credentials: 'same-origin' })
.then(function(r) { return r.ok ? r.json() : null; })
.catch(function() { return null; });
});
const memberBodies = await Promise.all(memberFetches);
_bulkAssignPackages = pkgs.map(function(p, i) {
const det = memberBodies[i] || {};
const ids = new Set((det.tables || []).map(function(t) { return t.id; }));
return { id: p.id, name: p.name, member_table_ids: ids };
});
// Inline "+ Create new package…" as a sentinel value. The change
// handler below intercepts __create__ and runs a tiny prompt-based
// flow (name → POST → re-list → re-select) so the user can spawn
// a target without leaving the bulk-assign workflow.
//
// v55 — also annotate each option with "(N of M tables already in)"
// so the admin sees the existing distribution before picking a
// target. M = number of visible/grantable tables in this modal;
// N = how many of those are already in that package. Surfaces the
// implicit "this package is half-full of what I'm about to assign"
// signal the admin previously had to derive by clicking through.
const totalVisible = _bulkAssignTables.length;
sel.innerHTML = '<option value="">— Choose a package —</option>'
+ '<option value="__create__" style="font-weight:600;">+ Create new package…</option>'
+ _bulkAssignPackages.map(function(p) {
let overlap = 0;
_bulkAssignTables.forEach(function(t) {
if (p.member_table_ids.has(String(t.id))) overlap++;
});
const distHint = overlap > 0
? ' (' + overlap + ' of ' + totalVisible + ' already in)'
: '';
return '<option value="' + escapeHtmlAttr(p.id) + '">'
+ escapeHtml(p.name) + escapeHtml(distHint) + '</option>';
}).join('');
sel.onchange = async function() {
if (sel.value !== '__create__') { refreshBulkAssignAvailability(); return; }
const name = prompt('Name for the new Data Package:');
if (!name || !name.trim()) { sel.value = ''; return; }
const slug = _deriveSlug(name.trim());
const r = await fetch('/api/admin/data-packages', {
method: 'POST',
credentials: 'same-origin',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
name: name.trim(),
slug,
description: null,
icon: null,
color: '#e0f2fe',
}),
});
if (!r.ok) {
const detail = await r.json().catch(function() { return {}; });
alert('Create failed: ' + (detail.detail || r.statusText));
sel.value = '';
return;
}
const created = await r.json();
// Append to the in-memory list + the <select>; preserve any
// already-checked table rows by NOT re-rendering the list.
_bulkAssignPackages.push({
id: created.id, name: name.trim(),
member_table_ids: new Set(),
});
const opt = document.createElement('option');
opt.value = created.id;
opt.textContent = name.trim();
sel.appendChild(opt);
sel.value = created.id;
refreshBulkAssignAvailability();
};
if (preselectPkgId && _bulkAssignPackages.some(function(p) { return p.id === preselectPkgId; })) {
sel.value = preselectPkgId;
}
renderBulkAssignList();
refreshBulkAssignAvailability();
}
function closeBulkAssignModal() {
document.getElementById('bulkAssignTablesModal').style.display = 'none';
// Drop the ?assign_to= query param so a manual refresh doesn't
// re-open the modal — but only when it was the param that opened
// us; otherwise leave the URL untouched.
if (window.location.search.indexOf('assign_to=') !== -1) {
history.replaceState(null, '', window.location.pathname + window.location.hash);
}
}
function renderBulkAssignList() {
const list = document.getElementById('bulk-assign-list');
if (!_bulkAssignTables.length) {
list.innerHTML = '<div style="padding:12px; color:#5f6368;">'
+ 'No tables registered yet — register some via the connector tabs first.</div>';
return;
}
list.innerHTML = _bulkAssignTables.map(function(t) {
const id = String(t.id || '');
const name = String(t.name || id);
const st = String(t.source_type || '');
const bucket = String(t.bucket || '');
const meta = [st, bucket].filter(Boolean).join(' · ');
return ''
+ '<label data-table-id="' + escapeHtmlAttr(id) + '" '
+ ' data-table-name="' + escapeHtmlAttr(name.toLowerCase()) + '" '
+ ' data-table-source="' + escapeHtmlAttr(st.toLowerCase()) + '" '
+ ' style="display:flex; gap:8px; align-items:center; padding:6px; border-radius:4px; cursor:pointer;">'
+ ' <input type="checkbox" class="bulk-assign-cb" value="' + escapeHtmlAttr(id) + '">'
+ ' <span style="flex:1; font-family:var(--font-mono);">' + escapeHtml(name) + '</span>'
+ (meta ? ' <span style="font-size:12px; color:#5f6368;">' + escapeHtml(meta) + '</span>' : '')
+ '</label>';
}).join('');
}
// Re-render the per-row "already assigned" hints whenever the
// target package changes — keeps the UI in sync with reality
// without a re-fetch (member sets were cached at open time).
function refreshBulkAssignAvailability() {
const pkgId = document.getElementById('bulk-assign-package').value;
const pkg = _bulkAssignPackages.find(function(p) { return p.id === pkgId; });
const members = pkg ? pkg.member_table_ids : new Set();
document.querySelectorAll('#bulk-assign-list label').forEach(function(row) {
const tid = row.getAttribute('data-table-id');
const cb = row.querySelector('.bulk-assign-cb');
const already = members.has(tid);
// Wipe any existing badge before re-applying.
const oldBadge = row.querySelector('.bulk-assign-badge');
if (oldBadge) oldBadge.remove();
if (already) {
cb.checked = true;
cb.disabled = true;
row.style.opacity = '0.5';
const badge = document.createElement('span');
badge.className = 'bulk-assign-badge';
badge.textContent = 'already in package';
badge.style.cssText = 'font-size:11px; color:#10b77f; padding:2px 6px; '
+ 'border:1px solid #10b77f33; border-radius:10px;';
row.appendChild(badge);
} else {
cb.disabled = false;
row.style.opacity = '';
// Only auto-uncheck rows that we previously force-checked —
// don't clobber the user's manual selection on package switch.
}
});
}
function filterBulkAssignList() {
const q = (document.getElementById('bulk-assign-search').value || '').toLowerCase();
document.querySelectorAll('#bulk-assign-list label').forEach(function(row) {
if (!q) { row.style.display = ''; return; }
const name = row.getAttribute('data-table-name') || '';
const src = row.getAttribute('data-table-source') || '';
row.style.display = (name.includes(q) || src.includes(q)) ? '' : 'none';
});
}
function toggleAllBulkAssign(checked) {
document.querySelectorAll('#bulk-assign-list label').forEach(function(row) {
if (row.style.display === 'none') return;
const cb = row.querySelector('.bulk-assign-cb');
if (cb && !cb.disabled) cb.checked = checked;
});
}
async function submitBulkAssign() {
const pkgId = document.getElementById('bulk-assign-package').value;
if (!pkgId) {
alert('Pick a target Data Package first.');
return;
}
const pkg = _bulkAssignPackages.find(function(p) { return p.id === pkgId; });
const alreadyMembers = pkg ? pkg.member_table_ids : new Set();
const tableIds = Array.from(
document.querySelectorAll('#bulk-assign-list .bulk-assign-cb:checked')
)
.map(function(cb) { return cb.value; })
// Skip the disabled-already-member rows — the server would no-op,
// but avoiding the call keeps audit_log noise down.
.filter(function(id) { return !alreadyMembers.has(id); });
if (!tableIds.length) {
alert('Nothing to assign — pick at least one table not already in the package.');
return;
}
const btn = document.getElementById('bulk-assign-submit-btn');
btn.disabled = true;
btn.textContent = 'Assigning ' + tableIds.length + '…';
try {
const calls = tableIds.map(function(tid) {
return fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId) + '/tables', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ table_id: tid }),
}).then(function(r) { return { tid: tid, ok: r.ok, status: r.status }; })
.catch(function(e) { return { tid: tid, ok: false, status: 0, err: String(e) }; });
});
const results = await Promise.all(calls);
const fails = results.filter(function(r) { return !r.ok; });
if (fails.length) {
alert(fails.length + ' of ' + results.length + ' failed. '
+ 'First failure: ' + fails[0].tid + ' (HTTP ' + fails[0].status + ').');
}
closeBulkAssignModal();
// Refresh the top-level grid so newly-bundled package counts update.
if (typeof loadDataPackagesSection === 'function') {
loadDataPackagesSection();
}
} finally {
btn.disabled = false;
btn.textContent = 'Assign selected';
}
}
// ── Auto-open Bulk Assign on ?assign_to=<pkg_id> ────────
// Lands here from the /catalog "0 tables — assign some →" CTA.
// Wait one tick so the loadDataPackagesSection hydrator has a
// chance to populate the package grid first (cosmetic — keeps
// the page background populated when the modal opens).
function initBulkAssignFromQuery() {
try {
const params = new URLSearchParams(window.location.search);
const pid = params.get('assign_to');
if (pid) {
setTimeout(function() { openBulkAssignModal(pid); }, 50);
}
} catch (e) { /* no-op */ }
}
document.addEventListener('DOMContentLoaded', initBulkAssignFromQuery);
</script>
{# (removed) Create Data Package RBAC step-2 modal — the per-group
Available|Required matrix is now an inline collapsible section
inside #createDataPackageModal (#cdp-rbac-details). The modal-on-
modal pattern was confusing per user feedback. #}
<!-- ═══════════════ Bulk-assign tables → package ═══════════════
Replaces the 50-clicks workflow ("edit each table individually
and add it via the chip-input inside its modal") with one
dropdown + multi-select submit. Driven by the "Bulk assign
tables" button on the Data Packages section header AND by the
/catalog "0 tables — assign some →" CTA which lands here with
?assign_to=<pkg_id> (see initBulkAssignFromQuery). -->
<!-- ═══════════════ Edit Data Package modal ═══════════════
Opened from the Edit button on each card in the Data Packages
section. Lets admin rename + change description / icon / color,
see + remove member tables, and delete the package. -->
<div class="modal-overlay" id="editDataPackageModal" style="display:none;">
<div class="modal" style="max-width:680px;">
<div class="modal-header">
<h2>Edit Data Package</h2>
<button class="modal-close" onclick="closeEditDataPackageModal()">×</button>
</div>
<div class="modal-body">
<input type="hidden" id="edp-id">
<div class="form-group">
<label class="form-label">Name</label>
<input id="edp-name" type="text" class="form-input">
</div>
<div class="form-group">
<label class="form-label">Slug</label>
<input id="edp-slug" type="text" class="form-input" disabled
style="opacity:0.6;" title="Slug is permanent — used in URLs and grants">
<div class="form-hint">Slug is permanent.</div>
</div>
<div class="form-group">
<label class="form-label">Description</label>
<textarea id="edp-desc" class="form-textarea" rows="2"></textarea>
</div>
<div class="form-group" style="display:flex; gap:12px;">
<div style="flex:1;">
<label class="form-label">Status</label>
<select id="edp-status" class="form-input" style="height:38px;">
<option value="prod">Prod — ready for analyst use</option>
<option value="poc">POC — try-before-you-buy</option>
<option value="coming-soon">Coming soon — visible but not usable yet</option>
<option value="draft">Draft — admin-only, hidden from analysts</option>
</select>
</div>
<div style="flex:1;">
<label class="form-label">Category <span class="optional">(optional)</span></label>
<input id="edp-category" type="text" class="form-input"
placeholder="e.g. Sessions &amp; Traffic">
</div>
</div>
<div class="form-group" style="display:flex; gap:12px;">
<div style="flex:1;">
<label class="form-label">Icon</label>
<input id="edp-icon" type="text" class="form-input" maxlength="4">
</div>
<div style="flex:1;">
<label class="form-label">Color</label>
<div class="cf-palette-row" data-target="edp-color"></div>
<input id="edp-color" type="color" class="form-input"
style="height:32px; padding:2px; cursor:pointer;">
</div>
</div>
{# Cover image upload (v50). Preview shows the current cover when
one is set, with an inline Remove button that sends ""
(empty-string-means-clear contract in the PUT endpoint). #}
<div class="form-group">
<label class="form-label">Cover image <span class="optional">(optional)</span></label>
<div style="display:flex; gap:12px; align-items:center;">
<div id="edp-cover-preview"
style="width:120px; height:72px; border-radius:8px;
border:1px solid var(--border); background:#f1f5f9;
display:flex; align-items:center; justify-content:center;
font-size:12px; color:var(--text-secondary); overflow:hidden;">
No image
</div>
<div style="flex:1;">
<input id="edp-cover-file" type="file"
accept="image/png,image/jpeg,image/gif,image/webp"
onchange="onCoverFilePicked(this, 'edp')">
<input id="edp-cover-url" type="hidden" value="">
<div class="form-hint">
PNG / JPEG / GIF / WebP, max 5 MiB.
<button type="button" class="btn btn-secondary btn-sm"
style="margin-left:6px; padding:2px 8px; font-size:11px;"
onclick="clearCoverImage('edp')">Remove image</button>
</div>
</div>
</div>
</div>
{# v50: inline +Add-tables picker. The old "Add tables" button
closed this Edit modal and opened Bulk Assign, losing the
admin's mental context. The picker below stays inline: a
multi-select of unassigned tables + an Add-selected button
that POSTs without closing the modal. The legacy Bulk Assign
modal stays available from the section header outside the
Edit modal. #}
<div class="form-group" style="border-top:1px solid var(--border); padding-top:14px; margin-top:14px;">
<label class="form-label" style="display:flex; align-items:center; justify-content:space-between;">
<span>Tables in this package</span>
<button class="btn btn-secondary" type="button" style="padding:4px 10px; font-size:12px;"
onclick="toggleEdpInlineAdd()">
+ Add tables
</button>
</label>
<div id="edp-inline-add" style="display:none; margin-bottom:8px; padding:8px;
border:1px dashed var(--border); border-radius:6px; background:var(--background);">
<div style="display:flex; gap:6px; margin-bottom:6px;">
<input id="edp-inline-add-search" type="search" class="form-input"
placeholder="Filter unassigned tables…"
style="flex:1;" oninput="filterEdpInlineAdd()">
</div>
<div id="edp-inline-add-list"
style="max-height:180px; overflow-y:auto; padding:4px;
background:var(--surface); border:1px solid var(--border-light);
border-radius:4px;">
<div style="padding:8px; color:var(--text-secondary); font-size:12px;">
Loading unassigned tables…
</div>
</div>
<div style="display:flex; justify-content:flex-end; gap:6px; margin-top:6px;">
<button class="btn btn-secondary btn-sm" type="button"
onclick="toggleEdpInlineAdd()">Cancel</button>
<button class="btn btn-primary btn-sm" type="button"
id="edp-inline-add-submit-btn"
onclick="submitEdpInlineAdd()">Add selected</button>
</div>
</div>
<div id="edp-tables-list" class="card-body" style="padding:0; max-height:240px; overflow-y:auto; border:1px solid var(--border); border-radius:8px;"></div>
</div>
{# Inline Group Access matrix — admin can mark this package as
available / required for each group right from the edit
modal (mirroring the Create flow). Lazy-loaded on first
open of the <details>; pre-filled from the package's
existing resource_grants rows so the dropdowns reflect
reality. On save, diffs vs the original snapshot and
emits POST / DELETE /api/admin/grants to bring the server
in sync. Closes the "Data Package nikde nejde nastavit
required" user-reported gap. #}
<details class="form-group" id="edp-rbac-details"
style="border:1px solid var(--border); border-radius:8px; padding:8px 12px;">
<summary style="cursor:pointer; font-weight:600; font-size:13px; user-select:none;">
Group access
<span style="color:var(--text-secondary); font-weight:normal; font-size:12px;">— mark required / available per group</span>
</summary>
<p class="form-hint" style="margin:8px 0;">
<em>required</em> auto-installs the package into the group's
stack on next pull; <em>available</em> lets the group see + opt
in from /catalog; <em>(no grant)</em> hides it. Changes apply
when you click <strong>Save changes</strong>.
</p>
<div id="edp-rbac-rows"
style="display:flex; flex-direction:column; gap:6px; max-height:30vh; overflow-y:auto;">
<div class="loading" style="padding:6px; color:var(--text-secondary); font-size:12px;">
Click to load groups…
</div>
</div>
</details>
</div>
<div class="modal-footer" style="justify-content:space-between;">
<button class="btn" type="button" style="color:#b91c1c; border-color:#fca5a5;"
onclick="deleteEditDataPackage()">Delete package</button>
<div style="display:flex; gap:8px;">
<button class="btn btn-secondary" onclick="closeEditDataPackageModal()">Cancel</button>
<button class="btn btn-primary" id="edp-save-btn" onclick="submitEditDataPackage()">Save changes</button>
</div>
</div>
</div>
</div>
{# Group-by-bucket preview modal — replaces the old confirm(). Lets
the admin see each distinct bucket, its table count, the resulting
slug, and whether it would be skipped because a package with that
slug already exists. Uncheck rows to opt out individually. #}
<div class="modal-overlay" id="groupByBucketPreviewModal" style="display:none;">
<div class="modal" style="max-width:640px;">
<div class="modal-header">
<h2>Create a Data Package per bucket</h2>
<button class="modal-close" onclick="closeGroupByBucketModal()">×</button>
</div>
<div class="modal-body">
<p class="form-hint" style="margin-bottom:12px;">
One package will be created per checked bucket below.
Existing tables in any package are left in place (the junction
INSERT is idempotent).
</p>
<div id="gbb-rows"
style="display:flex; flex-direction:column; gap:6px; max-height:50vh; overflow-y:auto;
border:1px solid var(--border); border-radius:8px; padding:8px;">
<div class="loading" style="padding:8px; color:var(--text-secondary); font-size:13px;">
Loading buckets…
</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeGroupByBucketModal()">Cancel</button>
<button class="btn btn-primary" id="gbb-submit-btn"
onclick="submitGroupByBucket()">Create checked</button>
</div>
</div>
</div>
<div class="modal-overlay" id="bulkAssignTablesModal" style="display:none;">
<div class="modal" style="max-width:680px;">
<div class="modal-header">
<h2>Bulk assign tables to package</h2>
<button class="modal-close" onclick="closeBulkAssignModal()">×</button>
</div>
<div class="modal-body">
<div class="form-group">
<label class="form-label" for="bulk-assign-package">Target Data Package</label>
<select id="bulk-assign-package" class="form-input"
style="width:100%;"
onchange="refreshBulkAssignAvailability()">
<option value="">— Loading packages —</option>
</select>
<div class="form-hint">Pick the package the selected tables should be bundled into.</div>
</div>
<div class="form-group">
<label class="form-label">Tables</label>
<div style="display:flex; gap:8px; align-items:center; margin:6px 0;">
<input id="bulk-assign-search" type="search" class="form-input"
placeholder="Filter tables by name / source…"
style="flex:1;" oninput="filterBulkAssignList()">
<button type="button" class="btn btn-secondary btn-sm"
onclick="toggleAllBulkAssign(true)">Select visible</button>
<button type="button" class="btn btn-secondary btn-sm"
onclick="toggleAllBulkAssign(false)">Clear</button>
</div>
<div id="bulk-assign-list"
style="border:1px solid var(--border); border-radius:6px;
max-height:50vh; overflow-y:auto; padding:6px;">
<div style="padding:12px; color:#5f6368;">Loading tables…</div>
</div>
<div class="form-hint" id="bulk-assign-hint">
Already-assigned tables are checked + disabled so re-submitting
this modal is a no-op against them.
</div>
</div>
</div>
<div class="modal-footer">
<button class="btn btn-secondary" onclick="closeBulkAssignModal()">Cancel</button>
<button class="btn btn-primary" id="bulk-assign-submit-btn"
onclick="submitBulkAssign()">
Assign selected
</button>
</div>
</div>
</div>
{% endblock %}
{% block scripts %}
<script>
/* ═══════════════════════════════════════════════════════════════
Admin Tables - JavaScript
═══════════════════════════════════════════════════════════════ */
// ── Register-new-table dropdown ─────────────────────────────
// Replaces the per-connector tab nav. The user picks the connector
// here only to choose which register modal to open; otherwise the
// page is package-centric and source_type is just an inline tag.
function toggleRegisterNewTableMenu(evt) {
if (evt) evt.stopPropagation();
var menu = document.getElementById('registerNewTableMenu');
if (!menu) return;
menu.style.display = (menu.style.display === 'block') ? 'none' : 'block';
}
function closeRegisterNewTableMenu() {
var menu = document.getElementById('registerNewTableMenu');
if (menu) menu.style.display = 'none';
}
document.addEventListener('click', function(e) {
var btn = document.getElementById('registerNewTableBtn');
var menu = document.getElementById('registerNewTableMenu');
if (!btn || !menu) return;
if (btn.contains(e.target) || menu.contains(e.target)) return;
menu.style.display = 'none';
});
// State
let registryData = null;
let registryVersion = null;
let currentEditTableId = null;
// ── Toast notification ──────────────────────────────────────
function showToast(message, type) {
var toast = document.getElementById('toast');
var icon = document.getElementById('toastIcon');
var msg = document.getElementById('toastMessage');
toast.className = 'toast toast-' + type;
msg.textContent = message;
if (type === 'success') {
icon.innerHTML = '<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="#10B77F" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M22 11.08V12a10 10 0 1 1-5.93-9.14"/><polyline points="22 4 12 14.01 9 11.01"/></svg>';
} else {
icon.innerHTML = '<svg width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="#EA580C" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="10"/><line x1="15" y1="9" x2="9" y2="15"/><line x1="9" y1="9" x2="15" y2="15"/></svg>';
}
// Show
requestAnimationFrame(function() {
toast.classList.add('visible');
});
// Hide after 4 seconds
setTimeout(function() {
toast.classList.remove('visible');
}, 4000);
}
// ── Format helpers ──────────────────────────────────────────
function formatNumber(n) {
if (n == null) return '-';
if (n >= 1000000) return (n / 1000000).toFixed(1) + 'M';
if (n >= 1000) return n.toLocaleString();
return String(n);
}
function formatSize(bytes) {
if (bytes == null || bytes === 0) return '-';
if (bytes >= 1073741824) return (bytes / 1073741824).toFixed(1) + ' GB';
if (bytes >= 1048576) return (bytes / 1048576).toFixed(1) + ' MB';
if (bytes >= 1024) return (bytes / 1024).toFixed(1) + ' KB';
return bytes + ' B';
}
function escapeHtml(str) {
if (!str) return '';
var div = document.createElement('div');
div.textContent = str;
return div.innerHTML;
}
/**
* Escape a string for safe inclusion inside a single- OR double-quoted
* HTML attribute. Unlike `escapeHtml` (which goes through textContent →
* innerHTML and only escapes `<`/`>`/`&`), this also escapes both quote
* characters so the value can't break out of the attribute. Issue #265.
*/
function escapeHtmlAttr(str) {
if (str === null || str === undefined) return '';
return String(str)
.replace(/&/g, '&amp;')
.replace(/</g, '&lt;')
.replace(/>/g, '&gt;')
.replace(/"/g, '&quot;')
.replace(/'/g, '&#39;');
}
// Defensive normalization for descriptions registered via shell-quoting
// tooling that injected literal backslash escapes (e.g. `Don\'t`, `\n`).
// Mirrors _unescape_shell_quoting in app/api/admin.py — applied at render
// time so already-stored corrupt rows still display readably.
function unescapeShellQuoting(s) {
if (!s) return s;
// Order matters: protect real backslashes via NUL sentinel first,
// unescape the well-known sequences, then restore real backslashes.
return s
.replace(/\\\\/g, '')
.replace(/\\n/g, '\n')
.replace(/\\r/g, '\r')
.replace(/\\t/g, '\t')
.replace(/\\'/g, "'")
.replace(/\\"/g, '"')
.replace(//g, '\\');
}
// C3: removed dead Discovery panel JS. The global Discovery card +
// its #discoverBtn / #discoveryResults DOM hooks were removed when
// the per-tab UI landed; per-tab Discover/List datalist helpers live
// in the per-source shims further down. The legacy "Register" button
// rendered per discovery row also went away — operators register
// through the per-tab Register modals.
// ── Registration Modal ──────────────────────────────────────
// Server-rendered marker so the JS knows the instance's data source
// type. Lives on <body> after C3 (previously on the now-removed
// #registerModal).
var DATA_SOURCE_TYPE = document.body.dataset.sourceType || 'keboola';
function openRegisterModal(arg) {
// Phase E + F: dispatch by string argument.
// 'bigquery' → BQ tab Register button → #registerBqModal.
// 'keboola' → Keboola tab Register button → #registerKeboolaModal.
if (arg === 'bigquery') {
return _openBqRegisterModal({});
}
if (arg === 'keboola') {
return _openKeboolaTabRegisterModal();
}
// Fallback when called with no argument — pick by instance type.
if (DATA_SOURCE_TYPE === 'bigquery') {
return _openBqRegisterModal({});
}
return _openKeboolaTabRegisterModal();
}
function _openBqRegisterModal(table) {
// BQ uses a manual-entry form (no discovery panel for BQ in M1).
// `table` may be partially populated by a future M2 prefill —
// tolerate either an empty call or a {bucket, source_table, ...}
// shape from a hypothetical future prefill.
table = table || {};
document.getElementById('bqDataset').value = table.bucket || '';
document.getElementById('bqSourceTable').value = table.source_table || table.name || '';
document.getElementById('bqViewName').value = table.name || '';
document.getElementById('bqDescription').value = '';
document.getElementById('bqFolder').value = '';
document.getElementById('bqSyncSchedule').value = '';
var summary = document.getElementById('bqPrecheckSummary');
if (summary) summary.style.display = 'none';
var btn = document.getElementById('registerBqSubmitBtn');
btn.disabled = false;
btn.textContent = 'Register Table';
btn.onclick = registerBqTable;
document.getElementById('registerBqModal').classList.add('active');
}
// C3: removed dead helpers _openKeboolaRegisterModal /
// closeRegisterModal that drove the now-deleted #registerModal.
// The Phase F #registerKeboolaModal owns the Keboola flow now.
function closeRegisterBqModal() {
document.getElementById('registerBqModal').classList.remove('active');
}
// ── Keboola tab register modal (Phase F1) ──────────────────────
function _openKeboolaTabRegisterModal() {
// Reset form to defaults each open. Whole mode is the default
// (Q2='whole'); the kb-source-table fields are visible.
var modal = document.getElementById('registerKeboolaModal');
if (!modal) return;
var radio = modal.querySelector('input[name="kbSyncMode"][value="whole"]');
if (radio) radio.checked = true;
['kbViewName', 'kbBucket', 'kbSourceTable', 'kbSourceQuery',
'kbSyncSchedule', 'kbDescription', 'kbFolder', 'kbPrimaryKey',
// v26 Direct-extract fields — reset so a stale value from a prior
// open doesn't leak into the next registration.
'kbIncrementalWindowDays', 'kbMaxHistoryDays', 'kbPartitionBy',
'kbInitialLoadChunkDays', 'kbWhereFilters'].forEach(function(id) {
var el = document.getElementById(id);
if (el) el.value = '';
});
// Default v26 dropdowns
var kbStrategyEl = document.getElementById('kbStrategy');
if (kbStrategyEl) kbStrategyEl.value = 'full_refresh';
var kbGranEl = document.getElementById('kbPartitionGranularity');
if (kbGranEl) kbGranEl.value = 'month';
onKbSyncModeChange(); // apply default visibility
var btn = document.getElementById('registerKeboolaSubmitBtn');
btn.disabled = false;
btn.textContent = 'Register';
modal.classList.add('active');
}
function closeRegisterKeboolaModal() {
document.getElementById('registerKeboolaModal').classList.remove('active');
}
function _getKbSyncMode() {
var el = document.querySelector('input[name="kbSyncMode"]:checked');
return el ? el.value : 'whole';
}
function onKbSyncModeChange() {
var mode = _getKbSyncMode();
// Whole + Direct both need bucket/source_table inputs; only Custom hides them
document.querySelectorAll('.kb-source-table').forEach(function(el) {
el.style.display = (mode === 'custom') ? 'none' : '';
});
document.querySelectorAll('.kb-source-custom').forEach(function(el) {
el.style.display = (mode === 'custom') ? '' : 'none';
});
// v26 strategy panel — Direct mode only
document.querySelectorAll('.kb-direct-only').forEach(function(el) {
el.style.display = (mode === 'direct') ? '' : 'none';
});
if (mode === 'direct') onKbStrategyChange();
}
function _getKbStrategy() {
var el = document.getElementById('kbStrategy');
return el ? el.value : 'full_refresh';
}
function _kbStrategyVisibility(el, s) {
// Single-pass visibility check that handles elements tagged with
// multiple strategy classes (e.g. window_days has both
// kb-strategy-incremental + kb-strategy-partitioned because the
// field is shared). Using two separate forEach loops would race —
// the second loop hiding what the first showed.
if (el.classList.contains('kb-strategy-not-incremental')) {
return s !== 'incremental';
}
// Element is visible if its class list mentions the active strategy
return el.classList.contains('kb-strategy-' + s);
}
function onKbStrategyChange() {
var s = _getKbStrategy();
document.querySelectorAll(
'.kb-direct-only .kb-strategy-incremental, ' +
'.kb-direct-only .kb-strategy-partitioned, ' +
'.kb-direct-only .kb-strategy-not-incremental'
).forEach(function(el) {
el.style.display = _kbStrategyVisibility(el, s) ? '' : 'none';
});
}
function _intOrNull(id) {
var raw = (document.getElementById(id).value || '').trim();
if (!raw) return null;
var n = parseInt(raw, 10);
return isNaN(n) ? null : n;
}
function _parseFiltersJSONOrThrow(id) {
var raw = (document.getElementById(id).value || '').trim();
if (!raw) return null;
try {
return JSON.parse(raw);
} catch (e) {
throw new Error('where_filters: invalid JSON — ' + e.message);
}
}
function _buildKeboolaPayload() {
// Three modes:
// whole / custom → query_mode='materialized' (DuckDB Keboola extension)
// direct → query_mode='local' + v26 sync_strategy fields
// (Storage API SDK, supports incremental/partitioned/where_filters)
var mode = _getKbSyncMode();
var viewName = (document.getElementById('kbViewName').value || '').trim();
var bucket = (document.getElementById('kbBucket').value || '').trim();
var sourceTable = (document.getElementById('kbSourceTable').value || '').trim();
var pk = (document.getElementById('kbPrimaryKey').value || '').trim();
var primaryKey = pk
? pk.split(',').map(function(s) { return s.trim(); }).filter(Boolean)
: [];
if (mode === 'direct') {
var strategy = _getKbStrategy();
var payload = {
name: viewName || sourceTable,
source_type: 'keboola',
query_mode: 'local',
bucket: bucket,
source_table: sourceTable,
primary_key: primaryKey,
sync_strategy: strategy,
sync_schedule: (document.getElementById('kbSyncSchedule').value || '').trim() || null,
description: (document.getElementById('kbDescription').value || '').trim() || null,
folder: (document.getElementById('kbFolder').value || '').trim() || null,
};
if (strategy === 'incremental' || strategy === 'partitioned') {
payload.incremental_window_days = _intOrNull('kbIncrementalWindowDays');
payload.max_history_days = _intOrNull('kbMaxHistoryDays');
}
if (strategy === 'partitioned') {
payload.partition_by = (document.getElementById('kbPartitionBy').value || '').trim() || null;
payload.partition_granularity = document.getElementById('kbPartitionGranularity').value || null;
payload.initial_load_chunk_days = _intOrNull('kbInitialLoadChunkDays');
}
if (strategy !== 'incremental') {
payload.where_filters = _parseFiltersJSONOrThrow('kbWhereFilters');
}
return payload;
}
// Materialized paths (whole / custom)
var common = {
name: viewName || sourceTable,
source_type: 'keboola',
query_mode: 'materialized',
primary_key: primaryKey,
sync_schedule: (document.getElementById('kbSyncSchedule').value || '').trim() || null,
description: (document.getElementById('kbDescription').value || '').trim() || null,
folder: (document.getElementById('kbFolder').value || '').trim() || null,
};
if (mode === 'custom') {
return Object.assign({}, common, {
source_query: (document.getElementById('kbSourceQuery').value || '').trim(),
});
}
// Whole — synthesize SELECT * FROM kbc."bucket"."table".
return Object.assign({}, common, {
bucket: bucket,
source_table: sourceTable,
source_query: 'SELECT * FROM kbc."' + bucket + '"."' + sourceTable + '"',
});
}
function registerKeboolaTable() {
var btn = document.getElementById('registerKeboolaSubmitBtn');
btn.disabled = true;
btn.textContent = 'Registering...';
var payload;
try {
payload = _buildKeboolaPayload();
} catch (e) {
// _parseFiltersJSONOrThrow surfaces malformed JSON before the
// POST so the operator sees the parse error inline rather than
// a 422 from the server. Re-enable the submit button.
showToast('' + e.message, 'error');
btn.disabled = false;
btn.textContent = 'Register';
return;
}
fetch('/api/admin/register-table', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
})
.then(function(r) {
if (!r.ok) {
return r.json().then(function(d) {
throw new Error(d.detail || d.error || 'Registration failed');
});
}
return r.json();
})
.then(function() {
closeRegisterKeboolaModal();
showToast('Table registered', 'success');
loadRegistry();
})
.catch(function(err) {
showToast('' + err.message, 'error');
})
.finally(function() {
btn.disabled = false;
btn.textContent = 'Register';
});
}
// Discovery shims — the existing /api/admin/discover-tables endpoint
// already routes by the instance's data_source.type (returning Keboola
// tables when the instance is Keboola-typed); the source=keboola query
// param is informational. Hidden behind `data_source_type == 'keboola'`
// because on a BQ-typed instance the endpoint would return BQ-shaped
// data (wrong shape, confusing); operators fall back to manual entry
// for cross-source registration. Future work: extend the endpoint to
// accept an explicit ?source= override so secondary-source registration
// works in both directions and we can remove this guard.
{% if data_source_type == 'keboola' %}
function discoverKeboolaBuckets(datalistId) {
fetch('/api/admin/discover-tables?source=keboola')
.then(function(r) {
if (!r.ok) return r.json().then(function(d) {
throw new Error(d.detail || d.error || 'Keboola discovery failed');
});
return r.json();
})
.then(function(data) {
var dl = document.getElementById(datalistId);
if (!dl) return;
dl.innerHTML = '';
// Endpoint may return either {buckets:[...]}, {datasets:[...]}
// or {tables:[...]} depending on routing; project to a flat
// bucket-id list. Keboola path returns tables → derive uniq
// bucket_ids.
var buckets = data.buckets || data.datasets;
if (!buckets && Array.isArray(data.tables)) {
var seen = {};
buckets = [];
data.tables.forEach(function(t) {
var b = t.bucket_id || (t.bucket && t.bucket.id);
if (b && !seen[b]) { seen[b] = 1; buckets.push(b); }
});
}
(buckets || []).forEach(function(b) {
var o = document.createElement('option');
o.value = (typeof b === 'string') ? b : (b.id || b.bucket_id || '');
dl.appendChild(o);
});
showToast('Loaded ' + (dl.children.length) + ' buckets', 'success');
})
.catch(function(err) {
showToast('' + err.message, 'error');
});
}
function discoverKeboolaTables(bucketInputId, tablesDatalistId) {
var bucketEl = document.getElementById(bucketInputId);
var bucket = bucketEl ? (bucketEl.value || '').trim() : '';
if (!bucket) {
showToast('Fill bucket first', 'error');
return;
}
fetch('/api/admin/discover-tables?source=keboola&bucket=' + encodeURIComponent(bucket))
.then(function(r) {
if (!r.ok) return r.json().then(function(d) {
throw new Error(d.detail || d.error || 'Keboola table discovery failed');
});
return r.json();
})
.then(function(data) {
var dl = document.getElementById(tablesDatalistId);
if (!dl) return;
dl.innerHTML = '';
var tables = data.tables || [];
// Filter to the selected bucket if endpoint didn't.
tables.filter(function(t) {
var b = t.bucket_id || (t.bucket && t.bucket.id);
return !bucket || !b || b === bucket;
}).forEach(function(t) {
var o = document.createElement('option');
var name = (typeof t === 'string') ? t : (t.name || t.id || '');
// Strip bucket prefix if present.
if (name.indexOf(bucket + '.') === 0) name = name.substring(bucket.length + 1);
o.value = name;
dl.appendChild(o);
});
showToast('Loaded ' + dl.children.length + ' tables in ' + bucket, 'success');
})
.catch(function(err) {
showToast('' + err.message, 'error');
});
}
{% endif %}{# data_source_type == 'keboola' discover helpers #}
// ── Keboola edit-modal sync-mode helpers (always needed — radio buttons
// are rendered for all instance types) ───────────────────────────────
// Pre-rewrite this whole block lived inside the Jinja
// data_source_type guard above, which was fine when the modal was
// only reachable from the (then) Keboola tab. With the package-
// centric layout, Edit on any Keboola row opens this modal
// regardless of the instance's primary data_source.type, so the
// lifecycle helpers must be available unconditionally. The
// discover/prefill helpers (which hit the keboola-typed
// /api/admin/discover-tables endpoint) stay gated above + below.
function _getEditKbSyncMode() {
var el = document.querySelector('input[name="editKbSyncMode"]:checked');
return el ? el.value : 'whole';
}
function onEditKbSyncModeChange() {
var mode = _getEditKbSyncMode();
document.querySelectorAll('.editkb-source-table').forEach(function(el) {
el.style.display = (mode === 'custom') ? 'none' : '';
});
document.querySelectorAll('.editkb-source-custom').forEach(function(el) {
el.style.display = (mode === 'custom') ? '' : 'none';
});
document.querySelectorAll('.editkb-direct-only').forEach(function(el) {
el.style.display = (mode === 'direct') ? '' : 'none';
});
if (mode === 'direct') onEditKbStrategyChange();
}
function _getEditKbStrategy() {
var el = document.getElementById('editKbStrategy');
return el ? el.value : 'full_refresh';
}
function _editKbStrategyVisibility(el, s) {
if (el.classList.contains('editkb-strategy-not-incremental')) {
return s !== 'incremental';
}
return el.classList.contains('editkb-strategy-' + s);
}
function onEditKbStrategyChange() {
var s = _getEditKbStrategy();
document.querySelectorAll(
'.editkb-direct-only .editkb-strategy-incremental, ' +
'.editkb-direct-only .editkb-strategy-partitioned, ' +
'.editkb-direct-only .editkb-strategy-not-incremental'
).forEach(function(el) {
el.style.display = _editKbStrategyVisibility(el, s) ? '' : 'none';
});
}
function _setEditKbRadio(value) {
var el = document.querySelector('input[name="editKbSyncMode"][value="' + value + '"]');
if (el) el.checked = true;
}
function openEditKeboolaModal(table) {
// Populate fields from a registry row. Mode detection:
// - query_mode='local' → 'direct' (v26 sync-strategy path)
// - query_mode='materialized' + auto SELECT * → 'whole'
// - query_mode='materialized' + custom SQL → 'custom'
table = table || {};
document.getElementById('editKbTableId').value = table.id || '';
var bucket = table.bucket || '';
var sourceTable = table.source_table || '';
var sourceQuery = table.source_query || '';
var queryMode = table.query_mode || 'materialized';
var isAutoSelectStar = false;
if (sourceQuery && bucket && sourceTable) {
var auto = 'SELECT * FROM kbc."' + bucket + '"."' + sourceTable + '"';
isAutoSelectStar = sourceQuery.replace(/\s+/g, ' ').trim() === auto;
}
var mode;
if (queryMode === 'local') {
mode = 'direct';
} else if (sourceQuery && !isAutoSelectStar) {
mode = 'custom';
} else {
mode = 'whole';
}
_setEditKbRadio(mode);
document.getElementById('editKbBucket').value = bucket;
document.getElementById('editKbSourceTable').value = sourceTable;
document.getElementById('editKbSourceQuery').value = sourceQuery;
document.getElementById('editKbSyncSchedule').value = table.sync_schedule || '';
document.getElementById('editKbDescription').value = table.description || '';
document.getElementById('editKbFolder').value = table.folder || '';
document.getElementById('editKbPrimaryKey').value = (table.primary_key || []).join(', ');
// v26 fields
document.getElementById('editKbStrategy').value = table.sync_strategy || 'full_refresh';
document.getElementById('editKbIncrementalWindowDays').value =
table.incremental_window_days != null ? table.incremental_window_days : '';
document.getElementById('editKbMaxHistoryDays').value =
table.max_history_days != null ? table.max_history_days : '';
document.getElementById('editKbPartitionBy').value = table.partition_by || '';
document.getElementById('editKbPartitionGranularity').value =
table.partition_granularity || 'month';
document.getElementById('editKbInitialLoadChunkDays').value =
table.initial_load_chunk_days != null ? table.initial_load_chunk_days : '';
document.getElementById('editKbWhereFilters').value =
table.where_filters ? JSON.stringify(table.where_filters, null, 2) : '';
onEditKbSyncModeChange();
var btn = document.getElementById('editKeboolaSubmitBtn');
btn.disabled = false;
btn.textContent = 'Save Changes';
// Reset + hydrate the Data Packages chip-input (parity with the
// legacy + BQ edit modals).
_editKbOriginalPackageIds = new Set();
var kbChipHost = document.getElementById('editKbPackagesChips');
if (kbChipHost) {
if (typeof kbChipHost.clearChips === 'function') kbChipHost.clearChips();
_hydrateEditPackagesChips(table.id, kbChipHost, _editKbOriginalPackageIds);
}
document.getElementById('editKeboolaModal').classList.add('active');
}
function closeEditKeboolaModal() {
_closeEditModalById('editKeboolaModal');
}
function _editIntOrNull(id) {
var raw = (document.getElementById(id).value || '').trim();
if (!raw) return null;
var n = parseInt(raw, 10);
return isNaN(n) ? null : n;
}
function _editParseFiltersJSONOrThrow(id) {
var raw = (document.getElementById(id).value || '').trim();
if (!raw) return null;
try {
return JSON.parse(raw);
} catch (e) {
throw new Error('where_filters: invalid JSON — ' + e.message);
}
}
function _buildKeboolaEditPayload() {
var mode = _getEditKbSyncMode();
var bucket = (document.getElementById('editKbBucket').value || '').trim();
var sourceTable = (document.getElementById('editKbSourceTable').value || '').trim();
var pk = (document.getElementById('editKbPrimaryKey').value || '').trim();
var primaryKey = pk
? pk.split(',').map(function(s) { return s.trim(); }).filter(Boolean)
: [];
if (mode === 'direct') {
var strategy = _getEditKbStrategy();
var payload = {
query_mode: 'local',
bucket: bucket,
source_table: sourceTable,
primary_key: primaryKey,
sync_strategy: strategy,
sync_schedule: (document.getElementById('editKbSyncSchedule').value || '').trim() || null,
description: (document.getElementById('editKbDescription').value || '').trim() || null,
folder: (document.getElementById('editKbFolder').value || '').trim() || null,
// PUT requires explicit nulls to clear v26 fields when switching
// away from a strategy that used them — otherwise the merged-dict
// path keeps the stale value.
incremental_window_days: null,
max_history_days: null,
partition_by: null,
partition_granularity: null,
initial_load_chunk_days: null,
where_filters: null,
};
if (strategy === 'incremental' || strategy === 'partitioned') {
payload.incremental_window_days = _editIntOrNull('editKbIncrementalWindowDays');
payload.max_history_days = _editIntOrNull('editKbMaxHistoryDays');
}
if (strategy === 'partitioned') {
payload.partition_by = (document.getElementById('editKbPartitionBy').value || '').trim() || null;
payload.partition_granularity = document.getElementById('editKbPartitionGranularity').value || null;
payload.initial_load_chunk_days = _editIntOrNull('editKbInitialLoadChunkDays');
}
if (strategy !== 'incremental') {
payload.where_filters = _editParseFiltersJSONOrThrow('editKbWhereFilters');
}
return payload;
}
var common = {
query_mode: 'materialized',
primary_key: primaryKey,
sync_schedule: (document.getElementById('editKbSyncSchedule').value || '').trim() || null,
description: (document.getElementById('editKbDescription').value || '').trim() || null,
folder: (document.getElementById('editKbFolder').value || '').trim() || null,
};
if (mode === 'custom') {
return Object.assign({}, common, {
source_query: (document.getElementById('editKbSourceQuery').value || '').trim(),
});
}
return Object.assign({}, common, {
bucket: bucket,
source_table: sourceTable,
source_query: 'SELECT * FROM kbc."' + bucket + '"."' + sourceTable + '"',
});
}
function saveKeboolaTabEdit() {
var btn = document.getElementById('editKeboolaSubmitBtn');
var tableId = document.getElementById('editKbTableId').value;
if (!tableId) {
showToast('Missing table id', 'error');
return;
}
btn.disabled = true;
btn.textContent = 'Saving...';
var payload;
try {
payload = _buildKeboolaEditPayload();
} catch (e) {
showToast('' + e.message, 'error');
btn.disabled = false;
btn.textContent = 'Save Changes';
return;
}
(async function () {
try {
var r = await fetch('/api/admin/registry/' + encodeURIComponent(tableId), {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
});
if (!r.ok) {
var d = await r.json().catch(function() { return {}; });
throw new Error(d.detail || d.error || 'Update failed');
}
// Diff Data Package chip selection vs original.
var kbFails = await _diffApplyPackageMembership(
tableId,
document.getElementById('editKbPackagesChips'),
_editKbOriginalPackageIds
);
if (kbFails > 0) {
showToast(kbFails + ' package membership change(s) failed', 'error');
}
closeEditKeboolaModal();
showToast('Table updated', 'success');
loadRegistry();
if (typeof loadAdminTablesLayout === 'function') loadAdminTablesLayout();
} catch (err) {
showToast('' + err.message, 'error');
} finally {
btn.disabled = false;
btn.textContent = 'Save Changes';
}
})();
}
// prefillFromKeboolaTable is already inside the outer Phase F2
// guard (the data_source_type keboola Jinja `if` above) — it only
// fires from a button inside the Keboola Edit modal that itself
// only renders when data_source_type == 'keboola'. The redundant
// nested guard was dropped during the rebase onto main; main
// never had it.
function prefillFromKeboolaTable(textareaId) {
// Edit-modal callers may pass an editKbBucket / editKbSourceTable
// pair instead of the Register modal's kbBucket / kbSourceTable.
// Detect the modal via the textarea id prefix.
var prefix = textareaId.indexOf('editKb') === 0 ? 'editKb' : 'kb';
var bucket = (document.getElementById(prefix + 'Bucket').value || '').trim();
var sourceTable = (document.getElementById(prefix + 'SourceTable').value || '').trim();
if (!bucket || !sourceTable) {
showToast('Fill bucket + source table first', 'error');
return;
}
var ta = document.getElementById(textareaId);
if (ta.value.trim()) {
if (!confirm('Replace existing SQL?')) return;
}
ta.value = 'SELECT *\nFROM kbc."' + bucket + '"."' + sourceTable + '"\nWHERE -- your filter here';
}
// C3: removed dead helper _buildKeboolaLegacyPayload — the Phase F
// _buildKeboolaPayload (above) replaced it.
function _getBqAccessMode() {
// Q1 radio. Default 'live' if nothing's checked yet (model-validator
// safety net for the initial render).
var el = document.querySelector('input[name="bqAccessMode"]:checked');
return el ? el.value : 'live';
}
function _getBqSyncMode() {
// Q2 radio (only meaningful when access mode is 'synced').
var el = document.querySelector('input[name="bqSyncMode"]:checked');
return el ? el.value : 'whole';
}
function _buildBigQueryPayload() {
// Two-question form maps to backend `query_mode`:
// live → query_mode='remote' (server auto-detects
// BASE TABLE vs VIEW)
// synced/whole → query_mode='materialized' (auto SELECT *)
// synced/custom → query_mode='materialized' (admin SQL)
// The UI never asks "Table vs View" — that's a server-side detail.
var accessMode = _getBqAccessMode();
var syncMode = _getBqSyncMode();
var viewName = document.getElementById('bqViewName').value.trim();
var description = document.getElementById('bqDescription').value.trim() || null;
var folder = document.getElementById('bqFolder').value.trim() || null;
var syncSchedule = document.getElementById('bqSyncSchedule').value.trim() || null;
if (accessMode === 'synced' && syncMode === 'custom') {
return {
name: viewName,
source_type: 'bigquery',
query_mode: 'materialized',
source_query: document.getElementById('bqSourceQuery').value.trim(),
profile_after_sync: false,
description: description,
folder: folder,
sync_schedule: syncSchedule,
};
}
var dataset = document.getElementById('bqDataset').value.trim();
var sourceTable = document.getElementById('bqSourceTable').value.trim();
if (accessMode === 'synced' && syncMode === 'whole') {
// Whole-table sync. We don't ship the project to the browser, so
// the SQL uses DuckDB three-part syntax against the materialize
// session's ATTACH alias. Native BQ dry-run can't parse this form
// (DuckDB identifier quoting), so the cost guardrail falls
// fail-open with a warning — operator who needs the cap to engage
// picks Custom query and writes backtick-quoted native BQ
// identifiers.
//
// Issue #266: also persist bucket+source_table on the row so the
// Edit modal can pre-fill those inputs on a subsequent open. Pre-
// #266 the register flow only sent source_query, leaving bucket=
// NULL in the registry; the Edit modal then loaded empty inputs
// and an admin saving with the visible empty fields would
// synthesize the broken `SELECT * FROM bq."".""` SQL.
return {
name: viewName || sourceTable,
source_type: 'bigquery',
query_mode: 'materialized',
bucket: dataset,
source_table: sourceTable,
source_query: 'SELECT * FROM bq."' + dataset + '"."' + sourceTable + '"',
profile_after_sync: false,
description: description,
folder: folder,
sync_schedule: syncSchedule,
};
}
// Live access — server auto-detects BASE TABLE vs VIEW at register
// time, so the UI doesn't make the operator pick.
return {
name: viewName || sourceTable,
source_type: 'bigquery',
bucket: dataset,
source_table: sourceTable,
query_mode: 'remote',
profile_after_sync: false,
description: description,
folder: folder,
sync_schedule: syncSchedule,
};
}
function onBqAccessModeChange() {
// Q1: toggle live ↔ synced. Q2 (sync mode) is meaningful only when
// synced; default it to 'whole' on first reveal so the form stays
// consistent without forcing the operator to click twice.
var accessMode = _getBqAccessMode();
var liveFields = document.querySelectorAll('.bq-access-live');
var syncedFields = document.querySelectorAll('.bq-access-synced');
liveFields.forEach(function(el) {
el.style.display = (accessMode === 'live') ? '' : 'none';
});
syncedFields.forEach(function(el) {
el.style.display = (accessMode === 'synced') ? '' : 'none';
});
// Q2 is fresh on first reveal; trigger its handler to apply the
// source-table vs source-custom visibility rules.
if (accessMode === 'synced') {
onBqSyncModeChange();
} else {
// Live mode: show source-table fields, hide custom-SQL textarea.
document.querySelectorAll('.bq-source-table').forEach(function(el) {
el.style.display = '';
});
document.querySelectorAll('.bq-source-custom').forEach(function(el) {
el.style.display = 'none';
});
}
}
function onBqSyncModeChange() {
// Q2: toggle whole-table ↔ custom-SQL. Whole reuses the
// dataset/source-table inputs (server-side SELECT *), Custom shows
// the SQL textarea. Only fires when access mode is already 'synced'.
var syncMode = _getBqSyncMode();
var tableFields = document.querySelectorAll('.bq-source-table');
var customFields = document.querySelectorAll('.bq-source-custom');
tableFields.forEach(function(el) {
el.style.display = (syncMode === 'whole') ? '' : 'none';
});
customFields.forEach(function(el) {
el.style.display = (syncMode === 'custom') ? '' : 'none';
});
}
// The discover / list-tables / prefill helpers are shared between the
// Register and Edit modals. The default datalist + input ids match the
// Register modal; Edit-modal callers pass their own ids so the
// autocomplete populates the right datalist and reads from the right
// dataset input.
function discoverBqDatasets(datalistId) {
// GET /api/admin/discover-tables (no `dataset`) → list datasets in
// the configured BQ project. Populate the named <datalist> so the
// dataset input shows them as autocomplete suggestions. Endpoint
// routes through BqAccess so config / auth errors come back as the
// standard {error, kind, details} shape.
var dlId = datalistId || 'bqDatasetList';
fetch('/api/admin/discover-tables')
.then(function(r) {
if (!r.ok) return r.json().then(function(d) {
var msg = (d && d.detail && d.detail.error) || (d && d.detail) || 'BQ discovery failed';
throw new Error(msg);
});
return r.json();
})
.then(function(data) {
var dl = document.getElementById(dlId);
dl.innerHTML = '';
(data.datasets || []).forEach(function(ds) {
var opt = document.createElement('option');
opt.value = ds.dataset_id;
opt.label = ds.full_id || ds.dataset_id;
dl.appendChild(opt);
});
showToast('Loaded ' + (data.count || 0) + ' datasets', 'success');
})
.catch(function(err) {
showToast(err.message || 'Discovery failed', 'error');
});
}
function discoverBqTables(datasetInputId, tablesDatalistId) {
// GET /api/admin/discover-tables?dataset=NAME → list tables + views
// in the dataset. Two-step (dataset → tables) avoids paying the
// per-dataset list_tables() cost up front on big projects.
var inId = datasetInputId || 'bqDataset';
var dlId = tablesDatalistId || 'bqTableList';
var dataset = document.getElementById(inId).value.trim();
if (!dataset) {
showToast('Fill Dataset first', 'error');
return;
}
fetch('/api/admin/discover-tables?dataset=' + encodeURIComponent(dataset))
.then(function(r) {
if (!r.ok) return r.json().then(function(d) {
var msg = (d && d.detail && d.detail.error) || (d && d.detail) || 'BQ table discovery failed';
throw new Error(msg);
});
return r.json();
})
.then(function(data) {
var dl = document.getElementById(dlId);
dl.innerHTML = '';
(data.tables || []).forEach(function(t) {
var opt = document.createElement('option');
opt.value = t.table_id;
// BQ entity types: TABLE, VIEW, MATERIALIZED_VIEW,
// EXTERNAL, SNAPSHOT — surface so the operator can
// pick the right entity option.
opt.label = t.table_id + (t.table_type ? ' (' + t.table_type + ')' : '');
dl.appendChild(opt);
});
showToast('Loaded ' + (data.count || 0) + ' tables in ' + dataset, 'success');
})
.catch(function(err) {
showToast(err.message || 'Table list failed', 'error');
});
}
function prefillFromTable(textareaId) {
// Convenience for the Custom SQL path: prompt for project.dataset.table
// and prefill `SELECT * FROM \`...\`` so the operator only edits the
// WHERE / projection. We can't know the project from the form (it's
// server-side config), so the prompt accepts both `dataset.table` and
// `project.dataset.table`. Empty / cancel → no change.
var taId = textareaId || 'bqSourceQuery';
var ta = document.getElementById(taId);
var existing = ta.value.trim();
if (existing && !confirm('SQL field is not empty — overwrite with prefill?')) {
return;
}
var ref = window.prompt(
'Enter the source table as `dataset.table` or `project.dataset.table` '
+ '(it will be wrapped in backticks):',
''
);
if (!ref) return;
ref = ref.trim().replace(/^`|`$/g, '');
var stub =
'SELECT *\n'
+ 'FROM `' + ref + '`\n'
+ 'WHERE -- TODO: filter or aggregate before the COPY runs (10 GiB cap)\n'
+ 'LIMIT 1000';
ta.value = stub;
}
// C3: removed dead entry points registerTable / _registerKeboolaTable
// that submitted the now-deleted #registerModal. The Keboola Register
// flow runs through #registerKeboolaModal → registerKeboolaTable().
function registerBqTable() {
// BQ tab modal entry point — runs the same two-step precheck →
// confirm flow against #registerBqSubmitBtn / #registerBqModal.
var btn = document.getElementById('registerBqSubmitBtn');
btn.disabled = true;
_registerBigQueryTable(btn);
}
function _registerBigQueryTable(btn) {
// Step 1 of two-step flow: precheck only. After precheck succeeds
// we surface row count / size / column count in the modal AND swap
// the primary button to "Register" — operator must explicitly
// click it to fire the actual register call. Pre-fix this chained
// precheck → register in a single promise, so the operator never
// got to review the summary before the row was committed (review
// IMPORTANT 4 in #119).
btn.textContent = 'Checking source...';
var payload = _buildBigQueryPayload();
var summary = document.getElementById('bqPrecheckSummary');
var summaryText = document.getElementById('bqPrecheckSummaryText');
fetch('/api/admin/register-table/precheck', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
})
.then(function(r) {
if (!r.ok) return r.json().then(function(d) { throw new Error(d.detail || 'Precheck failed'); });
return r.json();
})
.then(function(data) {
// Surface the source-side metrics so the operator can sanity-
// check the dataset/table before committing the row.
var t = data.table || {};
if (summary && summaryText) {
summary.style.display = '';
summaryText.textContent =
formatNumber(t.rows) + ' rows, ' +
formatSize(t.size_bytes) + ', ' +
(t.column_count != null ? t.column_count : (t.columns || []).length) + ' columns';
}
// Swap the primary button from "Register Table" (which fired
// the precheck) to "Register" (which fires the actual register
// call). Reusing the same modal means the operator can still
// hit Cancel or close without committing.
btn.disabled = false;
btn.textContent = 'Register';
btn.onclick = function() { _confirmRegisterBigQueryTable(btn, payload); };
})
.catch(function(err) {
showToast('Precheck failed: ' + err.message, 'error');
btn.disabled = false;
btn.textContent = 'Register Table';
});
}
function _confirmRegisterBigQueryTable(btn, payload) {
// Step 2 of the two-step BQ flow: actually POST /register-table.
// Reached only after the operator has reviewed the precheck
// summary in the modal and clicked the (relabeled) Register button.
btn.disabled = true;
btn.textContent = 'Registering...';
fetch('/api/admin/register-table', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
})
.then(function(r) {
// 200 (synchronous materialize) and 202 (BG materialize) are
// success; 500 with a `rebuild_failed` body is the new failure
// mode where the row is in the registry but the view wasn't
// created. Surface the error verbatim so the operator can fix
// it (typically a missing project in the overlay or auth
// missing bigquery.metadata.get).
return r.json().then(function(d) {
if (!r.ok) {
var msg = d.detail || d.message || 'Registration failed';
if (d.errors && d.errors.length) {
msg += ' (' + d.errors.length + ' error' + (d.errors.length > 1 ? 's' : '') + ')';
}
throw new Error(msg);
}
closeRegisterBqModal();
if (r.status === 202) {
showToast('Registration accepted, materializing in background', 'success');
} else {
showToast('Queryable as ' + (d.view_name || d.id || payload.name), 'success');
}
loadRegistry();
});
})
.catch(function(err) {
showToast('Registration failed: ' + err.message, 'error');
})
.finally(function() {
// Restore the modal's default button state — the operator may
// close + reopen the modal, and openRegisterModal also resets
// onclick back to registerBqTable.
btn.disabled = false;
btn.textContent = 'Register Table';
btn.onclick = registerBqTable;
});
}
// Back-compat entry point — kept for any external wiring that may
// still call openBigQueryRegisterModal(). Phase E: routes through
// the new BQ-tab modal.
function openBigQueryRegisterModal() {
openRegisterModal('bigquery');
}
// ── Edit Modal ──────────────────────────────────────────────
// Original mode captured at openEditModal-time so the access-mode
// change handler can detect "operator just switched it" vs "loaded
// from registry" and only surface the destructive-edit warning on a
// real change.
let _editOriginalQueryMode = null;
function _getEditBqAccessMode() {
var el = document.querySelector('input[name="editBqAccessMode"]:checked');
return el ? el.value : 'live';
}
function _getEditBqSyncMode() {
var el = document.querySelector('input[name="editBqSyncMode"]:checked');
return el ? el.value : 'whole';
}
function _setEditBqRadio(name, value) {
var el = document.querySelector('input[name="' + name + '"][value="' + value + '"]');
if (el) el.checked = true;
}
// ── Edit-modal architecture (v55 review of #L61) ──────────────────
// We dispatch by ``table.source_type`` to one of three per-source
// modals because their inner forms genuinely differ:
//
// * BQ: access-mode radios (live / synced), sync-mode radios
// (whole / custom SQL), dataset + source_table inputs.
// * Keboola: storage-API bucket + table inputs, sync_strategy enum,
// primary_key chip list, sync_schedule cron.
// * Generic: description, folder, strategy, primary_key (catch-all
// fallback for jira / future connectors).
//
// What's *already* shared lives in helpers:
// * ``_hydrateEditPackagesChips`` — Data Packages chip-input
// fill-in (all three modals).
// * ``_diffApplyPackageMembership`` — minimal POST/DELETE delta
// against /api/admin/data-packages/{id}/tables (all three save
// handlers).
// * ``_closeEditModalById`` — uniform close/reset (added v55).
//
// A fourth "one modal with conditional sections" design would shave
// ~40% of the form HTML but multiply state-hydration bugs (stale
// field state when source_type-switches re-show different inputs).
// The per-source modals + shared helpers pattern is the explicit
// architecture; see ``openEditModal`` for the dispatcher.
function openEditModal(table) {
var sourceType = (table.source_type || '').toLowerCase();
if (sourceType === 'bigquery') {
return _openEditBqModal(table);
}
if (sourceType === 'keboola') {
return openEditKeboolaModal(table);
}
return _openEditGenericModal(table);
}
// Single close/reset helper used by all three per-source modal
// wrappers. Resets currentEditTableId + _editOriginalQueryMode so a
// subsequent open() doesn't inherit stale state from the previous
// modal-source.
function _closeEditModalById(modalId) {
var el = document.getElementById(modalId);
if (el) el.classList.remove('active');
currentEditTableId = null;
_editOriginalQueryMode = null;
}
function _openEditBqModal(table) {
// Populate the BQ-tab Edit modal from a registry row. Mirror of
// populateEditModal's BQ branch from before C2 — same query_mode →
// radio mapping, same field bindings, but writes to the new
// #editBqModal scope.
currentEditTableId = table.id;
document.getElementById('editBqTableId').value = table.id || '';
var badge = document.getElementById('editBqSourceTypeBadge');
if (badge) badge.textContent = (table.source_type || 'bigquery');
var qmode = (table.query_mode || 'remote');
var sq = (table.source_query || '');
// Issue #266: register-time the whole-table branch omits
// bucket/source_table from the JSON payload (only source_query is
// sent), so existing whole-table materialized rows persist with
// bucket=NULL in the registry. When the Edit modal then reads
// table.bucket on those rows it gets null → empty input → admin
// saves a broken `SELECT * FROM bq."".""` SQL. Parse the source_query
// to recover the dataset+table for the pre-fill. Same regex shape
// we already use to detect whole-table mode below.
var SELECT_STAR_RE = /^\s*SELECT\s+\*\s+FROM\s+bq\."([^"]+)"\."([^"]+)"\s*$/i;
var isAutoSelectStar = SELECT_STAR_RE.test(sq);
if (qmode === 'materialized') {
_setEditBqRadio('editBqAccessMode', 'synced');
_setEditBqRadio('editBqSyncMode', isAutoSelectStar ? 'whole' : 'custom');
} else {
_setEditBqRadio('editBqAccessMode', 'live');
_setEditBqRadio('editBqSyncMode', 'whole');
}
// Pre-fill dataset/table inputs. For whole-table materialized rows
// without persisted bucket/source_table (pre-#266 register state),
// recover them from the SQL.
var preDataset = table.bucket || '';
var preSourceTable = table.source_table || '';
if (!preDataset && !preSourceTable && isAutoSelectStar) {
var m = sq.match(SELECT_STAR_RE);
if (m) { preDataset = m[1]; preSourceTable = m[2]; }
}
document.getElementById('editBqDataset').value = preDataset;
document.getElementById('editBqSourceTable').value = preSourceTable;
document.getElementById('editBqSourceQuery').value = sq;
document.getElementById('editBqSyncSchedule').value = table.sync_schedule || '';
document.getElementById('editBqDescription').value = table.description || '';
document.getElementById('editBqFolder').value = table.folder || '';
document.getElementById('editBqModeWarning').style.display = 'none';
_editOriginalQueryMode = qmode;
onEditBqAccessModeChange(); // fire to set field visibility
var btn = document.getElementById('editBqSubmitBtn');
btn.disabled = false;
btn.textContent = 'Save Changes';
// Reset + hydrate the Data Packages chip-input (parity with the
// legacy edit modal — see _openEditGenericModal for full notes).
_editBqOriginalPackageIds = new Set();
var bqChipHost = document.getElementById('editBqPackagesChips');
if (bqChipHost) {
if (typeof bqChipHost.clearChips === 'function') bqChipHost.clearChips();
_hydrateEditPackagesChips(table.id, bqChipHost, _editBqOriginalPackageIds);
}
document.getElementById('editBqModal').classList.add('active');
}
function closeEditBqModal() {
_closeEditModalById('editBqModal');
}
// Tracks the set of Data Package ids each edit modal was opened
// with, so each save handler can emit the minimal add/remove deltas
// against /api/admin/data-packages/{id}/tables instead of replaying
// every chip. One Set per modal because the BQ + Keboola + legacy
// modals can coexist in the DOM and need independent origin state.
var _editGenericOriginalPackageIds = new Set();
var _editBqOriginalPackageIds = new Set();
var _editKbOriginalPackageIds = new Set();
function _openEditGenericModal(table) {
// Catch-all fallback for source_types that aren't bigquery or keboola
// (e.g. jira, internal, future connectors). Carries the Strategy +
// Primary Key + Description + Folder fields and a Data Packages
// chip-input (parity with BQ). Renamed from _openEditLegacyModal
// — "Legacy" was misleading; this is the default, not a deprecated
// path.
currentEditTableId = table.id;
document.getElementById('editTableId').value = table.id || '';
var sourceType = (table.source_type || '').toLowerCase();
document.getElementById('editSourceTypeBadge').textContent = sourceType || '—';
document.getElementById('editDescription').value = table.description || '';
document.getElementById('editFolder').value = table.folder || '';
document.getElementById('editStrategy').value = table.sync_strategy || 'full_refresh';
document.getElementById('editPrimaryKey').value = (table.primary_key || []).join(', ');
_editOriginalQueryMode = null;
document.getElementById('editSubmitBtn').disabled = false;
// Hydrate Data Packages chip-input with current memberships.
// chip-input.js inits idempotently at DOMContentLoaded; we add
// chips via the public `.addChip()` API after a fresh memberships
// fetch, after clearing whatever was left from a prior edit.
_editGenericOriginalPackageIds = new Set();
var chipHost = document.getElementById('editGenericPackagesChips');
if (chipHost) {
// Reset chips between edits — clear hidden + visible state.
var hidden = chipHost.querySelector('input[type="hidden"]');
if (hidden) hidden.value = '[]';
var chipsRow = chipHost.querySelector('div');
if (chipsRow) chipsRow.innerHTML = '';
// selected[] is closure-private in chip-input.js — easiest way
// to clear it is Backspace simulation, but that's brittle.
// Instead we wire a `clearChips()` helper if exposed; fall
// back to wiping `dataset.selected` and re-init.
if (typeof chipHost.clearChips === 'function') {
chipHost.clearChips();
}
_hydrateGenericEditPackagesChips(table.id, chipHost);
}
document.getElementById('editModal').classList.add('active');
}
async function _hydrateGenericEditPackagesChips(tableId, chipHost) {
return _hydrateEditPackagesChips(tableId, chipHost, _editGenericOriginalPackageIds);
}
// Shared chip-input hydration for any edit modal. Pulls the package
// list, fetches members in parallel, and pre-fills `chipHost` with
// every package containing `tableId`. The matched ids are stashed
// in `originalIdsSet` so the save handler can diff against the
// current chip selection and emit minimal POST/DELETE calls.
// ~N small fetches; fine for instances with <50 packages.
async function _hydrateEditPackagesChips(tableId, chipHost, originalIdsSet) {
try {
var r = await fetch('/api/admin/data-packages', { credentials: 'same-origin' });
if (!r.ok) return;
var pkgs = await r.json();
var members = await Promise.all(pkgs.map(function(p) {
return fetch('/api/admin/data-packages/' + encodeURIComponent(p.id),
{ credentials: 'same-origin' })
.then(function(rr) { return rr.ok ? rr.json() : null; })
.catch(function() { return null; });
}));
members.forEach(function(det, i) {
if (!det) return;
var ids = (det.tables || []).map(function(t) { return t.id; });
if (ids.indexOf(tableId) === -1) return;
originalIdsSet.add(pkgs[i].id);
if (chipHost.addChip) {
chipHost.addChip({ id: pkgs[i].id, name: pkgs[i].name });
}
});
} catch (_) {
// Non-fatal: the field stays empty and admin can re-pick.
}
}
// Shared delta sync — diffs the chip-input's current selection vs
// the originally-loaded set, emits minimal POST/DELETE against the
// junction endpoint, and returns the failure count so the calling
// save handler can surface it in the toast.
async function _diffApplyPackageMembership(tableId, chipHost, originalIdsSet) {
if (!chipHost || !chipHost.getSelectedIds) return 0;
var nowIds = new Set(chipHost.getSelectedIds());
var calls = [];
nowIds.forEach(function(id) {
if (!originalIdsSet.has(id)) {
calls.push(fetch('/api/admin/data-packages/' + encodeURIComponent(id) + '/tables', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
credentials: 'same-origin',
body: JSON.stringify({ table_id: tableId }),
}));
}
});
originalIdsSet.forEach(function(id) {
if (!nowIds.has(id)) {
calls.push(fetch('/api/admin/data-packages/' + encodeURIComponent(id)
+ '/tables/' + encodeURIComponent(tableId),
{ method: 'DELETE', credentials: 'same-origin' }));
}
});
if (!calls.length) return 0;
var results = await Promise.allSettled(calls);
return results.filter(function(x) {
return x.status === 'rejected' || (x.value && !x.value.ok);
}).length;
}
function onEditBqAccessModeChange() {
var accessMode = _getEditBqAccessMode();
document.querySelectorAll('.bq-edit-access-synced').forEach(function(el) {
el.style.display = (accessMode === 'synced') ? '' : 'none';
});
if (accessMode === 'synced') {
onEditBqSyncModeChange();
} else {
// Live: dataset+table inputs visible, SQL textarea hidden.
document.querySelectorAll('.bq-edit-source-table').forEach(function(el) {
el.style.display = '';
});
document.querySelectorAll('.bq-edit-source-custom').forEach(function(el) {
el.style.display = 'none';
});
}
// Mode-switch warning: only fire when the operator actually flipped
// access mode from what was loaded — typo-fix edits stay quiet.
var newMode = (accessMode === 'synced') ? 'materialized' : 'remote';
var warn = document.getElementById('editBqModeWarning');
if (_editOriginalQueryMode && newMode !== _editOriginalQueryMode) {
var msg;
if (_editOriginalQueryMode === 'materialized' && newMode === 'remote') {
msg = '⚠ Switching Synced locally → Live from BigQuery will drop the materialized parquet on the next sync. Analysts who already pulled this table will start getting live BQ queries instead of a local copy; the sync schedule becomes ignored.';
} else {
msg = '⚠ Switching Live from BigQuery → Synced locally: the next scheduled sync runs a SELECT and writes a parquet. Analysts will start reading the local copy on their next `agnes pull`. Remember to set a sync schedule.';
}
warn.textContent = msg;
warn.style.display = '';
} else {
warn.style.display = 'none';
}
}
function onEditBqSyncModeChange() {
var syncMode = _getEditBqSyncMode();
document.querySelectorAll('.bq-edit-source-table').forEach(function(el) {
el.style.display = (syncMode === 'whole') ? '' : 'none';
});
document.querySelectorAll('.bq-edit-source-custom').forEach(function(el) {
el.style.display = (syncMode === 'custom') ? '' : 'none';
});
}
function closeEditModal() {
_closeEditModalById('editModal');
}
async function saveTableEdit() {
// Generic / catch-all save handler — runs for source_types that
// aren't bigquery (handled by saveBqTabEdit) or keboola (handled
// by saveKeboolaTabEdit). Currently covers jira-style rows.
if (!currentEditTableId) return;
var btn = document.getElementById('editSubmitBtn');
btn.disabled = true;
btn.textContent = 'Saving...';
var pk = document.getElementById('editPrimaryKey').value.trim();
var primaryKey = pk
? pk.split(',').map(function(s) { return s.trim(); }).filter(Boolean)
: [];
var payload = {
sync_strategy: document.getElementById('editStrategy').value,
primary_key: primaryKey,
description:
document.getElementById('editDescription').value.trim() || null,
folder: document.getElementById('editFolder').value.trim() || null,
};
try {
var r = await fetch('/api/admin/registry/' + encodeURIComponent(currentEditTableId), {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
if (!r.ok) {
var d = await r.json().catch(function() { return {}; });
throw new Error(d.detail || d.error || 'Update failed');
}
// Diff chip selection vs original — emit minimal POST/DELETE
// against the junction. Errors surface via toast but don't
// roll back the table PUT.
var legacyFails = await _diffApplyPackageMembership(
currentEditTableId,
document.getElementById('editGenericPackagesChips'),
_editGenericOriginalPackageIds
);
if (legacyFails > 0) {
showToast(legacyFails + ' package membership change(s) failed', 'error');
}
closeEditModal();
showToast('Table updated successfully', 'success');
loadRegistry();
if (typeof loadAdminTablesLayout === 'function') {
loadAdminTablesLayout();
}
} catch (err) {
showToast('Update failed: ' + err.message, 'error');
} finally {
btn.disabled = false;
btn.textContent = 'Save Changes';
}
}
function saveBqTabEdit() {
// Save handler for the relocated BQ Edit modal (#editBqModal).
// Mirrors the pre-C2 BQ branch of saveTableEdit() but reads from
// the BQ-modal-scoped fields (editBqDescription / editBqFolder).
if (!currentEditTableId) return;
var btn = document.getElementById('editBqSubmitBtn');
btn.disabled = true;
btn.textContent = 'Saving...';
// Two-question state machine — same as Register modal:
// live → query_mode='remote'
// synced/whole → materialized + auto SELECT *
// synced/custom → materialized + admin SQL
var accessMode = _getEditBqAccessMode();
var syncMode = _getEditBqSyncMode();
var dataset = document.getElementById('editBqDataset').value.trim();
var sourceTable = document.getElementById('editBqSourceTable').value.trim();
var payload = {
description: document.getElementById('editBqDescription').value.trim() || null,
folder: document.getElementById('editBqFolder').value.trim() || null,
source_type: 'bigquery',
sync_schedule:
document.getElementById('editBqSyncSchedule').value.trim() || null,
};
if (accessMode === 'synced' && syncMode === 'custom') {
payload.query_mode = 'materialized';
payload.source_query =
document.getElementById('editBqSourceQuery').value.trim();
// Issue #266: only clear bucket/source_table on a genuine mode
// flip (was non-materialized → now custom-SQL materialized). The
// original pre-#266 code nulled unconditionally on every save —
// so an admin editing only the description on an already-
// materialized custom-SQL row would silently wipe bucket/
// source_table. Destructive for rows that had them persisted
// (whole-table-then-switched-to-custom, or curl-set via API).
// On no-op saves (mode didn't change since modal load), omit
// the keys entirely so the PUT handler's `exclude_unset=True`
// preserves the existing values in the registry.
if (_editOriginalQueryMode !== 'materialized') {
payload.bucket = null;
payload.source_table = null;
}
} else if (accessMode === 'synced' && syncMode === 'whole') {
payload.query_mode = 'materialized';
payload.source_query =
'SELECT * FROM bq."' + dataset + '"."' + sourceTable + '"';
payload.bucket = dataset || null;
payload.source_table = sourceTable || null;
} else {
// Live.
payload.query_mode = 'remote';
payload.bucket = dataset || null;
payload.source_table = sourceTable || null;
payload.source_query = null;
}
(async function () {
try {
var r = await fetch('/api/admin/registry/' + encodeURIComponent(currentEditTableId), {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
if (!r.ok) {
var d = await r.json().catch(function() { return {}; });
throw new Error(d.detail || d.error || 'Update failed');
}
// Diff Data Package chip selection vs original.
var bqFails = await _diffApplyPackageMembership(
currentEditTableId,
document.getElementById('editBqPackagesChips'),
_editBqOriginalPackageIds
);
if (bqFails > 0) {
showToast(bqFails + ' package membership change(s) failed', 'error');
}
closeEditBqModal();
showToast('Table updated successfully', 'success');
loadRegistry();
if (typeof loadAdminTablesLayout === 'function') loadAdminTablesLayout();
} catch (err) {
showToast('Update failed: ' + err.message, 'error');
} finally {
btn.disabled = false;
btn.textContent = 'Save Changes';
}
})();
}
// ── Registry (legacy shim) ──────────────────────────────────
// Pre-rewrite, loadRegistry() filled four per-tab listing divs
// (bqTableListing, kbTableListing, jiraTableListing, internalTableListing).
// Those listings were deleted with the connector tabs. Existing register /
// edit / delete flows still call loadRegistry() to refresh the page after
// a mutation — delegate to loadAdminTablesLayout() which now renders the
// unified package-centric view from the same /api/admin/registry payload.
function loadRegistry() {
if (typeof loadAdminTablesLayout === 'function') {
return loadAdminTablesLayout();
}
}
// Per-row Mode badge — tooltip mirrors the explanation in the
// Register modal so admins can hover instead of opening
// docs/admin/query-modes.md to remember which mode does what.
var _MODE_TOOLTIPS = {
local: 'Synced locally — DuckDB downloads to parquet on a schedule; analysts query the local copy via agnes pull.',
remote: 'Live from BigQuery — no download; queries go through the DuckDB BQ extension with the 5 GiB scan cap.',
materialized: 'Materialised SQL — scheduler runs admin SQL through DuckDB and writes the result to a parquet, distributed like a local table.',
internal: 'Internal agnes_* table backed by system.duckdb; read-only, seeded on app boot.',
};
function renderModeBadge(mode) {
var m = (mode || 'local').toLowerCase();
var cls = 'mode-badge mode-' + (['local','remote','materialized','internal'].indexOf(m) >= 0 ? m : 'local');
var tip = _MODE_TOOLTIPS[m] || '';
return '<span class="' + cls + '" title="' + escapeHtmlAttr(tip) + '">' + escapeHtml(m) + '</span>';
}
function renderRegistryListing(target, tables) {
if (!target) return;
if (tables.length === 0) {
target.innerHTML = '<div class="panel-body-empty">No tables registered yet.</div>';
return;
}
var html = '<table class="registry-table">';
html += '<thead><tr>';
html += '<th class="col-id">Table ID</th>';
html += '<th class="col-mode">Mode</th>';
html += '<th class="col-source">Source</th>';
html += '<th class="col-pk">Primary Key</th>';
html += '<th class="col-schedule">Schedule</th>';
html += '<th class="col-folder">Folder</th>';
html += '<th>Description</th>';
html += '<th class="col-registered">Registered</th>';
html += '<th class="col-status"></th>';
html += '<th class="col-actions">Actions</th>';
html += '</tr></thead><tbody>';
tables.forEach(function(table) {
html += '<tr>';
html += '<td class="col-id" title="' + escapeHtml(table.id) + '">' + escapeHtml(table.id) + '</td>';
html += '<td class="col-mode">' + renderModeBadge(table.query_mode) + '</td>';
// Source: bucket / source_table; em-dash when both empty (custom-SQL row).
var bucket = table.bucket || '';
var srcTable = table.source_table || '';
var sourceText = '';
if (bucket && srcTable) {
sourceText = bucket + ' / ' + srcTable;
} else if (bucket || srcTable) {
sourceText = bucket || srcTable;
}
var sourceCell = sourceText ? escapeHtml(sourceText) : '—';
html += '<td class="col-source" title="' + escapeHtml(sourceText) + '">' + sourceCell + '</td>';
html += '<td class="col-pk">' + escapeHtml((table.primary_key || []).join(', ') || '—') + '</td>';
html += '<td class="col-schedule">' + escapeHtml(table.sync_schedule || '—') + '</td>';
// Folder badge — '—' when null/empty.
if (table.folder) {
html += '<td class="col-folder"><span class="folder-badge">' + escapeHtml(table.folder) + '</span></td>';
} else {
html += '<td class="col-folder">—</td>';
}
var desc = unescapeShellQuoting(table.description || '');
html += '<td class="col-description" title="' + escapeHtml(desc) + '">' + escapeHtml(desc || '—') + '</td>';
// Registered: stacked email + date (first 10 chars of ISO timestamp).
var regBy = table.registered_by || '';
var regByDisplay = regBy;
if (regBy.length > 24 && regBy.indexOf('@') > 0) {
regByDisplay = regBy.split('@')[0];
}
var regAt = table.registered_at ? String(table.registered_at).slice(0, 10) : '';
html += '<td class="col-registered">';
html += '<div class="registered-by" title="' + escapeHtml(regBy) + '">' + (regByDisplay ? escapeHtml(regByDisplay) : '—') + '</div>';
html += '<div class="registered-at">' + escapeHtml(regAt || '') + '</div>';
html += '</td>';
// Status: warning icon when the last sync errored.
html += '<td class="col-status">';
if (table.last_sync_error) {
html += '<span title="' + escapeHtml(table.last_sync_error) + '">';
html += '<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" style="color: var(--error);"><path d="M10.29 3.86 1.82 18a2 2 0 0 0 1.71 3h16.94a2 2 0 0 0 1.71-3L13.71 3.86a2 2 0 0 0-3.42 0z"/><line x1="12" y1="9" x2="12" y2="13"/><line x1="12" y1="17" x2="12.01" y2="17"/></svg>';
html += '</span>';
}
html += '</td>';
html += '<td class="col-actions"><div style="display:flex; gap:4px; justify-content:flex-end;">';
// Internal source tables (agnes_*) are seeded on every app
// boot from connectors/internal/registry.py — Edit/Delete
// would either no-op or be reverted on next start. Hide both
// actions; Manage access stays because RBAC is still
// per-table (internal tables auto-grant; the link still
// takes the admin to the access page if they want to inspect).
//
// Issue #265: HTML attribute values do not recognize backslash
// escapes the way JavaScript string literals do — use HTML-
// entity escape (`&#39;` for `'`, plus `"`, `<`, `>`, `&`)
// via escapeHtmlAttr, the right encoding for an HTML attribute.
var isInternal = (table.source_type === 'internal');
if (!isInternal) {
html += '<button class="btn-icon" title="Edit" onclick=\'openEditModal(' + escapeHtmlAttr(JSON.stringify(table)) + ')\'>';
html += '<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M11 4H4a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2v-7"/><path d="M18.5 2.5a2.121 2.121 0 0 1 3 3L12 15l-4 1 1-4 9.5-9.5z"/></svg>';
html += '</button>';
}
// Manage access: deep-link to /admin/access pre-filtered to this table.
html += '<button class="btn-icon" title="Manage access" onclick="manageAccess(&#39;' + escapeHtmlAttr(table.id) + '&#39;)">';
html += '<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="3" y="11" width="18" height="11" rx="2" ry="2"/><path d="M7 11V7a5 5 0 0 1 10 0v4"/></svg>';
html += '</button>';
if (!isInternal) {
html += '<button class="btn-icon danger" title="Delete" onclick="deleteTable(&#39;' + escapeHtmlAttr(table.id) + '&#39;)">';
html += '<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polyline points="3 6 5 6 21 6"/><path d="M19 6v14a2 2 0 0 1-2 2H7a2 2 0 0 1-2-2V6m3 0V4a2 2 0 0 1 2-2h4a2 2 0 0 1 2 2v2"/></svg>';
html += '</button>';
}
html += '</div></td></tr>';
});
html += '</tbody></table>';
target.innerHTML = html;
}
// Deep-link from a table row to /admin/access scoped to this table_id.
// The /admin/access page reads window.location.hash on bootstrap and,
// when it matches `table:<id>`, pre-fills the resource filter so the
// operator lands on the just-clicked table once they pick a group.
function manageAccess(tableId) {
window.location.href = '/admin/access#table:' + encodeURIComponent(tableId);
}
function deleteTable(tableId) {
if (!confirm('Are you sure you want to unregister "' + tableId + '"?\n\nThis will remove it from sync and clean up user subscriptions.')) {
return;
}
// DELETE doesn't need a body — the API has no `version` field.
fetch('/api/admin/registry/' + encodeURIComponent(tableId), {
method: 'DELETE',
})
.then(function(r) {
// 204 No Content has an empty body, so don't .json() it.
if (r.status === 204) return null;
if (!r.ok) return r.json().then(function(d) { throw new Error(d.detail || d.error || 'Delete failed'); });
return r.json();
})
.then(function() {
showToast('Table unregistered successfully', 'success');
loadRegistry();
})
.catch(function(err) {
showToast('Delete failed: ' + err.message, 'error');
});
}
// ── Modal close on overlay click ────────────────────────────
var _registerBqModalEl = document.getElementById('registerBqModal');
if (_registerBqModalEl) {
_registerBqModalEl.addEventListener('click', function(e) {
if (e.target === this) closeRegisterBqModal();
});
}
var _registerKeboolaModalEl = document.getElementById('registerKeboolaModal');
if (_registerKeboolaModalEl) {
_registerKeboolaModalEl.addEventListener('click', function(e) {
if (e.target === this) closeRegisterKeboolaModal();
});
}
var _editKeboolaModalEl = document.getElementById('editKeboolaModal');
if (_editKeboolaModalEl) {
_editKeboolaModalEl.addEventListener('click', function(e) {
if (e.target === this) closeEditKeboolaModal();
});
}
var _editBqModalEl = document.getElementById('editBqModal');
if (_editBqModalEl) {
_editBqModalEl.addEventListener('click', function(e) {
if (e.target === this) closeEditBqModal();
});
}
document.getElementById('editModal').addEventListener('click', function(e) {
if (e.target === this) closeEditModal();
});
// Close modals on Escape key
document.addEventListener('keydown', function(e) {
if (e.key === 'Escape') {
closeRegisterBqModal();
if (typeof closeRegisterKeboolaModal === 'function') closeRegisterKeboolaModal();
if (typeof closeEditKeboolaModal === 'function') closeEditKeboolaModal();
if (typeof closeEditBqModal === 'function') closeEditBqModal();
closeEditModal();
}
});
// ── Initialize ──────────────────────────────────────────────
loadRegistry();
// ── Cache warmup toolbar (issue #155 / #156) ────────────────
let cacheWarmupSource = null;
function _cacheWarmupClearPollFallback() {
if (window._cacheWarmupPollInterval) {
clearInterval(window._cacheWarmupPollInterval);
window._cacheWarmupPollInterval = null;
}
}
function cacheWarmupInit() {
cacheWarmupRefreshSnapshot();
cacheWarmupOpenStream();
}
function cacheWarmupRefreshSnapshot() {
fetch('/api/admin/cache-warmup/status')
.then(function(r) { return r.json(); })
.then(function(state) { cacheWarmupRender(state); })
.catch(function() { /* silent */ });
}
function cacheWarmupOpenStream() {
try {
cacheWarmupSource = new EventSource('/api/admin/cache-warmup/stream');
cacheWarmupSource.addEventListener('start', cacheWarmupOnStart);
cacheWarmupSource.addEventListener('row', cacheWarmupOnRow);
cacheWarmupSource.addEventListener('complete', cacheWarmupOnComplete);
cacheWarmupSource.addEventListener('snapshot', function(e) {
_cacheWarmupClearPollFallback();
cacheWarmupRender(JSON.parse(e.data));
});
cacheWarmupSource.onerror = function() {
if (cacheWarmupSource) {
cacheWarmupSource.close();
cacheWarmupSource = null;
}
// Continuous polling fallback. Try to re-open SSE every 30 s in
// case the proxy / network heals. Only one polling interval at a
// time (prevent stacking on repeated errors).
if (!window._cacheWarmupPollInterval) {
window._cacheWarmupPollInterval = setInterval(
cacheWarmupRefreshSnapshot, 3000
);
setTimeout(function tryReconnect() {
if (cacheWarmupSource) return; // already reconnected
try {
clearInterval(window._cacheWarmupPollInterval);
window._cacheWarmupPollInterval = null;
cacheWarmupOpenStream(); // recursive — onerror retries again
} catch (e) {
window._cacheWarmupPollInterval = setInterval(
cacheWarmupRefreshSnapshot, 3000
);
setTimeout(tryReconnect, 30000);
}
}, 30000);
}
};
} catch (e) {
setInterval(cacheWarmupRefreshSnapshot, 3000);
}
}
function cacheWarmupRender(state) {
var summary = document.getElementById('cacheWarmupSummary');
var bar = document.getElementById('cacheWarmupBar');
var btn = document.getElementById('cacheWarmupRunBtn');
if (!summary) return;
if (!state || state.state === 'never_run') {
summary.textContent = 'No cache warmup yet — click Re-warm all to start.';
bar.style.display = 'none';
btn.disabled = false;
return;
}
var inProgress = state.completed_at === null || state.completed_at === undefined;
var pct = state.total > 0 ? Math.round((state.completed * 100) / state.total) : 0;
summary.textContent = inProgress
? state.completed + ' / ' + state.total + ' fresh — running…'
: 'Last run: ' + state.completed + ' ok, ' + state.failed + ' errors';
bar.style.display = 'block';
bar.value = pct;
btn.disabled = inProgress;
if (state.rows) {
for (var tid in state.rows) {
cacheWarmupSetRowBadge(tid, state.rows[tid]);
}
}
}
function cacheWarmupOnStart(e) {
_cacheWarmupClearPollFallback();
var data = JSON.parse(e.data);
var log = document.getElementById('cacheWarmupLog');
log.textContent = '';
// Hide the inline hint paragraph as soon as a real log line lands
// — the empty <pre>'s placeholder context is no longer useful and
// the dark log block is what the operator wants to watch.
var hint = document.getElementById('cacheWarmupHint');
if (hint) hint.style.display = 'none';
// Auto-open the details so streamed events are visible immediately,
// even if the operator clicked Re-warm without expanding first.
var details = document.getElementById('cacheWarmupDetails');
if (details) details.open = true;
cacheWarmupAppendLog(
'[' + nowHHMMSS() + '] start trigger=' + data.trigger + ' total=' + data.total
);
cacheWarmupRefreshSnapshot();
}
function cacheWarmupOnRow(e) {
_cacheWarmupClearPollFallback();
var rs = JSON.parse(e.data);
cacheWarmupAppendLog(
'[' + nowHHMMSS() + '] ' + rs.status.padEnd(7) + ' ' + rs.table_id +
(rs.duration_ms ? ' (' + (rs.duration_ms / 1000).toFixed(1) + ' s)' : '') +
(rs.error ? ' ' + rs.error : '')
);
cacheWarmupSetRowBadge(rs.table_id, rs);
cacheWarmupRefreshSnapshot();
}
function cacheWarmupOnComplete(e) {
_cacheWarmupClearPollFallback();
var data = JSON.parse(e.data);
cacheWarmupAppendLog(
'[' + nowHHMMSS() + '] complete total=' + data.total +
' ok=' + data.completed + ' fail=' + data.failed
);
cacheWarmupRefreshSnapshot();
}
function cacheWarmupAppendLog(line) {
var log = document.getElementById('cacheWarmupLog');
if (!log) return;
log.textContent += line + '\n';
log.scrollTop = log.scrollHeight;
}
function cacheWarmupSetRowBadge(tableId, rs) {
document.querySelectorAll('tr').forEach(function(tr) {
var idCell = tr.querySelector('td.col-id');
if (!idCell || idCell.textContent.trim() !== tableId) return;
var statusCell = tr.querySelector('td.col-status');
if (!statusCell) return;
var color = {fresh: '#10B77F', warming: 'var(--primary)', pending: '#9CA3AF', error: '#EA580C'}[rs.status] || '#9CA3AF';
var label = rs.status === 'fresh' ? 'fresh' : rs.status;
// Build via DOM API so rs.error escapes safely into the title
// attribute (XSS guard — rs.error is server-derived, may contain
// quotes / angle brackets).
var span = document.createElement('span');
span.style.cssText =
'display:inline-block;padding:2px 6px;border-radius:3px;' +
'font-size:11px;background:' + color + ';color:white;';
if (rs.error) span.setAttribute('title', rs.error);
span.textContent = label;
statusCell.replaceChildren(span);
});
}
function nowHHMMSS() {
var d = new Date();
return d.toTimeString().slice(0, 8);
}
function cacheWarmupRun() {
var btn = document.getElementById('cacheWarmupRunBtn');
btn.disabled = true;
// Auto-open the log section so the operator sees streaming output
// instead of staring at a disabled button while ~24s pass per
// remote BQ row. The hint paragraph hides as soon as a real log
// line lands (see cacheWarmupOnStart).
var details = document.getElementById('cacheWarmupDetails');
if (details) details.open = true;
fetch('/api/admin/cache-warmup/run', {method: 'POST'})
.then(function(r) { return r.json(); })
.then(function() { /* SSE stream picks up the new run */ })
.catch(function() { btn.disabled = false; });
}
document.addEventListener('DOMContentLoaded', cacheWarmupInit);
// ── Top-level Data Packages section hydration ───────────────
// Discovery fix: before this section existed, admins had to open a
// table's edit modal to even know Data Packages were a thing (the
// chip-input lives inside the BQ register/edit form). This block
// surfaces the package list above the connector tabs + adds a
// "+ New" entry point that opens the existing Create Data Package
// modal directly. The card grid reuses the .stack-grid + .stack-card
// classes shared with /catalog + /memory.
// ── Package-centric layout hydrator ──────────────────────────
// Replaces the prior loadDataPackagesSection (card grid) + the four
// per-tab listing renders. Pulls /api/admin/data-packages and
// /api/admin/registry in parallel, then renders:
// 1. one collapsible <details> per package containing its member tables
// 2. an "Unpackaged tables" callout for everything else
// Source_type appears as an inline tag on each row but no longer
// drives the layout (the user's "everything must live within some
// group in data packages" feedback).
async function loadAdminTablesLayout() {
var pkgsEl = document.getElementById('adminTablesLayoutPackages');
var unpkEl = document.getElementById('adminTablesLayoutUnpackaged');
if (!pkgsEl || !unpkEl) return;
try {
// v54: single round-trip for packages + their member ids via
// ``?include_table_ids=true``. Was N+1 (one GET per package
// after the list); on instances with 50+ packages the page
// took multiple seconds before this collapsed fetch.
var [pkgResp, tableResp] = await Promise.all([
fetch('/api/admin/data-packages?include_table_ids=true',
{ credentials: 'same-origin' }),
fetch('/api/admin/registry', { credentials: 'same-origin' }),
]);
if (!pkgResp.ok) {
pkgsEl.innerHTML = '<div class="panel-body-empty" style="color:var(--error);">Failed to load Data Packages (HTTP ' + pkgResp.status + ').</div>';
unpkEl.innerHTML = '';
return;
}
if (!tableResp.ok) {
pkgsEl.innerHTML = '<div class="panel-body-empty" style="color:var(--error);">Failed to load registry (HTTP ' + tableResp.status + ').</div>';
unpkEl.innerHTML = '';
return;
}
var pkgs = await pkgResp.json();
var registryBody = await tableResp.json();
registryData = registryBody;
registryVersion = registryBody.version;
var allTables = Array.isArray(registryBody)
? registryBody
: (registryBody.tables || []);
var tableById = {};
allTables.forEach(function(t) { tableById[t.id] = t; });
var packagedIds = new Set();
var pkgsWithTables = pkgs.map(function(p) {
// table_ids comes from the new ?include_table_ids=true
// server-side join. Empty packages omit the field — default to [].
var memberIds = p.table_ids || [];
memberIds.forEach(function(id) { packagedIds.add(id); });
var memberRows = memberIds
.map(function(id) { return tableById[id]; })
.filter(Boolean);
return Object.assign({}, p, { _members: memberRows });
});
// ── Render packages ──
if (!pkgsWithTables.length) {
pkgsEl.innerHTML = ''
+ '<div style="padding:24px; text-align:center; color:var(--text-secondary); '
+ 'background:var(--background); border:1px dashed var(--border); '
+ 'border-radius:8px;">'
+ ' <p style="margin:0 0 8px; font-size:14px; color:var(--text-primary);">'
+ ' <strong>No Data Packages yet</strong>'
+ ' </p>'
+ ' <p style="margin:0 0 12px; font-size:13px;">'
+ ' Bundle related tables so analysts can opt-in to them as a group.'
+ ' </p>'
+ ' <button class="btn btn-primary" type="button" '
+ ' onclick="openCreateDataPackageModal(\'\', null)">+ New Data Package</button>'
+ ' <button class="btn btn-secondary" type="button" '
+ ' onclick="groupTablesByBucket()" style="margin-left:8px;">'
+ ' Group tables by bucket'
+ ' </button>'
+ '</div>';
} else {
pkgsEl.innerHTML = pkgsWithTables.map(function(p) {
var iconRaw = p.icon || '📦';
var color = p.color || '#f3f4f6';
var name = String(p.name || '');
var slug = String(p.slug || '');
var desc = String(p.description || '');
var tcount = (p._members || []).length;
var rowsHtml = (p._members || []).length
? _renderPackageTableRows(p._members, p.id)
: '<div class="panel-body-empty" style="padding:16px; font-size:12px;">'
+ 'No tables in this package yet. Click <strong>+ Add tables</strong> above.</div>';
return ''
+ '<details open data-package-id="' + escapeHtmlAttr(p.id) + '" '
+ 'style="margin-bottom:12px; border:1px solid var(--border); '
+ 'border-radius:10px; background:var(--surface);">'
+ ' <summary style="display:flex; align-items:center; gap:12px; padding:12px 16px; '
+ 'cursor:pointer; user-select:none; list-style:none;">'
+ ' <span style="display:inline-flex; align-items:center; justify-content:center; '
+ 'width:32px; height:32px; border-radius:8px; flex-shrink:0; '
+ 'background:' + escapeHtmlAttr(color) + ';">'
+ escapeHtml(iconRaw)
+ ' </span>'
+ ' <span style="flex:1; min-width:0;">'
+ ' <strong style="font-size:14px;">' + escapeHtml(name) + '</strong>'
+ (desc ? ('<div style="font-size:12px; color:var(--text-secondary); '
+ 'margin-top:2px;">' + escapeHtml(desc) + '</div>') : '')
+ ' </span>'
+ ' <span style="font-size:12px; color:var(--text-secondary);">'
+ tcount + ' table' + (tcount === 1 ? '' : 's') + '</span>'
+ ' <button class="btn btn-secondary btn-sm" type="button" '
+ 'onclick="event.preventDefault(); event.stopPropagation(); '
+ 'openBulkAssignModal(\'' + escapeHtmlAttr(p.id) + '\')">'
+ ' + Add tables'
+ ' </button>'
+ ' <button class="btn btn-secondary btn-sm" type="button" '
+ 'onclick="event.preventDefault(); event.stopPropagation(); '
+ 'openEditDataPackageModal(\'' + escapeHtmlAttr(p.id) + '\')">'
+ ' Edit package'
+ ' </button>'
+ ' <a class="btn btn-secondary btn-sm" '
+ 'href="/catalog/p/' + escapeHtmlAttr(slug) + '" '
+ 'onclick="event.stopPropagation()" '
+ 'title="See this package as analysts will">'
+ ' View as analyst'
+ ' </a>'
+ ' </summary>'
+ ' <div style="padding:0 12px 12px;">' + rowsHtml + '</div>'
+ '</details>';
}).join('');
}
// ── Render Unpackaged tables ──
// Exclude internal/agnes_* rows from "needs packaging" — they're
// auto-seeded and shouldn't be auto-bundled. Still listed in a
// separate read-only group below.
var unpackagedNeedsAttention = allTables.filter(function(t) {
return !packagedIds.has(t.id)
&& (t.source_type || '') !== 'internal';
});
var internalUnpackaged = allTables.filter(function(t) {
return !packagedIds.has(t.id)
&& (t.source_type || '') === 'internal';
});
var unpkParts = [];
if (unpackagedNeedsAttention.length) {
unpkParts.push(''
+ '<section id="adminTablesUnpackaged" style="border:1px solid #FCD34D; '
+ 'background:rgba(255,247,205,0.4); border-radius:10px;">'
+ ' <header style="display:flex; align-items:center; gap:12px; '
+ 'padding:12px 16px; border-bottom:1px solid #FCD34D;">'
+ ' <span style="font-size:16px;">⚠️</span>'
+ ' <strong style="flex:1; font-size:14px;">Unpackaged tables '
+ '<span style="color:var(--text-secondary); font-weight:500;">'
+ '(' + unpackagedNeedsAttention.length + ' — needs packaging)</span></strong>'
+ ' <button class="btn btn-secondary btn-sm" type="button" '
+ 'onclick="groupTablesByBucket()">Group by bucket</button>'
+ ' <button class="btn btn-primary btn-sm" type="button" '
+ 'onclick="openBulkAssignModal(\'\')">Bulk assign</button>'
+ ' </header>'
+ ' <div style="padding:0 12px 12px;">'
+ _renderPackageTableRows(unpackagedNeedsAttention, null)
+ ' </div>'
+ '</section>');
}
if (internalUnpackaged.length) {
unpkParts.push(''
+ '<section style="margin-top:16px; border:1px solid var(--border); '
+ 'background:var(--background); border-radius:10px;">'
+ ' <header style="display:flex; align-items:center; gap:12px; '
+ 'padding:12px 16px; border-bottom:1px solid var(--border);">'
+ ' <strong style="flex:1; font-size:13px; color:var(--text-secondary);">'
+ ' Agnes internal tables '
+ '<span style="font-weight:500;">(read-only; seeded on app boot)</span>'
+ ' </strong>'
+ ' </header>'
+ ' <div style="padding:0 12px 12px;">'
+ _renderPackageTableRows(internalUnpackaged, null)
+ ' </div>'
+ '</section>');
}
unpkEl.innerHTML = unpkParts.join('');
} catch (e) {
pkgsEl.innerHTML = '<div class="panel-body-empty" style="color:var(--error);">'
+ 'Network error: ' + escapeHtml(e.message) + '</div>';
unpkEl.innerHTML = '';
}
}
// Render a list of table rows for the package-centric layout. Each row:
// [name link → /admin/tables?edit=<id>] [source_type tag] [bucket] [mode]
// [Edit] [Remove from package | + Add to package ▾]
// pkgIdOrNull → when non-null, rows expose a "Remove from package" action;
// when null (Unpackaged section), rows expose "+ Add to package".
function _renderPackageTableRows(tables, pkgIdOrNull) {
if (!tables || !tables.length) {
return '<div class="panel-body-empty">No tables here yet.</div>';
}
var html = '<table class="registry-table" style="margin-top:8px;">';
html += '<thead><tr>';
html += '<th class="col-id">Table</th>';
html += '<th>Source</th>';
html += '<th>Bucket</th>';
html += '<th class="col-mode">Mode</th>';
html += '<th class="col-actions">Actions</th>';
html += '</tr></thead><tbody>';
tables.forEach(function(t) {
var id = String(t.id || '');
var name = String(t.name || id);
var src = String(t.source_type || '');
var bucket = String(t.bucket || '');
var editUrl = '/admin/tables?edit=' + encodeURIComponent(id);
var isInternal = (src === 'internal');
// data-table-row-{name,source,bucket} attributes power the
// filterAdminTablesLayout() search; lower-case once at render
// time so the filter can use plain substring matching.
html += '<tr data-table-id="' + escapeHtmlAttr(id) + '" '
+ 'data-table-row-name="' + escapeHtmlAttr(name.toLowerCase()) + '" '
+ 'data-table-row-source="' + escapeHtmlAttr(src.toLowerCase()) + '" '
+ 'data-table-row-bucket="' + escapeHtmlAttr(bucket.toLowerCase()) + '">';
html += '<td class="col-id" title="' + escapeHtml(id) + '">'
+ '<a href="' + escapeHtmlAttr(editUrl) + '" '
+ 'style="color:var(--primary); text-decoration:none;">'
+ escapeHtml(name) + '</a></td>';
html += '<td><span class="badge badge-available" style="font-size:11px;">'
+ escapeHtml(src || '—') + '</span></td>';
html += '<td style="font-family:var(--font-mono); font-size:12px; color:var(--text-secondary);">'
+ escapeHtml(bucket || '—') + '</td>';
html += '<td class="col-mode">' + renderModeBadge(t.query_mode) + '</td>';
html += '<td class="col-actions"><div style="display:flex; gap:4px; justify-content:flex-end;">';
if (!isInternal) {
html += '<button class="btn-icon" title="Edit" '
+ 'onclick=\'openEditModal(' + escapeHtmlAttr(JSON.stringify(t)) + ')\'>'
+ '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M11 4H4a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2v-7"/><path d="M18.5 2.5a2.121 2.121 0 0 1 3 3L12 15l-4 1 1-4 9.5-9.5z"/></svg>'
+ '</button>';
}
if (pkgIdOrNull && !isInternal) {
html += '<button class="btn-icon danger" title="Remove from package" '
+ 'onclick="removeTableFromPackageById(&#39;' + escapeHtmlAttr(pkgIdOrNull) + '&#39;, &#39;' + escapeHtmlAttr(id) + '&#39;)">'
+ '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><line x1="5" y1="12" x2="19" y2="12"/></svg>'
+ '</button>';
} else if (!pkgIdOrNull && !isInternal) {
// Icon button for visual rhythm with Edit (the wide
// "+ Add to package" text button used to overflow the
// 120px col-actions and bleed leftward into the Mode
// column — user-reported visual bug). Title attr keeps
// the explicit label discoverable on hover.
html += '<button class="btn-icon" type="button" '
+ 'title="Add to a Data Package" '
+ 'onclick="openBulkAssignFromRow(&#39;' + escapeHtmlAttr(id) + '&#39;)" '
+ 'style="color:var(--primary);">'
+ '<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">'
+ '<path d="M22 19a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h5l2 3h9a2 2 0 0 1 2 2z"/>'
+ '<line x1="12" y1="11" x2="12" y2="17"/>'
+ '<line x1="9" y1="14" x2="15" y2="14"/>'
+ '</svg>'
+ '</button>';
}
html += '</div></td></tr>';
});
html += '</tbody></table>';
return html;
}
// Remove a single table from a package, then refresh the layout.
async function removeTableFromPackageById(pkgId, tableId) {
if (!confirm('Remove this table from the package? (The table itself is not deleted.)')) return;
var r = await fetch('/api/admin/data-packages/' + encodeURIComponent(pkgId)
+ '/tables/' + encodeURIComponent(tableId),
{ method: 'DELETE', credentials: 'same-origin' });
if (!r.ok) { alert('Remove failed: HTTP ' + r.status); return; }
if (typeof loadAdminTablesLayout === 'function') loadAdminTablesLayout();
}
// Open bulk-assign modal pre-checking a single table — the "+ Add to
// package" CTA on each unpackaged row. Defers to the existing modal so
// the operator can pick the target package without leaving the page.
function openBulkAssignFromRow(tableId) {
openBulkAssignModal('');
// Once the modal's table list has rendered, pre-check the requested
// row. The modal renders synchronously via renderBulkAssignList after
// its async fetches resolve, so poll briefly.
var tries = 0;
var tick = setInterval(function() {
tries += 1;
var cb = document.querySelector(
'#bulk-assign-list .bulk-assign-cb[value="' + tableId.replace(/"/g, '') + '"]'
);
if (cb) {
cb.checked = true;
cb.scrollIntoView({ block: 'center' });
clearInterval(tick);
} else if (tries > 50) {
clearInterval(tick);
}
}, 50);
}
// Backward-compat alias — Edit/Create/Delete-package flows declared
// before the rewrite still call loadDataPackagesSection() to refresh.
var loadDataPackagesSection = loadAdminTablesLayout;
window.loadDataPackagesSection = loadDataPackagesSection;
// ── Search filter for the /admin/tables layout ───────────────────
// Mirrors the /catalog + /memory hero-search pattern: one input
// narrows both packages and table rows in place. A package is
// visible when its name matches OR any of its member tables match;
// an unpackaged row is visible when its name / source / bucket
// matches.
function filterAdminTablesLayout() {
var input = document.getElementById('adminTablesSearch');
var hint = document.getElementById('adminTablesSearchHint');
var q = (input && input.value || '').toLowerCase().trim();
var matched = 0;
var total = 0;
// Per-package <details> filtering — show the package if its name
// matches, OR any of the rows inside match.
document.querySelectorAll(
'#adminTablesLayoutPackages details[data-package-id]'
).forEach(function(d) {
var pkgName = ((d.querySelector('summary strong')||{}).textContent || '').toLowerCase();
var rows = d.querySelectorAll('[data-table-row-name]');
var anyRow = false;
rows.forEach(function(r) {
total += 1;
var rn = (r.getAttribute('data-table-row-name') || '').toLowerCase();
var rs = (r.getAttribute('data-table-row-source') || '').toLowerCase();
var rb = (r.getAttribute('data-table-row-bucket') || '').toLowerCase();
var hit = !q || rn.indexOf(q) !== -1 || rs.indexOf(q) !== -1 || rb.indexOf(q) !== -1;
r.style.display = hit ? '' : 'none';
if (hit) { anyRow = true; matched += 1; }
});
var pkgHit = !q || pkgName.indexOf(q) !== -1 || anyRow;
d.style.display = pkgHit ? '' : 'none';
// When the query hits the package name only, expand it; when the
// query is empty leave the user's open/closed state alone.
if (q && pkgHit && (anyRow || pkgName.indexOf(q) !== -1)) {
d.open = true;
}
});
// Unpackaged rows live outside the per-package <details>.
document.querySelectorAll(
'#adminTablesLayoutUnpackaged [data-table-row-name]'
).forEach(function(r) {
total += 1;
var rn = (r.getAttribute('data-table-row-name') || '').toLowerCase();
var rs = (r.getAttribute('data-table-row-source') || '').toLowerCase();
var rb = (r.getAttribute('data-table-row-bucket') || '').toLowerCase();
var hit = !q || rn.indexOf(q) !== -1 || rs.indexOf(q) !== -1 || rb.indexOf(q) !== -1;
r.style.display = hit ? '' : 'none';
if (hit) matched += 1;
});
if (hint) {
hint.textContent = q
? matched + ' of ' + total + ' table' + (total === 1 ? '' : 's')
: '';
}
}
window.filterAdminTablesLayout = filterAdminTablesLayout;
document.addEventListener('DOMContentLoaded', loadAdminTablesLayout);
// Auto-open the Create Data Package modal when the page is opened with
// `?new_package=1` — drives the /catalog right-of-tabs admin action
// (which redirects here rather than duplicating the modal scaffolding).
document.addEventListener('DOMContentLoaded', function() {
var qs = new URLSearchParams(window.location.search);
if (qs.get('new_package') !== '1') return;
// Wait a tick so the layout's deferred load completes (the modal
// doesn't actually depend on it, but the page feels less janky if
// the layout has rendered first).
setTimeout(function() {
if (typeof openCreateDataPackageModal === 'function') {
openCreateDataPackageModal('', null);
}
}, 50);
// Strip the query param so a manual refresh doesn't re-open the
// modal (mirrors closeBulkAssignModal's URL hygiene).
history.replaceState(null, '',
window.location.pathname + window.location.hash);
});
// Auto-open the Edit Data Package modal when the page is opened
// with `?edit_package=<pkg_id>` — closes the /catalog/p/<slug>
// drill-down Edit button (which previously linked to `#<pkg.id>`
// and did nothing past landing on the page).
document.addEventListener('DOMContentLoaded', function() {
var qs = new URLSearchParams(window.location.search);
var pkgId = qs.get('edit_package');
if (!pkgId) return;
// Wait for the layout's deferred load to settle so the modal
// hydrators (member tables, RBAC matrix) have something to hang
// their fetches on.
setTimeout(function() {
if (typeof openEditDataPackageModal === 'function') {
openEditDataPackageModal(pkgId);
}
}, 250);
// Strip the query param so a manual refresh doesn't re-open.
history.replaceState(null, '',
window.location.pathname + window.location.hash);
});
// Auto-open the source-typed edit modal when the page is opened with
// `?edit=<table_id>` — closes the /catalog/p/<slug> drill-down loop
// where each table name is now an admin-only link landing here.
// Package-centric rewrite: no more switchTab; the modal opens directly.
document.addEventListener('DOMContentLoaded', async function() {
var qs = new URLSearchParams(window.location.search);
var editId = qs.get('edit');
if (!editId) return;
try {
var r = await fetch('/api/admin/registry', { credentials: 'same-origin' });
if (!r.ok) return;
var body = await r.json();
var tables = Array.isArray(body) ? body : (body.tables || []);
var t = tables.find(function(x) { return String(x.id) === String(editId); });
if (!t) {
alert('Table not found (id=' + editId + '). It may have been deleted.');
return;
}
// Small delay so the layout has time to mount; then open the source-
// dispatched edit modal directly. No tab nav anymore.
setTimeout(function() {
if (typeof openEditModal === 'function') openEditModal(t);
}, 200);
} catch (e) {
console.warn('?edit auto-open failed:', e);
}
});
</script>
{% endblock %}
{# Chip-input component (v55 — was globally loaded in base.html, now
opt-in per template via this extra_scripts block; admin_tables mounts
it via the BQ Register modal's `data-name="bq_package_ids"` host). #}
{% block extra_scripts %}
<script type="module" src="{{ static_url('js/components/chip-input.js') }}"></script>
{% endblock %}