* feat(unified-stack): Browse + My Stack + Recipes + RBAC matrix (v49–v55)
Squash of 94 commits spanning the v49 → v55 unified-stack rewrite.
Full per-feature breakdown lives in CHANGELOG.md under [Unreleased].
Major buckets:
* v49 schema — first-class user_groups + user_group_members +
resource_grants; admin can CRUD groups and grants; Google
Workspace nightly sync writes into the new tables.
* v49 data_packages — admin-curated bundles of tables, RBAC-gated,
first-class section on /catalog Browse + My Stack.
* v49 memory_domains — row-backed (replaces hardcoded VALID_DOMAINS
enum); admin can CRUD; grants follow the same shape as tables and
packages.
* v50 cover_image_url + admin sidebar collapsibles + per-row Mode
tooltip + admin queue domain badges + admin "+ New Item" seed flow.
* v51 lifecycle status (prod/poc/coming-soon/draft) + category +
palette swatches on admin modals.
* v52 per-table detail page /catalog/t/<id>.
* v53 Recipes — admin-curated SQL templates as a second tab on
/catalog with full Edit/Delete admin affordances.
* v54 soft-delete (deleted_at) + Undo toast for packages, memory
domains, and recipes; hard_delete() retained as escape hatch.
* v55 Recipes RBAC — ResourceType.RECIPE registered, inline Group
Access matrix on Create + Edit Recipe modals (mirrors the Memory
Domain pattern).
* Activity Center per-resource filter (resource_prefix LIKE-anchored
on audit_log.resource); admin nav g+letter keyboard shortcuts;
loadAdminTablesLayout N+1 → single endpoint; /api/memory 30s
page-level cache.
* CI hardening — Keboola legacy tests pytest.importorskip; perf-
smoke threshold widened to stop cold-cache flake.
5002 tests passing, 35 skipped.
* feat(p2 backlog): Cmd-K palette + suggest-a-domain + nightly E2E + v55 schema
10-item P2 sweep on top of the unified-stack squash. New behaviour:
* Cmd-K admin command palette (base.html) — fuzzy-search overlay over
admin + user-facing routes. Arrows/Enter to navigate, Esc to close.
* Stack-tabs digit shortcuts — 1/2/3 switch Browse / My Stack /
Recipes on /catalog + /corporate-memory.
* Friendlier non-admin empty state on /corporate-memory, plus a
"Suggest a domain" CTA → POST /api/memory-domain-suggestions, admin
queue with approve/reject. Backed by a new memory_domain_suggestions
table (schema v55).
* /admin/corporate-memory 7-tab strip grouped under Moderation /
Catalog parent labels.
* Bulk-assign table → package dropdown annotates each option with
"(N of M tables already in)" so the existing distribution is visible
before picking a target.
* GET /api/memory + /tree accept is_required filter; admin status
dropdowns route the "Required" sentinel onto it (status no longer
holds 'mandatory' post-v49, so the old dropdown returned nothing).
* chip-input.js is now opt-in per template via {% block extra_scripts %}
instead of loaded globally on every page from base.html.
* Edit-modal close helpers consolidated onto _closeEditModalById();
docs the per-source-type modal architecture decision.
* New .github/workflows/e2e-nightly.yml runs agent-browser smoke
scripts (scripts/e2e/smoke_*.sh) against a docker-compose stack
nightly at 04:30 UTC; failures open an agent-browser-nightly issue.
5012 tests passing, 35 skipped.
* fix(visual audit): 6 page regressions on memory + data-package surfaces
agent-browser walkthrough of every memory + data-package page in the PR
turned up 6 real bugs. Fixes:
1. Admin memory modals were dead. Duplicate `let _cmdNewDomainId`
declarations from the deprecated step-2 RBAC stubs in
admin_corporate_memory.html collided with the live state vars
declared earlier in the same <script> → SyntaxError on parse →
the entire second script block silently failed → every inline
onclick= handler defined there (`+ New Memory Domain`, Edit, etc.)
was a no-op. Removed the duplicate stubs.
2. /catalog/t/<table_id> + /catalog/r/<slug> rendered unstyled.
Both templates injected their CSS via {% block head %} but
base.html exposes {% block head_extra %} — wrong block name
meant <style> rules never reached the rendered HTML. Renamed
to head_extra. Hero card, section cards, dark SQL block, proper
full-width inputs all now render as designed.
3. L49 leak — "MANDATORY" KPI label + "Make Mandatory" row buttons
on /admin/corporate-memory still used the old word. Renamed to
"Required" / "Mark as Required" so UI matches the data model
(v49 split moved the Required tier onto the orthogonal
is_required boolean; status no longer holds 'mandatory').
4. Activity Center Resource dropdown didn't know the v55
`memory_domain_suggestion:` namespace — added it.
5. Tab strip on /admin/corporate-memory wrapped text 2× per button
on narrow viewports after the L50 MODERATION/CATALOG group
labels pushed total width past most viewports. Switched the
strip to flex-wrap:nowrap + overflow-x:auto with
white-space:nowrap + flex-shrink:0 on every direct child so the
tabs stay one row and slide horizontally when they overflow.
5012 tests passing, 35 skipped.
* rebase-cleanup: align with main's 0.54.25-27 API design + comment fix
Three follow-on fixes after rebasing onto origin/main (0.54.27):
* admin_tables.html: dropped a stray nested ``{% if data_source_type
== 'keboola' %}`` around ``prefillFromKeboolaTable`` (main never had
it; the outer Phase F2 guard already covers it) and reworded a JS
comment that contained literal ``{% %}`` tokens which Jinja was
parsing as a real tag → unbalanced if/endif → 30 template render
failures across the suite.
* /api/stack/subscription/{type}/{id}: DELETE now returns 204 instead
of 200 per the 0.54.26 design rules. CLI client + parity tests
updated to accept 2xx / assert 204.
* Memory-domain suggestion approve/reject paths added to
``_VERB_PATH_ALLOWLIST`` — they are pending → approved/rejected
state-machine transitions (approve also creates the real
memory_domains row as a side effect), so the RPC shape is
intentional rather than a missed PATCH refactor.
5035 tests passing, 35 skipped.
* fix(catalog_table_detail): real polish pass — hero glyph, dedup pills, rows/size meta, scoped sync CTA
The previous fix only got the block-name typo so the existing CSS rendered.
The actual layout was still wireframe-tier on close inspection:
* No cover glyph in the hero (a flat white card with title + meta line);
data-package + memory-domain detail pages both have a colored icon
square. Restored parity — table.icon emoji if set, otherwise initials
on a colored square using table.color.
* "INTERNAL" pill rendered twice for agnes_audit etc. — the mode pill
and the source-type pill happened to be identical strings. Now skip
the source pill when it matches the mode (`internal == internal`).
* Bucket / source_table code chip showed `Agnes Internal.audit_log` for
internal rows — meaningless to a user. Hidden when source_type is
internal.
* `pairs_well_with` admin input was a comma-separated `<input>` always
visible. Wrapped all 4 sections in an Edit-on-demand toggle: read-
only display by default, "+ Add" / "Edit" button on the right edge
of each section header reveals the inline form, Cancel hides it.
* "Trigger sync now" was a cramped link squashed into the empty-state
flex row (visible as `Tr…` overflow before). Promoted to a proper
btn-primary button under the empty-state copy. Hidden entirely for
internal tables (which are server-managed — no upstream to pull).
* Hero meta now surfaces row count + payload size (when sync_state has
them) + last sync timestamp on a single line — was missing from the
original.
* Mode pills colored by tier (local=green, remote=amber, materialized=
blue, internal=gray) so the basic fact about a table reads at a
glance, not from upper-cased ALL-CAPS text alone.
* tests(v56): TDD baseline for extended data-packages content + per-table docs
68 failing tests across 8 files spec the v56 surface before any
implementation lands:
* test_schema_v55_to_v56_migration.py — schema bump, additive ALTERs
on data_packages + table_registry, idempotency, sequential-upgrade
preservation
* test_data_packages_repo_v56.py — repo create/update/get/list for
owner_name, owner_team, tags, long_description, when_to_use,
when_not_to_use, example_questions (JSON list round-trip, empty
defaults, partial-update preservation)
* test_table_registry_v56_docs.py — update_docs for grain, platforms,
partition_col, history, gotchas; preserves v52 docs columns
* test_api_data_packages_v56.py — PUT/POST/GET for all new fields,
field-level validation (tag count, bullet length, description size),
virtual badge derivation (curated/new)
* test_api_registry_docs_v56.py — PATCH /api/admin/registry/{id}/docs
for v56 fields, validation, RBAC unchanged
* test_web_catalog_package_detail_v56.py — /catalog/p/<slug> rewrite
asserts on rendered owner line, tag pills, badges, What it is,
Use it when, Skip it when, Example questions, per-table extended
detail in collapsible row, key-gotcha distinctness, admin-only Edit
* test_web_stack_card_v56_metadata.py — Browse-grid card additions
(owner chip, tag chips, badges) without breaking back-compat for
rows missing the new fields
* test_data_packages_no_vendor_content.py — CI guard: scans app/ +
src/ + cli/ + config/ + scripts/ for Groupon-specific tokens from
the colleague's spec MD; fails if any leak into OSS surfaces
* test_db_schema_version.py — bumped 55 → 56 with rationale
Plus updates schema-version assertion to 56. Implementation lands in
subsequent commits (schema migration → repo → API → templates).
* feat(v56): schema + repo for extended data-packages content
Schema additions (ALTER ADD COLUMN IF NOT EXISTS — additive + idempotent):
* data_packages: owner_name, owner_team, tags, long_description,
when_to_use, when_not_to_use, example_questions (JSON-as-VARCHAR for
the lists)
* table_registry: grain, platforms, partition_col, history, gotchas
(extends the v52 sample_questions / things_to_know / pairs_well_with
docs surface with structured per-table content)
Repo extensions:
* DataPackagesRepository.create + update accept the new fields with
the same Optional-is-no-op contract as v51 (pass an empty list to
clear a JSON column)
* _decode_row decodes the new JSON-list columns to Python lists; NULL
rounds back to [] so callers don't branch
* TableRegistryRepository.update_docs grew the v56 fields alongside
the existing v52 ones — single PATCH can write either tier
atomically
* TableRegistryRepository._decode_row picks up platforms + gotchas in
the same NULL-tolerant decoder
22 repo + migration tests passing. API + UI land in subsequent commits.
* feat(v56): API surface for extended data-packages + per-table docs
CreateDataPackageRequest + UpdateDataPackageRequest grew the v56 fields
(owner_name, owner_team, tags, long_description, when_to_use,
when_not_to_use, example_questions) with per-field validators that
match the Foundry spec checklist:
* tags: ≤8 entries × ≤30 chars
* long_description: ≤4000 chars
* use/skip: ≤8 bullets × ≤200 chars
* example_questions: ≤12 × ≤200 chars
_serialize emits all v56 fields plus a virtual ``badges`` list derived
server-side at render time (no DB column needed): "curated" when the
creator is in the Admin group, "new" within 30 days of created_at.
Backdating created_at or admin-status changes pick up automatically.
PATCH /api/admin/registry/{id}/docs extended with v56 structured
per-table fields (grain, platforms, partition_col, history, gotchas).
gotchas: list of {key: bool, body: str} Pydantic models with the same
≤8 cap; first key=true entry becomes the Key gotcha on the rendered
package detail page. PATCH echoes the fresh state so callers can
re-render without a second GET.
26 API tests passing (16 data-packages + 10 registry-docs).
* feat(v56): /catalog/p/<slug> rewrite + Browse-grid card augmentation
The third (and final) v56 commit lights up the UI surfaces backed by
the schema + API commits earlier in this PR:
* /catalog/p/<slug> template rebuilt around the Foundry spec's
section ladder — hero (icon + name + badges + owner + tags +
description + meta + Add-to-stack), "What it is" markdown body,
paired "Use it when / Skip it when" panels, "Tables in this
package" with collapsible per-table extended detail (grain /
platforms / partition_col / history / gotchas + sample questions),
and an "Example questions you can ask Claude" prompt panel. Each
section guarded by ``{% if pkg.<field> %}`` — empty content fields
hide the section entirely (no "No X yet" placeholder noise on the
public-facing drilldown).
* router catalog_package_detail hydrates per-table v56 fields onto
the tables list + derives the virtual badges (curated / new)
server-side from creator-in-Admin + 30-day created_at.
* StackResolver.ResourceEntry grew owner_name / owner_team / tags /
badges; _fetch_entries pulls the v56 columns + computes badges
once per fetch using a single Admin-group SELECT.
* _data_package_entry_dict adapter passes the new fields through to
the macro; tags are merged source-type pills + admin-authored
category tags per the spec convention.
* _stack_card.html renders the v56 badges (top-left, data-badge=
hooks) + the owner chip (data-card-owner hook) without breaking
back-compat — pre-v56 rows render unchanged.
* Admin PUT handler strips the v56 docs fields from the
read-modify-write merged dict so register() doesn't blow up
with the now-larger row shape (same pattern as the v52 docs
fields stripping).
5115 tests passing (+98 v56 + 18 fixed regressions from the merged-
register PUT path), 35 skipped.
* fix(rbac): Edit-on-package + Group-access 'required' persistence + CI vendor guard
Three related bugs reported on the merged-with-main branch:
1. Clicking Edit on a Data Package card landed on /admin/tables with
a `#<pkg.id>` hash that nothing listened to — admin saw the global
table listing, not the editor for that specific package. Added a
`?edit_package=<pkg_id>` query-param handler in admin_tables.html
(analog to the existing `?edit=<table_id>` and `?assign_to=<pkg_id>`
patterns) that calls openEditDataPackageModal on DOMContentLoaded
after a 250ms layout settle. Updated the package-detail Edit link
to use the new query param.
2. Setting Group Access to 'required' didn't persist — re-opening
the modal showed 'available'. Root cause was the v49
``resource_grants.requirement`` enum existing in the DB but the
POST /api/admin/grants endpoint not surfacing it: ``CreateGrantRequest``
declared only group_id + resource_type + resource_id, so Pydantic
silently dropped the matrix's ``requirement: 'required'`` payload
and the new row landed at the DB column default ('available').
Plumbed ``requirement`` through ``CreateGrantRequest`` →
``ResourceGrantsRepository.create`` so the value persists in one
round-trip. Plus a UNIQUE-constraint race in the matrix
diff-apply: DELETE-old + POST-new ran in parallel via
``Promise.allSettled``, so POST could fire first and trip the
unique check before DELETE freed the slot. Switched to sequential
(await all deletes; then await all writes) across all three
matrices (Edit Data Package, Edit Memory Domain, Edit Recipe).
3. CI vendor-content guard ``test_no_groupon_specific_strings_in_oss``
tripped on two of my own docstrings: a "Foundry Data team" mention
in two src/db.py comments + an ``s1_session_landings`` example in
cli/skills/agnes-table-registration.md. Rephrased the comments to
"extended-descriptions admin spec" and replaced the example with
a generic ``events_daily`` table name.
5164 tests passing, 35 skipped (+4 regression tests pinning the POST
/api/admin/grants requirement contract). Vendor guard back to green.
* fix(catalog): admin Browse path drops v58 card fields
The /catalog and /memory admin god-mode branch built ResourceEntry
instances inline from pkg_repo.list() / domains_repo.list() and skipped
owner_name, owner_team, tags, and derived badges (curated/new). Visible
symptom: a package with an owner + tags rendered with the v56 chrome
for non-admin viewers but as a bare card for admins.
Adds StackResolver.browse_admin(user_id, resource_type) — admin god-mode
Browse that walks the full table but routes through the same
_fetch_entries enrichment pass as browse(), so admin + non-admin Browse
stay visually consistent. Both /catalog and /corporate-memory routes
switch to it.
Regression test in tests/test_stack_resolver_browse_admin.py covers:
owner/tags propagation, new/curated badge derivation, in_stack from
admin subscriptions, all-packages-regardless-of-grants, and the
ValueError for unsupported resource types.
* fix(catalog): three /catalog tab-strip UX bugs
1. Required Remove → red toast
browse_admin passed empty required_ids to _fetch_entries, so the
admin's own required grants surfaced as 'available' and the macro
rendered an actionable Remove button that POST /unsubscribe 400'd
on. Now derives required_ids from the admin's own groups so
Required packages render with the disabled "In stack (required)"
button. Regression test in test_stack_resolver_browse_admin.py.
2. Remove green-toasts but card stays until refresh
The My-Stack empty-state placeholder was only emitted server-side
when stack_entries was empty at render time. Removing the last
card left the tab completely blank — users read that as "Remove
didn't work, let me refresh". Both grid + empty-state are now
always rendered with one of them initially hidden; the JS swaps
visibility on add/remove instead of injecting DOM. Same fix in
/corporate-memory.
3. "What are Recipes?" + ambiguous (admin) suffix
Recipes tab now carries its own curator-block explainer (the
shared one was moved inside Browse view so it doesn't bleed
across tabs). The grey "(admin)" suffix becomes a yellow
.admin-only-hint chip with a title tooltip — visibility hint is
now unambiguous: yellow chip = "only you see this", non-admins
don't see the affordance at all.
* schema: renumber v51..v58 → v52..v59 to make room for main's v51
Main 0.54.29 introduced a NEW v51 (table_registry.bq_fqn — issue #343)
that releases ahead of this branch. The unified-stack chain v51..v58
shifts up by one so main's v51 stays as the released schema and ours
become v52..v59. Function names, internal version bumps, dispatch
ladder thresholds, and the migration-test references all move
together. Subsequent merge with main lands the bq_fqn column at the
freed v51 slot.
* fix(seed): seed admin lands in BOTH Admin AND Everyone groups
The LOCAL_DEV_MODE / SEED_ADMIN_EMAIL bootstrap only added the seed
user to Admin. Everyone-scoped grants — the canonical "every-user-
sees-this" pattern for Required onboarding — didn't surface for the
seed admin's own /catalog because they weren't in Everyone. Symptom:
admin grants a Required-tier package to Everyone, then sees it on
/catalog still rendered with an "Add to stack" button (because the
admin's resolved required_ids was empty for that package).
The dual-membership keeps Admin (authorization) and Everyone
(default-grant target) intentionally separate per the design comment
on UserRepository.create — every membership remains traceable to a
concrete row, just now with a system_seed row in Everyone too. Both
INSERTs go through UserGroupMembersRepository.add_member which is
idempotent on (user_id, group_id), so re-fires on every lifespan
startup don't duplicate rows.
Regression test in test_main_seed_admin_everyone.py.
* style: unify admin-only hints across marketplace + memory detail pages
Replaces three stale ``(admin)`` parentheticals with the same yellow
``admin-only`` chip introduced for /catalog tab actions. Same tooltip
copy ("Visible only to admins — analysts won't see this …") so the
visibility hint is unmistakable wherever it appears:
- Hard delete on marketplace_plugin_detail (admin-only destructive
action — same gating as the original suffix conveyed).
- Hard delete on marketplace_item_detail (same).
- Edit link on memory_domain_detail (title-attr only before; now a
visible chip too).
Non-admin viewers never saw these affordances — the gates are
unchanged. Pure styling pass for consistency.
* fix(catalog): exclude soft-deleted data packages + memory domains from Browse
``StackResolver._fetch_entries`` and ``browse_admin`` were querying
data_packages / memory_domains without a ``deleted_at IS NULL`` guard.
A package soft-deleted via /admin/* (v54 soft-delete contract) stayed
visible on /catalog and /memory until either an Undo or a hard delete
— directly contradicting the soft-delete UX which is supposed to
remove the affordance immediately and only retain the row for the
Undo window.
The repository accessors (DataPackagesRepository.list,
MemoryDomainsRepository.list, list_packages_of_table, etc.) already
filter deleted rows; this commit brings the resolver's direct SQL in
line with that contract.
Regression test in test_stack_resolver_browse_admin.py.
* fix(catalog): Add/Remove updates full card chrome, not just button
The previous _applyStackChange flipped only the footer button label —
the card border (.is-in-stack class), top-right "In stack" badge, and
button color class (--add / --remove) stayed at their server-rendered
state. After Add the user saw the button checkmark but the rest of
the card still looked like "available, not in stack". They read this
as "the change didn't take — let me refresh".
This commit makes the optimistic update mirror what the server-side
macro renders for the new state:
* ``c.classList.toggle('is-in-stack', becameInStack)`` — flips the
border + visual state class.
* Top-right ``.stack-card__req-badge--instack`` badge is injected on
Add, removed on Remove (skipped when ``data-requirement='required'``
— that slot is owned by the Required badge).
* Button text is "Remove" / "+ Add to stack" matching the macro
(was "✓ In stack" which was visually nice but inconsistent).
* Button color class --add / --remove swaps so the destructive Remove
tint kicks in immediately.
The clone-into-My-Stack path applies the same updates so the new card
in My Stack reads identically to a server-rendered in_stack card.
Mirrored in /corporate-memory.
* fix(memory): four Devin-review bugs on /memory drill-down + manifest
PR #333 Devin review surfaced four real bugs that ship a broken
/memory experience even though the unit tests passed.
1. Manifest md5 omits is_required + content (app/api/sync.py:836-840)
_build_memory_domains_section hashed only (id|title|status) per
item. _build_per_domain_markdown routes items between "## Required"
and "## Approved" by is_required and embeds full content — so an
admin edit of either dimension left the manifest md5 unchanged,
`agnes pull` skipped the re-fetch, and the analyst kept a stale
bundle.md. Now both fields participate in the hash.
2. required_count always 0 (src/repositories/memory_domains.py)
list_items_of_domain only SELECTed (id, title, status) so the
`it.get("is_required")` in the manifest builder always evaluated
to None → required_count = 0 regardless of actual state. The
manifest builder advertised a count it could never compute. Now
projects is_required + content too (required by fix 1 anyway).
3. Vote URL 404 (memory_domain_detail.html:289-290)
Constructed `/api/memory/items/{id}/vote` but the route is
`/api/memory/{id}/vote`. Every upvote/downvote button was a
silent no-op.
4. Dismiss/undismiss URL + method both wrong (memory_domain_detail.html:296-305)
Constructed `/api/memory/items/{id}/dismiss` (extra /items/) and
/undismiss (no such route — undismiss is DELETE on /dismiss).
Both buttons silently 404'd. Now POST + DELETE on
`/api/memory/{id}/dismiss` per app/api/memory.py:635/675.
* fix: multi-agent reviewer findings — vendor-token scrubs + manifest md5 predicate + soft-delete filter
Three reviewer findings from the multi-agent review on PR #333,
fixed in-place per CLAUDE.md issue-economy rule.
Reviewer-rules (Important — vendor-agnostic OSS):
- app/main.py:218 comment: replaced 'foundryai-prod' with generic
'a customer prod instance' phrasing. Public OSS repo must not
carry customer-specific tokens (CLAUDE.md § Project conventions).
- tests/test_table_registry_v56_docs.py:70 fixture string:
replaced "user_brand_affiliation = 'groupon'" with 'acme' on
the same rule.
Reviewer-architecture (closes still-unresolved Devin 🚩 ANALYSIS):
- app/api/sync.py _build_memory_domains_section: md5 hash loop now
filters items to the SAME predicate the bundle renderer uses
(is_required OR status='approved'). Pre-fix the hash iterated ALL
items but _build_per_domain_markdown only rendered the union of
required items + approved-non-required items — so an admin edit
to a pending/rejected non-required item flipped the md5 against
an identical-bytes bundle, triggering a wasteful re-fetch on
every analyst's next 'agnes pull'. The earlier commit fixed the
hash-input fields (is_required + content); this closes the
set-of-items asymmetry Devin separately flagged.
Reviewer-RBAC (minor cleanup):
- app/resource_types.py _data_package_blocks and _memory_domain_blocks
now filter 'WHERE deleted_at IS NULL' (v54 soft-delete column) so
the /admin/access UI doesn't surface soft-deleted entities as
grantable. Mirrors the existing filter on _recipe_blocks. No
security leak pre-fix (resolver double-filters and re-checks at
serve time), just UI cleanliness.
- app/services/stack_resolver.py add_to_stack: docstring note
added explaining that authorization is enforced at the API layer
(app/api/stack.py can_access gate), not at the resolver. The
initial review suggested adding a defensive 403 here, but that
broke 5 existing tests that legitimately call add_to_stack
directly without setting up grants first; the docstring captures
the contract instead. stack() already intersects subscriptions
with current available_ids on every read, so a 'zombie' row from
a misuse never leaks into the user-facing manifest.
* release: 0.55.0 — unified Browse + My Stack (Data Packages + Memory), schema v48→v59, 3 BREAKING
1143 lines
42 KiB
Python
1143 lines
42 KiB
Python
"""Unified admin REST API for user_groups, members, and resource_grants.
|
|
|
|
Replaces ``app.api.role_management`` and ``app.api.plugin_access`` with a
|
|
single namespace under ``/api/admin``:
|
|
|
|
- ``GET/POST/DELETE /api/admin/groups``
|
|
- ``GET/POST/DELETE /api/admin/groups/{group_id}/members``
|
|
- ``GET/POST/DELETE /api/admin/grants``
|
|
- ``GET /api/admin/resource-types``
|
|
|
|
Every endpoint is gated by ``require_admin``. Audit log entries are written
|
|
for every mutation so an admin's group/grant changes are traceable.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
import os
|
|
from datetime import datetime
|
|
from typing import Any, List, Optional
|
|
|
|
import duckdb
|
|
from fastapi import APIRouter, Depends, HTTPException
|
|
from pydantic import BaseModel
|
|
|
|
from app.auth.access import is_user_admin, require_admin
|
|
from app.auth.dependencies import _get_db, get_current_user
|
|
from app.resource_types import RESOURCE_TYPES, ResourceType, list_resource_types
|
|
from src.repositories.audit import AuditRepository
|
|
from src.repositories.user_groups import (
|
|
SystemGroupProtected,
|
|
UserGroupsRepository,
|
|
)
|
|
from src.repositories.resource_grants import ResourceGrantsRepository
|
|
from src.repositories.user_group_members import UserGroupMembersRepository
|
|
from src.repositories.users import UserRepository
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
router = APIRouter(prefix="/api/admin", tags=["access"])
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _audit(
|
|
conn: duckdb.DuckDBPyConnection,
|
|
actor_id: str,
|
|
action: str,
|
|
resource: str,
|
|
params: Optional[dict] = None,
|
|
) -> None:
|
|
try:
|
|
safe = None
|
|
if params:
|
|
safe = {
|
|
k: (v.isoformat() if isinstance(v, datetime) else v)
|
|
for k, v in params.items()
|
|
}
|
|
AuditRepository(conn).log(
|
|
user_id=actor_id, action=action, resource=resource, params=safe
|
|
)
|
|
except Exception:
|
|
# Audit failures must never break the mutation. Logged at WARN.
|
|
logger.warning("audit log failed for %s/%s", action, resource)
|
|
|
|
|
|
def _is_google_managed(g: dict) -> bool:
|
|
"""Whether a group row is owned by Google sync — admin UI/API treat such
|
|
rows as read-only.
|
|
|
|
Two ways a group can be Google-managed:
|
|
|
|
1. ``created_by='system:google-sync'`` — auto-created by the OAuth
|
|
callback when the user belonged to a prefix-matching Workspace
|
|
group; ``name`` is the full Workspace email.
|
|
2. ``is_system=TRUE`` AND the group's name matches the env-configured
|
|
admin/everyone Workspace email — the OAuth callback routes
|
|
memberships from those Workspace groups into the seeded system
|
|
row instead of creating a separate user_groups row, so the system
|
|
row effectively *becomes* a Google-synced row in this deployment.
|
|
Without the env mapping, system groups stay regular admin-managed
|
|
rows (renaming Admin is still blocked separately by
|
|
``UserGroupsRepository`` for code-reference safety).
|
|
"""
|
|
if (g.get("created_by") or "") == "system:google-sync":
|
|
return True
|
|
if g.get("is_system"):
|
|
from src.db import SYSTEM_ADMIN_GROUP, SYSTEM_EVERYONE_GROUP
|
|
admin_email = os.environ.get(
|
|
"AGNES_GROUP_ADMIN_EMAIL", ""
|
|
).strip().lower()
|
|
everyone_email = os.environ.get(
|
|
"AGNES_GROUP_EVERYONE_EMAIL", ""
|
|
).strip().lower()
|
|
if admin_email and g.get("name") == SYSTEM_ADMIN_GROUP:
|
|
return True
|
|
if everyone_email and g.get("name") == SYSTEM_EVERYONE_GROUP:
|
|
return True
|
|
return False
|
|
|
|
|
|
def _guard_google_managed(g: dict) -> None:
|
|
"""Raise 409 google_managed_readonly when the group is Google-managed."""
|
|
if _is_google_managed(g):
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail={
|
|
"code": "google_managed_readonly",
|
|
"message": (
|
|
"This group is managed by Google Workspace and is "
|
|
"read-only here. Add or remove members via "
|
|
"admin.google.com, or sign in again to refresh."
|
|
),
|
|
},
|
|
)
|
|
|
|
|
|
def _validate_resource_type(value: str) -> ResourceType:
|
|
try:
|
|
return ResourceType(value)
|
|
except ValueError:
|
|
raise HTTPException(
|
|
status_code=400,
|
|
detail=(
|
|
f"Unknown resource_type {value!r}. Known types: "
|
|
f"{[rt.value for rt in ResourceType]}"
|
|
),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Resource types (read-only, from Python enum)
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@router.get("/resource-types", response_model=List[dict])
|
|
async def get_resource_types(
|
|
user: dict = Depends(require_admin),
|
|
):
|
|
"""List the resource types defined in app.resource_types.
|
|
|
|
No DB call — these come from the ``ResourceType`` StrEnum. The shape
|
|
is ``[{key, display_name, description, id_format}]`` so the admin UI
|
|
can render the create-grant form's resource_type dropdown plus a
|
|
placeholder hint for the ``resource_id`` input.
|
|
"""
|
|
return list_resource_types()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Access overview — single-shot payload for the /admin/access page
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@router.get("/access-overview", response_model=dict)
|
|
async def access_overview(
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""One-shot snapshot for the /admin/access page.
|
|
|
|
Returns:
|
|
- ``groups``: every user_group with member + grant counts
|
|
- ``grants``: every (group_id, resource_type, resource_id) row
|
|
- ``resources``: per-resource-type hierarchical layout, where each
|
|
type has a list of *blocks* (parent entities, e.g. a marketplace)
|
|
and each block has *items* (concrete grantable resources).
|
|
|
|
UI stitches the three pieces into the two-column layout: groups on
|
|
the left, resources tree on the right with per-item checkboxes whose
|
|
state derives from ``grants``.
|
|
"""
|
|
groups_rows = UserGroupsRepository(conn).list_all()
|
|
members_repo = UserGroupMembersRepository(conn)
|
|
grants_repo = ResourceGrantsRepository(conn)
|
|
|
|
groups = []
|
|
for g in groups_rows:
|
|
groups.append({
|
|
"id": g["id"],
|
|
"name": g["name"],
|
|
"description": g.get("description"),
|
|
"is_system": bool(g.get("is_system", False)),
|
|
"created_by": g.get("created_by"),
|
|
# Same origin / google-management surface as `/api/admin/groups`
|
|
# so the /admin/access sidebar can render the identical pill +
|
|
# subtitle treatment without a second source of truth.
|
|
"origin": _derive_origin(g),
|
|
"is_google_managed": _is_google_managed(g),
|
|
"mapped_email": _mapped_email(g),
|
|
"member_count": members_repo.count_members(g["id"]),
|
|
"grant_count": grants_repo.count_for_group(g["id"]),
|
|
})
|
|
|
|
grants = [
|
|
{
|
|
"id": r["id"],
|
|
"group_id": r["group_id"],
|
|
"resource_type": r["resource_type"],
|
|
"resource_id": r["resource_id"],
|
|
}
|
|
for r in grants_repo.list_all()
|
|
]
|
|
|
|
# Per-resource-type hierarchies. Driven by the registry in
|
|
# app.resource_types — adding a new type there is the one place that
|
|
# surfaces here, no extra wiring. Disabled types (e.g. TABLE without
|
|
# AGNES_ENABLE_TABLE_GRANTS) are skipped so the admin UI does not
|
|
# render a chip for grants the runtime cannot enforce yet.
|
|
from app.resource_types import enabled_resource_types
|
|
resources = [
|
|
{
|
|
"type_key": spec.key.value,
|
|
"type_display": spec.display_name,
|
|
"blocks": spec.list_blocks(conn),
|
|
}
|
|
for spec in enabled_resource_types()
|
|
]
|
|
|
|
return {"groups": groups, "grants": grants, "resources": resources}
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# User groups
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class GroupResponse(BaseModel):
|
|
id: str
|
|
name: str
|
|
description: Optional[str] = None
|
|
is_system: bool = False
|
|
# 'system' | 'custom' | 'google_sync'. ``custom`` = created by an admin
|
|
# via the UI/CLI (no system marker, no google-sync marker on
|
|
# ``created_by``). Mapped Admin/Everyone (system row wired to a
|
|
# Workspace group via AGNES_GROUP_{ADMIN,EVERYONE}_EMAIL) report
|
|
# 'google_sync' here — Workspace is the authoritative source of
|
|
# membership for those rows, so the chip should advertise that, not
|
|
# the seed mechanism. Unmapped Admin/Everyone stay 'system'.
|
|
origin: str = "custom"
|
|
created_at: Optional[str] = None
|
|
created_by: Optional[str] = None
|
|
member_count: int = 0
|
|
grant_count: int = 0
|
|
# True iff the row is owned by Google sync — admin UI hides edit/delete
|
|
# affordances and the API rejects mutations with 409 google_managed_readonly.
|
|
is_google_managed: bool = False
|
|
# When the row is the seeded Admin / Everyone system group AND the
|
|
# corresponding env-mapping is configured, this is the upstream
|
|
# Workspace group email that funnels members in. The admin UI renders
|
|
# it as a subtitle under the canonical name (`Admin / admins@...`)
|
|
# so operators can see *which* Workspace group is wired to the system
|
|
# row. Null for regular google_sync rows (their email is already in
|
|
# `name`) and for unmapped system rows.
|
|
mapped_email: Optional[str] = None
|
|
|
|
|
|
class CreateGroupRequest(BaseModel):
|
|
name: str
|
|
description: Optional[str] = None
|
|
|
|
|
|
class UpdateGroupRequest(BaseModel):
|
|
name: Optional[str] = None
|
|
description: Optional[str] = None
|
|
|
|
|
|
def _derive_origin(g: dict) -> str:
|
|
"""Project a 3-value origin tag from existing user_groups columns.
|
|
|
|
- mapped via ``AGNES_GROUP_{ADMIN,EVERYONE}_EMAIL`` → 'google_sync'
|
|
(the seed badge is suppressed when the row is wired to Workspace —
|
|
Workspace is the authoritative source of membership)
|
|
- ``is_system=TRUE`` (otherwise) → 'system'
|
|
- ``created_by`` starts with 'system:google' → 'google_sync'
|
|
- other ``system:`` prefixed creator → 'system'
|
|
- else → 'custom'
|
|
(admin-created via UI/CLI — the value is named after the *origin*,
|
|
not the creator's role, so it doesn't visually clash with the
|
|
seeded `Admin` system row in the chip layer)
|
|
"""
|
|
is_system = bool(g.get("is_system"))
|
|
cb = g.get("created_by") or ""
|
|
name = g.get("name") or ""
|
|
if is_system:
|
|
from src.db import SYSTEM_ADMIN_GROUP, SYSTEM_EVERYONE_GROUP
|
|
admin_email = os.environ.get("AGNES_GROUP_ADMIN_EMAIL", "").strip()
|
|
everyone_email = os.environ.get("AGNES_GROUP_EVERYONE_EMAIL", "").strip()
|
|
if (admin_email and name == SYSTEM_ADMIN_GROUP) or (
|
|
everyone_email and name == SYSTEM_EVERYONE_GROUP
|
|
):
|
|
return "google_sync"
|
|
return "system"
|
|
if cb.startswith("system:google"):
|
|
return "google_sync"
|
|
if cb.startswith("system:"):
|
|
return "system"
|
|
return "custom"
|
|
|
|
|
|
def _mapped_email(g: dict) -> Optional[str]:
|
|
"""The Workspace group email that funnels members into a system row.
|
|
|
|
Only returns a value when the row is the seeded ``Admin`` / ``Everyone``
|
|
system group AND the matching env var is configured. Null otherwise —
|
|
regular google_sync rows already carry the email in ``name``, and
|
|
unmapped system rows have nothing to show.
|
|
"""
|
|
if not g.get("is_system"):
|
|
return None
|
|
from src.db import SYSTEM_ADMIN_GROUP, SYSTEM_EVERYONE_GROUP
|
|
name = g.get("name")
|
|
if name == SYSTEM_ADMIN_GROUP:
|
|
v = os.environ.get("AGNES_GROUP_ADMIN_EMAIL", "").strip()
|
|
return v or None
|
|
if name == SYSTEM_EVERYONE_GROUP:
|
|
v = os.environ.get("AGNES_GROUP_EVERYONE_EMAIL", "").strip()
|
|
return v or None
|
|
return None
|
|
|
|
|
|
def _group_to_response(
|
|
g: dict,
|
|
members_repo: UserGroupMembersRepository,
|
|
grants_repo: ResourceGrantsRepository,
|
|
) -> GroupResponse:
|
|
return GroupResponse(
|
|
id=g["id"],
|
|
name=g["name"],
|
|
description=g.get("description"),
|
|
is_system=bool(g.get("is_system", False)),
|
|
origin=_derive_origin(g),
|
|
created_at=str(g["created_at"]) if g.get("created_at") else None,
|
|
created_by=g.get("created_by"),
|
|
member_count=members_repo.count_members(g["id"]),
|
|
grant_count=grants_repo.count_for_group(g["id"]),
|
|
is_google_managed=_is_google_managed(g),
|
|
mapped_email=_mapped_email(g),
|
|
)
|
|
|
|
|
|
@router.get("/groups", response_model=List[GroupResponse])
|
|
async def list_groups(
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
groups = UserGroupsRepository(conn).list_all()
|
|
members_repo = UserGroupMembersRepository(conn)
|
|
grants_repo = ResourceGrantsRepository(conn)
|
|
return [_group_to_response(g, members_repo, grants_repo) for g in groups]
|
|
|
|
|
|
@router.get("/groups/{group_id}", response_model=GroupResponse)
|
|
async def get_group(
|
|
group_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""Single-group payload for the /admin/groups/{id} detail page header."""
|
|
g = UserGroupsRepository(conn).get(group_id)
|
|
if not g:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
members_repo = UserGroupMembersRepository(conn)
|
|
grants_repo = ResourceGrantsRepository(conn)
|
|
return _group_to_response(g, members_repo, grants_repo)
|
|
|
|
|
|
@router.post("/groups", response_model=GroupResponse, status_code=201)
|
|
async def create_group(
|
|
payload: CreateGroupRequest,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
name = payload.name.strip()
|
|
if not name:
|
|
raise HTTPException(status_code=400, detail="Group name is required")
|
|
repo = UserGroupsRepository(conn)
|
|
if repo.get_by_name(name):
|
|
raise HTTPException(status_code=409, detail=f"Group {name!r} already exists")
|
|
g = repo.create(
|
|
name=name,
|
|
description=payload.description,
|
|
created_by=user.get("email"),
|
|
)
|
|
_audit(
|
|
conn, user["id"], "user_group.created", f"group:{g['id']}",
|
|
{"name": name},
|
|
)
|
|
members_repo = UserGroupMembersRepository(conn)
|
|
grants_repo = ResourceGrantsRepository(conn)
|
|
return _group_to_response(g, members_repo, grants_repo)
|
|
|
|
|
|
@router.patch("/groups/{group_id}", response_model=GroupResponse)
|
|
async def update_group(
|
|
group_id: str,
|
|
payload: UpdateGroupRequest,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
repo = UserGroupsRepository(conn)
|
|
g = repo.get(group_id)
|
|
if not g:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
_guard_google_managed(g)
|
|
if g.get("is_system") and payload.name is not None and payload.name.strip() != g["name"]:
|
|
# System groups: block renames (the canonical names "Admin" /
|
|
# "Everyone" are referenced from app.auth.access and the
|
|
# marketplace filter), but description edits are cosmetic and
|
|
# allowed (admins curate them in /admin/access). The repo
|
|
# layer's narrowed guard (src/repositories/user_groups.py) is
|
|
# the second line of defense.
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="System groups cannot be renamed",
|
|
)
|
|
updates: dict = {}
|
|
if payload.name is not None and payload.name.strip() != g["name"]:
|
|
updates["name"] = payload.name.strip()
|
|
if payload.description is not None:
|
|
updates["description"] = payload.description
|
|
if updates:
|
|
try:
|
|
repo.update(group_id, **updates)
|
|
except SystemGroupProtected:
|
|
raise HTTPException(
|
|
status_code=409, detail="System groups cannot be renamed",
|
|
)
|
|
_audit(conn, user["id"], "user_group.updated", f"group:{group_id}", updates)
|
|
g = repo.get(group_id)
|
|
members_repo = UserGroupMembersRepository(conn)
|
|
grants_repo = ResourceGrantsRepository(conn)
|
|
return _group_to_response(g, members_repo, grants_repo)
|
|
|
|
|
|
@router.delete("/groups/{group_id}", status_code=204)
|
|
async def delete_group(
|
|
group_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
repo = UserGroupsRepository(conn)
|
|
g = repo.get(group_id)
|
|
if not g:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
_guard_google_managed(g)
|
|
if g.get("is_system"):
|
|
raise HTTPException(status_code=409, detail="Cannot delete a system group")
|
|
# Cascade members + grants atomically with the group row so a partial
|
|
# failure cannot leave orphans pointing at a deleted group_id. There are
|
|
# no FK constraints (group_id is plain VARCHAR), so the application is
|
|
# responsible for the invariant — wrap in an explicit transaction.
|
|
try:
|
|
conn.execute("BEGIN TRANSACTION")
|
|
conn.execute(
|
|
"DELETE FROM user_group_members WHERE group_id = ?", [group_id]
|
|
)
|
|
conn.execute(
|
|
"DELETE FROM resource_grants WHERE group_id = ?", [group_id]
|
|
)
|
|
repo.delete(group_id)
|
|
conn.execute("COMMIT")
|
|
except SystemGroupProtected:
|
|
conn.execute("ROLLBACK")
|
|
raise HTTPException(status_code=409, detail="Cannot delete a system group")
|
|
except Exception:
|
|
conn.execute("ROLLBACK")
|
|
raise
|
|
_audit(
|
|
conn, user["id"], "user_group.deleted", f"group:{group_id}",
|
|
{"name": g["name"]},
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Group members
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class MemberResponse(BaseModel):
|
|
user_id: str
|
|
email: str
|
|
name: Optional[str] = None
|
|
active: bool = True
|
|
source: str
|
|
added_at: Optional[str] = None
|
|
added_by: Optional[str] = None
|
|
|
|
|
|
class AddMemberRequest(BaseModel):
|
|
email: str
|
|
|
|
|
|
@router.get("/groups/{group_id}/members", response_model=List[MemberResponse])
|
|
async def list_members(
|
|
group_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
if not UserGroupsRepository(conn).get(group_id):
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
rows = UserGroupMembersRepository(conn).list_members_for_group(group_id)
|
|
return [
|
|
MemberResponse(
|
|
user_id=r["id"],
|
|
email=r["email"],
|
|
name=r.get("name"),
|
|
active=bool(r.get("active", True)),
|
|
source=r["source"],
|
|
added_at=str(r["added_at"]) if r.get("added_at") else None,
|
|
added_by=r.get("added_by"),
|
|
)
|
|
for r in rows
|
|
]
|
|
|
|
|
|
@router.post("/groups/{group_id}/members", response_model=MemberResponse, status_code=201)
|
|
async def add_member(
|
|
group_id: str,
|
|
payload: AddMemberRequest,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
g = UserGroupsRepository(conn).get(group_id)
|
|
if not g:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
_guard_google_managed(g)
|
|
target = UserRepository(conn).get_by_email(payload.email)
|
|
if not target:
|
|
raise HTTPException(status_code=404, detail=f"User {payload.email!r} not found")
|
|
members = UserGroupMembersRepository(conn)
|
|
if members.has_membership(target["id"], group_id):
|
|
raise HTTPException(status_code=409, detail="User already a member")
|
|
members.add_member(
|
|
user_id=target["id"],
|
|
group_id=group_id,
|
|
source="admin",
|
|
added_by=user.get("email"),
|
|
)
|
|
_audit(
|
|
conn, user["id"], "user_group.member_added",
|
|
f"group:{group_id}",
|
|
{"user_email": payload.email},
|
|
)
|
|
return MemberResponse(
|
|
user_id=target["id"],
|
|
email=target["email"],
|
|
name=target.get("name"),
|
|
active=bool(target.get("active", True)),
|
|
source="admin",
|
|
added_at=None,
|
|
added_by=user.get("email"),
|
|
)
|
|
|
|
|
|
@router.delete("/groups/{group_id}/members/{user_id}", status_code=204)
|
|
async def remove_member(
|
|
group_id: str,
|
|
user_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
members = UserGroupMembersRepository(conn)
|
|
# Last-admin guard: refuse to remove anyone from the seeded Admin group
|
|
# when they are the only active admin — recovery from zero admins
|
|
# requires direct DB access. Same protection as delete_user / update_user
|
|
# (active=False) in app/api/users.py.
|
|
group = UserGroupsRepository(conn).get(group_id)
|
|
if not group:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
_guard_google_managed(group)
|
|
if (
|
|
group["name"] == "Admin"
|
|
and is_user_admin(user_id, conn)
|
|
and UserRepository(conn).count_admins(active_only=True) <= 1
|
|
):
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="Cannot remove the last admin — at least one user must remain in the Admin group",
|
|
)
|
|
# Only delete admin-source rows from this endpoint. Google-sync rows
|
|
# rebuild themselves on next login; system_seed rows survive deploys.
|
|
removed = members.remove_member(user_id, group_id, require_source="admin")
|
|
if not removed:
|
|
raise HTTPException(
|
|
status_code=404,
|
|
detail="No admin-managed membership for this user in this group",
|
|
)
|
|
_audit(
|
|
conn, user["id"], "user_group.member_removed",
|
|
f"group:{group_id}",
|
|
{"user_id": user_id},
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Resource grants
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class GrantResponse(BaseModel):
|
|
id: str
|
|
group_id: str
|
|
group_name: str
|
|
resource_type: str
|
|
resource_id: str
|
|
assigned_at: Optional[str] = None
|
|
assigned_by: Optional[str] = None
|
|
# v49: 'available' | 'required' — Required tier is in-stack by default
|
|
# for every group member without an explicit subscription.
|
|
requirement: str = "available"
|
|
|
|
|
|
class CreateGrantRequest(BaseModel):
|
|
group_id: str
|
|
resource_type: str
|
|
resource_id: str
|
|
# v49 added the ``requirement`` enum on ``resource_grants``; the POST
|
|
# endpoint must accept it so clients can create a grant at the
|
|
# ``required`` tier in one round-trip. Without this, /admin/access
|
|
# + the inline RBAC matrices (Edit Data Package / Edit Memory Domain
|
|
# / Edit Recipe) silently fell through to the column default
|
|
# (``available``), and a re-open of the same modal showed the
|
|
# admin's "required" pick as "available" — looks like the save
|
|
# silently failed. Default kept at None so callers that don't
|
|
# explicitly pass a value still land at DB's column default.
|
|
requirement: Optional[str] = None
|
|
|
|
|
|
def _grant_to_response(g: dict) -> GrantResponse:
|
|
return GrantResponse(
|
|
id=g["id"],
|
|
group_id=g["group_id"],
|
|
group_name=g.get("group_name", ""),
|
|
resource_type=g["resource_type"],
|
|
resource_id=g["resource_id"],
|
|
assigned_at=str(g["assigned_at"]) if g.get("assigned_at") else None,
|
|
assigned_by=g.get("assigned_by"),
|
|
requirement=g.get("requirement") or "available",
|
|
)
|
|
|
|
|
|
@router.get("/grants", response_model=List[GrantResponse])
|
|
async def list_grants(
|
|
resource_type: Optional[str] = None,
|
|
group_id: Optional[str] = None,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
if resource_type:
|
|
_validate_resource_type(resource_type)
|
|
rows = ResourceGrantsRepository(conn).list_all(
|
|
resource_type=resource_type, group_id=group_id,
|
|
)
|
|
return [_grant_to_response(r) for r in rows]
|
|
|
|
|
|
@router.post("/grants", response_model=GrantResponse, status_code=201)
|
|
async def create_grant(
|
|
payload: CreateGrantRequest,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
rt = _validate_resource_type(payload.resource_type)
|
|
# Feature gate: refuse to mint grants for resource types whose runtime
|
|
# enforcement is not wired up yet (e.g. ResourceType.TABLE without
|
|
# AGNES_ENABLE_TABLE_GRANTS). Listing + deleting existing rows still
|
|
# works so operators can clean up legacy data.
|
|
from app.resource_types import is_resource_type_enabled
|
|
if not is_resource_type_enabled(rt):
|
|
raise HTTPException(
|
|
status_code=422,
|
|
detail=(
|
|
f"resource_type {rt.value!r} is not currently enabled. "
|
|
"Set AGNES_ENABLE_TABLE_GRANTS=1 to opt in once the runtime "
|
|
"enforcement is in place (see docs/TODO-rbac-data-enforcement.md)."
|
|
),
|
|
)
|
|
if not payload.resource_id.strip():
|
|
raise HTTPException(status_code=400, detail="resource_id is required")
|
|
if not UserGroupsRepository(conn).get(payload.group_id):
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
grants = ResourceGrantsRepository(conn)
|
|
# v49 ``requirement`` is part of the create-grant contract. Validate
|
|
# the enum here so the 422 message matches the endpoint contract
|
|
# rather than leaking a ValueError from the repo layer.
|
|
if payload.requirement is not None and payload.requirement not in (
|
|
"available", "required",
|
|
):
|
|
raise HTTPException(
|
|
status_code=422,
|
|
detail="requirement must be 'available' or 'required'",
|
|
)
|
|
try:
|
|
grant_id = grants.create(
|
|
group_id=payload.group_id,
|
|
resource_type=rt.value,
|
|
resource_id=payload.resource_id,
|
|
assigned_by=user.get("email"),
|
|
requirement=payload.requirement,
|
|
)
|
|
except duckdb.ConstraintException:
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="Grant already exists for this group/resource_type/resource_id",
|
|
)
|
|
_audit(
|
|
conn, user["id"], "resource_grant.created",
|
|
f"grant:{grant_id}",
|
|
{
|
|
"group_id": payload.group_id,
|
|
"resource_type": rt.value,
|
|
"resource_id": payload.resource_id,
|
|
},
|
|
)
|
|
# Re-read with the group name joined for the response.
|
|
rows = grants.list_all()
|
|
fresh = next((r for r in rows if r["id"] == grant_id), None)
|
|
if not fresh:
|
|
raise HTTPException(status_code=500, detail="Grant created but lookup failed")
|
|
return _grant_to_response(fresh)
|
|
|
|
|
|
class UpdateGrantRequirementRequest(BaseModel):
|
|
requirement: str # 'available' | 'required'
|
|
|
|
|
|
@router.put("/grants/{grant_id}", response_model=GrantResponse)
|
|
async def update_grant_requirement(
|
|
grant_id: str,
|
|
payload: UpdateGrantRequirementRequest,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""Update the ``requirement`` enum on an existing grant.
|
|
|
|
v49 — Section 4.5 of the unified-stack design (soft downgrade): when
|
|
transitioning ``required → available`` we eagerly materialize a
|
|
``user_stack_subscriptions`` row for every user currently in the
|
|
granted group, so the resource stays in their stack instead of
|
|
silently disappearing on the next refresh. The two writes happen
|
|
inside a single DuckDB transaction.
|
|
|
|
Going the other direction (``available → required``) is a no-op for
|
|
subscriptions — required is the always-in-stack tier and the
|
|
StackResolver treats required ids as in_stack regardless of any
|
|
subscription row.
|
|
"""
|
|
if payload.requirement not in ("available", "required"):
|
|
raise HTTPException(
|
|
status_code=400,
|
|
detail="requirement must be 'available' or 'required'",
|
|
)
|
|
grants = ResourceGrantsRepository(conn)
|
|
existing = grants.get(grant_id)
|
|
if not existing:
|
|
raise HTTPException(status_code=404, detail="Grant not found")
|
|
|
|
# All-or-nothing transition under one transaction so a fan-out failure
|
|
# doesn't leave the requirement flipped without the materialized
|
|
# subscriptions in place.
|
|
conn.execute("BEGIN")
|
|
try:
|
|
prior = grants.update_requirement(grant_id, payload.requirement)
|
|
# Soft-downgrade: required → available eagerly subscribes every
|
|
# group member to preserve continuity. ON CONFLICT DO NOTHING
|
|
# makes this idempotent if any subscription already exists.
|
|
if prior == "required" and payload.requirement == "available":
|
|
conn.execute(
|
|
"""INSERT INTO user_stack_subscriptions
|
|
(user_id, resource_type, resource_id)
|
|
SELECT m.user_id, ?, ?
|
|
FROM user_group_members m
|
|
WHERE m.group_id = ?
|
|
ON CONFLICT DO NOTHING""",
|
|
[existing["resource_type"], existing["resource_id"],
|
|
existing["group_id"]],
|
|
)
|
|
conn.execute("COMMIT")
|
|
except Exception:
|
|
conn.execute("ROLLBACK")
|
|
raise
|
|
|
|
_audit(
|
|
conn, user["id"], "resource_grant.requirement_updated",
|
|
f"grant:{grant_id}",
|
|
{
|
|
"prior": prior,
|
|
"new": payload.requirement,
|
|
"resource_type": existing["resource_type"],
|
|
"resource_id": existing["resource_id"],
|
|
"group_id": existing["group_id"],
|
|
},
|
|
)
|
|
|
|
# Re-read with the group name joined for the response.
|
|
rows = grants.list_all()
|
|
fresh = next((r for r in rows if r["id"] == grant_id), None)
|
|
if not fresh:
|
|
raise HTTPException(status_code=500, detail="Grant updated but lookup failed")
|
|
return _grant_to_response(fresh)
|
|
|
|
|
|
@router.delete("/grants/{grant_id}", status_code=204)
|
|
async def delete_grant(
|
|
grant_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
grants = ResourceGrantsRepository(conn)
|
|
existing = grants.get(grant_id)
|
|
if not existing:
|
|
raise HTTPException(status_code=404, detail="Grant not found")
|
|
|
|
# v39: refuse to revoke a grant whose underlying plugin is system-marked.
|
|
# The mark_system endpoint materializes per-group rows precisely so the
|
|
# plugin reaches every user; allowing per-group revoke here would punch
|
|
# a hole in the mandatory tier silently. Admin must unmark on
|
|
# /admin/marketplaces first, then revoke individual groups.
|
|
if existing["resource_type"] == "marketplace_plugin":
|
|
rid = existing["resource_id"] or ""
|
|
if "/" in rid:
|
|
mp_id, plugin_name = rid.split("/", 1)
|
|
sys_row = conn.execute(
|
|
"SELECT is_system FROM marketplace_plugins "
|
|
"WHERE marketplace_id = ? AND name = ?",
|
|
[mp_id, plugin_name],
|
|
).fetchone()
|
|
if sys_row and bool(sys_row[0]):
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="cannot_revoke_system_grant",
|
|
)
|
|
|
|
grants.delete(grant_id)
|
|
|
|
# v24: re-grant of the same plugin must reset every user to the default
|
|
# (subscribed). Drop matching subscriptions at the same time we drop the
|
|
# grant so state stays consistent — see
|
|
# src/repositories/user_curated_subscriptions.py.
|
|
optouts_dropped = 0
|
|
if existing["resource_type"] == "marketplace_plugin":
|
|
rid = existing["resource_id"] or ""
|
|
if "/" in rid:
|
|
mp_id, plugin_name = rid.split("/", 1)
|
|
from src.repositories.user_curated_subscriptions import (
|
|
UserCuratedSubscriptionsRepository,
|
|
)
|
|
optouts_dropped = UserCuratedSubscriptionsRepository(
|
|
conn
|
|
).delete_for_plugin(mp_id, plugin_name)
|
|
try:
|
|
from app.marketplace_server import packager
|
|
packager.invalidate_etag_cache()
|
|
except Exception:
|
|
pass
|
|
|
|
_audit(
|
|
conn, user["id"], "resource_grant.deleted", f"grant:{grant_id}",
|
|
{
|
|
"group_id": existing["group_id"],
|
|
"resource_type": existing["resource_type"],
|
|
"resource_id": existing["resource_id"],
|
|
"optouts_dropped": optouts_dropped,
|
|
},
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# User-centric views — back the /admin/users/{id} detail page.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
class UserMembershipResponse(BaseModel):
|
|
group_id: str
|
|
group_name: str
|
|
is_system: bool = False
|
|
# 'system' | 'custom' | 'google_sync' — same shared helper as
|
|
# /api/admin/groups + /api/users so the user detail page colors the
|
|
# membership chips identically to the user list and the groups page.
|
|
origin: str = "custom"
|
|
source: str
|
|
added_at: Optional[str] = None
|
|
added_by: Optional[str] = None
|
|
|
|
|
|
class AddUserToGroupRequest(BaseModel):
|
|
group_id: str
|
|
|
|
|
|
@router.get(
|
|
"/users/{user_id}/memberships",
|
|
response_model=List[UserMembershipResponse],
|
|
)
|
|
async def list_user_memberships(
|
|
user_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""Groups a user belongs to, joined with group metadata for display.
|
|
|
|
Includes ``source`` so the UI can distinguish admin-managed memberships
|
|
(deletable from this page) from Google-synced or system-seeded ones
|
|
(read-only — managed by their own writer).
|
|
"""
|
|
if not UserRepository(conn).get_by_id(user_id):
|
|
raise HTTPException(status_code=404, detail="User not found")
|
|
rows = conn.execute(
|
|
"""SELECT m.group_id, g.name AS group_name, g.is_system,
|
|
g.created_by, m.source, m.added_at, m.added_by
|
|
FROM user_group_members m
|
|
JOIN user_groups g ON g.id = m.group_id
|
|
WHERE m.user_id = ?
|
|
ORDER BY g.is_system DESC, g.name""",
|
|
[user_id],
|
|
).fetchall()
|
|
return [
|
|
UserMembershipResponse(
|
|
group_id=r[0],
|
|
group_name=r[1],
|
|
is_system=bool(r[2]),
|
|
origin=_derive_origin(
|
|
{"is_system": bool(r[2]), "name": r[1], "created_by": r[3]}
|
|
),
|
|
source=r[4],
|
|
added_at=str(r[5]) if r[5] else None,
|
|
added_by=r[6],
|
|
)
|
|
for r in rows
|
|
]
|
|
|
|
|
|
@router.post(
|
|
"/users/{user_id}/memberships",
|
|
response_model=UserMembershipResponse,
|
|
status_code=201,
|
|
)
|
|
async def add_user_to_group(
|
|
user_id: str,
|
|
payload: AddUserToGroupRequest,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""Add a user to a group from the user-centric page.
|
|
|
|
Mirror of POST /api/admin/groups/{id}/members but keyed on the user.
|
|
Always writes ``source='admin'`` so the row survives Google sync.
|
|
"""
|
|
if not UserRepository(conn).get_by_id(user_id):
|
|
raise HTTPException(status_code=404, detail="User not found")
|
|
group = UserGroupsRepository(conn).get(payload.group_id)
|
|
if not group:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
_guard_google_managed(group)
|
|
members = UserGroupMembersRepository(conn)
|
|
if members.has_membership(user_id, payload.group_id):
|
|
raise HTTPException(status_code=409, detail="Already a member")
|
|
members.add_member(
|
|
user_id=user_id,
|
|
group_id=payload.group_id,
|
|
source="admin",
|
|
added_by=user.get("email"),
|
|
)
|
|
_audit(
|
|
conn, user["id"], "user_group.member_added",
|
|
f"user:{user_id}",
|
|
{"group_id": payload.group_id, "group_name": group["name"]},
|
|
)
|
|
return UserMembershipResponse(
|
|
group_id=payload.group_id,
|
|
group_name=group["name"],
|
|
is_system=bool(group.get("is_system", False)),
|
|
origin=_derive_origin(
|
|
{
|
|
"is_system": bool(group.get("is_system", False)),
|
|
"name": group["name"],
|
|
"created_by": group.get("created_by"),
|
|
}
|
|
),
|
|
source="admin",
|
|
added_at=None,
|
|
added_by=user.get("email"),
|
|
)
|
|
|
|
|
|
@router.delete(
|
|
"/users/{user_id}/memberships/{group_id}",
|
|
status_code=204,
|
|
)
|
|
async def remove_user_from_group(
|
|
user_id: str,
|
|
group_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""Remove a user from a group from the user-centric page.
|
|
|
|
Only deletes admin-source rows (Google-sync / system-seed managed
|
|
elsewhere). Last-admin guard: refuse to remove anyone from Admin
|
|
when they are the only active admin — recovery from zero admins
|
|
requires direct DB access.
|
|
"""
|
|
group = UserGroupsRepository(conn).get(group_id)
|
|
if not group:
|
|
raise HTTPException(status_code=404, detail="Group not found")
|
|
_guard_google_managed(group)
|
|
if (
|
|
group["name"] == "Admin"
|
|
and is_user_admin(user_id, conn)
|
|
and UserRepository(conn).count_admins(active_only=True) <= 1
|
|
):
|
|
raise HTTPException(
|
|
status_code=409,
|
|
detail="Cannot remove the last admin — at least one user must remain in the Admin group",
|
|
)
|
|
members = UserGroupMembersRepository(conn)
|
|
removed = members.remove_member(user_id, group_id, require_source="admin")
|
|
if not removed:
|
|
raise HTTPException(
|
|
status_code=404,
|
|
detail="No admin-managed membership for this user in this group",
|
|
)
|
|
_audit(
|
|
conn, user["id"], "user_group.member_removed",
|
|
f"user:{user_id}",
|
|
{"group_id": group_id, "group_name": group["name"]},
|
|
)
|
|
|
|
|
|
class EffectiveAccessItem(BaseModel):
|
|
resource_type: str
|
|
resource_id: str
|
|
via_groups: List[dict] # [{group_id, group_name}]
|
|
|
|
|
|
class EffectiveAccessResponse(BaseModel):
|
|
is_admin: bool
|
|
items: List[EffectiveAccessItem]
|
|
|
|
|
|
@router.get(
|
|
"/users/{user_id}/effective-access",
|
|
response_model=EffectiveAccessResponse,
|
|
)
|
|
async def user_effective_access(
|
|
user_id: str,
|
|
user: dict = Depends(require_admin),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""List resources the user effectively has access to, with which group
|
|
grants each one. ``is_admin`` reflects the real Admin-group check but
|
|
no longer short-circuits the response — admins get the same explicit
|
|
grant breakdown as everyone else, so the admin viewing a target user
|
|
can see precisely what's been granted via which group rather than a
|
|
flat "Full access" pill that hides the wiring.
|
|
|
|
Note: actual authorization at runtime still gives Admin-group members
|
|
god-mode (see ``app.auth.access.is_user_admin``); this endpoint is a
|
|
debugging/audit view of the explicit grant graph, not the enforcement
|
|
surface.
|
|
"""
|
|
if not UserRepository(conn).get_by_id(user_id):
|
|
raise HTTPException(status_code=404, detail="User not found")
|
|
|
|
# JOIN user's group memberships with their grants. group_concat-style
|
|
# aggregation isn't worth it — render side-by-side rows and let the UI
|
|
# collapse same (resource_type, resource_id) into a single line.
|
|
rows = conn.execute(
|
|
"""SELECT rg.resource_type, rg.resource_id,
|
|
g.id AS group_id, g.name AS group_name
|
|
FROM user_group_members m
|
|
JOIN user_groups g ON g.id = m.group_id
|
|
JOIN resource_grants rg ON rg.group_id = m.group_id
|
|
WHERE m.user_id = ?
|
|
ORDER BY rg.resource_type, rg.resource_id, g.name""",
|
|
[user_id],
|
|
).fetchall()
|
|
|
|
grouped: dict[tuple[str, str], EffectiveAccessItem] = {}
|
|
for rt, rid, gid, gname in rows:
|
|
key = (rt, rid)
|
|
if key not in grouped:
|
|
grouped[key] = EffectiveAccessItem(
|
|
resource_type=rt, resource_id=rid, via_groups=[],
|
|
)
|
|
grouped[key].via_groups.append({"group_id": gid, "group_name": gname})
|
|
|
|
return EffectiveAccessResponse(
|
|
is_admin=is_user_admin(user_id, conn),
|
|
items=list(grouped.values()),
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Self-service: /api/me/effective-access — non-admin can view their own.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Separate router so it bypasses the admin gate. Mounted at /api/me/...
|
|
me_router = APIRouter(prefix="/api/me", tags=["me"])
|
|
|
|
|
|
@me_router.get(
|
|
"/effective-access",
|
|
response_model=EffectiveAccessResponse,
|
|
)
|
|
async def my_effective_access(
|
|
user: dict = Depends(get_current_user),
|
|
conn: duckdb.DuckDBPyConnection = Depends(_get_db),
|
|
):
|
|
"""Same payload as /api/admin/users/{id}/effective-access but scoped to
|
|
the calling user. Drives the /me/profile page's read-only access summary —
|
|
so non-admin callers can self-audit without elevation. Admins get the
|
|
same explicit grant breakdown as everyone else (no short-circuit) so
|
|
the profile page audits the actual grant graph; runtime authorization
|
|
still gives Admin god-mode regardless of this list."""
|
|
user_id = user["id"]
|
|
rows = conn.execute(
|
|
"""SELECT rg.resource_type, rg.resource_id,
|
|
g.id AS group_id, g.name AS group_name
|
|
FROM user_group_members m
|
|
JOIN user_groups g ON g.id = m.group_id
|
|
JOIN resource_grants rg ON rg.group_id = m.group_id
|
|
WHERE m.user_id = ?
|
|
ORDER BY rg.resource_type, rg.resource_id, g.name""",
|
|
[user_id],
|
|
).fetchall()
|
|
|
|
grouped: dict[tuple[str, str], EffectiveAccessItem] = {}
|
|
for rt, rid, gid, gname in rows:
|
|
key = (rt, rid)
|
|
if key not in grouped:
|
|
grouped[key] = EffectiveAccessItem(
|
|
resource_type=rt, resource_id=rid, via_groups=[],
|
|
)
|
|
grouped[key].via_groups.append({"group_id": gid, "group_name": gname})
|
|
|
|
return EffectiveAccessResponse(
|
|
is_admin=is_user_admin(user_id, conn),
|
|
items=list(grouped.values()),
|
|
)
|