agnes-the-ai-analyst/tests/test_admin_server_config_renderer_depth.py
ZdenekSrotyr df7f5b1d9a feat(admin-ui): /admin/server-config known-fields registry + structured nested editor
Today /admin/server-config renders fields by iterating Object.keys(payload) on the YAML value — if a key isn't in instance.yaml, the operator can't see it. They have to know to type it via the JSON-patch textarea (which only renders for empty sections) or SSH and edit YAML.

Adds a known-fields registry (`_KNOWN_FIELDS` in app/api/admin.py) the UI consumes alongside the YAML payload. Renderer shows BOTH:
  - existing fields (from YAML) with current value
  - known-but-unset fields with dashed-border placeholder + hint, ready to fill in

Renderer (`renderField`, `renderSection`, `collectSection`):
  - kind="string"|"secret"|"bool"|"int"|"select"|"object"|"array"|"map" — picks input type
  - kind="object" with `fields` — recursive structured form, arbitrary depth (corporate_memory needs 3-4 levels)
  - kind="array" with `item_kind` — vertical stack of typed inputs + add/remove buttons
  - kind="map" with `key_kind` + `value_kind` — key:value rows + add/remove (used for confidence.base, domain_owners, entity_resolution.entities)
  - data-path encoded as JSON segment array so map keys with embedded dots (e.g. 'user_verification.correction') survive collect → patch round-trip
  - .cfg-field.is-unset CSS — dashed border, muted label, italic hint

Sections newly exposed (added to _EDITABLE_SECTIONS):
  - openmetadata: url, token (secret), cache_ttl_seconds, verify_ssl
  - desktop: jwt_issuer, jwt_secret (secret), url_scheme

Known fields populated for existing sections:
  - data_source.bigquery: billing_project (the cause of the 403 USER_PROJECT_DENIED footgun when SA can read but not bill the data project), legacy_wrap_views (bigquery_query() wrap for VIEWs — issue #101 default off, ON for view-heavy deployments), max_bytes_per_materialize (cost guardrail)
  - data_source.keboola: stack_url, project_id (hints; values already populated)
  - ai: base_url (required for openai_compat), structured_output (select)
  - corporate_memory: full schema from instance.yaml.example — distribution_mode, approval_mode, review_period_months, notify_on_new_items, sources.{claude_local_md,session_transcripts}, extraction.{model,sensitivity_check,contradiction_check}, confidence.{base,modifiers,decay.{mode,half_life_months,decay_rate_monthly,floor}}, contradiction_detection.{enabled,max_candidates}, entity_resolution.{enabled,entities}, domain_owners, domains
  - Known partial: confidence.modifiers is map<string, map<string, float>> — falls through to JSON-textarea with TODO; structured editor for that one shape needs more renderer work

Tests:
  - test_admin_server_config_known_fields — registry envelope shape, smoke fixture
  - test_admin_server_config_renderer_depth — 4-level nested objects, arrays of strings, maps of floats, dotted-key safety
  - test_admin_server_config_corp_memory — full corporate_memory schema, 12 fields incl. nested
  - test_admin_server_config — existing tests adjusted for new shape
2026-05-01 20:27:01 +02:00

171 lines
7.1 KiB
Python

"""Renderer depth/array/map tests for /admin/server-config.
The base renderer in `admin_server_config.html` already supports arbitrary
depth for `kind="object"` with `fields` (recursion is bounded only by the
browser stack). This file pins down the harder shapes corporate_memory
needs:
- Arrays of scalars (e.g. domains, detection_types) rendered as a
per-element stack with add/remove buttons rather than a single JSON
textarea.
- Maps of scalars (e.g. confidence.base) rendered as key:value rows with
add/remove.
- Maps whose values are arrays of strings (e.g. domain_owners,
entity_resolution.entities) rendered as key + nested array rows.
- Dotted keys present in *data* (e.g. confidence.base keys like
``user_verification.correction``) survive round-trip without being
mistaken for nested-path separators.
We assert structurally on the static template (the page is a shell — JS
fills the form from /api/admin/server-config). The markers we look for
are the JS function/identifier names that implement each shape.
"""
def _auth(token):
return {"Authorization": f"Bearer {token}"}
def test_renderer_supports_array_of_scalars(seeded_app, monkeypatch, tmp_path):
"""An array-of-strings registry leaf renders as a vertical stack of
text inputs, not a JSON textarea.
Marker: the JS contains a renderer entry point for arrays-of-scalars
that produces add/remove controls — `renderArrayField` or equivalent
plus an "addArrayItem" / "removeArrayItem" interaction handler.
"""
monkeypatch.setenv("DATA_DIR", str(tmp_path))
state = tmp_path / "state"
state.mkdir(parents=True, exist_ok=True)
import yaml as _yaml
(state / "instance.yaml").write_text(_yaml.dump({
"data_source": {"type": "bigquery", "bigquery": {"project": "p"}},
}))
import app.instance_config as ic
ic._instance_config = None
try:
c = seeded_app["client"]
token = seeded_app["admin_token"]
c.cookies.set("access_token", token)
try:
r = c.get("/admin/server-config", headers={"Accept": "text/html"})
finally:
c.cookies.clear()
assert r.status_code == 200, r.text
body = r.text
# The renderer ships a dedicated array-of-scalars path.
assert "renderArrayField" in body, \
"JS must implement renderArrayField for kind='array'+item_kind=scalar"
# Add/remove handlers for individual array items.
assert "data-array-add" in body, "missing add-row interaction marker"
assert "data-array-remove" in body, "missing remove-row interaction marker"
finally:
ic._instance_config = None
def test_renderer_supports_map_of_scalars(seeded_app, monkeypatch, tmp_path):
"""A map of string→float renders as key:value rows with add/remove,
not as a JSON textarea. Marker: `renderMapField` exists in the JS.
"""
monkeypatch.setenv("DATA_DIR", str(tmp_path))
state = tmp_path / "state"
state.mkdir(parents=True, exist_ok=True)
import yaml as _yaml
(state / "instance.yaml").write_text(_yaml.dump({
"data_source": {"type": "bigquery", "bigquery": {"project": "p"}},
}))
import app.instance_config as ic
ic._instance_config = None
try:
c = seeded_app["client"]
token = seeded_app["admin_token"]
c.cookies.set("access_token", token)
try:
r = c.get("/admin/server-config", headers={"Accept": "text/html"})
finally:
c.cookies.clear()
assert r.status_code == 200, r.text
body = r.text
assert "renderMapField" in body, \
"JS must implement renderMapField for kind='map'"
assert "data-map-add" in body, "missing map add-row interaction marker"
assert "data-map-remove" in body, "missing map remove-row interaction marker"
finally:
ic._instance_config = None
def test_renderer_path_is_json_encoded_not_dotted_string(seeded_app, monkeypatch, tmp_path):
"""When data keys themselves contain dots (e.g.
``confidence.base.user_verification.correction`` where
``user_verification.correction`` is one map key), the renderer must
NOT split on '.' to reconstruct the patch shape — that would break
the dotted data key into two path segments.
Implementation: leaf inputs carry a `data-path` attribute holding the
JSON-encoded array of segments. The collector reads that array
instead of splitting `data-key` on '.'. The dotted `data-key` stays
around for backward compatibility (existing nested object fields
use it), but maps emit JSON paths so their keys round-trip intact.
"""
monkeypatch.setenv("DATA_DIR", str(tmp_path))
state = tmp_path / "state"
state.mkdir(parents=True, exist_ok=True)
import yaml as _yaml
(state / "instance.yaml").write_text(_yaml.dump({
"data_source": {"type": "bigquery", "bigquery": {"project": "p"}},
}))
import app.instance_config as ic
ic._instance_config = None
try:
c = seeded_app["client"]
token = seeded_app["admin_token"]
c.cookies.set("access_token", token)
try:
r = c.get("/admin/server-config", headers={"Accept": "text/html"})
finally:
c.cookies.clear()
assert r.status_code == 200, r.text
body = r.text
# The collector must understand JSON-encoded path arrays so map
# keys with embedded dots survive round-trip.
assert "data-path" in body, "JSON path attribute missing from renderer"
# The collector should prefer data-path over splitting data-key on '.'
# Look for the parsing entry point.
assert "JSON.parse" in body and "data-path" in body, \
"collector must parse JSON-encoded data-path arrays"
finally:
ic._instance_config = None
def test_renderer_handles_4_level_object_nesting(seeded_app, monkeypatch, tmp_path):
"""Smoke check: the recursive renderer doesn't bail out at depth 4.
The renderer is `renderNestedField(... depth)`; recursion is unbounded
on the JS side. We assert by ensuring the renderer's nested-form path
is wired with a depth-incrementing recursion call (literal markers in
the JS).
"""
monkeypatch.setenv("DATA_DIR", str(tmp_path))
state = tmp_path / "state"
state.mkdir(parents=True, exist_ok=True)
import yaml as _yaml
(state / "instance.yaml").write_text(_yaml.dump({
"data_source": {"type": "bigquery", "bigquery": {"project": "p"}},
}))
import app.instance_config as ic
ic._instance_config = None
try:
c = seeded_app["client"]
token = seeded_app["admin_token"]
c.cookies.set("access_token", token)
try:
r = c.get("/admin/server-config", headers={"Accept": "text/html"})
finally:
c.cookies.clear()
assert r.status_code == 200, r.text
body = r.text
# The recursion marker — depth bumps in the recursive call.
assert "renderNestedField(" in body
assert "(depth || 0) + 1" in body, \
"recursion must increment depth on each nested call"
finally:
ic._instance_config = None