agnes-the-ai-analyst/tests/test_cli_error_render.py
ZdenekSrotyr 7f743d0392 fix(cli): #168 review iter 6 — render empty-string diagnostics
Devin Review iter #6 found 2 issues.

🟡 BUG: cli/error_render.py filtered out empty-string values via
`detail[key] not in (None, "")` and `value not in (None, "")` before
they could reach `_kv_line`. But `_kv_line` was specifically designed
to render empty strings as `(empty)` — the filter shadowed that
branch. The hidden field happens to be the most operator-actionable
one in `cross_project_forbidden`: `billing_project: ""` is the exact
diagnostic confirming WHY USER_PROJECT_DENIED fires.

Change filter to `is not None`. Empty strings now flow through
`_kv_line` and render as `billing_project: (empty)`.

📝 ANALYSIS: CHANGELOG wording for the test-connection endpoint said
"the saved data_source.bigquery config", which Devin flagged as
slightly misleading because `get_bq_access` is `@functools.cache`d —
"Test connection" tests the config in the running process, not the
just-saved YAML overlay. The save flow already returns
`restart_required: True` and the UI shows a banner, so the behavior
is documented; only the CHANGELOG wording was loose. Tightened to
"the **process-cached** BqAccess... Tests the config active in the
running process — after a save the response includes restart_required;
click Test AFTER restart to validate the freshly-saved values."

New test: test_renders_empty_string_as_empty_marker locks in the
empty-string-as-(empty) rendering for the cross_project_forbidden
case so a future filter change won't silently drop the diagnostic
again. 9 affected render tests pass.
2026-05-04 14:30:43 +02:00

118 lines
4.6 KiB
Python

"""CLI structured error renderer for typed BigQuery error responses.
The server side maps BQ Forbidden/auth/badrequest errors into typed
BqAccessError dicts that the FastAPI handler returns as `detail` JSON.
Today the CLI side flattens them via `f"HTTP {code}: {body[:200]}"`,
truncating the structured shape and hiding the operator-facing hints.
`cli.error_render.render_error` recognizes a few canonical shapes and
pretty-prints them; falls back to truncated form for anything else.
Closes part of #160 §4.7.
"""
from __future__ import annotations
import pytest
@pytest.fixture
def render_error():
from cli.error_render import render_error
return render_error
def test_renders_typed_bq_access_error(render_error):
"""`{detail: {kind, hint, billing_project, data_project}}` from
BqAccessError surfaces as a multi-line block with the kind line, the
key/value pairs, and the hint word-wrapped at 80 cols."""
body = {"detail": {
"kind": "cross_project_forbidden",
"message": "USER_PROJECT_DENIED on bigquery.googleapis.com",
"billing_project": "",
"data_project": "prj-example-data-001",
"hint": (
"Set data_source.bigquery.billing_project in /admin/server-config "
"to a project where the SA has serviceusage.services.use, or "
"grant the SA that role on the data project."
),
}}
out = render_error(502, body)
# Single-line `f"HTTP {code}: ..."` style is the OLD form. The new
# renderer must produce multi-line output.
assert "\n" in out
# Kind appears prominently in the output.
assert "cross_project_forbidden" in out
# Key/value pairs visible.
assert "billing_project" in out
assert "prj-example-data-001" in out
# Hint text included.
assert "serviceusage.services.use" in out
def test_renders_remote_scan_too_large(render_error):
"""`{detail: {reason: 'remote_scan_too_large', scan_bytes, limit_bytes,
tables, suggestion}}` from the new /api/query guardrail formats with
the bytes + tables + suggestion clearly visible."""
body = {"detail": {
"reason": "remote_scan_too_large",
"scan_bytes": 10737418240, # 10 GiB
"limit_bytes": 5368709120, # 5 GiB
"tables": ["finance.unit_economics"],
"suggestion": (
"Use `da fetch <id> --select <cols> --where <predicate> "
"--estimate` to materialize a filtered subset, then query "
"the snapshot locally."
),
}}
out = render_error(400, body)
assert "remote_scan_too_large" in out
assert "10737418240" in out or "10 GiB" in out or "10737418240" in str(out)
assert "finance.unit_economics" in out
assert "da fetch" in out
def test_renders_bq_path_not_registered(render_error):
"""`{detail: {reason: 'bq_path_not_registered', path, hint}}` from the
RBAC patch formats path + hint clearly."""
body = {"detail": {
"reason": "bq_path_not_registered",
"path": 'bq."secret_ds"."secret_tbl"',
"hint": "Direct bq.* references must point to a registered table.",
}}
out = render_error(403, body)
assert "bq_path_not_registered" in out
assert 'secret_ds' in out
assert "registered table" in out
def test_falls_back_to_truncated_for_unrecognized_shape(render_error):
"""Body without recognizable typed shape falls back to truncated form."""
body = "Internal Server Error: Something went wrong" * 20 # 800+ chars
out = render_error(500, body)
# Old-style truncation kicks in; output is single-line and short.
assert len(out) < 600
assert "500" in out
def test_falls_back_when_detail_is_string(render_error):
"""Many old endpoints return `detail: "<string message>"` — render that
as-is without trying to walk it as a structured error dict."""
body = {"detail": "Only single SELECT queries are allowed"}
out = render_error(400, body)
assert "Only single SELECT" in out
def test_renders_empty_string_as_empty_marker(render_error):
"""Devin Review iter #6: `billing_project: ""` in cross_project_forbidden
is the key diagnostic showing WHY the operator hits USER_PROJECT_DENIED.
The renderer must show empty strings as `(empty)`, not silently drop them."""
body = {"detail": {
"kind": "cross_project_forbidden",
"billing_project": "", # the key diagnostic
"data_project": "prj-example",
"hint": "Set data_source.bigquery.billing_project",
}}
out = render_error(502, body)
assert "billing_project: (empty)" in out, \
f"empty billing_project must render as (empty); got:\n{out}"
assert "data_project: prj-example" in out