agnes-the-ai-analyst/docs/archive/superpowers/plans/2026-04-29-dev-debug-toolbar.md

Dev Debug Toolbar + Centralized Logging Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add fastapi-debug-toolbar (gated by DEBUG=1) with a custom DuckDBPanel, and replace 23 scattered logging.basicConfig calls with a single centralized setup_logging() helper that uses RichHandler in dev and JSON in prod.

Architecture: Phase 1–4 introduce app/logging_config.py and migrate every entrypoint (services, scripts, app, connectors) to call setup_logging(__name__). Phase 5 mounts DebugToolbarMiddleware behind a DEBUG=1 env gate so production never imports the toolbar. Phase 6 adds a custom DuckDBPanel with an InstrumentedConnection wrapper that hooks src/db.py's three duckdb.connect() call sites. Phase 7 documents the DEBUG=1 flag in .env.template, CHANGELOG.md, and docs/development.md.

Tech Stack: Python 3.11+, FastAPI, Starlette, DuckDB, rich>=13 (already a dep), fastapi-debug-toolbar>=0.6.3 (new dev-only dep, pulls Jinja2 + pyinstrument), pytest with coverage.

Working directory for execution: /Users/vrysanek/foundry-ai/agnes-the-ai-analyst/.worktrees/main-dev-logging/ on branch vr/dev-logging (tracks origin/main). All paths below are relative to this worktree root.

---

File Structure

New files

Path	Responsibility
app/logging_config.py	setup_logging(), _JSONFormatter, _derive_slug(), request_id_var ContextVar
app/middleware/__init__.py	empty package marker
app/middleware/request_id.py	RequestIdMiddleware (Starlette BaseHTTPMiddleware)
app/debug/__init__.py	empty package marker
app/debug/duckdb_panel.py	DuckDBPanel, InstrumentedConnection, record_query, get_request_store
app/debug/templates/panels/duckdb.html	Jinja template rendering query table
tests/test_logging_config.py	unit tests for setup_logging and slug derivation
tests/test_request_id_middleware.py	unit tests for request-id middleware
tests/test_duckdb_panel.py	unit tests for InstrumentedConnection
tests/test_toolbar_integration.py	integration tests for toolbar + DuckDBPanel
docs/development.md	brief usage doc covering DEBUG=1 and toolbar

Modified files

Path	Change
app/main.py	call setup_logging("app"), mount RequestIdMiddleware always, mount DebugToolbarMiddleware only if DEBUG=1
app/api/sync.py:105	delete rogue module-level logging.basicConfig
src/db.py	add _maybe_instrument() helper; wrap returns of get_system_db(), get_analytics_db(), get_analytics_db_readonly()
src/catalog_export.py	replace module-level basicConfig with setup_logging(__name__) (only when run as script)
src/profiler.py	replace module-level basicConfig with setup_logging(__name__)
services/scheduler/__main__.py	replace basicConfig with setup_logging(__name__)
services/ws_gateway/gateway.py	replace basicConfig with setup_logging(__name__)
services/telegram_bot/bot.py	replace basicConfig with setup_logging(__name__)
services/corporate_memory/collector.py	replace basicConfig with setup_logging(__name__)
services/session_collector/collector.py	replace basicConfig with setup_logging(__name__)
services/verification_detector/__main__.py	replace basicConfig with setup_logging(__name__)
connectors/keboola/extractor.py	remove module-level _logging.basicConfig (only run via CLI block at bottom)
connectors/jira/transform.py	remove module-level basicConfig
connectors/jira/incremental_transform.py	remove module-level basicConfig
connectors/jira/extract_init.py	replace basicConfig with setup_logging(__name__) (only inside __main__ block)
connectors/jira/scripts/poll_sla.py	replace with setup_logging(__name__)
connectors/jira/scripts/backfill.py	replace with setup_logging(__name__)
connectors/jira/scripts/backfill_sla.py	replace with setup_logging(__name__)
connectors/jira/scripts/backfill_remote_links.py	replace with setup_logging(__name__)
connectors/jira/scripts/consistency_check.py	replace with setup_logging(__name__)
scripts/migrate_json_to_duckdb.py	replace with setup_logging(__name__)
scripts/migrate_metrics_to_duckdb.py	replace with setup_logging(__name__)
scripts/migrate_parquets_to_extracts.py	replace with setup_logging(__name__)
scripts/migrate_registry_to_duckdb.py	replace with setup_logging(__name__)
scripts/generate_sample_data.py	replace with setup_logging(__name__)
pyproject.toml	add fastapi-debug-toolbar>=0.6.3 to [project.optional-dependencies].dev
config/.env.template	document DEBUG=1 toggle
CHANGELOG.md	new ### Added entry under ## [Unreleased]

Discovered facts (verified during plan writing)

• src/db.py has three duckdb.connect() call sites: lines 413, 423, 581/587. Three public helpers: get_system_db() (line 394, returns _system_db_conn.cursor()), get_analytics_db() (line 419), get_analytics_db_readonly() (line 573).
• services/scheduler/__main__.py:31, services/verification_detector/__main__.py:49, services/corporate_memory/collector.py:52, services/session_collector/collector.py:36, services/telegram_bot/bot.py:55, services/ws_gateway/gateway.py:214 — all have multi-line basicConfig calls.
• app/api/sync.py:105 has a module-level basicConfig inside an api file — bug, will be removed (root logger is configured by the app entrypoint).
• 23 total basicConfig call sites enumerated.

---

Task 1: Centralized logging module (app/logging_config.py) — Phase P1

Files:

• Create: app/logging_config.py

• Test: tests/test_logging_config.py

• Step 1.1: Write the failing tests

Create tests/test_logging_config.py:

import json
import logging
import os
from io import StringIO

import pytest

from app.logging_config import (
    _CONFIGURED,
    _derive_slug,
    _JSONFormatter,
    request_id_var,
    setup_logging,
)


@pytest.fixture(autouse=True)
def _reset_logging(monkeypatch):
    """Reset global logging state between tests."""
    import app.logging_config as lc
    lc._CONFIGURED = False
    monkeypatch.delenv("DEBUG", raising=False)
    monkeypatch.delenv("LOG_LEVEL", raising=False)
    yield
    lc._CONFIGURED = False
    logging.getLogger().handlers.clear()


def test_dev_uses_rich_handler(monkeypatch):
    monkeypatch.setenv("DEBUG", "1")
    setup_logging("app")
    handlers = logging.getLogger().handlers
    assert len(handlers) == 1
    from rich.logging import RichHandler
    assert isinstance(handlers[0], RichHandler)


def test_prod_uses_json_formatter():
    setup_logging("app")
    handlers = logging.getLogger().handlers
    assert len(handlers) == 1
    assert isinstance(handlers[0], logging.StreamHandler)
    assert isinstance(handlers[0].formatter, _JSONFormatter)


def test_idempotent():
    setup_logging("app")
    setup_logging("app")
    setup_logging("app")
    assert len(logging.getLogger().handlers) == 1


def test_log_level_from_env(monkeypatch):
    monkeypatch.setenv("LOG_LEVEL", "DEBUG")
    setup_logging("app")
    assert logging.getLogger().level == logging.DEBUG


def test_log_level_default_prod():
    setup_logging("app")
    assert logging.getLogger().level == logging.INFO


def test_log_level_default_dev(monkeypatch):
    monkeypatch.setenv("DEBUG", "1")
    setup_logging("app")
    assert logging.getLogger().level == logging.DEBUG


def test_slug_explicit_short_name():
    assert _derive_slug("scheduler") == "scheduler"


def test_slug_strips_services_prefix():
    assert _derive_slug("services.scheduler.__main__") == "scheduler"


def test_slug_keeps_nested_module():
    assert _derive_slug("services.corporate_memory.collector") == "corporate_memory.collector"


def test_slug_strips_app_prefix():
    assert _derive_slug("app.main") == "app"


def test_slug_strips_connectors_prefix():
    assert _derive_slug("connectors.jira.transform") == "jira.transform"


def test_slug_explicit_app():
    assert _derive_slug("app") == "app"


def test_json_formatter_includes_service_field():
    rec = logging.LogRecord(
        name="test", level=logging.INFO, pathname=__file__, lineno=1,
        msg="hello world", args=(), exc_info=None,
    )
    fmt = _JSONFormatter(service="myservice")
    line = fmt.format(rec)
    parsed = json.loads(line)
    assert parsed["service"] == "myservice"
    assert parsed["msg"] == "hello world"
    assert parsed["lvl"] == "INFO"
    assert parsed["logger"] == "test"
    assert "ts" in parsed


def test_json_formatter_includes_request_id_when_set():
    fmt = _JSONFormatter(service="app")
    rec = logging.LogRecord(
        name="t", level=logging.INFO, pathname=__file__, lineno=1,
        msg="m", args=(), exc_info=None,
    )
    token = request_id_var.set("abc123")
    try:
        line = fmt.format(rec)
    finally:
        request_id_var.reset(token)
    parsed = json.loads(line)
    assert parsed["request_id"] == "abc123"


def test_json_formatter_omits_request_id_when_unset():
    fmt = _JSONFormatter(service="app")
    rec = logging.LogRecord(
        name="t", level=logging.INFO, pathname=__file__, lineno=1,
        msg="m", args=(), exc_info=None,
    )
    line = fmt.format(rec)
    parsed = json.loads(line)
    assert "request_id" not in parsed


def test_json_formatter_includes_exception():
    fmt = _JSONFormatter(service="app")
    try:
        raise ValueError("boom")
    except ValueError:
        import sys
        rec = logging.LogRecord(
            name="t", level=logging.ERROR, pathname=__file__, lineno=1,
            msg="oops", args=(), exc_info=sys.exc_info(),
        )
    line = fmt.format(rec)
    parsed = json.loads(line)
    assert "exc" in parsed
    assert "ValueError: boom" in parsed["exc"]


def test_setup_logging_emits_parsable_json_in_prod(capsys):
    setup_logging("app")
    logging.getLogger("test").info("hello %s", "world")
    out = capsys.readouterr().err
    parsed = json.loads(out.strip().splitlines()[-1])
    assert parsed["msg"] == "hello world"
    assert parsed["service"] == "app"


def test_setup_logging_silences_uvicorn_access_in_prod():
    setup_logging("app")
    assert logging.getLogger("uvicorn.access").level == logging.WARNING


def test_setup_logging_keeps_uvicorn_access_in_dev(monkeypatch):
    monkeypatch.setenv("DEBUG", "1")
    setup_logging("app")
    assert logging.getLogger("uvicorn.access").level == logging.INFO

• Step 1.2: Run tests, verify they fail

uv run pytest tests/test_logging_config.py -q

Expected: collection error or ImportError: cannot import name 'setup_logging' because the module does not exist yet.

• Step 1.3: Implement app/logging_config.py

Create the file:

"""Centralized logging configuration for FastAPI app and background services.

Each entrypoint (app/main.py, services/*/__main__.py or top-level script)
calls setup_logging(__name__) once. Library modules just do
`logger = logging.getLogger(__name__)` — they NEVER call setup_logging.

Dev (DEBUG=1): rich.logging.RichHandler with color, tracebacks, links.
Prod: stdlib StreamHandler with JSON formatter to stderr.
"""

from __future__ import annotations

import contextvars
import json
import logging
import os
import sys
from datetime import datetime, timezone
from pathlib import Path

request_id_var: contextvars.ContextVar[str | None] = contextvars.ContextVar(
    "request_id", default=None
)

_CONFIGURED = False


def setup_logging(service: str | None = None, level: str | None = None) -> None:
    """Configure root logger. Idempotent.

    Pass ``__name__`` (preferred) or an explicit short slug like ``"app"``.
    Multiple calls are no-ops.
    """
    global _CONFIGURED
    if _CONFIGURED:
        return

    debug = os.environ.get("DEBUG", "").lower() in ("1", "true", "yes")
    lvl = (
        level
        or os.environ.get("LOG_LEVEL")
        or ("DEBUG" if debug else "INFO")
    ).upper()
    slug = _derive_slug(service)

    if debug:
        from rich.logging import RichHandler

        handler: logging.Handler = RichHandler(
            rich_tracebacks=True,
            tracebacks_show_locals=False,
            show_time=True,
            show_path=True,
            markup=False,
            force_terminal=True,
        )
        handler.setFormatter(logging.Formatter("[%(name)s] %(message)s"))
    else:
        handler = logging.StreamHandler()
        handler.setFormatter(_JSONFormatter(service=slug))

    logging.basicConfig(level=lvl, handlers=[handler], force=True)
    logging.getLogger("uvicorn.access").setLevel(
        logging.INFO if debug else logging.WARNING
    )
    logging.getLogger("httpx").setLevel(logging.WARNING)
    _CONFIGURED = True


def _derive_slug(service: str | None) -> str:
    """Turn module name (``__name__``) or override into readable service slug.

    Examples:
        _derive_slug("app")                                  -> "app"
        _derive_slug("services.scheduler.__main__")          -> "scheduler"
        _derive_slug("services.corporate_memory.collector")  -> "corporate_memory.collector"
        _derive_slug("connectors.jira.transform")            -> "jira.transform"
    """
    if service and not service.startswith("_") and service != "__main__":
        s = (
            service.removeprefix("services.")
            .removeprefix("connectors.")
            .removeprefix("app.")
        )
        s = s.removesuffix(".__main__").removesuffix(".main")
        return s or "app"

    try:
        frame = sys._getframe(2)
        path = frame.f_globals.get("__file__")
        if path:
            p = Path(path)
            for top in ("services", "connectors", "app"):
                if top in p.parts:
                    i = p.parts.index(top) + 1
                    rest = p.parts[i:]
                    name = ".".join([*rest[:-1], p.stem])
                    return name.removesuffix(".__main__").removesuffix(".main") or top
            return p.stem
    except Exception:
        pass
    return "app"


class _JSONFormatter(logging.Formatter):
    def __init__(self, service: str) -> None:
        super().__init__()
        self.service = service

    def format(self, record: logging.LogRecord) -> str:
        payload: dict[str, object] = {
            "ts": datetime.fromtimestamp(record.created, tz=timezone.utc).isoformat(),
            "lvl": record.levelname,
            "logger": record.name,
            "service": self.service,
            "msg": record.getMessage(),
        }
        rid = request_id_var.get()
        if rid:
            payload["request_id"] = rid
        if record.exc_info:
            payload["exc"] = self.formatException(record.exc_info)
        return json.dumps(payload, default=str)

• Step 1.4: Run tests, verify they pass

uv run pytest tests/test_logging_config.py -q

Expected: all tests pass. If any fail, fix before proceeding.

• Step 1.5: Run quality checks

ruff format app/logging_config.py tests/test_logging_config.py
ruff check app/logging_config.py tests/test_logging_config.py --fix

Expected: no warnings.

• Step 1.6: Commit

git add app/logging_config.py tests/test_logging_config.py
git commit -m "feat(logging): add centralized setup_logging with dev/prod handlers"

---

Task 2: Request-ID middleware (app/middleware/request_id.py) — Phase P2

Files:

• Create: app/middleware/__init__.py (empty)

• Create: app/middleware/request_id.py

• Test: tests/test_request_id_middleware.py

• Step 2.1: Create empty package marker

mkdir -p app/middleware
: > app/middleware/__init__.py

• Step 2.2: Write the failing tests

Create tests/test_request_id_middleware.py:

import re

from fastapi import FastAPI
from fastapi.testclient import TestClient

from app.logging_config import request_id_var
from app.middleware.request_id import RequestIdMiddleware


def _make_app() -> FastAPI:
    app = FastAPI()
    app.add_middleware(RequestIdMiddleware)

    @app.get("/echo-rid")
    def echo_rid():
        return {"rid": request_id_var.get()}

    return app


def test_assigns_id_when_missing():
    client = TestClient(_make_app())
    resp = client.get("/echo-rid")
    assert resp.status_code == 200
    rid = resp.headers["x-request-id"]
    assert re.fullmatch(r"[0-9a-f]{12}", rid)
    assert resp.json()["rid"] == rid


def test_passes_through_provided_id():
    client = TestClient(_make_app())
    resp = client.get("/echo-rid", headers={"X-Request-ID": "given-1234"})
    assert resp.status_code == 200
    assert resp.headers["x-request-id"] == "given-1234"
    assert resp.json()["rid"] == "given-1234"


def test_request_id_resets_after_request():
    client = TestClient(_make_app())
    client.get("/echo-rid")
    assert request_id_var.get() is None

• Step 2.3: Run tests, verify they fail

uv run pytest tests/test_request_id_middleware.py -q

Expected: ModuleNotFoundError: No module named 'app.middleware.request_id'.

• Step 2.4: Implement the middleware

Create app/middleware/request_id.py:

"""Request-ID middleware. Assigns or propagates X-Request-ID per request."""

from __future__ import annotations

import uuid

from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response

from app.logging_config import request_id_var


class RequestIdMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next) -> Response:
        rid = request.headers.get("x-request-id") or uuid.uuid4().hex[:12]
        token = request_id_var.set(rid)
        try:
            response = await call_next(request)
        finally:
            request_id_var.reset(token)
        response.headers["x-request-id"] = rid
        return response

• Step 2.5: Run tests, verify they pass

uv run pytest tests/test_request_id_middleware.py -q

Expected: all 3 tests pass.

• Step 2.6: Run quality checks

ruff format app/middleware/ tests/test_request_id_middleware.py
ruff check app/middleware/ tests/test_request_id_middleware.py --fix

• Step 2.7: Commit

git add app/middleware/ tests/test_request_id_middleware.py
git commit -m "feat(http): add RequestIdMiddleware for cross-log correlation"

---

Task 3: Wire setup_logging + RequestIdMiddleware into app/main.py — Phase P2 (continued)

Files:

• Modify: app/main.py (add setup_logging call at top of module before any other import that emits logs; add RequestIdMiddleware)

• Modify: app/api/sync.py:105 (delete rogue basicConfig)

• Step 3.1: Read app/main.py to find safe insertion points

sed -n '1,40p' app/main.py

Expected: shows imports + module-level setup. Note the line where app = FastAPI(...) is created and where app.include_router(...) calls begin.

• Step 3.2: Add setup_logging("app") near the top of app/main.py

Locate the first non-from __future__ import in app/main.py. Above the FastAPI import, add:

from app.logging_config import setup_logging

setup_logging("app")

Keep it before any module that emits logs at import time.

• Step 3.3: Mount RequestIdMiddleware immediately after app = FastAPI(...) is created

Find the app = FastAPI(...) line. Immediately after it (before any app.add_middleware(...) or app.include_router(...)), add:

from app.middleware.request_id import RequestIdMiddleware

app.add_middleware(RequestIdMiddleware)

• Step 3.4: Delete rogue module-level basicConfig in app/api/sync.py

Read app/api/sync.py lines 100–110 to find the exact line and any logger usage:

sed -n '100,115p' app/api/sync.py

Then delete the line:

logging.basicConfig(level=logging.INFO, format="%(levelname)s: %(message)s")

If there is no other import logging usage in the file beyond this and a logger = logging.getLogger(__name__), leave the import alone (the logger reference still needs it).

• Step 3.5: Run the full test suite

uv run pytest -q

Expected: all tests still pass. If any test was relying on the rogue basicConfig formatter from sync.py, it will surface here — fix by switching that test's expectations to use caplog.

• Step 3.6: Sanity-check the app starts

uv run uvicorn app.main:app --port 8011 &
APP_PID=$!
sleep 3
curl -s -i http://localhost:8011/health 2>/dev/null | head -10
kill $APP_PID

Expected: response includes x-request-id: <12 hex chars> header.

• Step 3.7: Commit

git add app/main.py app/api/sync.py
git commit -m "feat(app): wire setup_logging + RequestIdMiddleware in app/main.py"

---

Task 4: Migrate service entrypoints — Phase P3

Six service entrypoints currently call logging.basicConfig(...) at module top level. Replace each with setup_logging(__name__).

Files:

• Modify: services/scheduler/__main__.py

• Modify: services/ws_gateway/gateway.py

• Modify: services/telegram_bot/bot.py

• Modify: services/corporate_memory/collector.py

• Modify: services/session_collector/collector.py

• Modify: services/verification_detector/__main__.py

• Step 4.1: Migrate services/scheduler/__main__.py

Line 31 onward currently is:

logging.basicConfig(
    level=os.environ.get("LOG_LEVEL", "INFO").upper(),
    format="%(asctime)s %(levelname)s [scheduler] %(message)s",
)
logger = logging.getLogger(__name__)

Replace the logging.basicConfig(...) block (multi-line) with:

from app.logging_config import setup_logging

setup_logging(__name__)
logger = logging.getLogger(__name__)

Keep the existing import logging and the logger = ... line. Remove import os only if it is unused after the change (verify by running ruff after).

• Step 4.2: Migrate services/ws_gateway/gateway.py

Find the logging.basicConfig(...) block at line 214 (inside an if __name__ == "__main__": or similar startup function). Replace:

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)

with:

from app.logging_config import setup_logging
setup_logging(__name__)

• Step 4.3: Migrate services/telegram_bot/bot.py

Find the multi-line logging.basicConfig(...) block at line 55. Replace the entire block with:

from app.logging_config import setup_logging
setup_logging(__name__)
logger = logging.getLogger("notify-bot")  # keep existing logger name to preserve log filters

Preserve any pre-existing custom logger name.

• Step 4.4: Migrate services/corporate_memory/collector.py

Find the multi-line logging.basicConfig(...) block at line 52. Replace with:

from app.logging_config import setup_logging
setup_logging(__name__)

Keep any subsequent logger = logging.getLogger(...) line.

• Step 4.5: Migrate services/session_collector/collector.py

Find the multi-line logging.basicConfig(...) block at line 36. Replace with:

from app.logging_config import setup_logging
setup_logging(__name__)

• Step 4.6: Migrate services/verification_detector/__main__.py

Find the multi-line logging.basicConfig(...) block at line 49 (inside the __main__ function, NOT module-level). Replace inline:

from app.logging_config import setup_logging
setup_logging(__name__)

• Step 4.7: Run the full test suite

uv run pytest -q

Expected: all tests pass. Service modules' tests (if any) may need monkeypatch.setattr("app.logging_config._CONFIGURED", False) to reset state.

• Step 4.8: Smoke-test each service starts

# Scheduler should start, log a single line, then idle
timeout 3 uv run python -m services.scheduler 2>&1 | head -5 || true
# Ws gateway should start
timeout 3 uv run python -m services.ws_gateway 2>&1 | head -5 || true

Expected: each service emits a startup log line in JSON (no DEBUG=1) or rich color (DEBUG=1 timeout 3 uv run …).

• Step 4.9: Run quality checks

ruff format services/
ruff check services/ --fix

• Step 4.10: Commit

git add services/
git commit -m "refactor(services): migrate entrypoints to centralized setup_logging"

---

Task 5: Migrate connectors/, src/, scripts/ — Phase P4

These 14 files have module-level or __main__-block basicConfig calls. The migration rules differ by location:

• Module-level basicConfig in a library file (e.g. connectors/jira/transform.py): just remove it — library modules should never configure root logger; the entrypoint that imports them is responsible.
• basicConfig inside if __name__ == "__main__": block: replace with setup_logging(__name__).

Files:

• Modify: src/catalog_export.py, src/profiler.py

• Modify: connectors/keboola/extractor.py

• Modify: connectors/jira/transform.py, incremental_transform.py, extract_init.py

• Modify: connectors/jira/scripts/poll_sla.py, backfill.py, backfill_sla.py, backfill_remote_links.py, consistency_check.py

• Modify: scripts/migrate_json_to_duckdb.py, migrate_metrics_to_duckdb.py, migrate_parquets_to_extracts.py, migrate_registry_to_duckdb.py, generate_sample_data.py

• Step 5.1: Determine library vs entrypoint per file

For each file, run:

grep -nE 'if __name__ == "__main__"|basicConfig' <file>

If basicConfig is outside any if __name__ block AND the file is imported by other modules → it is a library file. Remove the basicConfig line entirely.

If basicConfig is inside if __name__ == "__main__" OR the file is exclusively run as a script (the scripts/ dir, connectors/jira/scripts/) → replace with setup_logging(__name__).

File	Verdict	Action
src/catalog_export.py	library + has if __name__ block	check: if basicConfig is inside __main__, replace; else remove
src/profiler.py	library	remove
connectors/keboola/extractor.py:289	library, has CLI block at bottom	the _logging.basicConfig is inside the CLI block — replace with setup_logging(__name__)
connectors/jira/transform.py:20	library	remove (module-level)
connectors/jira/incremental_transform.py:38	library	remove (module-level)
connectors/jira/extract_init.py:124	inside if __name__ == "__main__"	replace
connectors/jira/scripts/*.py (5 files)	scripts	replace
scripts/migrate_*.py (4 files)	scripts	replace
scripts/generate_sample_data.py:1035	inside if __name__ == "__main__"	replace

• Step 5.2: Apply removals (library files)

For each library file with module-level basicConfig:

connectors/jira/transform.py:20:

• delete the line logging.basicConfig(level=logging.INFO).

connectors/jira/incremental_transform.py:38:

• delete the line logging.basicConfig(level=logging.INFO).

src/profiler.py:28:

• delete the multi-line logging.basicConfig(...) call.

src/catalog_export.py:43:

• inspect: if outside any __main__ block, delete the multi-line basicConfig; if inside, do step 5.3 instead.

sed -n '40,55p' src/catalog_export.py
sed -n '25,35p' src/profiler.py

• Step 5.3: Apply replacements (entrypoint files)

For each entrypoint file, replace the basicConfig(...) block with:

from app.logging_config import setup_logging
setup_logging(__name__)

Files: connectors/keboola/extractor.py:289 (note: uses _logging alias — replace the call inside the same scope), connectors/jira/extract_init.py:124, all 5 connectors/jira/scripts/*.py, all 4 scripts/migrate_*.py, scripts/generate_sample_data.py:1035.

For connectors/keboola/extractor.py:

# was:
#     _logging.basicConfig(level=_logging.INFO, format="%(levelname)s: %(message)s")
from app.logging_config import setup_logging
setup_logging(__name__)

• Step 5.4: Verify no basicConfig left in production code

grep -rEn 'logging\.basicConfig|_logging\.basicConfig' \
    src/ app/ services/ connectors/ scripts/ \
    --include="*.py" | grep -v "tests/" || echo "CLEAN"

Expected: prints CLEAN. If anything matches, decide library vs entrypoint and fix.

• Step 5.5: Run the full test suite

uv run pytest -q

Expected: all tests pass.

• Step 5.6: Smoke-test one CLI script

DATA_DIR=/tmp/agnes-smoke uv run python -m scripts.migrate_metrics_to_duckdb --help 2>&1 | head -10 || true

Expected: usage message appears with no double-config warnings.

• Step 5.7: Run quality checks

ruff format src/ connectors/ scripts/
ruff check src/ connectors/ scripts/ --fix

• Step 5.8: Commit

git add src/ connectors/ scripts/
git commit -m "refactor(logging): remove scattered basicConfig calls across libs and scripts"

---

Task 6: Add fastapi-debug-toolbar dev dep + wire middleware behind DEBUG=1 — Phase P5

Files:

• Modify: pyproject.toml (add dev dep)

• Modify: app/main.py (mount DebugToolbarMiddleware only if DEBUG=1)

• Step 6.1: Add the dev dependency

Edit pyproject.toml [project.optional-dependencies].dev block:

[project.optional-dependencies]
dev = [
    "pytest>=9.0.0",
    "pytest-timeout>=2.0.0",
    "pytest-xdist>=3.0.0",
    "faker>=24.0.0",
    "anthropic>=0.30.0",
    "openai>=1.30.0",
    "jsonschema>=4.0.0",
    "fastapi-debug-toolbar>=0.6.3",
]

Also mirror under [tool.uv].dev-dependencies if present:

[tool.uv]
dev-dependencies = [
    "pytest>=9.0.0",
    "pytest-timeout>=2.0.0",
    "pytest-xdist>=3.0.0",
    "faker>=24.0.0",
    "anthropic>=0.30.0",
    "openai>=1.30.0",
    "fastapi-debug-toolbar>=0.6.3",
]

• Step 6.2: Install the new dep

uv pip install ".[dev]"

Expected: installs fastapi-debug-toolbar, Jinja2 (already a dep), pyinstrument.

• Step 6.3: Wire DebugToolbarMiddleware in app/main.py

Locate the app.add_middleware(RequestIdMiddleware) line added in Task 3. Immediately after it, add:

import os

DEBUG = os.environ.get("DEBUG", "").lower() in ("1", "true", "yes")

if DEBUG:
    try:
        from debug_toolbar.middleware import DebugToolbarMiddleware
        app.add_middleware(
            DebugToolbarMiddleware,
            panels=[
                "debug_toolbar.panels.headers.HeadersPanel",
                "debug_toolbar.panels.routes.RoutesPanel",
                "debug_toolbar.panels.settings.SettingsPanel",
                "debug_toolbar.panels.versions.VersionsPanel",
                "debug_toolbar.panels.timer.TimerPanel",
                "debug_toolbar.panels.logging.LoggingPanel",
                "debug_toolbar.panels.profiling.ProfilingPanel",
            ],
            settings={"REFRESH_INTERVAL": 5000},
        )
    except ImportError:
        import logging
        logging.getLogger(__name__).warning(
            "DEBUG=1 but fastapi-debug-toolbar not installed; toolbar disabled"
        )

The DuckDBPanel is added in Task 8, not here.

Also adjust the app = FastAPI(...) call to honor DEBUG:

app = FastAPI(debug=DEBUG, ...existing kwargs...)

• Step 6.4: Manual browser verification

DEBUG=1 uv run uvicorn app.main:app --port 8011 &
APP_PID=$!
sleep 3

Use playwright-cli for verification (per browser-automation.md):

playwright-cli -s=$PILOT_SESSION_ID open http://localhost:8011/dashboard
playwright-cli -s=$PILOT_SESSION_ID snapshot
# Expect to see a "djdt" or toolbar tab in the accessibility tree
playwright-cli -s=$PILOT_SESSION_ID screenshot /tmp/toolbar.png
playwright-cli -s=$PILOT_SESSION_ID close

kill $APP_PID

Expected: screenshot shows toolbar tab on right edge of the page. If the route requires auth, set LOCAL_DEV_MODE=1 too.

If you don't have playwright-cli, use this curl + grep verification:

DEBUG=1 uv run uvicorn app.main:app --port 8011 &
APP_PID=$!
sleep 3
LOCAL_DEV_MODE=1 curl -s http://localhost:8011/dashboard 2>/dev/null | grep -i "fastDebug\|debug.*toolbar" | head -3
kill $APP_PID

Expected: at least one match showing the toolbar HTML was injected.

• Step 6.5: Verify toolbar is OFF without DEBUG

uv run uvicorn app.main:app --port 8011 &
APP_PID=$!
sleep 3
curl -s http://localhost:8011/dashboard 2>/dev/null | grep -ci "fastDebug" || echo "no toolbar (correct)"
kill $APP_PID

Expected: no toolbar (correct).

• Step 6.6: Verify under docker compose up

Compose runs uvicorn under a non-TTY stdout, which is the case force_terminal=True exists for. This step catches docker-only failure modes that uvicorn-direct misses (env propagation, log multiplexing).

DEBUG=1 LOCAL_DEV_MODE=1 docker compose up -d app
sleep 8
docker compose logs --tail=30 app | grep -iE "rich|debug|toolbar|color" || true
docker compose exec -T app sh -lc 'python -c "import os; print(\"DEBUG=\", os.environ.get(\"DEBUG\"))"'
curl -s http://localhost:8000/health -i 2>/dev/null | head -5
docker compose down

Expected:

• Compose log lines appear with ANSI color codes (e.g. \e[) — proves force_terminal=True is doing its job.
• The python eval prints DEBUG= 1.
• /health returns 200 with x-request-id header.

If colors are missing, force_terminal=True is being overridden — check whether the compose service has tty: true set, or whether stdout is being piped through a process that strips ANSI.

• Step 6.7: Run the full test suite

uv run pytest -q

Expected: all tests pass.

• Step 6.8: Commit

git add pyproject.toml app/main.py uv.lock
git commit -m "feat(dev): mount fastapi-debug-toolbar behind DEBUG=1"

---

Task 7: InstrumentedConnection — Phase P6 (part 1)

Files:

• Create: app/debug/__init__.py (empty)

• Create: app/debug/duckdb_panel.py (just InstrumentedConnection and helpers — DuckDBPanel itself comes in Task 8)

• Test: tests/test_duckdb_panel.py

• Step 7.1: Create empty package marker

mkdir -p app/debug/templates/panels
: > app/debug/__init__.py

• Step 7.2: Write the failing tests

Create tests/test_duckdb_panel.py:

import duckdb
import pytest

from app.debug.duckdb_panel import (
    InstrumentedConnection,
    _request_store,
    get_request_store,
    record_query,
)


@pytest.fixture
def store_token():
    """Provide a fresh request-scoped store."""
    token = _request_store.set([])
    yield
    _request_store.reset(token)


@pytest.fixture
def conn():
    return duckdb.connect(":memory:")


def test_records_query(store_token, conn):
    inst = InstrumentedConnection(conn, "system")
    inst.execute("SELECT 1")
    store = get_request_store()
    assert len(store) == 1
    q = store[0]
    assert q.db == "system"
    assert q.sql == "SELECT 1"
    assert q.error is None
    assert q.ms >= 0


def test_records_query_with_params(store_token, conn):
    inst = InstrumentedConnection(conn, "analytics")
    inst.execute("SELECT $1::INT", [42])
    store = get_request_store()
    assert len(store) == 1
    assert store[0].params == [42]


def test_records_error(store_token, conn):
    inst = InstrumentedConnection(conn, "system")
    with pytest.raises(duckdb.Error):
        inst.execute("SELECT FROM bogus_syntax")
    store = get_request_store()
    assert len(store) == 1
    assert store[0].error is not None
    assert "bogus_syntax" in store[0].error or "syntax" in store[0].error.lower()


def test_db_tag_preserved(store_token):
    a = duckdb.connect(":memory:")
    b = duckdb.connect(":memory:")
    InstrumentedConnection(a, "system").execute("SELECT 1")
    InstrumentedConnection(b, "analytics").execute("SELECT 2")
    store = get_request_store()
    assert {q.db for q in store} == {"system", "analytics"}


def test_no_op_outside_request(conn):
    """When _request_store is None (outside a debug request), do not raise."""
    assert get_request_store() is None
    inst = InstrumentedConnection(conn, "system")
    inst.execute("SELECT 1")  # must not raise
    assert get_request_store() is None


def test_passthrough_attributes(conn):
    """Wrapper must delegate non-execute methods to the real connection."""
    inst = InstrumentedConnection(conn, "system")
    inst.execute("CREATE TABLE t (x INT)")
    inst.execute("INSERT INTO t VALUES (1), (2), (3)")
    rows = inst.execute("SELECT x FROM t ORDER BY x").fetchall()
    assert rows == [(1,), (2,), (3,)]


def test_cursor_returns_instrumented(store_token, conn):
    """A cursor() call returns an InstrumentedConnection wrapping the real cursor."""
    inst = InstrumentedConnection(conn, "system")
    cur = inst.cursor()
    assert isinstance(cur, InstrumentedConnection)
    cur.execute("SELECT 99")
    store = get_request_store()
    assert len(store) == 1
    assert store[0].db == "system"


def test_record_query_no_op_when_store_none():
    """record_query is safe to call when no store is set."""
    record_query("system", "SELECT 1", None, 0.0, None)  # must not raise

• Step 7.3: Run tests, verify they fail

uv run pytest tests/test_duckdb_panel.py -q

Expected: ImportError: cannot import name 'InstrumentedConnection'.

• Step 7.4: Implement InstrumentedConnection and helpers

Create app/debug/duckdb_panel.py (panel class will be added in Task 8):

"""Per-request DuckDB query capture for the dev debug toolbar.

`InstrumentedConnection` wraps a `duckdb.DuckDBPyConnection` and records
every `.execute()` call into a contextvar-scoped store. The toolbar's
`DuckDBPanel` reads that store at response time.

When the contextvar is unset (outside a debug request, or in prod), all
recording is a no-op — `_maybe_instrument()` in `src/db.py` returns the
raw connection, so the wrapper isn't even constructed in prod paths.
"""

from __future__ import annotations

import contextvars
import time
from dataclasses import dataclass
from typing import Any

import duckdb


@dataclass
class Query:
    db: str
    sql: str
    params: Any
    ms: float
    rows: int | None
    error: str | None = None


_request_store: contextvars.ContextVar[list[Query] | None] = contextvars.ContextVar(
    "duckdb_panel_store", default=None
)


def get_request_store() -> list[Query] | None:
    """Return the current request's query buffer, or None outside a debug request."""
    return _request_store.get()


def record_query(
    db: str,
    sql: str,
    params: Any,
    started: float,
    rows: int | None,
    error: str | None = None,
) -> None:
    store = _request_store.get()
    if store is None:
        return
    store.append(
        Query(
            db=db,
            sql=sql,
            params=params,
            ms=(time.perf_counter() - started) * 1000.0,
            rows=rows,
            error=error,
        )
    )


class InstrumentedConnection:
    """`duckdb.DuckDBPyConnection`-compatible wrapper that records queries."""

    def __init__(self, real: duckdb.DuckDBPyConnection, db_tag: str) -> None:
        self._real = real
        self._db = db_tag

    def execute(self, sql: str, params: Any = None, *args: Any, **kwargs: Any):
        started = time.perf_counter()
        err: str | None = None
        result = None
        try:
            if params is not None:
                result = self._real.execute(sql, params, *args, **kwargs)
            else:
                result = self._real.execute(sql, *args, **kwargs)
            return result
        except Exception as e:
            err = repr(e)
            raise
        finally:
            rows: int | None = None
            try:
                if result is not None and hasattr(result, "rowcount"):
                    rows = result.rowcount
            except Exception:
                pass
            record_query(self._db, sql, params, started, rows, err)

    def cursor(self) -> "InstrumentedConnection":
        return InstrumentedConnection(self._real.cursor(), self._db)

    def __getattr__(self, name: str) -> Any:
        return getattr(self._real, name)

• Step 7.5: Run tests, verify they pass

uv run pytest tests/test_duckdb_panel.py -q

Expected: all 8 tests pass.

• Step 7.6: Run quality checks

ruff format app/debug/ tests/test_duckdb_panel.py
ruff check app/debug/ tests/test_duckdb_panel.py --fix

• Step 7.7: Commit

git add app/debug/__init__.py app/debug/duckdb_panel.py tests/test_duckdb_panel.py
git commit -m "feat(debug): add InstrumentedConnection for per-request DuckDB query capture"

---

Task 8: DuckDBPanel + Jinja template + integration in src/db.py — Phase P6 (part 2)

Files:

• Modify: app/debug/duckdb_panel.py (append DuckDBPanel class)

• Create: app/debug/templates/panels/duckdb.html

• Modify: src/db.py (add _maybe_instrument; wrap returns)

• Modify: app/main.py (add app.debug.duckdb_panel.DuckDBPanel to the panels list)

• Test: tests/test_toolbar_integration.py

• Step 8.1: Append DuckDBPanel to app/debug/duckdb_panel.py

Add at the bottom of the file:

# Toolbar Panel (only imported when DEBUG=1, so import inside the class block)
try:
    from debug_toolbar.panels import Panel
    from debug_toolbar.types import ServerTiming, Stats

    class DuckDBPanel(Panel):
        """fastapi-debug-toolbar panel rendering captured DuckDB queries."""

        title = "DuckDB"
        template = "panels/duckdb.html"

        @property
        def nav_subtitle(self) -> str:
            store = get_request_store() or []
            total_ms = sum(q.ms for q in store)
            return f"{len(store)} queries · {total_ms:.1f} ms"

        async def process_request(self, request):
            _request_store.set([])
            return await super().process_request(request)

        async def generate_stats(self, request, response) -> Stats | None:
            store = get_request_store() or []
            return {
                "queries": [q.__dict__ for q in store],
                "total_ms": sum(q.ms for q in store),
                "by_db": {
                    db: sum(q.ms for q in store if q.db == db)
                    for db in {q.db for q in store}
                },
            }

        async def generate_server_timing(self, request, response) -> ServerTiming:
            store = get_request_store() or []
            return [("DuckDB", "DuckDB queries", sum(q.ms for q in store))]
except ImportError:
    # fastapi-debug-toolbar not installed in this environment (e.g. prod).
    # The class is only referenced when DEBUG=1, where the dep is required.
    DuckDBPanel = None  # type: ignore[assignment, misc]

• Step 8.2: Create the Jinja template

Create app/debug/templates/panels/duckdb.html:

<h4>DuckDB queries — {{ stats.queries|length }} ({{ "%.1f"|format(stats.total_ms) }} ms total)</h4>
{% if stats.by_db %}
<p>By DB:
  {% for db, ms in stats.by_db.items() %}<code>{{ db }}</code>: {{ "%.1f"|format(ms) }} ms{% if not loop.last %} · {% endif %}{% endfor %}
</p>
{% endif %}
<table class="djdt-table">
  <thead>
    <tr><th>#</th><th>DB</th><th>ms</th><th>rows</th><th>SQL</th><th>params</th></tr>
  </thead>
  <tbody>
  {% for q in stats.queries %}
    <tr class="{{ 'djdt-error' if q.error else '' }}">
      <td>{{ loop.index }}</td>
      <td>{{ q.db }}</td>
      <td>{{ "%.2f"|format(q.ms) }}</td>
      <td>{{ q.rows if q.rows is not none else '—' }}</td>
      <td>
        <pre>{{ q.sql }}</pre>
        {% if q.error %}<div class="djdt-error">{{ q.error }}</div>{% endif %}
      </td>
      <td><code>{{ q.params }}</code></td>
    </tr>
  {% endfor %}
  </tbody>
</table>

• Step 8.3: Wrap connections in src/db.py

The file has three duckdb.connect() call sites: lines 413, 423, 581, 587. The three public helpers return:

• get_system_db() — returns _system_db_conn.cursor() (line 416). Wrap the cursor.
• get_analytics_db() — returns the connection directly (line 423). Wrap it.
• get_analytics_db_readonly() — two return paths (lines 586, 587 onward). Wrap both.

Add this helper near the top of src/db.py (just after the imports):

import os as _os_for_debug

_DEBUG = _os_for_debug.environ.get("DEBUG", "").lower() in ("1", "true", "yes")


def _maybe_instrument(con, db_tag: str):
    """Wrap a duckdb connection with InstrumentedConnection when DEBUG=1, else return as-is."""
    if not _DEBUG:
        return con
    from app.debug.duckdb_panel import InstrumentedConnection
    return InstrumentedConnection(con, db_tag)

Then modify each helper:

get_system_db() at line 394 — the return _system_db_conn.cursor() becomes:

return _maybe_instrument(_system_db_conn.cursor(), "system")

get_analytics_db() at line 419 — the return duckdb.connect(str(db_path)) becomes:

return _maybe_instrument(duckdb.connect(str(db_path)), "analytics")

get_analytics_db_readonly() at line 573 — both return statements (line 586 with return conn and line 587 with conn = duckdb.connect(str(db_path), read_only=True) followed by ATTACH + final return conn) get the wrap applied at the final return only:

The function ends with something like return conn after ATTACH loop. Replace that final return conn with return _maybe_instrument(conn, "analytics_ro").

For the early-return path (lines 581–586) where the file doesn't exist: replace the early return conn with return _maybe_instrument(conn, "analytics_ro").

sed -n '580,605p' src/db.py  # verify exact return sites before editing

• Step 8.4: Mount DuckDBPanel in app/main.py

Edit the panel list created in Task 6.3. Add the DuckDBPanel string to the list:

panels=[
    "debug_toolbar.panels.headers.HeadersPanel",
    "debug_toolbar.panels.routes.RoutesPanel",
    "debug_toolbar.panels.settings.SettingsPanel",
    "debug_toolbar.panels.versions.VersionsPanel",
    "debug_toolbar.panels.timer.TimerPanel",
    "debug_toolbar.panels.logging.LoggingPanel",
    "debug_toolbar.panels.profiling.ProfilingPanel",
    "app.debug.duckdb_panel.DuckDBPanel",
],

The toolbar resolves panels by import string. Confirm app/debug/duckdb_panel.py exposes DuckDBPanel at module level (it does — see Step 8.1).

• Step 8.5: Configure Jinja template lookup

The toolbar needs to find panels/duckdb.html. Wire the template path by adding a conftest-style env or by passing the path to the middleware via settings. The simplest path: copy/symlink the template into the toolbar's expected lookup. Verify by reading fastapi-debug-toolbar docs:

uv run python -c "import debug_toolbar.middleware as m; import inspect; print(inspect.getfile(m))"

Then check how panels with custom templates register lookup paths. If the toolbar uses Jinja PackageLoader, set template = "panels/duckdb.html" AND ensure app/debug/templates/ is on its lookup path. The standard way (per upstream) is to set the template attribute and rely on the panel's own package; since DuckDBPanel lives in app.debug.duckdb_panel, the toolbar should resolve app/debug/templates/panels/duckdb.html automatically via package introspection.

If the toolbar fails to find the template, add this class attribute on DuckDBPanel:

template_dirs = ["app/debug/templates"]

Verify by hitting an HTML route with DEBUG=1 and inspecting the rendered toolbar HTML.

• Step 8.6: Write the integration tests

Create tests/test_toolbar_integration.py:

import os

import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient


@pytest.fixture
def reset_logging_state():
    import app.logging_config as lc
    lc._CONFIGURED = False
    yield
    lc._CONFIGURED = False
    import logging
    logging.getLogger().handlers.clear()


@pytest.fixture
def app_with_toolbar(monkeypatch, reset_logging_state):
    monkeypatch.setenv("DEBUG", "1")
    # Reload app.main fresh so DEBUG gate is honored
    import importlib
    import app.main as main_mod
    importlib.reload(main_mod)
    return main_mod.app


@pytest.fixture
def app_no_toolbar(monkeypatch, reset_logging_state):
    monkeypatch.delenv("DEBUG", raising=False)
    import importlib
    import app.main as main_mod
    importlib.reload(main_mod)
    return main_mod.app


@pytest.mark.integration
def test_toolbar_html_present_when_debug(app_with_toolbar):
    client = TestClient(app_with_toolbar)
    # Hit any HTML response endpoint (the dashboard or login page)
    resp = client.get("/dashboard", follow_redirects=False)
    if resp.status_code in (302, 401):
        pytest.skip("Dashboard requires auth in this env; visual verify covered separately")
    assert resp.status_code == 200
    body = resp.text.lower()
    assert "djdt" in body or "fastdebug" in body, "Toolbar markup not injected"


@pytest.mark.integration
def test_no_toolbar_when_debug_off(app_no_toolbar):
    client = TestClient(app_no_toolbar)
    resp = client.get("/dashboard", follow_redirects=False)
    if resp.status_code in (302, 401):
        return
    body = resp.text.lower()
    assert "djdt" not in body
    assert "fastdebug" not in body


@pytest.mark.integration
def test_request_id_header_always_present(app_no_toolbar):
    client = TestClient(app_no_toolbar)
    resp = client.get("/health")
    assert "x-request-id" in resp.headers

• Step 8.7: Run integration tests

uv run pytest tests/test_toolbar_integration.py -q

Expected: tests pass (or skip when auth required). If dashboard is hidden behind login, test against another HTML route (/login or /setup) — adjust the fixture to a route known to render HTML without auth.

• Step 8.8: Manual browser verification with playwright-cli

DEBUG=1 LOCAL_DEV_MODE=1 uv run uvicorn app.main:app --port 8011 &
APP_PID=$!
sleep 4

playwright-cli -s=$PILOT_SESSION_ID open http://localhost:8011/admin/access
playwright-cli -s=$PILOT_SESSION_ID snapshot
playwright-cli -s=$PILOT_SESSION_ID screenshot /tmp/toolbar-duckdb.png
playwright-cli -s=$PILOT_SESSION_ID close

kill $APP_PID
ls -la /tmp/toolbar-duckdb.png

Open /tmp/toolbar-duckdb.png (or read with the Read tool — it accepts images). Confirm:

• Toolbar tab visible on right edge
• Clicking opens panels list including DuckDB
• DuckDB panel shows queries with system / analytics_ro tags

If toolbar tab not visible: check uvicorn logs for fastapi-debug-toolbar not installed warning or template-loader errors.

• Step 8.9: Run the full test suite

uv run pytest -q --cov=app.logging_config --cov=app.middleware --cov=app.debug --cov-fail-under=80

Expected: tests pass and coverage on the new modules ≥ 80%.

• Step 8.10: Verify DuckDBPanel under docker compose up

DEBUG=1 LOCAL_DEV_MODE=1 docker compose up -d app
sleep 8
# Trigger a request that hits the DB
curl -s http://localhost:8000/dashboard -o /tmp/dash.html
docker compose logs --tail=40 app | head -40
grep -i "duckdb\|djdt" /tmp/dash.html | head -5
docker compose down

Expected: dash.html contains the toolbar HTML with the DuckDB panel slot. Logs show DB queries when DEBUG=1 (debug-level log output captures execute calls indirectly through any logger.debug in src/db.py).

• Step 8.11: Run quality checks

ruff format app/ src/db.py tests/test_toolbar_integration.py
ruff check app/ src/db.py tests/test_toolbar_integration.py --fix

• Step 8.12: Commit

git add app/debug/ src/db.py app/main.py tests/test_toolbar_integration.py
git commit -m "feat(debug): add DuckDBPanel rendering per-request queries with db tag"

---

Task 9: Documentation, .env.template, CHANGELOG — Phase P7

Files:

• Modify: config/.env.template (or .env.template at repo root — verify location)

• Create: docs/development.md

• Modify: CHANGELOG.md

• Step 9.1: Locate the env template

ls -la config/.env.template .env.template 2>/dev/null

The CLAUDE.md references config/.env.template. Verify path before editing.

• Step 9.2: Document DEBUG=1 in env template

Append to config/.env.template (or wherever it lives):

# === Local development ===
# DEBUG=1 enables:
#   - FastAPI debug=True (richer 500 pages)
#   - rich.logging.RichHandler (colored, with tracebacks)
#   - fastapi-debug-toolbar mounted at right edge of HTML pages
#   - DuckDB query capture in the toolbar
# Never set in production. Keep separate from LOCAL_DEV_MODE (auth bypass).
# DEBUG=1

• Step 9.3: Create developer docs

Create docs/development.md:

# Development guide

## Logging

All processes (FastAPI app, scheduler, telegram_bot, ws_gateway, corporate_memory,
session_collector, CLI scripts) use `app.logging_config.setup_logging` to configure
the root logger. Each entrypoint calls it once:

```python
from app.logging_config import setup_logging
setup_logging(__name__)

Library modules just do logger = logging.getLogger(__name__) — they never configure root.

Env	Handler	Format
DEBUG=1	rich.logging.RichHandler	colored, clickable file:line, pretty tracebacks
(default)	stdlib StreamHandler	JSON to stderr (ts, lvl, logger, service, msg, optional request_id)

LOG_LEVEL overrides the level (default DEBUG when DEBUG=1, else INFO).

Request correlation

RequestIdMiddleware is mounted unconditionally on the FastAPI app. It assigns or propagates X-Request-ID and exposes it via the request_id_var ContextVar so the JSON formatter and the debug-toolbar logging panel see the same id.

Debug toolbar

Set DEBUG=1 to mount fastapi-debug-toolbar. Visit any HTML page (e.g. /admin/access, /dashboard, /login) and click the small toolbar tab on the right edge.

Panels:

• Headers — request/response headers
• Routes — registered FastAPI routes
• Settings — Pydantic / instance-config values
• Versions — installed package versions
• Timer — request duration
• Logging — log records emitted during the request
• Profiling — pyinstrument flame graph
• DuckDB — every con.execute(sql, params) from src/db.py, tagged by system / analytics / analytics_ro, with timing and row count

JSON-only endpoints (Swagger UI at /docs) replay the most recent request's panels via a cookie-based mechanism.

The toolbar is never imported in production: the import is gated by if DEBUG: ... in app/main.py, and fastapi-debug-toolbar lives only in the dev optional-dependency group.

Running locally

uv pip install ".[dev]"
DEBUG=1 LOCAL_DEV_MODE=1 uv run uvicorn app.main:app --reload --port 8000

Open http://localhost:8000/dashboard.

- [ ] **Step 9.4: Update CHANGELOG.md**

Read the top of `CHANGELOG.md`:

```bash
sed -n '1,30p' CHANGELOG.md

If a ## [Unreleased] heading exists at the top, add bullets under it. If not, create one above the topmost released version.

Add under ### Added:

## [Unreleased]

### Added
- Dev debug toolbar gated by `DEBUG=1`. Mounts `fastapi-debug-toolbar` with panels
  for headers, routes, settings, versions, timer, logging, profiling, and a custom
  DuckDB panel that captures every `con.execute()` from `src/db.py` (tagged by
  `system` / `analytics` / `analytics_ro`). See `docs/development.md`.
- `X-Request-ID` request header / response header on every FastAPI response, plus
  a `request_id` field in JSON logs for cross-process correlation.
- Centralized `app.logging_config.setup_logging()` — replaces 23 scattered
  `logging.basicConfig(...)` calls. Uses `rich.logging.RichHandler` in dev
  (`DEBUG=1`) and JSON to stderr in prod.

### Changed
- All service entrypoints (`services/scheduler/__main__.py`, `ws_gateway`,
  `telegram_bot`, `corporate_memory`, `session_collector`, `verification_detector`)
  and CLI scripts under `scripts/` and `connectors/jira/scripts/` now call
  `setup_logging(__name__)` instead of inline `basicConfig`. Library modules no
  longer configure root logger at import time.

### Fixed
- Removed rogue module-level `logging.basicConfig` from `app/api/sync.py` that
  was reconfiguring root logger every time the api module was imported.

• Step 9.5: Run the full test suite one more time

uv run pytest -q

Expected: all tests pass.

• Step 9.6: Final verification

Confirm no remaining basicConfig outside tests:

grep -rEn 'logging\.basicConfig|_logging\.basicConfig' \
    src/ app/ services/ connectors/ scripts/ \
    --include="*.py" | grep -v "tests/" || echo "CLEAN"

Expected: CLEAN.

Confirm DEBUG=1 smoke test:

DEBUG=1 LOCAL_DEV_MODE=1 uv run uvicorn app.main:app --port 8011 &
APP_PID=$!
sleep 4
curl -s http://localhost:8011/health -i 2>/dev/null | head -5
kill $APP_PID

Expected: response with x-request-id header. Server log line is rich-colored.

• Step 9.7: Commit

git add config/.env.template docs/development.md CHANGELOG.md
git commit -m "docs: document DEBUG=1 toggle and centralized logging"

• Step 9.8: Push branch

git push -u origin vr/dev-logging

Open a PR via gh pr create (out of scope for this plan — the user runs PR creation separately).

---

Self-Review Checklist (already applied during plan writing)

• Spec coverage: every section in the spec maps to one of Tasks 1–9.
  • logging_config + RichHandler → Task 1
  • RequestIdMiddleware → Tasks 2–3
  • migration of 23 basicConfig calls → Tasks 3, 4, 5
  • DuckDBPanel + InstrumentedConnection → Tasks 7–8
  • production safeguards (DEBUG gate, dev-only dep) → Task 6
  • docs / CHANGELOG / .env.template → Task 9
• Placeholder scan: no TBD, TODO, or "implement later" left in the plan.
• Type consistency: Query dataclass used in tests and in DuckDBPanel.generate_stats; InstrumentedConnection.cursor() returns InstrumentedConnection (matches test).
• Files-not-to-modify rule: plan does NOT touch connectors/jira/file_lock.py, connectors/jira/transform.py is library-level — only the basicConfig line is removed (plan respects "core logic" stability), services/ws_gateway/ is touched only at the entrypoint gateway.py:214 for the basicConfig swap (no logic change).

---

Plan complete and saved to docs/superpowers/plans/2026-04-29-dev-debug-toolbar.md.