* feat(initial-workspace): per-instance agnes init override
Adds Initial Workspace Template — an admin-configurable per-instance
override for the agnes init analyst workspace. When configured, agnes
init downloads a server-rendered zip from a Git repo the admin registered
and extracts it into the analyst's workspace, fully bypassing Agnes-default
CLAUDE.md / settings.json / hooks / slash commands / AGNES_WORKSPACE.md.
Repo layout convention: only the contents of a top-level `workspace/`
subdirectory ship to analysts; admin docs (README, CI configs) at the
repo root stay in the repo and never reach an analyst. Sync rejects
repos without `workspace/` at root.
Server side:
- src/initial_workspace.py — clone (or fetch+reset), validate, build zip
with strict path checks and reserved-path rejection
(workspace/.claude/init-complete reserved by Agnes)
- app/api/initial_workspace.py — admin CRUD + sync endpoint + analyst-
facing status/zip/applied endpoints; config persists to instance.yaml
overlay, PAT to .env_overlay
- app/secrets.py — refactor: persist_overlay_token shared helper with
threading.Lock for .env_overlay writes (closes pre-existing race
between concurrent marketplaces saves)
- app/web/templates/admin_server_config.html — new "Initial Workspace
Template" section + modal + Sync/Edit/Delete/Download buttons (matches
existing cfg-section visual language)
CLI side:
- cli/lib/override.py — single source of truth for is_override_workspace
sentinel detection
- cli/lib/initial_workspace.py — probe status, safe zip extraction with
../absolute/symlink rejection, typed-YES force confirmation
- cli/commands/init.py — override branch (skips Agnes-default workspace
writes); extended sentinel with override:true, template_source,
template_sha so future agnes self-upgrade does not auto-refresh hooks
- cli/lib/hooks.py + cli/lib/commands.py — short-circuit on override
workspaces (install_claude_hooks, install_claude_commands,
maybe_refresh_claude_hooks)
Audit-event strategy: server writes initial_workspace.fetch_started
inside GET /api/initial-workspace.zip (cannot be spoofed by PAT-holder);
CLI POST /applied writes initial_workspace.applied as best-effort
confirmation. Admin mutations log via the existing _audit pattern.
Tests: 27 server (clone/validate/zip + workspace-subdir convention +
concurrent persist_overlay_token + endpoint shapes + audit rows) + 29
CLI (override sentinel parse + probe fall-through + safe extraction +
YES strictness + hook guards + e2e mocked init).
Risk acceptance — documented in docs/initial-workspace-override.md +
CHANGELOG Internal section so AI reviewers understand the deviations
from defaults are intentional:
- maybe_refresh_claude_hooks deliberately no-ops on override workspaces
- --force on override does NOT back up CLAUDE.md (admin's repo is the
source of truth)
- .claude/CLAUDE.local.md IS overwritten by override extraction when
admin's repo ships one
* test+vendor-agnostic: drop Groupon tokens from #292 fixtures + extend admin-gate coverage
Two fixes from the takeover review on #292:
1. **Vendor-agnostic OSS rule**: Replace `Groupon` / `groupon/template`
tokens in test fixtures with `Acme` / `acme/template` (8 sites in
test_cli_init_override.py + 1 in test_initial_workspace_api.py).
Per CLAUDE.md "Vendor-agnostic OSS — no customer-specific content"
rule: customer-specific tokens don't belong in shipped artifacts,
even in test fixtures. The pre-existing FoundryAI mentions in
test_instance_config.py + test_setup_instructions.py are out of
scope for this PR (didn't introduce them).
2. **Admin-gate coverage gap**: `test_admin_endpoints_require_admin`
only covered GET /api/admin/initial-workspace + POST .../sync. The
register-write (POST .../initial-workspace) and delete (DELETE
.../initial-workspace) endpoints used the same `Depends(require_admin)`
wiring but had no regression test. Loop now covers all 4 verbs so
a future refactor that drops the dependency from one endpoint
fails here instead of silently exposing the write/delete paths to
any analyst with a PAT.
* release: 0.54.9 — Initial Workspace Template (per-instance agnes init override)
Last commit on the PR per CLAUDE.md hard rule. Patch bump (0.54.8 →
0.54.9) for Mina's Initial Workspace Template feature.
No DB migration (config lives in instance.yaml overlay). No
mandatory operator action — empty default keeps OSS-default
agnes init behavior. Operators wanting full template control link a
Git repo on /admin/server-config → "Initial Workspace Template".
See docs/initial-workspace-override.md for the full
responsibility-transfer contract.
---------
Co-authored-by: Minas Arustamyan <arustamyan.minas@gmail.com>
Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
* feat(store-guardrails): admin-configurable content thresholds
Adds the flea-market content guardrail floors to the /admin/server-config
editor so operators can tune the bar without code changes. Defaults are
unchanged (60 chars description, 25 chars command, 5 distinct words, 200
chars body) — patching guardrails.* in instance.yaml or via the admin UI
overrides any of them and the next inline check picks up the new value.
src/store_guardrails/content_check.py now resolves the four floors via
helper functions (_min_desc_chars / _min_command_desc_chars /
_min_distinct_words / _min_body_chars) that read app.instance_config at
call time. Module-level _DEFAULT_* constants remain as fallbacks if
the import fails (defensive — keeps the guardrail module loadable
without the app package on its path).
app/instance_config.py grows four matching getters returning the live
value with sane defaults + integer coercion.
app/api/admin.py registers 'guardrails' as an editable section + ships
nine known-fields entries (min_description_chars,
min_command_description_chars, min_distinct_words, min_body_chars,
enabled, review_model, blocked_quota_per_day, blocked_bundle_ttl_days,
stuck_review_grace_seconds) with operator-facing hint copy explaining
what each knob does.
app/web/templates/admin_server_config.html gets a SECTION_META entry
so the section renders as 'Flea-market guardrails' with a help string
instead of a bare section ID.
app/web/router.py threads the live thresholds into /store/new and
/store/examples via a small _guardrail_thresholds() helper so the
disclosure copy, char counter, and "Why these limits" table render
the configured value (not a hardcoded 60). End-to-end smoke verified:
PATCH guardrails.min_description_chars=90 → /store/new immediately
renders "90 characters" + JS DESC_MIN=90 on the next request, no
restart required (helpers read live config per call).
* chore(store-guardrails): address PR review safe-fix findings
Code-review safe_auto findings on PR #281 (review run
20260513-100126-64052520):
- CHANGELOG: add Unreleased entry covering the new
/admin/server-config Flea-market guardrails section, the four live
threshold getters, and the route-helper rendering knobs. Required by
the project's non-negotiable "Changelog discipline" rule.
- content_check.py: narrow `except Exception` to `except ImportError`
on the four `_min_*()` resolver helpers. Surface-level TypeError /
ValueError on a malformed YAML value belongs to the
instance_config getters' own try/except — the resolvers should only
defend against the in-tree import itself failing, not silently
swallow real bugs in the getters.
- store_upload.html: refresh the stale "30-char threshold" comment to
reflect the configurable floor (default 60), and add `|default(60)`
/ `|default(25)` / `|default(5)` filters to the disclosure-copy
bindings so the upload form matches store_examples.html's
belt-and-suspenders rendering if a future route ever renders the
template without populating the `guardrail` context.
- router.py: tighten `_guardrail_thresholds()` return annotation
from bare `dict` to `dict[str, int]`.
Residual work (left for separate change after operator direction):
- Add round-trip test (PATCH guardrails -> next inline check uses
new value) — primary testing gap.
- Decide policy on `min_*=0` (currently coerced to 1 via
`max(1, int(val))`) vs treating 0 as a disable sentinel like
neighbour getters (`blocked_quota_per_day`,
`blocked_bundle_ttl_days`).
- Add POST-time integer validation for `guardrails.*` so a typo'd
YAML value (bool / string / float) errors loudly instead of
silently falling back to the default.
* test(store-guardrails): cover admin-configurable thresholds + PATCH round-trip
Closes the "primary testing gap" Vojta noted in the safe-fix commit
on PR #281 — the four new `get_guardrails_min_*` getters and the
PATCH-takes-effect-on-next-check live-config flow had no direct
coverage.
10 new tests in `tests/test_store_guardrails_admin_config.py`:
- TestGuardrailGetterDefaults (4 tests) — each new getter returns the
documented default (60 / 25 / 5 / 200) when nothing is configured.
- TestGuardrailGetterOverlay (5 tests) — overlay-driven overrides win,
string values that look numeric coerce via int(), garbage strings
fall back to default via the (TypeError, ValueError) branch, and the
`max(1, int(val))` floor pins zero/negative inputs to 1.
- TestPatchRoundTrip (1 test) — PATCH `/api/admin/server-config`
`guardrails.min_description_chars=90`, then call content_check
against a 75-char description that previously passed: must now fail
with `too_short`. Then PATCH back to 60 and verify the next check
passes again. Closes the cache-invalidation contract Vojta relies on
for the "no app restart" claim — broken without the
reset_cache() bracket in /api/admin/server-config.
The TestGuardrailGetterOverlay.test_zero_or_negative_floored_to_one
test pins the current `max(1, int(val))` policy. Vojta's safe-fix
commit explicitly left "policy on min_*=0 vs disable-sentinel" as
residual work — pinning the current behavior here ensures any future
change to use 0 as a disable sentinel must update this test (and the
reviewer sees the policy decision).
Verified: 4509 tests pass locally (4499 existing + 10 new).
* release: 0.54.2 — admin-configurable flea-market guardrail thresholds + tests
Last commit on the PR per CLAUDE.md hard rule. Patch bump (0.54.1 →
0.54.2) bundling Vojta's admin-configurable thresholds for the
flea-market content guardrail (9 knobs in /admin/server-config) plus
the test coverage closing the "primary testing gap" he punted in the
safe-fix commit.
No DB migration; defaults unchanged from PR #276 — instances that
don't set `guardrails.*` keep the original bar transparently.
---------
Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
Co-authored-by: ZdenekSrotyr <139972147+ZdenekSrotyr@users.noreply.github.com>
New top-level 'materialize' section, single field (lock_ttl_seconds).
Default 86400 (24h). Backs the file-lock TTL reclaim added in the
per-table-mutex change. Editable via PUT /api/admin/server-config and
the /admin/server-config UI.
Devin Review on PR #168 found 5 issues — all real, all addressed.
🚩 ANALYSIS_001 (architectural): concurrent-slot guard didn't protect
actual BQ query execution. Earlier `_enforce_remote_bq_quota_and_cap`
ran dry-run + cap check inside `with quota.acquire(user_id):`, then
returned — releasing the slot BEFORE `analytics.execute(...)` ran. Spec
§4.3.3 explicitly designs the slot to wrap execute so the per-user
concurrent cap limits BQ scans, not just dry-runs.
Refactor to a context manager `_bq_quota_and_cap_guard`. Caller's `with`
block now holds the slot through dry-run, cap check, the actual
`analytics.execute(...)` (which is what triggers the BQ scan when DuckDB
resolves the master view), AND the post-flight record_bytes. Slot
released only when caller's `with` body exits.
🟡 BUG_001: placeholder JS walked `original` (full GET payload root)
instead of `original.sections`. `placeholder_from: ["data_source",
"bigquery", "project"]` is a section-relative path, so billing_project
placeholder NEVER rendered. Fix: walk `original.sections` (with fallback
to `original` for safety).
🟡 BUG_002 + BUG_003: admin_tables.html register and edit modals'
operator help text referenced `max_bytes_per_remote_query` (the old
name from the spec) but the actual config key is `bq_max_scan_bytes`
after the fix-up commit `6423888d` moved it. Replace both occurrences.
🟡 BUG_004: CHANGELOG entry said `api.query.bq_max_scan_bytes` (the
old path) but the read at app/api/query.py:53 is
`get_value("data_source", "bigquery", "bq_max_scan_bytes", ...)`. An
operator who set it under `api.query` in their yaml would have no
effect. Correct path in CHANGELOG.
All 95 #160-affected tests pass after the changes.
Closes the operator-side half of the reporter's loop. The CLI fix in
the previous commit makes USER_PROJECT_DENIED errors readable to
analysts; this commit lets admins verify reachability proactively
from /admin/server-config without waiting for analyst reports.
New endpoint POST /api/admin/bigquery/test-connection
(app/api/admin_bigquery_test.py, ~110 LOC):
- Depends(require_admin); registered in app/main.py.
- Builds BqAccess via existing get_bq_access(), runs `SELECT 1 AS ok`
with a 10s polling timeout.
- 200 with {ok, billing_project, data_project, elapsed_ms} on success.
- 400 for `BqAccessError(not_configured)` (operator config issue).
- 502 for any other typed BqAccessError or unknown upstream exception.
- 504 for concurrent.futures.TimeoutError; best-effort cancel_job
invoked (BQ-side cancel may still run; documented caveat).
Server-config placeholder (app/api/admin.py + admin_server_config.html):
- `data_source.bigquery.billing_project` field-spec gains
`placeholder_from: ["data_source", "bigquery", "project"]`.
- renderLeafInput's text branch reads `opts.spec.placeholder_from`,
walks the loaded `original` config dict, injects
`placeholder="(defaults to <project>)"` into the input HTML at
construction time. Admin sees the access.py:339-340 fallback rule
visible directly in the UI without reading source.
UI button:
- "Test BigQuery connection" button next to data_source's Save button.
- onTestBigQuery() POSTs to the endpoint, renders structured result
inline (green check + elapsed_ms on success; red kind + hint on
failure).
Tests: 6 endpoint cases + 1 placeholder payload test = 7 GREEN. 62
total across the affected admin server-config test files.
Today /admin/server-config renders fields by iterating Object.keys(payload) on the YAML value — if a key isn't in instance.yaml, the operator can't see it. They have to know to type it via the JSON-patch textarea (which only renders for empty sections) or SSH and edit YAML.
Adds a known-fields registry (`_KNOWN_FIELDS` in app/api/admin.py) the UI consumes alongside the YAML payload. Renderer shows BOTH:
- existing fields (from YAML) with current value
- known-but-unset fields with dashed-border placeholder + hint, ready to fill in
Renderer (`renderField`, `renderSection`, `collectSection`):
- kind="string"|"secret"|"bool"|"int"|"select"|"object"|"array"|"map" — picks input type
- kind="object" with `fields` — recursive structured form, arbitrary depth (corporate_memory needs 3-4 levels)
- kind="array" with `item_kind` — vertical stack of typed inputs + add/remove buttons
- kind="map" with `key_kind` + `value_kind` — key:value rows + add/remove (used for confidence.base, domain_owners, entity_resolution.entities)
- data-path encoded as JSON segment array so map keys with embedded dots (e.g. 'user_verification.correction') survive collect → patch round-trip
- .cfg-field.is-unset CSS — dashed border, muted label, italic hint
Sections newly exposed (added to _EDITABLE_SECTIONS):
- openmetadata: url, token (secret), cache_ttl_seconds, verify_ssl
- desktop: jwt_issuer, jwt_secret (secret), url_scheme
Known fields populated for existing sections:
- data_source.bigquery: billing_project (the cause of the 403 USER_PROJECT_DENIED footgun when SA can read but not bill the data project), legacy_wrap_views (bigquery_query() wrap for VIEWs — issue #101 default off, ON for view-heavy deployments), max_bytes_per_materialize (cost guardrail)
- data_source.keboola: stack_url, project_id (hints; values already populated)
- ai: base_url (required for openai_compat), structured_output (select)
- corporate_memory: full schema from instance.yaml.example — distribution_mode, approval_mode, review_period_months, notify_on_new_items, sources.{claude_local_md,session_transcripts}, extraction.{model,sensitivity_check,contradiction_check}, confidence.{base,modifiers,decay.{mode,half_life_months,decay_rate_monthly,floor}}, contradiction_detection.{enabled,max_candidates}, entity_resolution.{enabled,entities}, domain_owners, domains
- Known partial: confidence.modifiers is map<string, map<string, float>> — falls through to JSON-textarea with TODO; structured editor for that one shape needs more renderer work
Tests:
- test_admin_server_config_known_fields — registry envelope shape, smoke fixture
- test_admin_server_config_renderer_depth — 4-level nested objects, arrays of strings, maps of floats, dotted-key safety
- test_admin_server_config_corp_memory — full corporate_memory schema, 12 fields incl. nested
- test_admin_server_config — existing tests adjusted for new shape
Adds /admin/server-config UI for editing instance.yaml from the web. Hardening: SSRF gate on data_source URLs, narrow-overlay write strategy, atomic writes, audit log with secret masking on shape changes, threading lock on read-modify-write, corrupt-overlay refusal on write side + louder log on read side, modal Promise resolution on backdrop dismiss, sentinel scrub on save (defense-in-depth client+server). Bundles Windows PowerShell wrapper from #80. Cuts release v0.13.0.
minasarustamyan