agnes-the-ai-analyst/scripts
ZdenekSrotyr b627de8344 feat(diagnose) + docs: warn on USER_PROJECT_DENIED footgun + document all newly-exposed knobs
Diagnostic + operator-facing documentation that closes the loop on the work in this PR.

`da diagnose` (via /api/health/detailed):
  - New _check_bq_billing_project() helper. When data_source.type='bigquery' and BqProjects.billing == .data, surface a yellow warning: 'BigQuery billing project equals data project'. Hint includes the YAML field path + the /admin/server-config UI shortcut. Diagnose's overall status promotes warning → degraded so the CLI echoes it.
  - Non-BQ instances (Keboola-only, etc.) skip the check.
  - Implementation hooks into the existing /api/health/detailed surface — no new endpoint, no CLI changes.

config/instance.yaml.example documentation:
  - data_source.bigquery.billing_project: USER_PROJECT_DENIED hint, /admin/server-config UI reference
  - data_source.bigquery.legacy_wrap_views: analyst-side discipline note (use `da fetch` / `da query --remote`), issue #101 history, view-heavy deployment guidance
  - data_source.bigquery.max_bytes_per_materialize: cost guardrail block (NEW — wasn't documented in .example before)
  - ai.base_url: provider list + UI hint
  - openmetadata + desktop: 'configurable via /admin/server-config UI' headers
  - corporate_memory: leading note that the schema is editable via UI

Other docs:
  - CHANGELOG.md: comprehensive Unreleased section
  - CLAUDE.md: schema chain → v20 + Materialized SQL connector mode + per-connector tab UI mention
  - README.md: mode-first source table summary
  - docs/architecture.md: per-connector tab UI mention
  - cli/skills/connectors.md: bootstrap rails (parallel to #154)
  - docs/superpowers/plans/2026-05-01-admin-tables-form-cleanup.md: implementation plan archive (2515 lines)
  - scripts/seed_dummy_tables.py: drop is_public after #150 RBAC migration (column gone)

Tests:
  - test_diagnose_billing.py — 3 cases (BQ with billing==data warns, BQ with billing!=data clean, non-BQ skips)
2026-05-01 20:27:24 +02:00
..
debug chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
dev feat(setup): cross-platform TLS bootstrap + marketplace plugin install (#137) 2026-04-30 08:56:45 +02:00
ops fix(tls-rotate): self-signed fallback sets basicConstraints=critical,CA:FALSE (#159) 2026-05-01 12:23:14 +02:00
bootstrap-gcp.sh fix(bootstrap): grant monitoring.editor + enable monitoring API 2026-04-21 20:32:50 +02:00
duckdb_manager.py chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
fetch-env-from-secrets.sh chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
generate_openapi.py feat: multi-instance deployment — all 14 must-have items from spec 2026-04-10 11:57:42 +02:00
generate_sample_data.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
init.sh refactor: final cleanup — delete legacy auth, clean deps, fix hash, migrate to uv 2026-03-31 19:18:30 +02:00
migrate_json_to_duckdb.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
migrate_metrics_to_duckdb.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
migrate_parquets_to_extracts.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
migrate_registry_to_duckdb.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
README.md fix: rewrite Makefile and scripts/README.md 2026-04-09 17:16:04 +02:00
run-local-dev.ps1 feat(dev): add Windows PowerShell wrapper for local development (#80) 2026-04-28 23:59:11 +02:00
run-local-dev.sh fix(security+ops) + release(0.12.1): #82 #85 #87 hardening + cut 0.12.1 (#104) 2026-04-28 19:57:30 +02:00
seed_corporate_memory.py feat(memory): corporate memory v1+v1.5 + 0.15.0 (#72) 2026-04-29 07:16:22 +02:00
seed_dummy_tables.py feat(diagnose) + docs: warn on USER_PROJECT_DENIED footgun + document all newly-exposed knobs 2026-05-01 20:27:24 +02:00
smoke-test-materialized-bq.sh feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
smoke-test.sh fix(ci): smoke-test stale route + rollback ghcr auth + issues:write (#140) 2026-04-30 09:42:27 +02:00
tls-fetch.sh feat(tls): corporate-CA HTTPS with URL-driven rotation, on-VM CSR gen, self-signed fallback (#51) 2026-04-25 19:51:25 +00:00

Scripts

Utility and migration scripts for Agnes AI Data Analyst.

Active Scripts

Script Purpose
generate_sample_data.py Generate sample data for development/demo
duckdb_manager.py DuckDB database management utilities
init.sh Initial server setup (install deps, create dirs)

Migration Scripts (one-time use)

Script Purpose
migrate_json_to_duckdb.py Migrate v1 JSON state files to DuckDB
migrate_parquets_to_extracts.py Migrate v1 parquet layout to extract.duckdb
migrate_registry_to_duckdb.py Migrate v1 table registry to DuckDB