agnes-the-ai-analyst/scripts
Vojtech 0bbbf3e40b
feat(tls): corporate-CA HTTPS with URL-driven rotation, on-VM CSR gen, self-signed fallback (#51)
Replaces the implicit Let's Encrypt flow with a general corporate-CA HTTPS path:

- Caddy switches to cert-file mode (`tls /certs/fullchain.pem /certs/privkey.pem`) with HSTS + TLS 1.2/1.3 floor
- New `docker-compose.tls.yml` overlay closes host `:8000` when Caddy fronts (no TLS bypass)
- New `scripts/tls-fetch.sh` — generic URL fetcher for `sm://`, `gs://`, `https://`, `file://` with redirect refusal + PEM validation
- New `scripts/grpn/agnes-tls-rotate.sh` — daily rotation, self-signed fallback against same key (zero key churn), on-VM RSA-2048 + CSR auto-gen, atomic swap, SIGUSR1 reload
- `scripts/grpn/agnes-auto-upgrade.sh` becomes cert-aware (auto-enables tls overlay when certs present)
- Compose profile `production` renamed to `tls` (aligns with DEPLOYMENT.md and infra startup)

Pairs with FoundryAI/agnes-the-ai-analyst-infra#27 (merged) which wires per-VM `local.vm_tls`, writes `TLS_*` env vars into `.env`, auto-creates Secret Manager containers for `sm://` privkey URLs, and installs `agnes-tls-rotate.{service,timer}` for daily polling.

Includes hardening + docs follow-ups from code review:
- `TLS_CSR_SUBJECT` env-var parametrisation applied to both CSR and self-signed cert paths
- curl `--max-redirs 0 --proto '=https'` + post-fetch PEM validation in `tls-fetch.sh`
- `ulimit -c 0` + array-form `COMPOSE_FILES` (POSIX-safe, bash 3.2 compatible)
- TLS section added to `config/.env.template`
- Historical-note headers in `docs/superpowers/{plans,specs}/2026-04-09-*.md` flagging the profile rename
2026-04-25 19:51:25 +00:00
..
grpn feat(tls): corporate-CA HTTPS with URL-driven rotation, on-VM CSR gen, self-signed fallback (#51) 2026-04-25 19:51:25 +00:00
bootstrap-gcp.sh fix(bootstrap): grant monitoring.editor + enable monitoring API 2026-04-21 20:32:50 +02:00
duckdb_manager.py docs: update references to deleted files 2026-04-09 17:15:19 +02:00
fetch-env-from-secrets.sh infra: add fetch-env-from-secrets.sh for VM-side .env generation 2026-04-21 16:18:35 +02:00
generate_openapi.py feat: multi-instance deployment — all 14 must-have items from spec 2026-04-10 11:57:42 +02:00
generate_sample_data.py refactor: delete old sync pipeline — 9,500 lines removed 2026-03-31 07:50:37 +02:00
init.sh refactor: final cleanup — delete legacy auth, clean deps, fix hash, migrate to uv 2026-03-31 19:18:30 +02:00
migrate_json_to_duckdb.py feat: add JSON to DuckDB migration script with tests 2026-03-27 15:09:06 +01:00
migrate_metrics_to_duckdb.py feat: add standalone metric YAML → DuckDB migration script 2026-04-10 19:35:36 +02:00
migrate_parquets_to_extracts.py feat: add migration scripts for extract.duckdb transition 2026-03-30 20:21:12 +02:00
migrate_registry_to_duckdb.py feat: add migration scripts for extract.duckdb transition 2026-03-30 20:21:12 +02:00
README.md fix: rewrite Makefile and scripts/README.md 2026-04-09 17:16:04 +02:00
run-local-dev.sh feat(dev): LOCAL_DEV_MODE for one-command local dev + magic-link fixes (#32) 2026-04-22 14:47:33 +02:00
smoke-test.sh fix: address Devin review round 4 — bash arithmetic, CalVer max, docs 2026-04-10 14:39:16 +02:00
switch-dev-vm.sh chore: add switch-dev-vm.sh helper for hackathon (#20) 2026-04-21 21:33:02 +02:00
tls-fetch.sh feat(tls): corporate-CA HTTPS with URL-driven rotation, on-VM CSR gen, self-signed fallback (#51) 2026-04-25 19:51:25 +00:00

Scripts

Utility and migration scripts for Agnes AI Data Analyst.

Active Scripts

Script Purpose
generate_sample_data.py Generate sample data for development/demo
duckdb_manager.py DuckDB database management utilities
init.sh Initial server setup (install deps, create dirs)

Migration Scripts (one-time use)

Script Purpose
migrate_json_to_duckdb.py Migrate v1 JSON state files to DuckDB
migrate_parquets_to_extracts.py Migrate v1 parquet layout to extract.duckdb
migrate_registry_to_duckdb.py Migrate v1 table registry to DuckDB