* fix(security+ops): #82 #85 #87 — auth hardening, API validation, deploy posture Security and operational hardening across three issue groups: - M23: docker-compose.override.yml → docker-compose.dev.yml (BREAKING, prod foot-gun) - C13: Container runs as non-root user 'agnes' (USER directive in Dockerfile) - M21: Docker resource limits (mem_limit, cpus) on app + scheduler - M22: Caddyfile security headers (X-Frame-Options, X-Content-Type-Options, Referrer-Policy, -Server) - M17: /api/health split into minimal (unauth) + /api/health/detailed (auth) (BREAKING) - M26: release.yml restricts build-and-push to main + workflow_dispatch; paths-ignore for docs - C2: table_id traversal validation on /api/data/{table_id}/download - M4: Upload streaming (chunk-read + temp file) instead of full-buffer; /local-md hashed filename - C5: reset_token removed from POST /api/users/{id}/reset-password response - C8: Startup WARNING when no user has password_hash (bootstrap window visible) - M9: Audit log on failed web form login (mirrors /auth/token endpoint) - M10: Atomic magic-link consume via compare-and-swap (CONSUMED: marker + DuckDB conflict catch) Also: SSRF protection on /api/admin/configure (#46), memory stats SQL aggregation (#90) Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(review): SSRF 169.254.x.x + IPv6 multicast; M10 marker cleanup safety Review fixes: - Add 169.254.0.0/16 (link-local, cloud metadata) to SSRF regex — was missing, allowing requests to AWS/GCP/Azure metadata endpoints - Add ff[0-9a-f]{2}: (IPv6 multicast) to SSRF regex - M10: wrap Step 3 (CONSUMED marker cleanup) in try-except with warning log — prevents unhandled exception if DB write fails after successful token consumption - Add test for 169.254.169.254 SSRF rejection Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(review): SSRF IPv6 bypass, CLI health endpoint, upload FD leak Address Devin Review findings on PR #104: 1. SSRF IPv6 bypass: Replace hostname regex with DNS resolution + ipaddress module checks. The old regex patterns like `fe80:` only matched up to the first colon, missing real IPv6 addresses like `fe80::1`, `fc00::1`, `ff02::1`. The new approach resolves the hostname via getaddrinfo and checks each resulting IP against ipaddress.is_private/is_loopback/is_link_local/is_reserved/is_multicast. 2. CLI commands broken: `da setup test-connection`, `da setup verify`, `da diagnose`, `da status` all called /api/health expecting the old format (status=="healthy", services dict). Now they call /api/health/detailed for service-level checks (with graceful fallback to the minimal endpoint when auth is not configured). 3. Temp file handle leak: _stream_to_temp returns an open NamedTemporaryFile; callers now close it before shutil.move() to prevent FD leaks until GC. Also adds IPv6 SSRF test cases (loopback, link-local, unique-local, multicast) with mocked DNS resolution for test environment independence. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(review): download regex blocks hyphenated IDs; document health split Address Devin Review round-3 findings on PR #104: 1. _SAFE_IDENTIFIER regex blocked hyphenated table IDs: The download endpoint used the strict SQL-identifier regex which does not allow dots or hyphens, but Keboola table IDs like in.c-crm.orders contain both. Switched to _SAFE_QUOTED_IDENTIFIER which allows dots and hyphens while still blocking path-traversal chars (/, .., \) and quote/control characters. Added test for hyphenated/dotted IDs. 2. Documented health endpoint split in DEPLOYMENT.md: Added Health checks & external monitoring section explaining both endpoints (minimal unauth /api/health vs authenticated /api/health/detailed) and how to wire external monitoring tools to the detailed endpoint with a PAT. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * release(0.12.1): cut hotfix for snapshot integrity + #82/#85/#87 hardening * fix(security): apply CAS pattern to password reset confirm (#82/M10 follow-up) Devin review on the rebased PR flagged the asymmetry: magic-link verify got the atomic compare-and-swap pattern in the original M10 fix, but password reset confirm at /auth/password/reset/confirm was still using read-validate-clear. Two concurrent POSTs with the same valid reset token could both succeed in setting different new passwords (last-write- wins). Lower severity than the magic-link race because the attacker would need the reset token AND to race the legitimate user, but the asymmetry was a polish gap. Mirrors app/auth/providers/email.py::_consume_token CAS exactly: write unique CONSUMED:<random> marker via UPDATE...WHERE token=old_token, then SELECT to verify our marker won, then proceed. Only the winner clears the marker and applies the password change. New regression test_concurrent_reset_only_one_wins in tests/test_password_flows.py::TestResetConfirm pins the contract: two ThreadPoolExecutor workers + Barrier hit /reset/confirm with the same token; exactly one gets 302 (password applied), the other gets 200 with 'Invalid or expired'. Sanity-checked against the pre-CAS code — both POSTs got 302 (race confirmed). --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
151 lines
3.8 KiB
YAML
151 lines
3.8 KiB
YAML
services:
|
|
app:
|
|
build: .
|
|
# --proxy-headers + --forwarded-allow-ips make uvicorn honor the
|
|
# X-Forwarded-Proto / X-Forwarded-Host headers any reverse proxy (Caddy,
|
|
# nginx, Cloudflare Tunnel) sets. Without it, request.url_for() emits
|
|
# http://localhost:8000/... even when the user is on https://, which
|
|
# breaks OAuth callbacks (redirect_uri_mismatch). Belt-and-suspenders —
|
|
# FORWARDED_ALLOW_IPS=* in .env does the same via env var.
|
|
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --proxy-headers --forwarded-allow-ips='*'
|
|
ports:
|
|
- "8000:8000"
|
|
volumes:
|
|
- data:/data
|
|
- ./config:/app/config:ro
|
|
# - ./custom-connectors:/app/connectors/custom:ro # Tier A: AI-generated connectors
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
healthcheck:
|
|
test: ["CMD", "curl", "-sf", "http://localhost:8000/api/health"]
|
|
interval: 30s
|
|
timeout: 5s
|
|
retries: 3
|
|
restart: unless-stopped
|
|
mem_limit: 4g
|
|
mem_reservation: 1g
|
|
cpus: 2.0
|
|
|
|
# One-shot: run extractor then rebuild orchestrator views
|
|
extract:
|
|
build: .
|
|
command: >
|
|
sh -c "python -m connectors.keboola.extractor &&
|
|
python -c 'from src.orchestrator import SyncOrchestrator; print(SyncOrchestrator().rebuild())'"
|
|
volumes:
|
|
- data:/data
|
|
- ./config:/app/config:ro
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
profiles:
|
|
- extract
|
|
|
|
scheduler:
|
|
build: .
|
|
command: python -m services.scheduler
|
|
volumes:
|
|
- data:/data
|
|
- ./config:/app/config:ro
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
- API_URL=http://app:8000
|
|
- SEED_ADMIN_EMAIL=${SEED_ADMIN_EMAIL:-}
|
|
depends_on:
|
|
app:
|
|
condition: service_healthy
|
|
restart: unless-stopped
|
|
mem_limit: 2g
|
|
cpus: 1.0
|
|
|
|
telegram-bot:
|
|
build: .
|
|
command: python -m services.telegram_bot
|
|
volumes:
|
|
- data:/data
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
depends_on:
|
|
- app
|
|
profiles:
|
|
- full
|
|
restart: unless-stopped
|
|
|
|
ws-gateway:
|
|
build: .
|
|
command: python -m services.ws_gateway
|
|
volumes:
|
|
- data:/data
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
depends_on:
|
|
- app
|
|
profiles:
|
|
- full
|
|
restart: unless-stopped
|
|
|
|
corporate-memory:
|
|
build: .
|
|
command: python -m services.corporate_memory
|
|
volumes:
|
|
- data:/data
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
depends_on:
|
|
- app
|
|
profiles:
|
|
- full
|
|
restart: unless-stopped
|
|
|
|
session-collector:
|
|
build: .
|
|
command: python -m services.session_collector
|
|
volumes:
|
|
- data:/data
|
|
env_file: .env
|
|
environment:
|
|
- DATA_DIR=/data
|
|
depends_on:
|
|
- app
|
|
profiles:
|
|
- full
|
|
restart: unless-stopped
|
|
|
|
# TLS reverse proxy. Corporate-CA certs mounted from /data/state/certs
|
|
# (managed by scripts/ops/agnes-tls-rotate.sh on the VM). For local
|
|
# development without certs, run without --profile tls and hit :8000
|
|
# directly.
|
|
caddy:
|
|
image: caddy:2-alpine
|
|
ports:
|
|
- "80:80"
|
|
- "443:443"
|
|
volumes:
|
|
- ./Caddyfile:/etc/caddy/Caddyfile:ro
|
|
- /data/state/certs:/certs:ro
|
|
- caddy_data:/data
|
|
- caddy_config:/config
|
|
environment:
|
|
- DOMAIN=${DOMAIN:-localhost}
|
|
# Passes through whatever the operator set in .env. Caddyfile uses
|
|
# {$CADDY_TLS:tls /certs/fullchain.pem /certs/privkey.pem} so:
|
|
# - unset → cert-file mode (corp PKI rotated by tls-rotate.sh)
|
|
# - "tls <email>" → Let's Encrypt auto-issue
|
|
# - "tls internal" → Caddy-managed self-signed
|
|
- CADDY_TLS
|
|
depends_on:
|
|
app:
|
|
condition: service_healthy
|
|
restart: unless-stopped
|
|
profiles:
|
|
- tls
|
|
|
|
volumes:
|
|
data:
|
|
caddy_data:
|
|
caddy_config:
|