Commit graph

61 commits

Author SHA1 Message Date
ZdenekSrotyr
ee7d5630ef fix: keep external_access enabled — views need read_parquet on local files
File access attacks blocked by SQL blocklist instead of DuckDB pragma
(pragma also blocks legitimate view resolution via read_parquet).
2026-04-08 12:33:05 +02:00
ZdenekSrotyr
f2f9a62803 fix: set enable_external_access=false AFTER ATTACHing extracts 2026-04-08 12:29:27 +02:00
ZdenekSrotyr
6efdf4ca64 fix: read-only analytics DB ATTACHes extract.duckdb files for view resolution 2026-04-08 12:27:12 +02:00
ZdenekSrotyr
05a1b452e9 security: harden query (read-only DB), uploads (path sanitization), scripts (AST validation) 2026-04-08 12:09:19 +02:00
ZdenekSrotyr
2d6a94fb6f fix: DuckDB concurrency — WAL mode, subprocess sync, temp+rename
Three-pronged fix for DuckDB lock conflicts:

1. WAL mode on system.duckdb — enables concurrent readers + writer
2. Sync trigger runs extractor as subprocess (not background task) —
   separate process = separate DuckDB connections, no lock conflict
3. Both extractor and orchestrator write to .tmp then atomic rename —
   avoids lock conflict with API reads on extract.duckdb/analytics.duckdb

Fixes #9 permanently.
2026-03-31 13:19:57 +02:00
ZdenekSrotyr
675a29c1c7 fix: DuckDB connection pool — shared connection avoids lock conflicts
Fixes #9 — background sync tasks could not access system.duckdb
because FastAPI held an exclusive lock. Now uses single shared
connection per DATA_DIR with cursor() for thread safety.
2026-03-31 13:01:04 +02:00
ZdenekSrotyr
2e7d5d1fe9 feat: access request UI — catalog badges, request modal, admin approval page
Backend:
- access_requests table in DuckDB schema
- AccessRequestRepository with create/approve/deny/list
- API: POST/GET /api/access-requests (submit, my requests, pending, approve, deny)

UI:
- Catalog: lock icon on private tables, "Request Access" button + modal
- Catalog: "Pending" badge for tables with pending requests
- Admin permissions page (/admin/permissions): approve/deny requests,
  grant/revoke permissions, view all user permissions
- Cross-navigation between admin/tables and admin/permissions

733 tests passing.
2026-03-31 12:45:29 +02:00
ZdenekSrotyr
1074d5ec49 feat: implement data access control — table-level permissions
Schema v3: add is_public column to table_registry (default true).

src/rbac.py: can_access_table() checks admin bypass, public flag,
explicit permissions, wildcard bucket permissions.

API enforcement:
- manifest: filters tables by user access
- download: 403 if no access
- catalog: filters table list
- query: validates referenced tables against allowed list

New admin permissions API (/api/admin/permissions) for grant/revoke.

28 access control tests + 733 total tests passing.
2026-03-31 12:33:31 +02:00
ZdenekSrotyr
18e5f0b6e8 feat: implement extract.duckdb contract — orchestrator + extractors
Phase 0: extend table_registry schema (v1→v2 migration), add
source_type/bucket/source_table/query_mode columns.

Phase 1: SyncOrchestrator ATTACHes extract.duckdb files into master
analytics.duckdb. Keboola extractor uses DuckDB extension with
legacy client fallback. BigQuery extractor is remote-only via
DuckDB BQ extension (no data download).

62 tests passing.
2026-03-30 20:12:56 +02:00
ZdenekSrotyr
79b0b66f2e feat: add DuckDB state layer with all repository classes
- src/db.py: schema with 14 tables matching design spec
- 7 repository classes: SyncState, Users, Knowledge, Audit,
  Telegram, PendingCode, Script, TableRegistry, Profiles
- 37 tests covering all CRUD operations
2026-03-27 15:06:55 +01:00
ZdenekSrotyr
f76411c603 feat: add DuckDB state layer with schema management
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 13:55:54 +01:00