Petr
e17dd85504
Remove hardcoded Jira/Keboola references from sync_data.sh
...
- Silent fallback when no sync settings exist (no 'Jira disabled' message)
- Generic dataset exclude/include loop driven by sync_settings.yaml
- Generic cleanup loop for disabled datasets
- Replaces 100+ lines of hardcoded Jira/kbc_telemetry_expert blocks
2026-03-15 01:02:37 +01:00
Petr
49adbe26ec
Move server venv setup to bootstrap, remove cross-platform pip freeze sync
...
Server venv is created during bootstrap via SSH (same package list,
installed natively on Linux). Removes sync_data.sh section that copied
pip freeze output across platforms (Windows/macOS freeze is incompatible
with Linux).
2026-03-15 00:59:45 +01:00
Petr
22a1bb5847
Auto-restart sync_data.sh after self-update (exec replaces process)
2026-03-15 00:53:17 +01:00
Petr
6f9de274fb
Fix: CLAUDE.md template vars overwrote $SSH_HOST used by rsync
...
The CLAUDE.md generation section reused SSH_HOST variable name to store
the server IP, overwriting the SSH alias needed for rsync. Renamed to
TMPL_SSH_ALIAS/TMPL_SERVER_HOST/TMPL_WEBAPP_URL to avoid collision.
2026-03-15 00:51:49 +01:00
Petr
b0e4749b0d
Replace hardcoded 'data-analyst' SSH alias with configurable $SSH_HOST
...
Read SSH alias from .sync_connection file at script start (default:
'data-analyst' for backward compatibility). All 32 occurrences of
hardcoded 'data-analyst:' and 'ssh data-analyst' replaced with $SSH_HOST.
2026-03-15 00:03:07 +01:00
Petr
2237334b05
Make CLAUDE.md template generic and instance-aware
...
- Remove all Keboola-specific content (metric categories, MRR/ARR refs,
corporate memory, hardcoded server IP)
- Add {ssh_alias}, {server_host}, {webapp_url} placeholders
- Bootstrap saves .sync_connection file with instance details
- sync_data.sh reads .sync_connection to substitute all placeholders
- Text instructions in dashboard include .sync_connection step
2026-03-14 23:57:58 +01:00
Petr
b99ec576ca
Add self-service data onboarding system
...
Table Registry as central source of truth (JSON) with atomic writes,
optimistic locking, audit logging, and data_description.md generation.
Existing readers (config.py, profiler.py) need zero changes.
Phase 1 - Discovery API:
- discover_tables() on DataSource ABC + Keboola implementation
- admin_required decorator with server-side recomputation
- GET /api/admin/discover-tables endpoint
Phase 2 - Table Registry:
- src/table_registry.py with CRUD, validation, migration from MD
- Admin API: register/update/unregister with version locking
- DELETE cascade cleans up per-user subscriptions
Phase 3 - Auto-Profiling:
- profile_changed_tables() for incremental profiling
- Non-fatal hook in sync_all() after successful sync
Phase 4 - Per-Table Subscriptions:
- table_mode (all/explicit) with per-table toggles
- GET/POST /api/table-subscriptions endpoints
- Subscription status in catalog and dashboard views
Phase 5 - Smart Sync:
- Python-generated rsync filter files (not shell YAML parsing)
- sync_data.sh uses --filter="merge ..." for explicit mode
Phase 6 - Admin UI:
- /admin/tables with discovery, registration modal, registry mgmt
- Vanilla JS, matching existing design system
2026-03-09 14:25:37 +01:00
Petr
c56905d34f
Initial commit: OSS data distribution platform
...
Open-source AI data analyst platform extracted from internal repo.
Includes data sync engine, Keboola adapter, Flask web portal,
server deployment scripts, and configuration templates.
2026-03-08 23:31:28 +01:00
