26c4e0934d
Phase 1 - Internal reference cleanup:
- Delete dev_docs/meetings/ (internal meeting notes/transcripts)
- Replace hardcoded usernames (padak/matejkys/dasa) with deploy/generic
- Replace "Internal AI Data Analyst" with "AI Data Analyst"
- Replace keboola/internal_ai_data_analyst URLs with your-org/ai-data-analyst
- Replace /tmp/keboola_load/ with /tmp/data_analyst_staging/ in dev_docs
Phase 2 - Deployment hardening:
- Tighten sudoers wildcards to explicit paths (visudo, sudoers cp)
- setup.sh creates all groups (data-ops, dataread, data-private) and deploy user
- webapp-setup.sh copies sudoers-webapp from repo instead of inline definition
- deploy.sh conditional copy for data_description.md (not in git for OSS)
- deploy.sh ownership changed to deploy:data-ops for /data/{scripts,docs,examples}
Phase 3 - Config and misc:
- Add ${ENV_VAR} interpolation to config/loader.py
- Expand config/instance.yaml.example with all sections (admins, deployment, auth, etc.)
- Create config/.env.template for secret values
- Add MIT LICENSE
- Fix .gitignore: add .venv/, docs/data_description.md
- Fix README.md: CSV status Planned, remove metrics/, update license text
- Translate Czech comments in requirements.txt to English
- Fix test_account_service.py: mock username mapping instead of relying on instance config
All 118 tests pass.
|
||
|---|---|---|
| .. | ||
| activate_venv.sh | ||
| backfill_gap.sh | ||
| collect_session.py | ||
| duckdb_manager.py | ||
| generate_user_sync_configs.py | ||
| init.sh | ||
| jira_backfill.py | ||
| jira_backfill_remote_links.py | ||
| jira_backfill_sla.py | ||
| jira_consistency_check.py | ||
| jira_poll_sla.py | ||
| README.md | ||
| setup_views.sh | ||
| sync_config_template.yaml | ||
| sync_data.sh | ||
| sync_jira.sh | ||
| update.sh | ||
Scripts
Helper scripts for working with AI Data Analyst project.
These scripts are synced from the server into server/scripts/ on the analyst's machine.
Available Scripts
setup_views.sh
Initialize or refresh DuckDB views on Parquet files.
bash server/scripts/setup_views.sh
sync_data.sh
Synchronize data from server, upload user files, and refresh DuckDB.
# Recommended: update scripts first, then sync
rsync -avz data-analyst:server/scripts/ ./server/scripts/ # Linux/macOS
scp -r data-analyst:server/scripts/* ./server/scripts/ # Windows fallback
bash server/scripts/sync_data.sh
# Other options:
bash server/scripts/sync_data.sh --dry-run # Preview what would be synced (no changes)
bash server/scripts/sync_data.sh --push # Only upload user/ to server
What sync does:
- Self-update check - detects if sync_data.sh changed, asks to re-run if so
- Downloads
server/docs/,server/scripts/,server/metadata/from server - Updates
CLAUDE.mdfrom latest template - Downloads
server/parquet/data files (with--deleteto remove old files) - Uploads
user/directory to server (backup, no--delete) - Syncs Python environment to server
- Validates DuckDB - if corrupted, deletes and recreates from parquets
- Reinitializes DuckDB views (
CREATE OR REPLACE VIEWfor all tables)
Self-update mechanism: The script checks its own checksum before and after syncing scripts. If it detects it was updated, it exits with a message asking you to run sync again. This ensures you always run the latest sync logic.
DuckDB corruption recovery: If DuckDB file is corrupted (e.g., interrupted sync), it's automatically detected and recreated. All data is safe in parquet files - DuckDB only contains VIEW definitions.
Typical Workflow
- First time setup: Follow bootstrap.yaml instructions
- Before analysis: Sync latest data
bash server/scripts/sync_data.sh - Analyze: Use DuckDB database at
user/duckdb/analytics.duckdb