266e8573d3
Move all Keboola-specific code out of src/ into connectors/keboola/: - git mv src/keboola_client.py -> connectors/keboola/client.py - Extract LocalKeboolaSource (855 lines) from data_sync.py -> connectors/keboola/adapter.py - Rename to KeboolaDataSource with full env var validation - Extend DataSource ABC with get_column_metadata() and get_source_name() - Add dynamic connector registry via importlib in create_data_source() - Refactor _generate_schema_yaml to use ABC methods (source_type, _schema_version: 2) - Remove src/adapters/ (redundant facade layer) - Remove Keboola validation from src/config.py (connector validates itself) - Add 14 tests for factory, ABC defaults, env validation, dynamic lookup
17 lines
533 B
Python
17 lines
533 B
Python
"""
|
|
AI Data Analyst - Data Sync Engine
|
|
|
|
Downloads data from configured sources, converts to Parquet files,
|
|
and syncs to analysts via rsync.
|
|
|
|
Main modules:
|
|
- config: Configuration management and data_description.md parsing
|
|
- data_sync: Data synchronization orchestration and DataSource ABC
|
|
- parquet_manager: CSV -> Parquet conversion and file management
|
|
|
|
Data source connectors live in connectors/ (e.g. connectors/keboola/).
|
|
|
|
Note: DuckDB management is handled by scripts/duckdb_manager.py (analyst-side)
|
|
"""
|
|
|
|
__version__ = "0.1.0"
|