Petr 38b86127ed Branding cleanup: remove Keboola-specific references from docs and config

- server/deploy.sh: KEBOOLA_ENV_FILE -> SYNC_ENV_FILE
- server/ws-gateway.service, notify-bot.service: remove Keboola from descriptions
- .gitignore: generic comment for data directory
- CLAUDE.md, README.md, ARCHITECTURE.md: update paths from src/adapters to connectors/
- docs/DATA_SOURCES.md: update custom connector guide to connectors/ pattern
- connectors/jira/README.md: keboola-analyst -> data-analyst in config paths
- dev_docs/desktop-app.md: KeboolaAnalyst -> DataAnalyst branding

2026-03-09 12:22:27 +01:00

1.8 KiB

Raw Blame History

Data Sources

Overview

AI Data Analyst uses a pluggable adapter system for data sources. Configure the adapter type in config/instance.yaml:

data_source:
  type: "keboola"  # Options: keboola, csv, bigquery (future)

Keboola Adapter

Syncs tables from Keboola Storage API.

Requirements

kbcstorage Python package (included in requirements.txt)
Keboola Storage API token with read access

Configuration

In .env:

KEBOOLA_STORAGE_TOKEN=your-token-here
KEBOOLA_STACK_URL=https://connection.your-region.keboola.com
KEBOOLA_PROJECT_ID=12345
DATA_SOURCE=keboola

Sync Strategies

Define in docs/data_description.md:

full_refresh: Downloads entire table each sync
incremental: Downloads only changed rows (using changedSince)
partitioned: Splits data into time-based partitions (month/day/year)

Data Description Format

folder_mapping:
  "in.c-crm": "sales"
  "in.c-hr": "hr"

tables:
  - id: "in.c-crm.company"
    name: "company"
    description: "Company master data from CRM"
    primary_key: "id"
    sync_strategy: "full_refresh"

Writing a Custom Connector

Create a new connector module in connectors/<name>/adapter.py:

from src.data_sync import DataSource

class MyDataSource(DataSource):
    def sync_table(self, table_config, sync_state):
        # Download data, convert to Parquet
        # Return {"success": True, "rows": N, "strategy": "..."}
        pass

The create_data_source() function in src/data_sync.py auto-discovers connectors from the connectors/ directory. Set data_source.type in config/instance.yaml to match the connector directory name (e.g., keboola for connectors/keboola/).

See connectors/keboola/ for a complete reference implementation.

1.8 KiB Raw Blame History