AI-Cognitive-Leap/agnes-the-ai-analyst
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
agnes-the-ai-analyst/tests
History
ZdenekSrotyr 506a378c3a
release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 
## Summary

Brings the Keboola connector to feature parity with the legacy internal data-analyst's per-table sync strategies. Closes the four documented gaps from the spec branch (`zs/keboola-connector-specs`):

- **Typed parquet** in the legacy SDK extraction path — column types from Keboola Storage metadata (provider cascade `user > ai-metadata-enrichment > keboola.snowflake-transformation`) survive the CSV → parquet roundtrip; invalid date strings (`'0000-00-00'`) and invalid numeric strings (`'Non-Manager'`) become NULL while keeping the column's typed schema. Pre-fix everything was VARCHAR.
- **Incremental sync** via Storage API `changedSince` — opt-in per table; pulls only delta rows, merges into the existing parquet by `primary_key` (drop_duplicates with keep='last'). Cuts daily extraction from O(full table) to O(delta).
- **Partitioned sync** — flat per-partition layout `data/<table>/<key>.parquet` (e.g. `2026_05.parquet`), per-affected-partition merge for daily updates, chunked initial load with 1-day overlap and 2-empty-chunk stop heuristic.
- **`where_filters`** — server-side row filter with date placeholders (`{{today}}`, `{{last_3_months}}`, `{{start_of_3_months_ago}}`, etc.) resolved at sync time. Force the SDK path; reject `incremental + where_filters` combination at API layer (changedSince already filters temporally).

## Architecture

- **Schema migration v25 → v26**: 7 new columns on `table_registry`. Existing `sync_strategy` column reused (pre-v26 it was inert catalog metadata; post-v26 the extractor dispatches off it).
- **Per-table dispatcher** in `extractor.run()` routes to one of `_extract_via_extension` (full_refresh + extension), `_extract_via_legacy` (full_refresh + filters or extension fallback), `extract_incremental`, or `extract_partitioned`.
- **API conflict policy**: `incremental + where_filters` → 422; `partitioned + query_mode='remote'` → 422; `partitioned ⇒ partition_by required`.
- **Admin UI**: third "Direct extract (Storage API)" radio in the Keboola Register / Edit modals, alongside existing "Whole table (extension)" and "Custom SQL". When selected, exposes a v26 sync-strategy panel with conditional fields per strategy.

## Test plan

- [x] **Unit + module** — 134 v26 tests covering migration, repo, parquet_io, where_filters, incremental (compute_changed_since + merge_parquet + extract_incremental E2E), partitioned (key derivation + merge_partition + chunked windows + extract_partitioned E2E), extractor dispatcher, admin API validators, PUT field clearing, registry-shape → dispatcher bridge
- [x] **HTML form structure** — all v26 inputs + visibility classes + JS payload fields verified in rendered template
- [x] **Real Keboola roundtrip** — registered a small test table as `sync_strategy='incremental'` against a test Storage project, triggered two syncs:
  - Sync 1: `changedSince=None` → full pull → 9 rows typed parquet
  - Sync 2: `changedSince=last_sync - 1d window` → 9 delta rows merged with 9 existing → 9 after dedup on primary_key (PK merge confirmed)
- [x] **Browser UX** — agent-browser session against a local uvicorn: login → admin/tables → register modal → switch radios → verify field visibility per strategy → submit → edit existing row → switch to Direct/Incremental → save → confirm DB persistence
- [x] **Regression** — no regressions in the broader 3252-test suite (3 pre-v26 tests updated for the deprecation-marker removal + schema-version bump; 2 pre-existing environment-sensitive test failures unrelated to this change)

## Bugs caught + fixed during E2E

The browser + real-Keboola roundtrip exposed four bugs the unit tests missed:

1. **JS visibility race** — two competing `forEach` loops set `display=''` then `display='none'` on form elements sharing `kb-strategy-incremental kb-strategy-partitioned` classes (window_days + max_history_days are reused across strategies). Fix: single-pass selector with class-based visibility resolver.
2. **PUT cannot clear field** — pre-v26 `updates = {k: v ... if v is not None}` collapsed "omitted from body" and "sent as null" into the same case, so admin couldn't switch a partitioned row back to full_refresh and have stale `partition_by` clear. Fix: `model_dump(exclude_unset=True)`.
3. **Subprocess DB lock conflict** — `_read_last_sync` reopened `system.duckdb` while the parent server held the write lock (subprocess contract at `app/api/sync.py:_run_sync` line 260). Fix: parent injects `__last_sync__` into table_config before subprocess spawn.
4. **Wrong KBC table_id** — `extract_incremental` / `extract_partitioned` built the Storage API table_id from the registry row's slugified `id` (`circle_inc`) instead of `bucket.source_table` (`in.c-finance.circle`), producing 404s. Fix: prefer `bucket+source_table`; fall back to `id` only when bucket empty.

## Operator notes

- Existing tables stay on `full_refresh` after migration; admins opt individual tables in via `agnes admin register-table --sync-strategy ...`, the Keboola Edit modal, or `POST/PUT /api/admin/registry`.
- `merge_parquet` and `merge_partition` use `pd.concat + drop_duplicates`, loading both existing and delta into pandas RAM. For tables in the multi-million-row range this may OOM — switch to `partitioned` strategy for those (per-partition merge keeps memory bounded). Documented in `### Internal` of the changelog entry.
- Date placeholders are resolved at **sync time**, not register time — a typo'd `{{lasst_week}}` is accepted at register and surfaces only when the next sync runs. By design (rolling windows need late-binding).

## Spec source

The four corresponding plans on the `zs/keboola-connector-specs` branch under `docs/superpowers/plans/2026-05-07-0[1-4]-*.md` capture the design rationale and link back to internal repo references for each subsystem.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/keboola/agnes-the-ai-analyst/pull/217" target="_blank">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-open-in-devin-review-dark.svg?v=1">
    <img src="https://static.devin.ai/assets/gh-open-in-devin-review-light.svg?v=1" alt="Open in Devin Review">
  </picture>
</a>
<!-- devin-review-badge-end -->
2026-05-07 19:01:27 +02:00
..
fixtures chore(lint): drop unused imports from new test files (ruff F401) 2026-05-04 19:32:31 +02:00
helpers feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
snapshots feat(api,web,cli): /admin/workspace-prompt + /api/welcome restored + da analyst writes CLAUDE.md 2026-05-03 22:44:14 +02:00
__init__.py
conftest.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_access_control.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_admin_bigquery_test_connection.py test(admin): #160 RED tests for BQ test-connection + server-config placeholder 2026-05-04 10:31:35 +02:00
test_admin_bq_register.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_admin_configure_api.py feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
test_admin_discover_bigquery.py feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
test_admin_discover_keboola_plan.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_admin_keboola_materialized.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_admin_phase_c_deprecation.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_admin_put_preservation.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_admin_register_materialized_server_generated_sql.py feat(admin): server-generate materialized source_query, allow BQ backticks 2026-05-04 18:37:27 +02:00
test_admin_register_source_type_validation.py feat(admin-api): reject register-table for source_type not configured on instance 2026-05-01 23:04:51 +02:00
test_admin_register_v26_validation.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_admin_run_endpoints.py fix(session-collector-api): mirror sibling endpoints' audit-on-exception (Devin Review on #179) 2026-05-05 09:31:33 +02:00
test_admin_server_config.py fix(query): #168 review iter 3 — RBAC name-vs-id, placeholder dead code 2026-05-04 13:51:36 +02:00
test_admin_server_config_corp_memory.py feat(admin-ui): /admin/server-config known-fields registry + structured nested editor 2026-05-01 20:27:01 +02:00
test_admin_server_config_known_fields.py refactor(bq): #160 remove legacy_wrap_views config knob (always-wrap) 2026-05-04 10:31:35 +02:00
test_admin_server_config_materialize_section.py feat(config): expose materialize.lock_ttl_seconds in server-config 2026-05-04 18:52:54 +02:00
test_admin_server_config_placeholder.py test(admin): #160 RED tests for BQ test-connection + server-config placeholder 2026-05-04 10:31:35 +02:00
test_admin_server_config_renderer_depth.py feat(admin-ui): /admin/server-config known-fields registry + structured nested editor 2026-05-01 20:27:01 +02:00
test_admin_tables_tab_ui.py feat(admin-ui): /admin/tables per-connector tabs + Keboola materialized parity + form cleanup + Manage access deep link 2026-05-01 20:26:29 +02:00
test_admin_tables_ui_materialized.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_admin_tables_warmup_ui.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_admin_tokens_ui.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_admin_unescape_shell_quoting.py Merge remote-tracking branch 'origin/main' into pr198-review 2026-05-06 11:35:45 +02:00
test_admin_unregister_cleanup.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_admin_user_capabilities_ui.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_admin_validator_backtick_relaxed_for_materialized.py tests: rename 'prj-grp' placeholder to 'my-project' for vendor-agnostic OSS 2026-05-04 20:38:47 +02:00
test_api.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_api_admin_materialized.py merge: pull #174 (BQ materialize view fix + concurrency, 0.33.0) into bootstrap branch 2026-05-04 20:53:00 +02:00
test_api_complete.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_api_query_guardrail.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_api_query_quota.py fix(query): #168 review iter 2 — quota user_id parity + concurrent-slot 429 2026-05-04 13:38:31 +02:00
test_api_query_rbac_bq_path.py sec(query): #160 BQ_PATH catches quoted "bq" catalog token (Phase 3 review) 2026-05-04 10:31:35 +02:00
test_api_scripts.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_app_version.py feat(server): expose APP_VERSION + MIN_COMPAT_CLI_VERSION on /api/* response headers 2026-05-06 23:23:23 +02:00
test_auth_providers.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_auth_rate_limit.py security(auth): per-IP rate limit + last-admin guard (#165) 2026-05-02 21:08:33 +02:00
test_auth_scheduler_token.py fix(scheduler): HTTP marketplaces job + SCHEDULER_API_TOKEN shared secret (#127) 2026-04-29 11:44:00 +02:00
test_auto_profiling.py
test_bigquery_auth.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_bigquery_extractor.py refactor(bq): #160 remove legacy_wrap_views config knob (always-wrap) 2026-05-04 10:31:35 +02:00
test_bigquery_extractor_full.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_bootstrap.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_bq_access.py fix(bq): map 'Response too large' to its own error class instead of generic bad_request 2026-05-06 13:09:31 +02:00
test_bq_cost_guardrail.py fix(bq-materialize): code-review follow-ups for d8a22996 2026-05-04 16:52:18 +02:00
test_bq_init_extract_skips.py feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
test_bq_materialize.py release: 0.40.0 — materialize_query writes _meta + inner view so master views appear 2026-05-06 16:04:58 +02:00
test_bq_materialize_concurrency.py fix(bq-materialize): TTL reclaim was dead code (Devin Review on extractor.py:166) 2026-05-04 22:36:56 +02:00
test_bq_materialize_query_wrapping.py fix(bq-materialize): wrap admin SQL in bigquery_query() so views work 2026-05-04 16:40:40 +02:00
test_bq_query_timeout.py fix(bigquery): apply bq_query_timeout_ms on every BQ-extension attach + surface silent failures 2026-05-06 11:24:14 +04:00
test_cache_warmup.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_catalog_export.py chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
test_check_access_endpoint.py feat(caddy): file_server for parquet downloads — bypass uvicorn 2026-05-05 16:41:33 +02:00
test_claude_md_api.py feat(api,web,cli): /admin/workspace-prompt + /api/welcome restored + da analyst writes CLAUDE.md 2026-05-03 22:44:14 +02:00
test_claude_md_renderer.py fix(claude_md): RBAC-filter tables; align today with now (UTC) 2026-05-04 05:57:22 +02:00
test_claude_md_template_repo.py feat(db,repo,renderer): schema v23 + claude_md_template + ClaudeMd renderer 2026-05-03 22:43:56 +02:00
test_claude_sessions.py fix(push): read sessions from ~/.claude/projects/<encoded-cwd>/ 2026-05-04 20:29:59 +02:00
test_clean_install_integration.py test: clean-install integration suite (minimal/zero grants, force, pre-init) 2026-05-04 19:22:24 +02:00
test_cli.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_cli_admin.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_cli_admin_materialized.py fix(admin): register-table real-world UX gaps for materialized BQ 2026-05-04 23:06:17 +02:00
test_cli_admin_metrics.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_cli_artifacts.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_auth.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_binary_rename.py feat(cli): rename binary from da to agnes (BREAKING) 2026-05-04 16:05:14 +02:00
test_cli_catalog.py release: 0.46.5 — agnes describe -n parses, server sanitizes NaN (#224) 2026-05-07 18:16:21 +02:00
test_cli_catalog_metrics.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_cli_describe.py release: 0.46.5 — agnes describe -n parses, server sanitizes NaN (#224) 2026-05-07 18:16:21 +02:00
test_cli_diagnose.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_diagnose_system.py fix(tests): strip ANSI escapes from --help output before substring asserts 2026-05-04 19:43:47 +02:00
test_cli_disk_info.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_error_render.py fix: update legacy-string assertions in tests + onboarding template 2026-05-04 20:08:07 +02:00
test_cli_explore.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_init.py release: 0.45.0 — easy-wins bundle (#84 #164 #177 #178 #203 #204) 2026-05-07 11:43:16 +02:00
test_cli_pull.py fix(tests): strip ANSI escapes from --help output before substring asserts 2026-05-04 19:43:47 +02:00
test_cli_push.py fix(tests): strip ANSI escapes from --help output before substring asserts 2026-05-04 19:43:47 +02:00
test_cli_query.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_cli_query_render.py fix(post-merge): clean up stale da verbs introduced via #174 merge 2026-05-04 20:57:36 +02:00
test_cli_refresh_marketplace.py fix(refresh-marketplace): align manual-mode hint with hook JSON 2026-05-07 06:59:13 +02:00
test_cli_server.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_snapshot.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_cli_snapshot_create.py fix(post-merge): clean up stale da verbs introduced via #174 merge 2026-05-04 20:57:36 +02:00
test_cli_status.py fix(tests): strip ANSI escapes from --help output before substring asserts 2026-05-04 19:43:47 +02:00
test_cli_store.py refactor(cli-store): pull/info → agnes admin store; add agnes store mine 2026-05-05 13:49:18 +02:00
test_cli_update_check.py feat(cli): add agnes self-upgrade with smoke test + rollback 2026-05-06 23:23:23 +02:00
test_client_version_check.py perf+test(cli): cache User-Agent at module scope; pin local==min boundary 2026-05-06 23:23:23 +02:00
test_column_metadata.py
test_connector_kit_poc.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_connectors_bigquery_metadata.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_connectors_keboola_metadata.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_corporate_memory_collector.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00
test_corporate_memory_page.py fix(ui): admin pending-review banner on /corporate-memory (#176) 2026-05-05 00:01:22 +02:00
test_corporate_memory_relations.py feat(admin): users/groups UI polish + SSO lock + v18 migration (#142) 2026-04-30 15:16:04 +02:00
test_corporate_memory_v1.py feat(memory): corporate memory v1+v1.5 + 0.15.0 (#72) 2026-04-29 07:16:22 +02:00
test_db.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_db_remote_attach_security.py fix(security): #81 Group A — orchestrator attach hardening (squashed) (#95) 2026-04-27 21:34:04 +02:00
test_db_schema_version.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_db_wal_recovery.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_diagnose_billing.py release: 0.45.0 — easy-wins bundle (#84 #164 #177 #178 #203 #204) 2026-05-07 11:43:16 +02:00
test_docker_compose.py fix(compose): drop corporate-memory + session-collector services (#176) 2026-05-04 23:59:44 +02:00
test_docker_full.py fix(tests): align docker-e2e health asserts with current /api/health shape 2026-05-03 11:21:19 +02:00
test_duckdb_manager.py chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
test_duckdb_panel.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
test_e2e_api.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_e2e_corporate_memory.py feat(memory): corporate memory v1+v1.5 + 0.15.0 (#72) 2026-04-29 07:16:22 +02:00
test_e2e_docker.py fix(tests): drop stale 'healthy' from /api/health status assert 2026-05-03 15:40:41 +02:00
test_e2e_extract.py
test_extractor_identifier_validation.py fix(security): #81 Group D — extractor-side identifier validation (squashed) (#97) 2026-04-27 21:46:17 +02:00
test_fixtures_smoke.py test: clean-bootstrap fixtures (fastapi_test_server, test_pat, zero_grants_workspace) 2026-05-04 19:11:54 +02:00
test_generate_sample_data.py
test_google_group_prefix_sync.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_group_sync.py feat(auth): Google Workspace group prefix filter + system mapping (#131) 2026-04-29 14:08:04 +02:00
test_groups_mapped_email.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_health_schema_gate.py release: 0.45.0 — easy-wins bundle (#84 #164 #177 #178 #203 #204) 2026-05-07 11:43:16 +02:00
test_health_session_pipeline.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_instance_config.py fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
test_instance_config_overlay.py fix(config): overlay-aware LLM consumers + env-ref resolution (#179 review) 2026-05-05 05:57:22 +02:00
test_jira_incremental.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_jira_service.py
test_jira_service_full.py fix(security): close Jira webhook fail-open + path traversal (#83) (#93) 2026-04-27 19:53:55 +02:00
test_jira_validation.py fix(security): close Jira webhook fail-open + path traversal (#83) (#93) 2026-04-27 19:53:55 +02:00
test_jira_webhooks.py feat(ci+tests): deploy safety audit — linting, rollback, smoke tests, 50+ new tests (#120) 2026-04-29 09:18:55 +02:00
test_journey_analyst.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_bootstrap_auth.py fix(security+ops) + release(0.12.1): #82 #85 #87 hardening + cut 0.12.1 (#104) 2026-04-28 19:57:30 +02:00
test_journey_hybrid.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_jira.py fix(security): close Jira webhook fail-open + path traversal (#83) (#93) 2026-04-27 19:53:55 +02:00
test_journey_memory.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_multisource.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_rbac.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_journey_sync_query.py fix(admin-api): keep source_type validator permissive when primary is 'local' (bootstrap) 2026-05-01 23:09:15 +02:00
test_keboola_access.py feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
test_keboola_extension_query_passthrough.py feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
test_keboola_extractor.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_keboola_extractor_dispatch.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_extractor_exit_codes.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_keboola_extractor_full.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_keboola_extractor_typed.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_incremental.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_incremental_e2e.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_init_extract_skips.py feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
test_keboola_materialize.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_keboola_materialized_e2e.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_keboola_parquet_io.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_partitioned.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_partitioned_e2e.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_registry_extended.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_storage_api.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_keboola_v27_migration.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_keboola_where_filters.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_legacy_strings_scan.py feat(admin): yellow banner for legacy CLI verbs in workspace-prompt override 2026-05-04 17:46:50 +02:00
test_lib_hooks.py release: 0.46.4 — detach SessionEnd push so it survives claude -p SIGTERM (#222) 2026-05-07 17:59:27 +02:00
test_lib_pull.py fix(pull): re-download parquet when file missing despite matching hash 2026-05-04 21:12:06 +02:00
test_live_bigquery.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_live_jira.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_live_keboola.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_llm_connector.py fix(anthropic): strict json_schema (additionalProperties=false) + add /admin/scheduler-runs UI 2026-05-05 08:00:57 +02:00
test_llm_provider_env_fallback.py fix(setup): seed default ai: block + env-var fallback (#176) 2026-05-04 23:55:19 +02:00
test_llm_providers_full.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_logging_config.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
test_main_startup_warmup.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_marketplace.py feat(rbac+marketplace): RBAC v13 + Claude Code marketplace + #81/#83/#44 hardening 2026-04-28 14:25:04 +02:00
test_marketplace_filter.py fix(marketplace): use plugin.json name in synth marketplace.json (#133) 2026-04-29 19:25:57 +02:00
test_marketplace_filter_store.py feat(store): /store + /my-ai-stack — community marketplace + per-user composition 2026-05-05 02:53:49 +02:00
test_marketplace_server_git.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_marketplace_server_zip.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_materialized_e2e.py fix(bq-materialize): wrap admin SQL in bigquery_query() so views work 2026-05-04 16:40:40 +02:00
test_me_debug.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_memory_api.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_metadata_api.py feat(admin): #108 M1 — BigQuery table registration in UI + CLI (#119) 2026-04-29 13:18:31 +02:00
test_metadata_models.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_metrics.py
test_migration.py
test_no_override_file.py fix(security+ops) + release(0.12.1): #82 #85 #87 hardening + cut 0.12.1 (#104) 2026-04-28 19:57:30 +02:00
test_openapi_snapshot.py
test_openmetadata_client.py feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
test_openmetadata_enricher.py chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
test_openmetadata_transformer.py chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
test_orchestrator.py fix(0.41.0): orphan parquet skip in filesystem fallback (CI regression) 2026-05-06 17:06:20 +02:00
test_orchestrator_remote_attach_security.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_orchestrator_sync_state_hash.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_packaging.py fix(deps): promote anthropic + openai to core dependencies (#176) 2026-05-04 23:52:30 +02:00
test_password_flows.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_pat.py refactor(cli): hard-cutover env vars + config dir to AGNES_* 2026-05-04 16:35:44 +02:00
test_profiler.py
test_pull_chunked.py fix: devil's advocate R3 — reap PID-suffixed leftovers from dead processes 2026-05-06 14:04:47 +02:00
test_pull_progress.py release: 0.45.0 — easy-wins bundle (#84 #164 #177 #178 #203 #204) 2026-05-07 11:43:16 +02:00
test_pull_shared_client.py perf(cli): persistent HTTP/2 client across pull invocation 2026-05-06 13:06:36 +02:00
test_query_bigquery_query_blocked.py test(query): #160 RED tests for guardrail+quota+RBAC+blocklist 2026-05-04 10:31:35 +02:00
test_query_bq_regex.py sec(query): #160 BQ_PATH catches quoted "bq" catalog token (Phase 3 review) 2026-05-04 10:31:35 +02:00
test_query_materialized_error_message.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_query_remote_rewrite.py fix: Devin Review #1 — apply backtick mask to wrapping rewriter 2026-05-06 21:06:21 +02:00
test_rbac.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_reader_smoke_matrix.py fix(tests): smoke matrix asserts no-traceback only (per-command rc varies) 2026-05-04 19:47:18 +02:00
test_remote_query.py fix(v2): #134 BigQuery cross-project errors return structured 502/400 + BqAccess facade (#138) 2026-04-30 10:11:20 +02:00
test_remote_query_error_details.py feat(cli): #160 shared structured error renderer for BQ-typed responses 2026-05-04 10:31:35 +02:00
test_repositories.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_request_id_middleware.py feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
test_resource_types.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_router_ca_pem.py feat(setup): cross-platform TLS bootstrap + marketplace plugin install (#137) 2026-04-30 08:56:45 +02:00
test_run_due_scripts.py feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
test_run_materialized_pass_in_flight_skip.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_scheduler.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_scheduler_full.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00
test_scheduler_sidecar.py feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
test_schema_v24_source_query_rewrite.py fix(schema-v24): raise on deferred migration so retry path actually runs (Devin Review on db.py:1757) 2026-05-04 23:11:34 +02:00
test_scripts_api.py feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
test_security.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_selective_gzip.py feat(web): rename /install → /setup; nav label 'Setup local agent' 2026-05-03 16:12:13 +02:00
test_self_upgrade.py feat(cli): add agnes self-upgrade with smoke test + rollback 2026-05-06 23:23:23 +02:00
test_session_collector.py fix(session-collector): argv-free run() helper, drop SystemExit footgun (Devin Review on #179) 2026-05-05 06:31:55 +02:00
test_settings_api.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_setup_ai_block.py fix(setup): seed default ai: block + env-var fallback (#176) 2026-05-04 23:55:19 +02:00
test_setup_hooks_template.py chore(docs): replace stale da verbs and vendor-specific install paths 2026-05-04 21:22:19 +02:00
test_setup_instructions.py feat(marketplace): clone-based plugin setup + auto-refresh SessionStart hook 2026-05-07 06:59:13 +02:00
test_setup_page_unified.py feat(marketplace): clone-based plugin setup + auto-refresh SessionStart hook 2026-05-07 06:59:13 +02:00
test_snapshot_meta.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_sql_safe.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_store_api.py refactor(cli-store): pull/info → agnes admin store; add agnes store mine 2026-05-05 13:49:18 +02:00
test_store_naming.py chore(store): genericize email examples in docstring + test 2026-05-05 05:48:32 +02:00
test_store_repositories.py feat(store): /store + /my-ai-stack — community marketplace + per-user composition 2026-05-05 02:53:49 +02:00
test_sync_filter.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_sync_manifest.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_sync_trigger_keboola_materialized.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_sync_trigger_materialized.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_sync_trigger_singleton.py Keboola cutover: native parquet path + sync correctness + auto-discover protection (#190) 2026-05-07 12:12:14 +02:00
test_table_registry_find_by_bq_path.py feat(repo): #160 add find_by_bq_path lookup for direct bq.* RBAC enforcement 2026-05-04 10:31:35 +02:00
test_table_registry_source_query.py feat(materialized): query_mode='materialized' for BigQuery + Keboola — admin SELECT → parquet → analyst 2026-05-01 20:25:56 +02:00
test_tagger.py feat(memory): corporate memory v1+v1.5 + 0.15.0 (#72) 2026-04-29 07:16:22 +02:00
test_telegram_api.py test: add telegram API endpoint tests (verify, unlink, status) 2026-04-12 14:12:28 +02:00
test_telegram_bot.py fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
test_telegram_bot_runner.py release: 0.45.0 — easy-wins bundle (#84 #164 #177 #178 #203 #204) 2026-05-07 11:43:16 +02:00
test_telegram_storage.py release: 0.45.0 — easy-wins bundle (#84 #164 #177 #178 #203 #204) 2026-05-07 11:43:16 +02:00
test_tokens_bootstrap_scope.py test(tokens): tighten scope-default + add precedence + audit + reserved-key tests 2026-05-04 17:07:02 +02:00
test_toolbar_integration.py feat(web): rename /install → /setup; nav label 'Setup local agent' 2026-05-03 16:12:13 +02:00
test_upload_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_user_management.py security(auth): per-IP rate limit + last-admin guard (#165) 2026-05-02 21:08:33 +02:00
test_users_sso_flag.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_v2_arrow.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_v2_cache.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_v2_catalog.py feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
test_v2_catalog_dispatcher.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_v2_catalog_invalidation.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_v2_catalog_remote_metadata.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_v2_client.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_v2_quota.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_v2_sample.py release: 0.46.5 — agnes describe -n parses, server sanitizes NaN (#224) 2026-05-07 18:16:21 +02:00
test_v2_scan.py perf: Tier 1 event-loop unblocking — async def → def on BQ-bound handlers 2026-05-05 17:44:08 +02:00
test_v2_scan_estimate.py perf: Tier 1 event-loop unblocking — async def → def on BQ-bound handlers 2026-05-05 17:44:08 +02:00
test_v2_schema.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_v2_schema_columns_consolidation.py release: 0.47.0 — source-agnostic catalog metadata + cache discipline (#223) 2026-05-07 18:33:55 +02:00
test_v26_keboola_e2e.py release: 0.47.1 — Keboola connector v27 (incremental, partitioned, where_filters, typed parquet) (#217) 2026-05-07 19:01:27 +02:00
test_version_headers_middleware.py feat(server): expose APP_VERSION + MIN_COMPAT_CLI_VERSION on /api/* response headers 2026-05-06 23:23:23 +02:00
test_view_collision_detection.py fix: #81 Group C — view-name collision detection (schema v10, squashed) (#100) 2026-04-27 22:09:49 +02:00
test_web_ui.py fix(profile-sessions): tolerate stat() failures on individual jsonl (Devin Review on #179) 2026-05-05 09:53:06 +02:00
test_welcome_template_api.py refactor(welcome-template): drop role param; resolve plugins per-user unconditionally 2026-05-04 22:13:46 +02:00
test_welcome_template_migration.py feat(db): schema v15 — welcome_template singleton table 2026-05-03 16:10:48 +02:00
test_welcome_template_renderer.py refactor(welcome-template): drop role param; resolve plugins per-user unconditionally 2026-05-04 22:13:46 +02:00
test_welcome_template_repo.py feat(repo): WelcomeTemplateRepository singleton CRUD 2026-05-03 16:10:48 +02:00
test_where_validator.py feat(v2): claude-driven fetch primitives + 0.14.0 (#102) 2026-04-29 01:07:19 +02:00
test_ws_gateway.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00