agnes-the-ai-analyst/tests
Petr Simecek 1bbbe58ea0
release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43)
* fix(cli): versioned wheel URL in setup instructions; drop broken /cli/agnes.whl alias (#36)

* fix(cli): inline PEP 427 wheel filename in setup instructions

`uv tool install <server>/cli/agnes.whl` fails with

    error: The wheel filename "agnes.whl" is invalid: Must have a version

because uv validates the filename in the URL path *before* fetching — so
the server-side Content-Disposition header (which has the real versioned
filename) is never consulted, and an HTTP redirect does not help either:
uv resolves the filename from the initial URL.

Fix the root cause by inlining the real PEP 427 filename into the setup
snippet the dashboard copies to the clipboard. The wheel filename is
resolved server-side via `_find_wheel()` and substituted into the lines
returned from `setup_instructions.resolve_lines()`, so both the read-only
HTML preview and the JS clipboard renderer get byte-identical output.

Also added `/cli/wheel/{filename}` to serve wheels at their PEP 427 path,
and kept `/cli/agnes.whl` as a 302 redirect for manual/legacy callers —
though that redirect alone is NOT sufficient for `uv tool install` (uv
validates before following redirects) and is there only as defense-in-depth.

Verified locally:
- `uv tool install <server>/cli/wheel/agnes_the_ai_analyst-2.0.0-py3-none-any.whl` succeeds
- `/install` HTML now renders the versioned URL; `/cli/agnes.whl` no longer appears in the rendered snippet

* fix(cli): remove /cli/agnes.whl alias entirely — it only confused users

The bareword alias was never actually usable:

- `uv tool install <server>/cli/agnes.whl` fails at filename validation
  before any HTTP fetch, so neither the Content-Disposition header nor a
  302 redirect rescued it.
- The 302-to-versioned-path fallback left a visibly "working" URL in
  browser / curl -L contexts, which is exactly how the original bug got
  reported in the first place ("the URL loads, why doesn't install work?").

Remove the endpoint and scrub all remaining references. The only CLI wheel
URL is now `/cli/wheel/{filename}` with the real PEP 427 filename, which
the setup-instructions template already generates server-side.

Existing tests that referenced /cli/agnes.whl become negative tests
("must not appear") so we don't regress.

* feat(cli): --version flag; sync --dry-run + progress indicator (#38)

* feat(cli): add --version / -V flag

Prints `da <version>` from package metadata (importlib.metadata). Falls
back to "unknown" when the package is not installed (e.g. running from a
source checkout without `uv pip install -e .`), instead of crashing.

Eager typer callback, so `da --version` exits before subcommand
resolution and does not require any auth/config.

* feat(cli): da sync --dry-run + X/N progress indicator

--dry-run reports what would be downloaded/uploaded without hitting the
API or writing local state. Supports the full flag set (--table, --json,
--upload-only); JSON shape is {"dry_run": true, "would_download": [...],
"summary": {...}}.

Progress bar now shows "[X/N] Downloading <table>..." with a Rich
BarColumn + TaskProgressColumn + TimeElapsedColumn instead of a bare
spinner — makes long syncs visible.

* feat(cli): durable sync + server gzip + auto-update check (#41)

* fix(sync): atomic writes + manifest hash verification + retry on transient errors

Three durability hooks around stream_download and the sync command:

1. Atomic writes. stream_download now streams into `<target>.tmp` and
   calls os.replace() on success, so the real target file never exists
   in a half-written state. On failure the tmp is unlinked — no cleanup
   leftovers, no guard needed at read time.

2. Retry with backoff. Transient errors (ConnectError, ReadError,
   WriteError, RemoteProtocolError, TimeoutException, 5xx) are retried
   up to 3× with 0.3s / 1s / 3s backoff. 4xx (auth, 404) surfaces
   immediately — retrying those is pointless.

3. Manifest-hash verification. After download, sync.py computes MD5 of
   the target (same 8KiB chunking as app/api/sync.py:_file_hash) and
   compares against `server_tables[tid]["hash"]`. Mismatch ⇒ unlink,
   record error, skip state commit. The PAR1 structural check survives
   as a fallback for legacy manifests without a hash.

Also makes _rebuild_duckdb_views tolerant: single broken parquet is
skipped with a stderr warning instead of killing the whole rebuild.

Supersedes #40 — this commit is a strict super-set (hash check + PAR1
fallback + atomic write + retry). #40 can be closed without merging.

* perf(server): enable GZipMiddleware for JSON / HTML responses

GZipMiddleware at minimum_size=1024 shaves bandwidth on manifest-style
JSON endpoints (/api/sync/manifest, /api/version, …) and the /install
HTML preview. Parquet file downloads are already columnar-compressed so
the middleware sees limited benefit there — but it doesn't hurt, httpx
on the client side decompresses transparently.

Placed after session middleware so gzip wraps the session-Set-Cookie
response too, and before CORSMiddleware so compression is applied to
both cross-origin and same-origin responses.

* feat(cli): auto-check for newer CLI version on startup

Server side
- GET /cli/latest returns {version, wheel_filename, download_url_path}
  for whatever wheel is currently in AGNES_CLI_DIST_DIR. Public,
  cacheable, no secrets — consumed by the CLI auto-update probe.

Client side
- New cli/update_check.py: reads /cli/latest with a 3s timeout, caches
  the result in $DA_CONFIG_DIR/update_check.json for 24h. Cache is
  invalidated when the installed version changes (e.g. after a fresh
  `uv tool install`) so stale "you're behind" warnings don't linger.
- Root typer callback fires the probe before subcommand dispatch; any
  failure is swallowed so a bad network never blocks a working command.
- Outdated → one-line stderr warning:
    [update] da 2.0.0 is out of date — latest on this server is 2.1.0.
    Upgrade: uv tool install --force <server>/cli/wheel/<…>.whl
- Disable with DA_NO_UPDATE_CHECK=1.

* fix(pr-review): None-guard the upgrade line + skip gzip on parquet paths

Two follow-ups from Devin review on #41.

1. format_outdated_notice(UpdateInfo(download_url=None)) emitted literal
   "uv tool install --force None" — copy-pasting that fails. Drop the
   upgrade snippet when the URL is absent and keep only the version line.

2. GZipMiddleware compressed everything over 1024 bytes, including the
   parquet FileResponses served by /api/data/{tid}/download,
   /cli/wheel/{name}, and /cli/download. Parquet is already columnar-
   compressed — gzip there is pure CPU + latency with no size win, and
   /api/data bodies can reach hundreds of MB. Wrap GZipMiddleware in a
   small _SelectiveGZipMiddleware that skips those path prefixes and
   delegates the rest to the stock middleware. JSON / HTML endpoints
   (manifest, /install, /api/version, …) still get compressed.

* release: bump to 2.1.0 — unify AGNES_VERSION with pyproject.toml version (#42)

Before: two independent version systems. pyproject.toml carried semver
(2.0.0 → wheel filename → `da --version`) while release.yml injected
CalVer into AGNES_VERSION (e.g. 2026.04.155 → /api/version). Users saw
different strings in the CLI vs. the /install page, and the CLI auto-
update check couldn't tell "new deploy, same package version" apart
from "new package version".

Make pyproject.toml [project].version the single product-version source
of truth. release.yml extracts it and feeds AGNES_VERSION, so every
surface (/api/version, /api/health, /cli/latest, `da --version`) agrees
on one number. The CalVer tag keeps doing what CalVer is for: release
identity on the git tag and Docker image tag (versioned_tag).

Also wires AGNES_TAG through the build: release.yml → Dockerfile ARG →
env, so /api/version.image_tag finally reports the actual image tag
instead of the "unknown" fallback.

Bump to 2.1.0 to reflect the PRs shipped on ps/wheel-name-fix: durable
sync (atomic writes + manifest MD5 + retry), server GZip, CLI auto-
update probe, setup snippet PEP 427 URL.

* fix(pr-review): directional version compare in is_outdated()

UpdateInfo.is_outdated() used `self.latest != self.installed`, which
fires in both directions. If the server is rolled back or the user
connects to an older deployment, the CLI would warn "out of date"
and — worse — the formatted notice would prompt

    uv tool install --force <older-version>.whl

i.e. an unintended downgrade.

Compare with packaging.version.Version (PEP 440 aware, handles pre-
release tags). Fall back to dotted-int tuple compare if packaging is
somehow missing, and return False on unparseable strings — better to
miss an upgrade hint than to silently suggest a downgrade.

Adds 4 test cases: installed older (True), installed newer (False),
10.0.0 vs 2.1.0 lexical-compare trap (correct), unparseable strings
(False).

Addresses Devin review on #43.

* fix(pr-review): read FastAPI app version from package metadata

app/main.py:80 hardcoded `version="2.0.0"` in the FastAPI constructor.
After #42 bumped pyproject.toml to 2.1.0, /api/version, /cli/latest,
and `da --version` all reported 2.1.0 while /openapi.json and the
/docs UI still advertised 2.0.0.

Read `agnes-the-ai-analyst` version via importlib.metadata (same
pattern cli/main.py:_cli_version already uses), with a `"dev"`
fallback when the package is not installed (source checkout). This
way pyproject.toml stays the single source of truth across every
version surface — /openapi.json now tracks the bump automatically.

Adds a dedicated test file to pin this behavior so a future
regression to a hardcoded literal fails at CI.

Addresses second Devin finding on #43.

* fix(pr-review): _fmt_bytes PiB label + negative cache in update_check

Two more follow-ups from Devin review on #43.

1. _fmt_bytes off-by-unit. The old loop exited at TiB but the fallback
   labelled PiB, so 1 PiB rendered as "1024.0 PiB". Restructure: put
   every unit inside the loop (KiB through EiB) so the division count
   always matches the label. Covers up to 1 ZiB cleanly; anything
   beyond renders as "<big>.0 EiB" rather than crashing.

2. Negative cache for failed /cli/latest probes. On a corporate
   firewall / VPN that silently drops packets, the 3s HTTP timeout
   fired on *every* `da` invocation. Writing a `latest=None` cache
   entry with a 5-minute TTL caps that at one probe per 5min. Successful
   probes still use the 24h TTL. Reading logic branches on whether the
   cached `latest` is None.

Adds TestFmtBytes (2 cases: small/medium sizes and the PiB/EiB fallback
regression), plus two TestSync update-check cases covering negative-
cache reuse and TTL expiry.
2026-04-22 21:18:18 +02:00
..
helpers fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
snapshots feat: multi-instance deployment — all 14 must-have items from spec 2026-04-10 11:57:42 +02:00
__init__.py
conftest.py fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
test_access_control.py refactor: remove legacy webapp + add missing tests + housekeeping 2026-03-31 13:44:06 +02:00
test_access_requests_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_admin_configure_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_admin_tokens_ui.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_analyst_bootstrap.py feat: add da analyst setup command with bootstrap flow 2026-04-10 19:43:36 +02:00
test_api.py feat: add POST /api/query/hybrid endpoint for two-phase BQ+DuckDB queries 2026-04-11 11:09:42 +02:00
test_api_complete.py fix: return filename instead of absolute path in upload responses 2026-04-12 14:23:51 +02:00
test_api_scripts.py fix: restrict script execution endpoints to analyst/admin roles 2026-04-09 16:31:42 +02:00
test_app_version.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_auth_providers.py feat(auth): password reset & invite flows for web + admin (#34) (#37) 2026-04-22 17:43:57 +02:00
test_auto_profiling.py
test_bigquery_extractor.py feat: add graceful shutdown handler 2026-04-09 07:03:45 +02:00
test_bigquery_extractor_full.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_bootstrap.py fix(auth): /auth/bootstrap activates seed users, disabled only by real password 2026-04-21 20:01:20 +02:00
test_catalog_export.py refactor: delete old sync pipeline — 9,500 lines removed 2026-03-31 07:50:37 +02:00
test_cli.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_cli_admin.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_cli_analyst.py test: add CLI gap tests for all 9 command groups 2026-04-12 11:13:15 +02:00
test_cli_artifacts.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_cli_auth.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_cli_diagnose.py test: add CLI gap tests for all 9 command groups 2026-04-12 11:13:15 +02:00
test_cli_explore.py test: add CLI gap tests for all 9 command groups 2026-04-12 11:13:15 +02:00
test_cli_metrics.py test: add CLI gap tests for all 9 command groups 2026-04-12 11:13:15 +02:00
test_cli_query.py test: add CLI gap tests for all 9 command groups 2026-04-12 11:13:15 +02:00
test_cli_server.py test: add CLI gap tests for all 9 command groups 2026-04-12 11:13:15 +02:00
test_cli_sync.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_cli_update_check.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_column_metadata.py feat: add ColumnMetadataRepository with CRUD and proposal import 2026-04-10 19:41:53 +02:00
test_connector_kit_poc.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_corporate_memory_collector.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00
test_db.py test: add correctness test for _reattach_remote_extensions 2026-04-12 08:40:12 +02:00
test_docker_full.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_duckdb_manager.py Add per-partition streaming sync and hybrid query architecture 2026-03-12 13:20:41 +01:00
test_e2e_api.py refactor: remove legacy webapp + add missing tests + housekeeping 2026-03-31 13:44:06 +02:00
test_e2e_docker.py feat: add E2E test suite — API, extractor, Docker 2026-03-31 08:18:54 +02:00
test_e2e_extract.py fix: use SCHEMA_VERSION constant in e2e migration test 2026-04-10 19:39:19 +02:00
test_generate_sample_data.py Add --format parquet using project's ParquetManager 2026-03-10 21:46:20 +01:00
test_instance_config.py fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
test_jira_incremental.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_jira_service.py test: add missing coverage for web UI, Jira extract, instance config, and concurrent rebuild 2026-04-09 07:15:14 +02:00
test_jira_service_full.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_jira_webhooks.py fix: address Devin review — docker-e2e .env, jira webhook test isolation 2026-04-13 14:36:31 +02:00
test_journey_analyst.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_bootstrap_auth.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_hybrid.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_jira.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_memory.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_multisource.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_rbac.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_journey_sync_query.py test: add E2E journey tests (J1-J8) covering full user flows 2026-04-12 11:13:51 +02:00
test_keboola_extractor.py feat: add graceful shutdown handler 2026-04-09 07:03:45 +02:00
test_keboola_extractor_full.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_live_bigquery.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_live_jira.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_live_keboola.py test: add Docker E2E and live connector test files 2026-04-12 11:10:06 +02:00
test_llm_connector.py Add modular LLM connector for Corporate Memory 2026-03-23 12:08:33 +01:00
test_llm_providers_full.py test: add connector test suite (Block D) — 5 files, 58 tests 2026-04-12 11:12:50 +02:00
test_memory_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_metadata_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_metrics.py fix: address code review — path injection, multi-table search, metrics import API, error handling 2026-04-10 19:56:00 +02:00
test_migration.py fix: replace os.environ direct assignment with monkeypatch.setenv in test fixtures 2026-04-09 07:11:36 +02:00
test_openapi_snapshot.py feat: multi-instance deployment — all 14 must-have items from spec 2026-04-10 11:57:42 +02:00
test_openmetadata_client.py Implement OpenMetadata catalog integration (Phase 1) 2026-03-12 14:07:13 +01:00
test_openmetadata_enricher.py refactor: delete old sync pipeline — 9,500 lines removed 2026-03-31 07:50:37 +02:00
test_openmetadata_transformer.py docs,tests: anonymize customer references 2026-04-21 11:56:19 +02:00
test_orchestrator.py test: add missing coverage for web UI, Jira extract, instance config, and concurrent rebuild 2026-04-09 07:15:14 +02:00
test_password_flows.py feat(auth): password reset & invite flows for web + admin (#34) (#37) 2026-04-22 17:43:57 +02:00
test_pat.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_permissions.py fix: replace os.environ direct assignment with monkeypatch.setenv in test fixtures 2026-04-09 07:11:36 +02:00
test_permissions_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_profiler.py
test_rbac.py fix: replace os.environ direct assignment with monkeypatch.setenv in test fixtures 2026-04-09 07:11:36 +02:00
test_remote_query.py fix: BQ COUNT subquery alias, wrap ImportError in RemoteQueryError 2026-04-11 20:29:03 +02:00
test_repositories.py fix: replace os.environ direct assignment with monkeypatch.setenv in test fixtures 2026-04-09 07:11:36 +02:00
test_scheduler.py Support multiple daily sync times (e.g., "daily 07:00,13:00,18:00") 2026-03-16 23:09:48 +01:00
test_scheduler_full.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00
test_scripts_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_security.py fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
test_selective_gzip.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_session_collector.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00
test_settings_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_setup_instructions.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_telegram_api.py test: add telegram API endpoint tests (verify, unlink, status) 2026-04-12 14:12:28 +02:00
test_telegram_bot.py fix: address code review findings — duplicate fixture, JWT key length, async deprecation 2026-04-13 13:47:51 +02:00
test_telegram_storage.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00
test_upload_api.py test: add 132 API gap tests across 8 endpoint modules 2026-04-12 11:13:24 +02:00
test_user_management.py User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
test_web_ui.py release(2.1.0): durable sync, CLI auto-update, versioned wheel URL, version unification (#43) 2026-04-22 21:18:18 +02:00
test_ws_gateway.py test: add Block C services tests (68 tests across 6 files) 2026-04-12 11:11:48 +02:00