* fix(cli): versioned wheel URL in setup instructions; drop broken /cli/agnes.whl alias (#36) * fix(cli): inline PEP 427 wheel filename in setup instructions `uv tool install <server>/cli/agnes.whl` fails with error: The wheel filename "agnes.whl" is invalid: Must have a version because uv validates the filename in the URL path *before* fetching — so the server-side Content-Disposition header (which has the real versioned filename) is never consulted, and an HTTP redirect does not help either: uv resolves the filename from the initial URL. Fix the root cause by inlining the real PEP 427 filename into the setup snippet the dashboard copies to the clipboard. The wheel filename is resolved server-side via `_find_wheel()` and substituted into the lines returned from `setup_instructions.resolve_lines()`, so both the read-only HTML preview and the JS clipboard renderer get byte-identical output. Also added `/cli/wheel/{filename}` to serve wheels at their PEP 427 path, and kept `/cli/agnes.whl` as a 302 redirect for manual/legacy callers — though that redirect alone is NOT sufficient for `uv tool install` (uv validates before following redirects) and is there only as defense-in-depth. Verified locally: - `uv tool install <server>/cli/wheel/agnes_the_ai_analyst-2.0.0-py3-none-any.whl` succeeds - `/install` HTML now renders the versioned URL; `/cli/agnes.whl` no longer appears in the rendered snippet * fix(cli): remove /cli/agnes.whl alias entirely — it only confused users The bareword alias was never actually usable: - `uv tool install <server>/cli/agnes.whl` fails at filename validation before any HTTP fetch, so neither the Content-Disposition header nor a 302 redirect rescued it. - The 302-to-versioned-path fallback left a visibly "working" URL in browser / curl -L contexts, which is exactly how the original bug got reported in the first place ("the URL loads, why doesn't install work?"). Remove the endpoint and scrub all remaining references. The only CLI wheel URL is now `/cli/wheel/{filename}` with the real PEP 427 filename, which the setup-instructions template already generates server-side. Existing tests that referenced /cli/agnes.whl become negative tests ("must not appear") so we don't regress. * feat(cli): --version flag; sync --dry-run + progress indicator (#38) * feat(cli): add --version / -V flag Prints `da <version>` from package metadata (importlib.metadata). Falls back to "unknown" when the package is not installed (e.g. running from a source checkout without `uv pip install -e .`), instead of crashing. Eager typer callback, so `da --version` exits before subcommand resolution and does not require any auth/config. * feat(cli): da sync --dry-run + X/N progress indicator --dry-run reports what would be downloaded/uploaded without hitting the API or writing local state. Supports the full flag set (--table, --json, --upload-only); JSON shape is {"dry_run": true, "would_download": [...], "summary": {...}}. Progress bar now shows "[X/N] Downloading <table>..." with a Rich BarColumn + TaskProgressColumn + TimeElapsedColumn instead of a bare spinner — makes long syncs visible. * feat(cli): durable sync + server gzip + auto-update check (#41) * fix(sync): atomic writes + manifest hash verification + retry on transient errors Three durability hooks around stream_download and the sync command: 1. Atomic writes. stream_download now streams into `<target>.tmp` and calls os.replace() on success, so the real target file never exists in a half-written state. On failure the tmp is unlinked — no cleanup leftovers, no guard needed at read time. 2. Retry with backoff. Transient errors (ConnectError, ReadError, WriteError, RemoteProtocolError, TimeoutException, 5xx) are retried up to 3× with 0.3s / 1s / 3s backoff. 4xx (auth, 404) surfaces immediately — retrying those is pointless. 3. Manifest-hash verification. After download, sync.py computes MD5 of the target (same 8KiB chunking as app/api/sync.py:_file_hash) and compares against `server_tables[tid]["hash"]`. Mismatch ⇒ unlink, record error, skip state commit. The PAR1 structural check survives as a fallback for legacy manifests without a hash. Also makes _rebuild_duckdb_views tolerant: single broken parquet is skipped with a stderr warning instead of killing the whole rebuild. Supersedes #40 — this commit is a strict super-set (hash check + PAR1 fallback + atomic write + retry). #40 can be closed without merging. * perf(server): enable GZipMiddleware for JSON / HTML responses GZipMiddleware at minimum_size=1024 shaves bandwidth on manifest-style JSON endpoints (/api/sync/manifest, /api/version, …) and the /install HTML preview. Parquet file downloads are already columnar-compressed so the middleware sees limited benefit there — but it doesn't hurt, httpx on the client side decompresses transparently. Placed after session middleware so gzip wraps the session-Set-Cookie response too, and before CORSMiddleware so compression is applied to both cross-origin and same-origin responses. * feat(cli): auto-check for newer CLI version on startup Server side - GET /cli/latest returns {version, wheel_filename, download_url_path} for whatever wheel is currently in AGNES_CLI_DIST_DIR. Public, cacheable, no secrets — consumed by the CLI auto-update probe. Client side - New cli/update_check.py: reads /cli/latest with a 3s timeout, caches the result in $DA_CONFIG_DIR/update_check.json for 24h. Cache is invalidated when the installed version changes (e.g. after a fresh `uv tool install`) so stale "you're behind" warnings don't linger. - Root typer callback fires the probe before subcommand dispatch; any failure is swallowed so a bad network never blocks a working command. - Outdated → one-line stderr warning: [update] da 2.0.0 is out of date — latest on this server is 2.1.0. Upgrade: uv tool install --force <server>/cli/wheel/<…>.whl - Disable with DA_NO_UPDATE_CHECK=1. * fix(pr-review): None-guard the upgrade line + skip gzip on parquet paths Two follow-ups from Devin review on #41. 1. format_outdated_notice(UpdateInfo(download_url=None)) emitted literal "uv tool install --force None" — copy-pasting that fails. Drop the upgrade snippet when the URL is absent and keep only the version line. 2. GZipMiddleware compressed everything over 1024 bytes, including the parquet FileResponses served by /api/data/{tid}/download, /cli/wheel/{name}, and /cli/download. Parquet is already columnar- compressed — gzip there is pure CPU + latency with no size win, and /api/data bodies can reach hundreds of MB. Wrap GZipMiddleware in a small _SelectiveGZipMiddleware that skips those path prefixes and delegates the rest to the stock middleware. JSON / HTML endpoints (manifest, /install, /api/version, …) still get compressed. * release: bump to 2.1.0 — unify AGNES_VERSION with pyproject.toml version (#42) Before: two independent version systems. pyproject.toml carried semver (2.0.0 → wheel filename → `da --version`) while release.yml injected CalVer into AGNES_VERSION (e.g. 2026.04.155 → /api/version). Users saw different strings in the CLI vs. the /install page, and the CLI auto- update check couldn't tell "new deploy, same package version" apart from "new package version". Make pyproject.toml [project].version the single product-version source of truth. release.yml extracts it and feeds AGNES_VERSION, so every surface (/api/version, /api/health, /cli/latest, `da --version`) agrees on one number. The CalVer tag keeps doing what CalVer is for: release identity on the git tag and Docker image tag (versioned_tag). Also wires AGNES_TAG through the build: release.yml → Dockerfile ARG → env, so /api/version.image_tag finally reports the actual image tag instead of the "unknown" fallback. Bump to 2.1.0 to reflect the PRs shipped on ps/wheel-name-fix: durable sync (atomic writes + manifest MD5 + retry), server GZip, CLI auto- update probe, setup snippet PEP 427 URL. * fix(pr-review): directional version compare in is_outdated() UpdateInfo.is_outdated() used `self.latest != self.installed`, which fires in both directions. If the server is rolled back or the user connects to an older deployment, the CLI would warn "out of date" and — worse — the formatted notice would prompt uv tool install --force <older-version>.whl i.e. an unintended downgrade. Compare with packaging.version.Version (PEP 440 aware, handles pre- release tags). Fall back to dotted-int tuple compare if packaging is somehow missing, and return False on unparseable strings — better to miss an upgrade hint than to silently suggest a downgrade. Adds 4 test cases: installed older (True), installed newer (False), 10.0.0 vs 2.1.0 lexical-compare trap (correct), unparseable strings (False). Addresses Devin review on #43. * fix(pr-review): read FastAPI app version from package metadata app/main.py:80 hardcoded `version="2.0.0"` in the FastAPI constructor. After #42 bumped pyproject.toml to 2.1.0, /api/version, /cli/latest, and `da --version` all reported 2.1.0 while /openapi.json and the /docs UI still advertised 2.0.0. Read `agnes-the-ai-analyst` version via importlib.metadata (same pattern cli/main.py:_cli_version already uses), with a `"dev"` fallback when the package is not installed (source checkout). This way pyproject.toml stays the single source of truth across every version surface — /openapi.json now tracks the bump automatically. Adds a dedicated test file to pin this behavior so a future regression to a hardcoded literal fails at CI. Addresses second Devin finding on #43. * fix(pr-review): _fmt_bytes PiB label + negative cache in update_check Two more follow-ups from Devin review on #43. 1. _fmt_bytes off-by-unit. The old loop exited at TiB but the fallback labelled PiB, so 1 PiB rendered as "1024.0 PiB". Restructure: put every unit inside the loop (KiB through EiB) so the division count always matches the label. Covers up to 1 ZiB cleanly; anything beyond renders as "<big>.0 EiB" rather than crashing. 2. Negative cache for failed /cli/latest probes. On a corporate firewall / VPN that silently drops packets, the 3s HTTP timeout fired on *every* `da` invocation. Writing a `latest=None` cache entry with a 5-minute TTL caps that at one probe per 5min. Successful probes still use the 24h TTL. Reading logic branches on whether the cached `latest` is None. Adds TestFmtBytes (2 cases: small/medium sizes and the PiB/EiB fallback regression), plus two TestSync update-check cases covering negative- cache reuse and TTL expiry.
250 lines
9.8 KiB
Python
250 lines
9.8 KiB
Python
"""Tests for the CLI auto-update check (cli/update_check.py)."""
|
|
|
|
import json
|
|
from unittest.mock import patch
|
|
|
|
import pytest
|
|
from typer.testing import CliRunner
|
|
|
|
from cli.main import app
|
|
|
|
runner = CliRunner()
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def tmp_config(tmp_path, monkeypatch):
|
|
monkeypatch.setenv("DA_CONFIG_DIR", str(tmp_path))
|
|
# Point CLI at a fake server so get_server_url() returns something stable.
|
|
monkeypatch.setenv("DA_SERVER", "http://server.test:8000")
|
|
yield tmp_path
|
|
|
|
|
|
def test_check_returns_none_when_disabled(tmp_config):
|
|
import os
|
|
os.environ["DA_NO_UPDATE_CHECK"] = "1"
|
|
try:
|
|
from cli import update_check
|
|
assert update_check.check("http://server.test:8000") is None
|
|
finally:
|
|
del os.environ["DA_NO_UPDATE_CHECK"]
|
|
|
|
|
|
def test_check_returns_none_when_server_url_missing(tmp_config):
|
|
from cli import update_check
|
|
assert update_check.check("") is None
|
|
assert update_check.check(None) is None # type: ignore[arg-type]
|
|
|
|
|
|
def test_check_returns_none_when_installed_version_unknown(tmp_config):
|
|
from cli import update_check
|
|
with patch("cli.update_check._installed_version", return_value="unknown"):
|
|
assert update_check.check("http://server.test:8000") is None
|
|
|
|
|
|
def test_check_fresh_fetch_and_cache_write(tmp_config):
|
|
from cli import update_check
|
|
|
|
payload = {
|
|
"version": "2.1.0",
|
|
"wheel_filename": "agnes_the_ai_analyst-2.1.0-py3-none-any.whl",
|
|
"download_url_path": "/cli/wheel/agnes_the_ai_analyst-2.1.0-py3-none-any.whl",
|
|
}
|
|
with patch("cli.update_check._installed_version", return_value="2.0.0"):
|
|
with patch("cli.update_check._fetch_latest", return_value=payload):
|
|
info = update_check.check("http://server.test:8000")
|
|
|
|
assert info is not None
|
|
assert info.installed == "2.0.0"
|
|
assert info.latest == "2.1.0"
|
|
assert info.download_url == (
|
|
"http://server.test:8000/cli/wheel/agnes_the_ai_analyst-2.1.0-py3-none-any.whl"
|
|
)
|
|
assert info.is_outdated() is True
|
|
|
|
# Cache file was written and re-reading it returns the same latest.
|
|
cache = json.loads((tmp_config / "update_check.json").read_text())
|
|
assert cache["installed"] == "2.0.0"
|
|
assert cache["latest"] == "2.1.0"
|
|
|
|
|
|
def test_check_uses_cache_within_ttl(tmp_config):
|
|
"""Cached entry within 24h skips the network fetch."""
|
|
from cli import update_check
|
|
|
|
# Seed a fresh cache entry.
|
|
(tmp_config / "update_check.json").write_text(json.dumps({
|
|
"installed": "2.0.0",
|
|
"server_url": "http://server.test:8000",
|
|
"latest": "2.0.5",
|
|
"download_url": "http://server.test:8000/cli/wheel/agnes_the_ai_analyst-2.0.5-py3-none-any.whl",
|
|
"checked_at": __import__("time").time(), # now
|
|
}))
|
|
|
|
with patch("cli.update_check._installed_version", return_value="2.0.0"):
|
|
with patch("cli.update_check._fetch_latest") as mock_fetch:
|
|
info = update_check.check("http://server.test:8000")
|
|
|
|
assert mock_fetch.call_count == 0 # cache hit
|
|
assert info.latest == "2.0.5"
|
|
assert info.is_outdated() is True
|
|
|
|
|
|
def test_check_invalidates_cache_when_installed_version_changed(tmp_config):
|
|
"""User ran a fresh install after the cache was written — re-probe."""
|
|
from cli import update_check
|
|
|
|
# Seed cache claiming the installed version was 1.9.0.
|
|
(tmp_config / "update_check.json").write_text(json.dumps({
|
|
"installed": "1.9.0",
|
|
"server_url": "http://server.test:8000",
|
|
"latest": "2.0.0",
|
|
"download_url": "http://server.test:8000/cli/wheel/x.whl",
|
|
"checked_at": __import__("time").time(),
|
|
}))
|
|
|
|
payload = {"version": "2.1.0", "download_url_path": "/cli/wheel/y.whl"}
|
|
with patch("cli.update_check._installed_version", return_value="2.0.0"):
|
|
with patch("cli.update_check._fetch_latest", return_value=payload) as mock_fetch:
|
|
info = update_check.check("http://server.test:8000")
|
|
|
|
assert mock_fetch.call_count == 1 # cache was invalidated
|
|
assert info.latest == "2.1.0"
|
|
|
|
|
|
def test_check_handles_network_failure_silently(tmp_config):
|
|
"""A probe that errors out returns None; no exception leaks."""
|
|
from cli import update_check
|
|
with patch("cli.update_check._installed_version", return_value="2.0.0"):
|
|
with patch("cli.update_check._fetch_latest", return_value=None):
|
|
assert update_check.check("http://server.test:8000") is None
|
|
|
|
|
|
def test_negative_cache_avoids_reprobe_on_repeated_failure(tmp_config):
|
|
"""Two consecutive check() calls after a failed probe must fire the
|
|
network once — the second call hits the 5-minute negative cache."""
|
|
from cli import update_check
|
|
|
|
with patch("cli.update_check._installed_version", return_value="2.0.0"):
|
|
with patch("cli.update_check._fetch_latest", return_value=None) as mock_fetch:
|
|
assert update_check.check("http://server.test:8000") is None
|
|
# Second call within the negative-cache window.
|
|
assert update_check.check("http://server.test:8000") is None
|
|
|
|
assert mock_fetch.call_count == 1 # no re-probe
|
|
|
|
|
|
def test_negative_cache_expires_after_ttl(tmp_config):
|
|
"""After the negative TTL elapses, the probe fires again."""
|
|
import time
|
|
import json as _json
|
|
|
|
from cli import update_check
|
|
|
|
# Seed a stale negative-cache entry (older than 5min).
|
|
stale_ts = time.time() - (update_check._NEGATIVE_CACHE_TTL_SECONDS + 60)
|
|
(tmp_config / "update_check.json").write_text(_json.dumps({
|
|
"installed": "2.0.0",
|
|
"server_url": "http://server.test:8000",
|
|
"latest": None,
|
|
"download_url": None,
|
|
"checked_at": stale_ts,
|
|
}))
|
|
|
|
payload = {"version": "2.1.0", "download_url_path": "/cli/wheel/x.whl"}
|
|
with patch("cli.update_check._installed_version", return_value="2.0.0"):
|
|
with patch("cli.update_check._fetch_latest", return_value=payload) as mock_fetch:
|
|
info = update_check.check("http://server.test:8000")
|
|
|
|
assert mock_fetch.call_count == 1 # cache expired, refetch
|
|
assert info is not None
|
|
assert info.latest == "2.1.0"
|
|
|
|
|
|
def test_is_outdated_false_when_same_version(tmp_config):
|
|
from cli.update_check import UpdateInfo
|
|
info = UpdateInfo(installed="2.0.0", latest="2.0.0", download_url="…")
|
|
assert info.is_outdated() is False
|
|
|
|
|
|
def test_is_outdated_false_when_latest_unknown(tmp_config):
|
|
from cli.update_check import UpdateInfo
|
|
info = UpdateInfo(installed="2.0.0", latest=None, download_url=None)
|
|
assert info.is_outdated() is False
|
|
|
|
|
|
def test_is_outdated_true_when_installed_older(tmp_config):
|
|
from cli.update_check import UpdateInfo
|
|
info = UpdateInfo(installed="2.0.0", latest="2.1.0", download_url="…")
|
|
assert info.is_outdated() is True
|
|
|
|
|
|
def test_is_outdated_false_when_installed_newer_than_server(tmp_config):
|
|
"""After a server rollback the CLI may be ahead — don't prompt a downgrade."""
|
|
from cli.update_check import UpdateInfo
|
|
info = UpdateInfo(installed="2.1.0", latest="2.0.0", download_url="…")
|
|
assert info.is_outdated() is False
|
|
|
|
|
|
def test_is_outdated_uses_pep440_comparison(tmp_config):
|
|
"""`10.0.0 > 2.1.0` — must not be tripped by lexicographic string compare."""
|
|
from cli.update_check import UpdateInfo
|
|
newer_on_server = UpdateInfo(installed="2.1.0", latest="10.0.0", download_url="…")
|
|
older_on_server = UpdateInfo(installed="10.0.0", latest="2.1.0", download_url="…")
|
|
assert newer_on_server.is_outdated() is True
|
|
assert older_on_server.is_outdated() is False
|
|
|
|
|
|
def test_is_outdated_false_for_unparseable_strings(tmp_config):
|
|
"""Unparseable versions default to False — we'd rather miss an upgrade
|
|
hint than suggest a bogus downgrade."""
|
|
from cli.update_check import UpdateInfo
|
|
info = UpdateInfo(installed="nightly-abc", latest="nightly-def", download_url="…")
|
|
assert info.is_outdated() is False
|
|
|
|
|
|
def test_format_outdated_notice_drops_upgrade_line_when_no_download_url(tmp_config):
|
|
"""`download_url=None` must NOT produce literal "None" in the copy-pasteable command."""
|
|
from cli.update_check import UpdateInfo, format_outdated_notice
|
|
info = UpdateInfo(installed="2.0.0", latest="2.1.0", download_url=None)
|
|
msg = format_outdated_notice(info)
|
|
assert "None" not in msg
|
|
assert "uv tool install" not in msg
|
|
assert "2.0.0" in msg and "2.1.0" in msg
|
|
|
|
|
|
def test_format_outdated_notice_includes_upgrade_command_when_url_present(tmp_config):
|
|
from cli.update_check import UpdateInfo, format_outdated_notice
|
|
info = UpdateInfo(
|
|
installed="2.0.0",
|
|
latest="2.1.0",
|
|
download_url="http://s/cli/wheel/a-2.1.0-py3-none-any.whl",
|
|
)
|
|
msg = format_outdated_notice(info)
|
|
assert "uv tool install --force http://s/cli/wheel/a-2.1.0-py3-none-any.whl" in msg
|
|
|
|
|
|
class TestRootCallbackIntegration:
|
|
"""The root callback must not crash a command when the probe fails, and
|
|
must emit a stderr warning when the server advertises a newer version."""
|
|
|
|
def test_probe_failure_does_not_break_command(self, tmp_config):
|
|
with patch("cli.update_check.check", side_effect=RuntimeError("boom")):
|
|
result = runner.invoke(app, ["--help"])
|
|
assert result.exit_code == 0
|
|
|
|
def test_outdated_warning_is_emitted(self, tmp_config, capsys):
|
|
"""Unit-test the warning hook directly: `--help` is eager and bypasses
|
|
the callback body, so we test `_maybe_warn_outdated` itself, which
|
|
is what every real subcommand dispatch triggers."""
|
|
from cli.main import _maybe_warn_outdated
|
|
from cli.update_check import UpdateInfo
|
|
info = UpdateInfo(
|
|
installed="2.0.0",
|
|
latest="2.1.0",
|
|
download_url="http://server.test:8000/cli/wheel/x.whl",
|
|
)
|
|
with patch("cli.update_check.check", return_value=info):
|
|
_maybe_warn_outdated()
|
|
captured = capsys.readouterr()
|
|
assert "[update]" in captured.err
|
|
assert "2.1.0" in captured.err
|