agnes-the-ai-analyst/.github/workflows/keboola-deploy.yml
ZdenekSrotyr 103669dafd
fix(cli-install): move kbcstorage to [server] extra so wheel installs cleanly (P0 onboarding hotfix → 0.53.4) (#272)
* fix(cli-install): move kbcstorage to [server] extra so wheel installs cleanly

The 0.53.3 wheel served at /cli/wheel/ is unsatisfiable on a clean machine:
analyst runs `uv tool install <wheel-url>` per the published /setup
instructions and the resolver immediately fails with

    Because kbcstorage<=0.9.5 depends on urllib3<2.0.0 and
    agnes-the-ai-analyst==0.53.3 depends on kbcstorage>=0.9.0 and
    urllib3>=2.7.0, we can conclude that agnes-the-ai-analyst==0.53.3
    cannot be used.

The `[tool.uv] override-dependencies = ["urllib3>=2.7.0"]` in pyproject.toml
masked the conflict in workspace contexts (Dockerfile + dev install) but
does NOT propagate to the wheel — wheel METADATA is plain PEP 621
Requires-Dist, and a fresh resolver context (uv tool install <wheel-url>)
never sees the override. Every existing test passed because the dev venv
already has kbcstorage 0.9.5 + urllib3 2.7.0 coexisting under workspace
overrides; the break only surfaces on the next analyst's first install.

Fix: kbcstorage moved out of [project] dependencies into
[project.optional-dependencies].server, since it is server-side only
(connectors/keboola/client.py is the sole import site, called from admin
endpoints, server connectors, and integration tests — never from the CLI
install path). Server install picks it up via Dockerfile's
`uv pip install --system --no-cache ".[server]"`. CI installs `.[dev,server]`
so workspace tests still cover the kbcstorage path. Analyst CLI wheel
METADATA now lists `kbcstorage>=0.9.0; extra == 'server'` (gated) and
`uv tool install <wheel>` resolves cleanly.

Verified end-to-end:
- Built wheel locally; inspected METADATA — kbcstorage line is now `; extra == 'server'`.
- `docker run --rm python:3.13-slim` + `uv tool install <wheel>`: agnes 0.53.4 installs, `agnes --version` works, `agnes catalog --help` renders, kbcstorage absent from CLI venv, urllib3 = 2.7.0.
- Same container with `.[server]` install path: kbcstorage present, urllib3 = 2.7.0 (override applies in workspace context).
- Full pytest suite green locally (4157 passed, 25 skipped).

* release: 0.53.4 — analyst CLI install hotfix (urllib3/kbcstorage resolver conflict)

Patch bump shipping the [server] extra split + new clean-install CI lane.
No DB migration; no API change; no operator-facing config change.
Operator side (Dockerfile path) auto-picks `.[server]` so the production
image gains kbcstorage transparently. Analyst onboarding (uv tool install
<wheel>) starts working again.
2026-05-12 17:09:44 +00:00

140 lines
4.9 KiB
YAML

name: Keboola Deploy
# Tag-triggered build for Keboola's internal dev instance.
#
# Why a separate workflow: the default release.yml builds an image for *every* push
# to *every* branch, which means a shared dev VM pinned to a floating tag like
# `:dev` sees whoever pushed last. That convenience for per-developer dev VMs
# (`dev-<prefix>-latest` aliases) is a footgun for shared instances.
#
# This workflow runs ONLY when an operator explicitly creates a `keboola-deploy-*`
# git tag. The image is published with two tags:
# - keboola-deploy-<git-tag-suffix> (immutable, audit trail in git)
# - keboola-deploy-latest (floating alias the VM tracks)
#
# Operator workflow:
# git checkout <commit>
# git tag keboola-deploy-2026-04-25-groups-test
# git push origin keboola-deploy-2026-04-25-groups-test
# # → image built, alias updated, agnes-dev cron picks it up within 5 min
on:
push:
tags:
- "keboola-deploy-*"
permissions:
contents: read
packages: write
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Install dependencies
run: uv pip install --system ".[dev,server]"
- name: Lint with ruff
run: |
pip install ruff
ruff check . || true
continue-on-error: true # Don't block on pre-existing lint issues; can tighten later
- name: Type check with mypy
run: |
pip install mypy
mypy src/ app/ cli/ connectors/ --ignore-missing-imports --no-error-summary || true
continue-on-error: true # Don't block on mypy initially, can tighten later
- name: Run tests
run: pytest tests/ -v --tb=short
env:
TESTING: "1"
build-and-push:
needs: test
runs-on: ubuntu-latest
outputs:
image_tag: ${{ steps.meta.outputs.tag }}
steps:
- uses: actions/checkout@v6
- name: Resolve tag + version
id: meta
run: |
TAG="${GITHUB_REF#refs/tags/}"
# Sanity: tag must start with keboola-deploy- (the `on:` filter already
# enforces this, but cheap belt-and-braces against future workflow edits).
case "$TAG" in
keboola-deploy-*) ;;
*) echo "::error::Tag $TAG does not match keboola-deploy-* — refusing to build"; exit 1 ;;
esac
# Package version: source of truth is pyproject.toml (same convention as
# release.yml). The git tag is the *deploy identifier*, package version
# is the *product identifier*.
PKG_VERSION=$(grep '^version' pyproject.toml | head -1 | sed -E 's/^version\s*=\s*"([^"]+)".*/\1/')
if [ -z "$PKG_VERSION" ]; then
echo "::error::Could not extract version from pyproject.toml"; exit 1
fi
echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
echo "pkg_version=${PKG_VERSION}" >> "$GITHUB_OUTPUT"
echo "Building image for git tag: ${TAG} (package version ${PKG_VERSION})"
- name: Log in to GHCR
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
uses: docker/build-push-action@v7
with:
push: true
build-args: |
AGNES_VERSION=${{ steps.meta.outputs.pkg_version }}
RELEASE_CHANNEL=keboola-deploy
AGNES_COMMIT_SHA=${{ github.sha }}
AGNES_TAG=${{ steps.meta.outputs.tag }}
tags: |
ghcr.io/${{ github.repository }}:${{ steps.meta.outputs.tag }}
ghcr.io/${{ github.repository }}:keboola-deploy-latest
smoke-test:
needs: build-and-push
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Start Agnes from built image
run: |
touch .env
export AGNES_TAG="${{ needs.build-and-push.outputs.image_tag }}"
docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.ci.yml up -d app
timeout 60 bash -c 'until curl -sf http://localhost:8000/api/health | python3 -c "import sys,json; d=json.load(sys.stdin); sys.exit(0 if d[\"status\"]!=\"unhealthy\" else 1)"; do sleep 3; done'
- name: Run smoke tests
run: bash scripts/smoke-test.sh http://localhost:8000
- name: Collect logs on failure
if: failure()
run: docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.ci.yml logs > smoke-test-logs.txt
- name: Upload logs
if: failure()
uses: actions/upload-artifact@v7
with:
name: smoke-test-logs
path: smoke-test-logs.txt
- name: Teardown
if: always()
run: docker compose -f docker-compose.yml -f docker-compose.prod.yml -f docker-compose.ci.yml down -v