4799119c81
* feat(deploy): keboola-deploy tag-triggered workflow + Caddyfile LE/internal modes + dev_instances TLS support
Three coordinated changes that together unblock Keboola's internal Agnes
deployment from the foot-gun where the dev VM tracks `:dev` (= last push
from anyone in the upstream repo).
1. .github/workflows/keboola-deploy.yml — new workflow
Triggered ONLY on `keboola-deploy-*` git tag pushes (not on every branch
push like release.yml). Builds an image and publishes two GHCR tags:
ghcr.io/keboola/agnes-the-ai-analyst:keboola-deploy-<git-tag-suffix>
ghcr.io/keboola/agnes-the-ai-analyst:keboola-deploy-latest
The Keboola dev VM pins to `keboola-deploy-latest`; an operator deploys
by `git tag keboola-deploy-foo && git push origin keboola-deploy-foo`.
Audit trail lives in git tags (immutable, who-tagged-what-when), no
PR-cycle needed for each deploy.
Doesn't touch Vojta/Minas/David workflow — release.yml still builds
`:dev-<slug>` for every branch push as before.
2. Caddyfile — parametrize TLS directive via $CADDY_TLS env var
PR #51 hardcoded cert-file mode (`tls /certs/fullchain.pem ...`) for
Groupon's corporate CA flow. That broke the Let's Encrypt path the
module previously supported. Now:
CADDY_TLS unset (default) → cert-file mode (Groupon corp PKI)
CADDY_TLS="tls user@x.com" → Let's Encrypt auto-issue
CADDY_TLS="tls internal" → Caddy-managed self-signed (lab/dev)
Single Caddyfile, three regimes, no per-deployment fork. Validated with
`caddy validate` in all three modes.
3. customer-instance module — dev_instances TLS + auto-set CADDY_TLS
- variables.tf: dev_instances object schema gains optional tls_mode +
domain (mirroring prod_instance). Defaults to "none" + "" so existing
callers without those fields keep current behavior.
- startup-script.sh.tpl: when tls_mode="caddy" and DOMAIN is set, write
CADDY_TLS=tls <ACME_EMAIL> (or "tls internal" when ACME_EMAIL empty)
into /opt/agnes/.env. Caddy then picks it up and the Caddyfile
substitution flips the cert source.
For an LE deploy: set tls_mode="caddy", domain="agnes-dev.example.com",
ensure DNS A-record points at the VM, and acme_email is set on the
module (or seed_admin_email is, since acme_email defaults to it).
After this lands, tag as infra-v1.6.0 so downstream infra repos can bump
their module ref without needing the upstream change tracking.
* feat(deploy): fetch optional Google OAuth credentials from Secret Manager
Mirrors the existing keboola-storage-token / agnes-<customer>-jwt-secret
pattern: VM SA reads google-oauth-client-{id,secret} secrets at boot
(if they exist + IAM is wired by caller via runtime_secrets) and writes
them into /opt/agnes/.env. Empty / missing / 403 → silent fallback
to "" so password and email auth keep working untouched.
Pairs with downstream change in agnes-infra-keboola which adds the two
secret names to runtime_secrets, granting the Keboola VM SA secretAccessor
on them. Operator pre-creates the SM containers via gcloud secrets create
google-oauth-client-{id,secret} (one-time, out of band) — values stay
in SM forever; rotation = `gcloud secrets versions add`.
This unblocks the Keboola agnes-dev deploy from PR #3 (infra) — without
GOOGLE_CLIENT_{ID,SECRET} in .env, app/auth/providers/google.is_available()
returns False and the Google sign-in button never even appears.
98 lines
3.4 KiB
YAML
98 lines
3.4 KiB
YAML
name: Keboola Deploy
|
|
|
|
# Tag-triggered build for Keboola's internal dev instance.
|
|
#
|
|
# Why a separate workflow: the default release.yml builds an image for *every* push
|
|
# to *every* branch, which means Keboola's `agnes-dev` VM (pinned to `:dev` or
|
|
# similar floating tag) sees whoever pushed last — Vojta, Minas, anyone. That
|
|
# convenience for Groupon-side dev VMs (per-developer `dev-<prefix>-latest` aliases)
|
|
# is a footgun for shared instances.
|
|
#
|
|
# This workflow runs ONLY when an operator explicitly creates a `keboola-deploy-*`
|
|
# git tag. The image is published with two tags:
|
|
# - keboola-deploy-<git-tag-suffix> (immutable, audit trail in git)
|
|
# - keboola-deploy-latest (floating alias the VM tracks)
|
|
#
|
|
# Operator workflow:
|
|
# git checkout <commit>
|
|
# git tag keboola-deploy-2026-04-25-groups-test
|
|
# git push origin keboola-deploy-2026-04-25-groups-test
|
|
# # → image built, alias updated, agnes-dev cron picks it up within 5 min
|
|
on:
|
|
push:
|
|
tags:
|
|
- "keboola-deploy-*"
|
|
|
|
permissions:
|
|
contents: read
|
|
packages: write
|
|
|
|
jobs:
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
|
|
- uses: actions/setup-python@v6
|
|
with:
|
|
python-version: "3.13"
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v7
|
|
|
|
- name: Install dependencies
|
|
run: uv pip install --system ".[dev]"
|
|
|
|
- name: Run tests
|
|
run: pytest tests/ -v --tb=short
|
|
env:
|
|
TESTING: "1"
|
|
|
|
build-and-push:
|
|
needs: test
|
|
runs-on: ubuntu-latest
|
|
outputs:
|
|
image_tag: ${{ steps.meta.outputs.tag }}
|
|
steps:
|
|
- uses: actions/checkout@v5
|
|
|
|
- name: Resolve tag + version
|
|
id: meta
|
|
run: |
|
|
TAG="${GITHUB_REF#refs/tags/}"
|
|
# Sanity: tag must start with keboola-deploy- (the `on:` filter already
|
|
# enforces this, but cheap belt-and-braces against future workflow edits).
|
|
case "$TAG" in
|
|
keboola-deploy-*) ;;
|
|
*) echo "::error::Tag $TAG does not match keboola-deploy-* — refusing to build"; exit 1 ;;
|
|
esac
|
|
# Package version: source of truth is pyproject.toml (same convention as
|
|
# release.yml). The git tag is the *deploy identifier*, package version
|
|
# is the *product identifier*.
|
|
PKG_VERSION=$(grep '^version' pyproject.toml | head -1 | sed -E 's/^version\s*=\s*"([^"]+)".*/\1/')
|
|
if [ -z "$PKG_VERSION" ]; then
|
|
echo "::error::Could not extract version from pyproject.toml"; exit 1
|
|
fi
|
|
echo "tag=${TAG}" >> "$GITHUB_OUTPUT"
|
|
echo "pkg_version=${PKG_VERSION}" >> "$GITHUB_OUTPUT"
|
|
echo "Building image for git tag: ${TAG} (package version ${PKG_VERSION})"
|
|
|
|
- name: Log in to GHCR
|
|
uses: docker/login-action@v4
|
|
with:
|
|
registry: ghcr.io
|
|
username: ${{ github.actor }}
|
|
password: ${{ secrets.GITHUB_TOKEN }}
|
|
|
|
- name: Build and push
|
|
uses: docker/build-push-action@v7
|
|
with:
|
|
push: true
|
|
build-args: |
|
|
AGNES_VERSION=${{ steps.meta.outputs.pkg_version }}
|
|
RELEASE_CHANNEL=keboola-deploy
|
|
AGNES_COMMIT_SHA=${{ github.sha }}
|
|
AGNES_TAG=${{ steps.meta.outputs.tag }}
|
|
tags: |
|
|
ghcr.io/${{ github.repository }}:${{ steps.meta.outputs.tag }}
|
|
ghcr.io/${{ github.repository }}:keboola-deploy-latest
|