b6543c9c55
1. .env_overlay write paths now match read path under STATE_DIR. app/main.py:343 reads via _state_dir() (post-PR #194), but two write sites still hardcoded ${DATA_DIR}/state/.env_overlay: - app/api/admin.py:2687 — configure endpoint secrets persistence - app/api/marketplaces.py:152 — marketplace PAT persistence Under flat-mount layout (STATE_DIR=/data-state) the admin UI wrote secrets to /data/state/.env_overlay while the app read from /data-state/.env_overlay, silently dropping the value on next restart. Both write sites now go through _state_dir(). 2. host-mount.yml: caddy inherits data:/srv:ro from base, but with no service populating the data: named volume (other services switched to direct /data binds), the inherited mount points at an empty Docker volume — try_files finds nothing, every parquet download falls through to uvicorn, defeating the v0.36.0 file_server bypass under the host-mount layout. Added a caddy override that restates all mounts including a direct /data:/srv:ro bind. Mirrors the comment + treatment already in flat-mount.yml.
96 lines
3.6 KiB
YAML
96 lines
3.6 KiB
YAML
# Bind-mount overlay — replaces the `data` named volume with a direct
|
|
# host bind mount per service.
|
|
#
|
|
# Why direct service-level bind, not driver_opts on the named volume
|
|
# ------------------------------------------------------------------
|
|
# The previous version of this file modified the `data` named volume's
|
|
# `driver_opts` to point at /data with `o: bind,rbind`. Docker named
|
|
# volumes have an immutability footgun: once a volume is created, its
|
|
# driver options are fixed for the life of the volume. Editing this
|
|
# file and re-running `docker compose up -d` does NOT propagate the
|
|
# new options to existing volumes — they keep whatever options were
|
|
# in effect at create time.
|
|
#
|
|
# This bit a deployer (Groupon FoundryAI) on 2026-05-05: the volume
|
|
# was created before this overlay had `bind,rbind`, kept the old
|
|
# `bind` (non-recursive) propagation, and containers wrote to a
|
|
# shadowed subdirectory of the parent disk instead of the nested
|
|
# child mount. DuckDB went FATAL on a root-owned WAL during a
|
|
# routine container recreate; sign-in broke.
|
|
#
|
|
# Direct service-level bind mounts (`/host/path:/container/path`)
|
|
# don't go through Docker's volume layer at all. They re-evaluate
|
|
# the mount options every container start, and modern Docker Engine
|
|
# (20.10+) defaults to recursive bind for these. No options to
|
|
# forget, no immutable state to migrate, no shadow-mount class.
|
|
#
|
|
# What this overlay does
|
|
# ----------------------
|
|
# `volumes: !override` on each service replaces the base
|
|
# `data:/data` named-volume mount with a direct `/data:/data` host
|
|
# bind. The named volume `data:` declared at the bottom of
|
|
# docker-compose.yml is left intact (still useful for local-dev
|
|
# `compose up` without this overlay) but is no longer referenced
|
|
# by any service when the overlay is active.
|
|
#
|
|
# When the operator's host has a nested mount under /data (e.g. a
|
|
# separate state disk mounted at /data/state), the recursive bind
|
|
# carries that nested mount into every container automatically.
|
|
#
|
|
# Usage (combined with docker-compose.prod.yml):
|
|
# docker compose \
|
|
# -f docker-compose.yml \
|
|
# -f docker-compose.prod.yml \
|
|
# -f docker-compose.host-mount.yml \
|
|
# up -d
|
|
#
|
|
# Do NOT use this overlay in CI — /data does not exist on GitHub
|
|
# runners.
|
|
#
|
|
# Compose-spec version requirement: !override merge tag is part of
|
|
# the Compose Specification supported by Docker Compose v2.20+ and
|
|
# the compose-go library used by Compose v5+. If you need to support
|
|
# older clients, fork this overlay into per-service files.
|
|
|
|
services:
|
|
app:
|
|
volumes: !override
|
|
- /data:/data
|
|
- ./config:/app/config:ro
|
|
|
|
extract:
|
|
volumes: !override
|
|
- /data:/data
|
|
- ./config:/app/config:ro
|
|
|
|
scheduler:
|
|
volumes: !override
|
|
- /data:/data
|
|
- ./config:/app/config:ro
|
|
|
|
telegram-bot:
|
|
volumes: !override
|
|
- /data:/data
|
|
|
|
ws-gateway:
|
|
volumes: !override
|
|
- /data:/data
|
|
|
|
caddy:
|
|
# Caddy was originally inheriting `data:/srv:ro` from the base
|
|
# service. Once the other services switch to direct binds and
|
|
# nothing populates the `data:` named volume, that inherited
|
|
# mount points at an empty Docker-managed volume — and the
|
|
# @download `try_files /bigquery/data/<id>.parquet …` block
|
|
# in Caddyfile finds nothing, so every parquet download falls
|
|
# through to the app's uvicorn worker, defeating the v0.36.0
|
|
# file_server bypass.
|
|
#
|
|
# Restate every mount the base caddy service depends on; mirror
|
|
# the same caveat that lives in flat-mount.yml.
|
|
volumes: !override
|
|
- ./Caddyfile:/etc/caddy/Caddyfile:ro
|
|
- /data/state/certs:/certs:ro
|
|
- caddy_data:/data
|
|
- caddy_config:/config
|
|
- /data:/srv:ro
|