agnes-the-ai-analyst/docs
Vojtech bd7b8c3233
fix(analyst): document BigQuery remote-query capability in bootstrap CLAUDE.md template (#154)
* fix(analyst): document BigQuery remote-query capability in bootstrap CLAUDE.md template

Closes #153.

The CLAUDE.md template generated by `da analyst bootstrap` (config/claude_md_template.txt)
covered metrics, sync, corporate memory, and directory layout — but had ZERO
mention of query_mode: "remote", da fetch, da query --remote, or --register-bq.
Result: the AI analyst running in a freshly-bootstrapped workspace had no
idea BigQuery-backed tables existed, no path to fetch unsynced data, and no
fallback for tables not in the catalog.

Validated against /Users/<user>/foundry-ai/foundryai-data-analyst/CLAUDE.md
on 2026-05-01: section confirmed missing. Workspace-level (parent-dir)
CLAUDE.md carried legacy SSH-heredoc instructions but the analyst-level
file (which Claude reads as primary project context) had nothing.

## Changes

### config/claude_md_template.txt (+83)

Added a `## Remote Queries (BigQuery)` section covering:

- Discovery first — `da catalog --json | jq '...'` to see all tables
  with their query_mode, then `da schema` and `da describe` for shape.
- Three query patterns:
  - `da fetch` (preferred) — materialize a filtered subset locally,
    query the snapshot, drop when done.
  - `da query --remote` — one-shot server-side execution (cheap probes).
  - `da query --register-bq` — hybrid joins between local + ad-hoc BQ.
- `da fetch` estimate-first discipline — rules of thumb on
  --select / --where / --estimate / snapshot reuse.
- BigQuery SQL flavor cheat sheet for `--where` (DATE literal,
  DATE_SUB, REGEXP_CONTAINS, CAST AS INT64).
- Unknown-table fallback: when a table isn't in `da catalog` at all,
  use ad-hoc `--register-bq` if the agnes server SA has BQ access, or
  ask admin to register with `query_mode: "remote"` for ongoing use.
- Pointer to `da skills show agnes-data-querying` for deeper guidance.

### docs/setup/claude_md_template.txt (deleted)

Stale 359-line template that documented the deprecated SSH-heredoc
remote_query.sh protocol. No code references it (verified via grep
across .py / .sh / .yml / .md). Removing eliminates two failure
modes:
1. A future refactor accidentally pulling it into a workspace and
   shipping deprecated guidance to analyst Claude sessions.
2. Reviewer confusion over which template is canonical.

### CHANGELOG.md

`### Fixed` and `### Removed` entries under [Unreleased].

## Tested

- Manually walked the diff against `da skills show agnes-data-querying`
  output on a live VM (foundryai-development) — patterns + flags
  match the modern CLI exactly.
- Re-bootstrap test deferred: requires network round-trip; pattern
  is identical to existing template substitution path so render is
  not at risk.

## Out of scope

- The companion gap that data_description.md often only enumerates
  query_mode: "local" tables (no signal that other modes exist) —
  separate concern, fix likely belongs in the metadata generator
  on the server side, not in the analyst template.
- Encouraging admins to register frequently-queried BQ tables as
  `query_mode: "remote"` in the registry — workflow improvement, not
  a code bug.

* chore(release): cut 0.28.0

---------

Co-authored-by: ZdenekSrotyr <zdenek.srotyr@keboola.com>
2026-05-01 12:06:41 +02:00
..
future docs: update references to deleted files 2026-04-09 17:15:19 +02:00
metrics feat: add 10 starter pack metrics (revenue, usage, sales, operations) 2026-04-10 19:35:28 +02:00
setup fix(analyst): document BigQuery remote-query capability in bootstrap CLAUDE.md template (#154) 2026-05-01 12:06:41 +02:00
superpowers fix(v2): #134 BigQuery cross-project errors return structured 502/400 + BqAccess facade (#138) 2026-04-30 10:11:20 +02:00
testing chore: clean repo for public release — fix references, remove drafts 2026-04-08 19:27:25 +02:00
ADR-corporate-memory-v1.md feat(memory): corporate memory v1+v1.5 + 0.15.0 (#72) 2026-04-29 07:16:22 +02:00
architecture.md feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
auth-google-oauth.md feat(rbac): drop dataset_permissions + users.role + is_public; v19 migration (#150) 2026-04-30 22:02:16 +02:00
auth-groups.md feat(auth): Google Workspace group prefix filter + system mapping (#131) 2026-04-29 14:08:04 +02:00
auto-install.md chore: clean stale docs — rewrite architecture.md, remove old plans 2026-04-09 09:06:13 +02:00
CONFIGURATION.md docs: update stale v1 docs to v2 Docker/FastAPI/DuckDB architecture 2026-04-09 18:44:25 +02:00
corporate-memory-governance.md Add Corporate Memory governance — Phase 1 (data model + admin API) 2026-03-23 19:15:33 +01:00
DATA_SOURCES.md feat(admin): #108 M1 — BigQuery table registration in UI + CLI (#119) 2026-04-29 13:18:31 +02:00
DEPLOYMENT.md feat(scheduler): re-wire sync_schedule + script.schedule; tune via env; OpenMetadata TLS (#135) 2026-04-29 22:06:30 +02:00
development.md feat(observability): request_id end-to-end + dev debug toolbar + centralized logging (#136) 2026-04-29 22:54:21 +02:00
HACKATHON.md docs: add HACKATHON.md — condensed deploy + dev playbooks (#21) 2026-04-21 21:33:06 +02:00
HEADLESS_USAGE.md User management + PAT + CLI distribution + HTML auth redirect (#9 #10 #11 #12) (#28) 2026-04-22 14:24:28 +02:00
llm-routing.md docs,tests: anonymize customer references 2026-04-21 11:56:19 +02:00
local-development.md feat(dev): add Windows PowerShell wrapper for local development (#80) 2026-04-28 23:59:11 +02:00
ONBOARDING.md ci: propagate infra-v* tags to template repo + auto-merge rules (#17) 2026-04-21 21:32:58 +02:00
pd-ps-comments.md feat(memory): corporate memory v1+v1.5 + 0.15.0 (#72) 2026-04-29 07:16:22 +02:00
QUICKSTART.md feat(ci+tests): deploy safety audit — linting, rollback, smoke tests, 50+ new tests (#120) 2026-04-29 09:18:55 +02:00
RBAC.md feat(ci+tests): deploy safety audit — linting, rollback, smoke tests, 50+ new tests (#120) 2026-04-29 09:18:55 +02:00
RELEASE_TEMPLATE.md feat: multi-instance deployment — all 14 must-have items from spec 2026-04-10 11:57:42 +02:00
sample-data.md
security-audit-2026-04.md chore(oss): isolate customer-specific deploy bits from scripts/grpn/ (#88, wave 1) (#94) 2026-04-27 20:24:34 +02:00
theme-reference.html Fix clipped annotation badges in theme-reference.html 2026-03-11 14:09:04 +01:00