107195730d
* feat(observability): optional PostHog integration (errors, LLM traces, replay, flags)
Off by default. Activates when POSTHOG_API_KEY is set in env. Defaults
to PostHog Cloud EU; override host for US Cloud or self-hosted.
Coverage:
- FastAPI 500 handler captures unhandled exceptions
- src/orchestrator.py rebuild + rebuild_source failures
- services/scheduler/ HTTP-job failures
- cli/main.py uncaught CLI errors (Typer.Exit/SystemExit/KeyboardInterrupt
skipped; flushes before re-raise so short-lived CLI invocations don't
drop events)
- connectors/llm/anthropic_provider.py + openai_compat.py emit
$ai_generation events with provider, model, latency, token counts
(prompt/completion bodies stay off unless POSTHOG_LLM_PAYLOADS=1
because LLM prompts here routinely include customer SQL/data)
- Browser snippet injected into every text/html response by
PosthogInjectionMiddleware — registered inside the GZip layer so it
sees uncompressed HTML before compression. Many templates are
standalone (their own DOCTYPE) and never extend base.html, so a
per-template include would miss them.
- Frontend: $pageview, $pageleave, JS error capture via window.error
and unhandledrejection handlers, masked session replay
(maskAllInputs: true plus CSS-selector mask for known data surfaces),
feature flags (browser posthog.isFeatureEnabled + server-side
feature_enabled with fallback for older SDKs).
Identification mode operator-configurable: none / id / email / full.
Default email ships user.id + email but never name. CLI entry point
moves from cli.main:app to cli.main:main (Typer wrapper).
Files:
- src/observability/posthog_client.py — lazy singleton, no network
when disabled, single-process flush on shutdown
- src/observability/llm_tracing.py — trace_generation context manager
- app/middleware/posthog_inject.py — HTML rewrite middleware
- app/web/templates/_posthog.html — browser snippet template
- docs/observability.md — operator guide
- config/.env.template — documented POSTHOG_* knobs
- tests/test_posthog_disabled.py + tests/test_posthog_client.py +
tests/test_llm_tracing.py — 18 tests covering disabled state,
identify-mode payloads, $ai_generation shape, error variant.
CHANGELOG entry under [Unreleased] Added.
* feat(observability): tag every PostHog event with environment + release
Splits PostHog dashboards cleanly between localhost / dev / staging /
production without manual tagging on every capture call.
- POSTHOG_ENVIRONMENT explicit override; auto-resolves to "local" when
LOCAL_DEV_MODE=1, else RELEASE_CHANNEL, else AGNES_DEPLOYMENT_ENV,
else "unknown".
- AGNES_VERSION → RELEASE_CHANNEL fallback feeds the `release` property
for "is this error new in this release?" cohorting.
- Backend gets both via the PostHog SDK's super_properties constructor
arg (every captured event picks them up automatically).
- Browser snippet calls posthog.register({environment, release}) inside
the loaded callback so $pageview, $exception, autocapture, etc. all
carry the same labels.
- request.state.user now populated by auth dependencies so the snippet
can actually call posthog.identify(user_id, {email}) for logged-in
users (previously the user block always resolved to None because
nothing wrote to request.state.user).
4 new tests cover env resolution: explicit > LOCAL_DEV_MODE > channel
> unknown, plus super-properties forwarding into the SDK constructor.
* feat(observability): inline user attrs on every PostHog event + debug throw route
PostHog's UI shows person properties on the Person profile page, not
inline on each event — so a reviewer triaging an exception couldn't tell
which user hit the bug without clicking through. Fix it on both sides.
- Backend capture_exception merges user_id / user_email / user_name into
the event properties (gated by POSTHOG_IDENTIFY_PII: none/id/email/full).
Backed by a new _user_props_for_event helper on PosthogClient.
- Browser snippet registers user_id + user_email + user_name as super-
properties via posthog.register({...}) so every $exception, $pageview,
and custom event coming from posthog.captureException() carries them
inline. Mirrors the backend so cross-referencing client/server events
doesn't require a person-profile lookup.
- /api/debug/throw — debug-only endpoint gated by DEBUG=1 (404 in prod).
Runs Depends(get_current_user) first so request.state.user is set when
the unhandled-exception handler captures the event. Lets operators
exercise the full observability path end-to-end without hand-rolling
a TestClient script. Configurable via ?kind=ValueError&msg=...
7 new tests cover: backend user-attr merge across identify modes,
anonymous request fall-through, browser snippet super-prop emission for
logged-in / anonymous / id-only / full-name cases.
* fix(observability): address minasarustamyan PR #231 review
Two bugs caught in review.
1. PosthogInjectionMiddleware dropped Response.background on every
return path. BaseHTTPMiddleware materialises the body and asks
subclasses to return a fresh Response — three paths in dispatch()
omitted background=, silently cancelling any BackgroundTask /
BackgroundTasks the route attached (audit logging, async webhooks,
email sends) with no log line. Fix: route every return through a
_passthrough() helper that forwards background.
Also adds a _MAX_BUFFER_BYTES (4 MB) cap so a streamed-HTML response
can't balloon RSS during buffering. Bigger bodies short-circuit
through with a warning rather than being injected.
Regression tests in tests/test_posthog_inject_middleware.py exercise
four return paths (snippet present, render-fail, double-injection
guard, non-HTML passthrough) plus the streaming-guard short-circuit.
2. $ai_input / $ai_output_choices were emitted without truncation, so
POSTHOG_LLM_PAYLOADS=1 silently dropped events past PostHog's ~32 KB
per-event ingest limit — exactly the calls (large prompts with
schemas / sample rows / SQL) an operator would want to inspect.
Fix: clip both at POSTHOG_LLM_PAYLOAD_MAX_CHARS (default 30000) with
an explicit "…[truncated N chars]" marker so readers don't mistake
truncated captures for complete ones. Metadata (provider, model,
tokens, latency, error) flows regardless. Three new tests cover
default-cap clipping, env-override, and pass-through under the cap.
37 PostHog tests pass.
77 lines
4.9 KiB
HTML
77 lines
4.9 KiB
HTML
{# PostHog browser snippet — included from <head> in base.html / base_login.html.
|
|
Renders nothing when the integration is disabled (no POSTHOG_API_KEY set on
|
|
the server). The `posthog_config` Jinja global is wired up once at app
|
|
startup in app/web/router.py from src.observability.get_posthog().
|
|
|
|
Privacy posture:
|
|
* Session replay is masked-by-default (`maskAllInputs: true` plus a CSS
|
|
selector covering data cells / inputs). Operator can append a custom
|
|
selector via POSTHOG_REPLAY_MASK_SELECTOR.
|
|
* `person_profiles: 'identified_only'` keeps anonymous visits out of the
|
|
people table.
|
|
* Identification respects POSTHOG_IDENTIFY_PII (none/id/email/full).
|
|
#}
|
|
{% if posthog_config and posthog_config.enabled %}
|
|
<script>
|
|
!function(t,e){var o,n,p,r;e.__SV||(window.posthog=e,e._i=[],e.init=function(i,s,a){function g(t,e){var o=e.split(".");2==o.length&&(t=t[o[0]],e=o[1]);t[e]=function(){t.push([e].concat(Array.prototype.slice.call(arguments,0)))}}(p=t.createElement("script")).type="text/javascript",p.crossOrigin="anonymous",p.async=!0,p.src=s.api_host.replace(".i.posthog.com","-assets.i.posthog.com")+"/static/array.js",(r=t.getElementsByTagName("script")[0]).parentNode.insertBefore(p,r);var u=e;for(void 0!==a?u=e[a]=[]:a="posthog",u.people=u.people||[],u.toString=function(t){var e="posthog";return"posthog"!==a&&(e+="."+a),t||(e+=" (stub)"),e},u.people.toString=function(){return u.toString(1)+".people (stub)"},o="init me ws ys ps bs capture je Di ks register register_once register_for_session unregister unregister_for_session getFeatureFlag getFeatureFlagPayload isFeatureEnabled reloadFeatureFlags updateEarlyAccessFeatureEnrollment getEarlyAccessFeatures on onFeatureFlags onSessionId getSurveys getActiveMatchingSurveys renderSurvey canRenderSurvey identify setPersonProperties group resetGroups setPersonPropertiesForFlags resetPersonPropertiesForFlags setGroupPropertiesForFlags resetGroupPropertiesForFlags reset get_distinct_id getGroups get_session_id get_session_replay_url alias set_config startSessionRecording stopSessionRecording sessionRecordingStarted captureException loadToolbar get_property getSessionProperty Es $s createPersonProfile Is opt_in_capturing opt_out_capturing has_opted_in_capturing has_opted_out_capturing clear_opt_in_out_capturing Ss debug I As getPageViewId captureTraceFeedback captureTraceMetric".split(" "),n=0;n<o.length;n++)g(u,o[n]);e._i.push([i,s,a])},e.__SV=1)}(document,window.posthog||[]);
|
|
posthog.init("{{ posthog_config.api_key_public }}", {
|
|
api_host: "{{ posthog_config.host }}",
|
|
person_profiles: "identified_only",
|
|
capture_pageview: true,
|
|
capture_pageleave: true,
|
|
autocapture: false,
|
|
{% if posthog_config.replay_enabled %}
|
|
session_recording: {
|
|
maskAllInputs: true,
|
|
maskTextSelector: "[data-sensitive], .data-cell, .query-result, .sql-output, code, pre{% if posthog_config.replay_mask_selector_extra %}, {{ posthog_config.replay_mask_selector_extra }}{% endif %}",
|
|
recordCrossOriginIframes: false
|
|
},
|
|
disable_session_recording: false,
|
|
{% else %}
|
|
disable_session_recording: true,
|
|
{% endif %}
|
|
loaded: function (ph) {
|
|
// Tag every browser event with deployment environment + release
|
|
// and (when a user is logged in) user_id + email/name. register()
|
|
// values are sticky (localStorage-persisted) and apply to every
|
|
// event — $pageview, $exception, $autocapture, etc. — so the
|
|
// event-detail view in PostHog shows who the user was inline,
|
|
// without clicking through to the person profile. Mirrors the
|
|
// backend's super_properties + user_props_for_event so cross-
|
|
// referencing client and server events doesn't require a join.
|
|
var _superProps = {
|
|
environment: {{ posthog_config.environment|tojson }}{% if posthog_config.release %},
|
|
release: {{ posthog_config.release|tojson }}{% endif %}
|
|
};
|
|
{% set _u = posthog_user_block(request) %}
|
|
{% if _u %}
|
|
_superProps.user_id = {{ _u.distinct_id|tojson }};
|
|
{% if _u.props.email %}
|
|
_superProps.user_email = {{ _u.props.email|tojson }};
|
|
{% endif %}
|
|
{% if _u.props.name %}
|
|
_superProps.user_name = {{ _u.props.name|tojson }};
|
|
{% endif %}
|
|
{% endif %}
|
|
ph.register(_superProps);
|
|
{% if _u %}
|
|
ph.identify({{ _u.distinct_id|tojson }}, {{ _u.props|tojson }});
|
|
{% endif %}
|
|
}
|
|
});
|
|
window.addEventListener("error", function (e) {
|
|
try {
|
|
if (window.posthog && typeof posthog.captureException === "function") {
|
|
posthog.captureException(e.error || new Error(e.message || "window.error"));
|
|
}
|
|
} catch (_err) {}
|
|
});
|
|
window.addEventListener("unhandledrejection", function (e) {
|
|
try {
|
|
if (window.posthog && typeof posthog.captureException === "function") {
|
|
posthog.captureException(e.reason || new Error("unhandledrejection"));
|
|
}
|
|
} catch (_err) {}
|
|
});
|
|
</script>
|
|
{% endif %}
|