95358448e6
Replace hardwired Anthropic API calls with a pluggable provider system. Each deployment configures its AI provider in instance.yaml — switching between Anthropic, LiteLLM, OpenRouter, or any OpenAI-compatible proxy is a config change, not a code change. New connectors/llm/ module: - StructuredExtractor Protocol with extract_json() interface - AnthropicExtractor: direct Anthropic SDK with retry + backoff - OpenAICompatExtractor: any OpenAI-compatible proxy with three-layer structured output fallback (json_schema -> json_object -> prompt) - Configurable structured_output policy (strict/json/auto) - Custom exception hierarchy (auth/rate_limit/timeout/format/refusal) - Zero secrets in logs: no API keys, prompts, or responses logged Reviewed by: Google Gemini, Claude Sonnet, OpenAI GPT-5.4. Security audit passed with all critical findings resolved.
41 lines
1.4 KiB
Python
41 lines
1.4 KiB
Python
"""Base protocol for structured extraction from LLMs."""
|
|
|
|
from typing import Protocol
|
|
|
|
|
|
class StructuredExtractor(Protocol):
|
|
"""Protocol for structured JSON extraction from language models.
|
|
|
|
This is a structured extraction interface, NOT a general LLM chat
|
|
interface. Implementations must return parsed JSON matching the
|
|
provided schema.
|
|
"""
|
|
|
|
def extract_json(
|
|
self,
|
|
prompt: str,
|
|
max_tokens: int,
|
|
json_schema: dict,
|
|
schema_name: str,
|
|
) -> dict:
|
|
"""Extract structured JSON from a prompt.
|
|
|
|
Args:
|
|
prompt: The extraction prompt to send to the model.
|
|
max_tokens: Maximum tokens in the response.
|
|
json_schema: JSON Schema that the response must conform to.
|
|
schema_name: Human-readable name for the schema (used in
|
|
logging and error messages).
|
|
|
|
Returns:
|
|
Parsed JSON dictionary conforming to the provided schema.
|
|
|
|
Raises:
|
|
LLMError: Base class for all LLM-related errors.
|
|
LLMAuthError: Invalid API key (permanent, do not retry).
|
|
LLMRateLimitError: Rate limited (transient, retry with backoff).
|
|
LLMTimeoutError: Timeout or connection error (transient, retry).
|
|
LLMFormatError: Invalid JSON or unexpected structure.
|
|
LLMRefusalError: Model refused due to safety filter.
|
|
"""
|
|
...
|