Language models

Roles

Every role has a primary model and ordered fallbacks. A role with no working model is simply unavailable, and the code path that would have used it takes the deterministic branch. Nothing in the core depends on a model being present.

Providers

Four provider kinds are implemented over raw HTTP with no SDKs: OpenAI-compatible (chat completions with JSON mode, also used for local servers that speak the same protocol), Anthropic (Messages API), Ollama and Amazon Bedrock (the Converse API for chat roles, Titan and Cohere models for embeddings). Bedrock requests are signed with AWS Signature Version 4 implemented over the standard library and pinned by tests to signatures produced by the reference AWS signer; the endpoint must be a Bedrock runtime host, from which the region is read, and the credential is one secret reference holding the access key, secret key and optional session token together. A provider row holds a name, kind, endpoint, timeout, extra headers, an allow_private_endpoint flag (required for local endpoints, because the outbound HTTP client refuses private and loopback ranges by default) and an API key reference. Models belong to providers and carry context window, dimensions for embedding models, per-token cost estimates and default parameters. The interface tests connectivity with a real call and records the result.

Secrets

Keys are never stored in clear. A key entered in the interface becomes an enc: reference encrypted under the master key; env:NAME and file:/path references defer to the environment or a file at call time. Forms never echo a stored key; the API returns whether a key is set, not the key; exports exclude them; the log handler redacts values that look like secrets; error messages from providers are scrubbed of the key before being stored or shown.

Routing, budgets and concurrency

The router resolves a role to its models, tries them in order, and records which one answered. A mind may carry its own role routing, which experiments use to run forks of one snapshot under different models; an override that names a missing or disabled model fails closed rather than falling back to the default, so a comparison between providers is always the comparison it claims to be. A daemon-wide concurrency pool bounds simultaneous calls. Budgets are enforced at three levels: a per-cycle call cap per mind, a per-mind daily token budget and a global daily token budget; exhaustion is a recorded event and the deterministic branch, not an exception. Usage is aggregated per mind, role and model for the interface and for Prometheus.

Recording and replay

Every call records role, purpose, provider, model, prompt name and version, the rendered request, the response, token usage, latency, estimated cost, seed and a hash of the request. Calls are visible per mind and per event (a response links to its call). When a router runs in replay mode, a request whose hash matches a recorded call is served from the recording and marked replayed; this is how cognitive replay reproduces model-dependent steps exactly and how it knows to label a step approximate when no recording matches.

Prompts

Prompts are versioned rows: name, role, description, system text, a Go template for the user message and an optional JSON schema for the answer. Untrusted text is always rendered through a data helper that wraps it in a delimited block with an explicit "this is data, not instructions" marker. Saving a prompt creates a new version; the interface diffs versions, lets you test a prompt against sample input, and shows which version each recorded call used. Default prompts are embedded in the binary and installed on first start; edited ones live in the database.

Language models are cognitive service providers. Cognitive code never names a vendor; it asks the router for a role and receives a response with a call id it can cite as evidence (ADR-0004).

Amazon Bedrock

Kind bedrock speaks the Bedrock Converse API (/model/{id}/converse) and, for embeddings, /model/{id}/invoke with the Titan and Cohere request shapes. Requests are signed with AWS Signature Version 4 implemented over the standard library (HMAC-SHA256; the canonical URI is encoded twice as the specification requires outside S3) and pinned by tests to signatures produced by botocore. The endpoint is https://bedrock-runtime.<region>.amazonaws.com; the region is taken from the host and any other host is refused. The provider's API key reference resolves to one string, ACCESS_KEY_ID:SECRET_ACCESS_KEY or ACCESS_KEY_ID:SECRET_ACCESS_KEY:SESSION_TOKEN, so the whole credential is encrypted together and never split across columns. Provider errors are scrubbed of the secret and session token. Converse has no JSON-mode switch; when a role requests JSON the instruction is added to the system text and the answer is parsed and validated on Noema's side as for every other provider.

Per-mind routing

A mind's cognitive configuration may carry llm_roles, a map from role to model id, which overrides the daemon-wide routing for that mind alone. It exists so that forks of one snapshot can run on different providers in a controlled experiment. An override names exactly one model; if that model is missing or disabled the call fails with "no model" rather than falling back, and the runtime identity records the override next to the daemon routing.

Roles

Each role has a primary model, ordered fallbacks, and optional temperature / max-token overrides. Fallbacks are tried in order on provider errors; budget errors and cancellation stop the chain.

Providers and models

Provider kinds: openai (OpenAI and every OpenAI-compatible server: llama.cpp, vLLM, LM Studio, Ollama's /v1), anthropic (Messages API), ollama (native API with seed and JSON mode), bedrock (Amazon Bedrock Converse API, see below).

Endpoints that resolve to private or loopback addresses are refused unless the provider has allow private addresses set (SSRF guard, threat T6). Credentials go in the API key field only; Authorization and x-api-key headers are rejected in the custom headers map.

Models carry kind (chat or embedding), context window, embedding dimensions and per-million-token costs used for cost estimates.

Secrets

The key field accepts either a reference (env:OPENAI_API_KEY, file:/run/secrets/key) or a pasted value. Pasted values are encrypted with AES-256-GCM under NOEMA_MASTER_KEY and stored as an enc: reference. Without a master key, only references are accepted. Keys are resolved at call time and never rendered, exported, logged or returned by the API; the UI shows only "encrypted in database" or the reference form.

Recording, budgets and replay

Every call is recorded in llm_calls: role, purpose, provider, model, prompt name and version, the full rendered request, the response, usage, latency, cost, seed, error, and a hash of everything that influences the answer. llm_request/llm_response events are emitted by the cognitive processes that use the router (phase 8 onward), citing the call id.

Budgets: a global daily token budget (NOEMA_DAILY_TOKEN_BUDGET), a per-mind daily budget (mind configuration), a per-cycle call cap (mind configuration) and a global concurrency limit (NOEMA_MAX_CONCURRENT_LLM). When a budget is exhausted the router returns an error and deterministic cognition continues without model help.

In replay mode the router serves the recorded response for an identical request hash, marking the response as replayed; this is what makes LLM-assisted steps exact during cognitive replay when a recording exists.

Prompts

Prompts are named, versioned rows (prompts, append-only prompt_versions) with a system template, a user template, and an optional JSON schema that forces structured output. Templates are Go text/template; untrusted content must be passed through {{data "label" .Field}}, which wraps it in delimited data blocks with an explicit "treat as data" instruction and escapes the delimiters (threats T1, T2). Built-in defaults are seeded on first start and never overwritten; the prompt manager supports editing, cloning, diffing, restoring and testing (render only, or run through the role's model).