AI Models

One pinned model (2026-10-07)

Every model call the platform makes — Cortex chat, log analysis, the Warden judge, image transcription (vision) and code fixes — goes to one model, deepseek-v4.1-flash, on NeuralWatt. It is pinned by its own id, never by the provider’s tracking alias nw-flash: an alias moves to a newer model with no deploy here, and the answers move with it.

Service	Where the id is set	Default
ai-service	src/config/env.ts PRIMARY_MODEL = LLM_MODEL, else legacy GROQ_MODEL	deepseek-v4.1-flash
ai-service vision	src/config/values.attachments.ts llmVisionModel = LLM_VISION_MODEL	PRIMARY_MODEL. Set to "" to make the deployment decline images
ai-service cheap / strong tiers	src/config/values.models.ts LLM_CHEAP_MODEL, LLM_STRONG_MODEL	PRIMARY_MODEL — cost is split by lane, not by model
warden (ai-service repo, Dockerfile.warden)	src/warden/config.ts WARDEN_JUDGE_MODEL	the ai-service LLM_MODEL
analyzer	src/core/providers/NeuralWattProvider.ts DEFAULT_MODEL; LLM_MODEL, else NEURALWATT_MODEL	deepseek-v4.1-flash
analyzer Warden judge	src/warden/config.ts WARDEN_JUDGE_MODEL	the analyzer LLM_MODEL
backend	ByoLlmService byo.platform.model (what Settings shows as “Logrite (built-in)”)	deepseek-v4.1-flash

The measurement behind the choice is in the comment on PRIMARY_MODEL (2026-09-24, 5 models × 14 real transcripts × 2 reps on the production findings prompt): deepseek-v4.1-flash 100.0 % correct, 92.9 % run-to-run consistent, 0 false positives; the previous default deepseek-v4-flash 92.9 % / 92.9 %.

It is a reasoning model

At its default (high) effort deepseek-v4.1-flash spends 590–1,351 completion tokens on reasoning_content before the first word of the answer, and the provider’s max_tokens caps reasoning and answer together. Under a 1,500 ceiling it returned empty answers on large evidence tables. That is why every call names its reasoning budget (below) and why callOptions adds a reasoning allowance on top of the answer budget.

Model resolution order

Both ai-service (src/core/registry/modelResolver.ts) and the analyzer resolve a role’s model in this order, highest first:

1. a workspace’s own BYO-LLM credential (ai-service: the request’s provider bundle wins before the registry is consulted);
2. an env var for the role — LLM_MODEL / GROQ_MODEL (chat), LLM_CHEAP_MODEL, LLM_STRONG_MODEL, WARDEN_JUDGE_MODEL (judge), LLM_ANALYSIS_MODEL, LLM_CODE_FIX_MODEL. Any of these being set, even to the value the registry would give, means the registry gets no vote for that role;
3. the backend model registry’s rank-0 row for the role (ai_model_role);
4. the compiled-in default above.

The analyzer’s analysis role is baseline-gated: a registry row that would move the analysis model while it has requires_baseline set is refused (logged REFUSED analysis model change) unless ALLOW_UNVERIFIED_ANALYSIS_MODEL=true, because the 28-language quality harness pins its digests to the model that produced them.

One key, five slots (2026-10-14)

Every estate (prod and dev ai-service, the analyzers, and the demo seeders’ relay provider) uses the same NeuralWatt key (identical sha256 prefix), and that key has 5 concurrent slots. A burst anywhere starves everything: on 2026-10-14 the dev analyzer’s agent-scan judge made ~1,100 calls in 10 minutes and prod Cortex answered “The AI step is unavailable right now” (429, concurrent_budget_exceeded: 5/5 slots in use). See Analyzer → the judge flood. The provider’s x-budget-remaining-usd header read 0.54 that day: check the balance before a demo.

The analyzer now holds itself to 3 of the 5 (analyzer 9ee4eda). providers/providerSlots.ts caps generateCompletion at LLM_MAX_CONCURRENT (default 3, 0 = no cap) per process, serves an interactive call (a custom-rule preview) before queued background work, and treats a 429 as busy, not down: it retries with backoff (Retry-After honoured, 15 s cap) for 20 s (interactive) or 90 s (background), then throws ProviderBusyError. /rules/propose answers that as 503 model_busy with Retry-After: 60, and the custom-rule preview says “the assistant is busy (the AI model is at capacity). Try again in a minute.” instead of “unreachable” (backend CustomRuleService.providerBusy). The warden judge’s chatCompletion path is not capped — a queue in front of a live request’s decision would be worse than a fast refusal.

The flex lane (2026-10-07)

The flex lane is the same model and weights on NeuralWatt’s discounted best-effort lane, sent as service_tier:"flex". It may queue for capacity; it never changes what writes the answer. Omitting service_tier is the standard lane.

Nothing a request waits for runs on flex. Measured 2026-10-07 from inside the ai-service container on both estates, an 8-token reply took 55–65 s on flex against 3–6 s on standard. Three awaited calls had been filed as background work and were moved off flex in ai-service 3302f72: the rolling conversation summary (the chat turn awaits it once a conversation passes the verbatim window), the hand-off brief (“continue in a new conversation” awaits it) and the upload summary prose (the sync upload response awaits it inside a 12 s budget, so on flex every upload waited 12 s and kept its original text).

Interactive chat never falls back to flex (2026-10-09). For the chat call class the flex rung of ResilientProvider is skipped: a failed standard call throws, and Cortex answers from the facts it already holds with answered:false, unanswered_reason:"provider_unavailable" (see AI service → Cortex close-out). Before, a chat turn could wait ~69 s on flex. Background work keeps the flex lane; the out-of-band Warden judge has its own 120 s budget (WARDEN_JUDGE_ASYNC_TIMEOUT_MS).

Request body fields (ai-service src/core/providers/wireOptions.ts, used by OpenAICompatibleProvider.baseBody):

Field	Value
max_tokens	maxTokens, else 4096
temperature	the call class’s value, else 0
top_p	1
reasoning_effort	the call class’s level
service_tier	"flex" only on the flex lane
response_format	{"type":"json_schema","json_schema":{"name":…,"schema":…,"strict":true}} when the call passes a schema
seed	still sent when the caller passes one (LLM_SEED, default 42). NeuralWatt documents no seed; nothing relies on it

The analyzer builds the same shape in src/core/providers/neuralWattBody.ts: temperature 0, top_p 1, seed 7, reasoning_effort defaults to low, service_tier:"flex" only when asked, strict json_schema when a schema is passed (else the older json_object hint), max_tokens default 8000.

Call policy (2026-10-07)

CALL_POLICY in ai-service src/config/values.models.ts has one row per class of model call. Every call site passes its row instead of literal options. The model is the same for every row.

Class	Lane	reasoning_effort	Temperature	Call sites
chat	standard	low	0	Agent loop and graph final pass (core/AgentLoop.ts, core/agent/graph.ts), chatEngine/runV1.ts, sync.ts, v2/run.ts, agentRun.ts, emit.ts (synthesis, re-address), activity.ts, v2/uploadAnswer.ts, synthesize/synthesize.ts, fuseTurn.ts, verify/critic/llmCritic.ts, verify/repair.ts, routes/agentBuilder.routes.ts, rolling and hand-off summaries (core/context/rollingSummary.ts, handoffSummary.ts)
route	standard	none	0	intent/llmBackstop.ts, synthesize/openEndedPlanner.ts, safety/guardrail.ts, utils/dynamicFollowUps.ts
smalltalk	standard	none	0.5	chatEngine/conversational.ts
judgeSync	standard	low	0	Warden inline judge (warden/judge.ts, the default class)
judgeAsync	flex	low	0	Warden out-of-band judge queue (warden/asyncJudgeQueue.ts)
analysisAsync	flex	high	0	Log-analysis findings when no caller is waiting (services/logAnalysis/llmFindings.ts, background = no deadline), compliance rule synthesis (compliance-streams/synthesize/ruleSynthesizer.ts)
analysisSync	standard	high	0	Log-analysis findings when a caller holds the connection open (llmFindings.ts), upload summary prose (execSummary.prose.ts)
vision	standard	low	0	Image transcription (core/attachments/visionExtract.ts)
codeFix	standard	high	0	routes/codeFixes/runCodeFix.ts

In the analyzer, the coding-issue write-up (src/api/routes/describeCodingIssue.ts) runs on the flex lane with reasoning_effort: "low"; the Warden judge runs low, temperature 0, standard lane first. KeyFindings generation (KeyFindingsGenerator, PostGenerationKeyFindingsGenerator, ComplianceKeyFindingsGenerator) sends temperature: 0.

The judge's tool_choice must be 'auto' (2026-10-08)

NeuralWatt answers a named ({type:"function", function:{name:"emit_verdicts"}}) or "required" tool_choice for deepseek-v4.1-flash with a Cloudflare 502, on every call. Both Warden judges forced the named form, so from the 2026-10-07 consolidation until ai-service 58df213 / analyzer e027863 no inline judge reached a verdict: the ai-service judge retried the 502 until its 6-second lane budget aborted it (judge.outcome: timeout, 52 of 52 CareVault requests on prod), and every judge-only policy — 42 CFR Part 2, instruction overrides — failed open. Both now send tool_choice: "auto" (measured 2.7 s, one emit_verdicts call) and treat a reply without that call as a lane error, never as an empty verdict that composes to ALLOW. Check judge.outcome in warden-requests-<ws>: decided is healthy, a run of timeout is this. not_requested means the binding had nothing for the judge (since 2026-10-16 a deterministic-only binding no longer reports decided).

callOptions and the reasoning allowance

callOptions(cls, answerTokens?) returns the class’s row plus maxTokens = answerTokens + REASONING_TOKEN_ALLOWANCE[reasoning_effort]. Omit answerTokens to take the provider default ceiling. A call site states the answer it needs; the allowance covers the thinking before it:

reasoning_effort	Allowance (tokens)
none	0
low	768
high	2048
xhigh	4096
max	8192

PROMPT_VERSION and ANALYSIS_VERSION

• PROMPT_VERSION (src/config/values.models.ts, currently 2026-10-07.1) is part of every Cortex answer-cache key. Bump it whenever a prompt changes what an answer can say.
• ANALYSIS_VERSION (src/config/values.logAnalysis.ts, currently 2026-10-07.1) is part of the upload-reuse key: an identical re-upload is answered from the stored analysis only when that analysis was produced by this version. The reuse key does not contain the model id.

Same-model fallback only (2026-10-07)

There is no fallback model anywhere. ai-service AIProviderFactory.buildResilient(model) builds [model on the standard lane, the same model on the flex lane] — only for AI_PROVIDER=neuralwatt; any other provider runs with no chain. Usage is metered once per logical call, outside the chain.

ResilientProvider (src/core/providers/ResilientProvider.ts):

1. calls the standard lane;
2. on a rate-limit error (429, rate limit, too many requests, over capacity, overloaded, resource_exhausted, quota) waits a jittered 250–600 ms and retries the same lane once (non-streaming calls only);
3. on a transient error (429/500/502/503/504, timeouts, connection resets, unavailable…) sends the call to the same model on the flex lane (FlexLaneProvider), which notes tier:"flex" on the turn. A streaming call falls back only if nothing was streamed yet.

Vision uses the same two rungs (AIProviderFactory.getVisionProviders()); a BYO workspace gets one rung, its own model, and only if that model can see. The ai-service Warden inline judge runs standard then flex; the out-of-band judge runs flex only. The analyzer’s judge walks JUDGE_TIERS = ['standard', 'flex'] on the same model and retries a 429 after 1 s, 2 s, 4 s.

Deleted env keys — no longer read by any service; remove them from overlays: LLM_FALLBACK_MODEL, LLM_FALLBACK_MODELS, LLM_VISION_FALLBACK_MODELS, WARDEN_JUDGE_LADDER, WARDEN_JUDGE_FALLBACK_MODEL. The analyzer’s Warden judge builds a NeuralWattProvider only; it has no Groq path and does not read GROQ_MODEL (the analyzer’s AI_PROVIDER=groq option for analysis still does).

The sample overlay still carries a deleted key

MasterControl ops/deploy/docker-compose.env.yml still sets LLM_VISION_FALLBACK_MODELS under ai-service. The code no longer reads it; it changes nothing.

When both lanes fail

Interactive paths that already hold measured facts answer with those facts followed by AI_UNAVAILABLE_NOTE (src/core/consistency/aiUnavailable.ts, re-exported from core/chatEngine/emit.ts):

The AI step is unavailable right now; these are the measured facts.

Used by the open-ended synthesis (model: "deterministic-facts"), the upload answer (v2/uploadAnswer.ts, model: "upload-findings") and the activity answer (chatEngine/activity.ts). The upload-answer cache never keeps an answer that carries this note, so the next asking tries the model again.

Vision and the live model catalogue (2026-10-07)

Vision is the same model (LLM_VISION_MODEL defaults to PRIMARY_MODEL). Probing showed a non-vision model given an image does not fail — one returned five invented lines — so a cross-model vision ladder could replace a refusal with fabrication.

src/core/providers/modelCatalogue.ts (ai-service and warden) loads the provider’s catalogue at boot and every 6 h:

GET {NEURALWATT_BASE_URL}/models     # default base https://api.neuralwatt.com/v1, so …/v1/models
Authorization: Bearer $NEURALWATT_API_KEY

• Only an answer with scope:"customer" counts. Without a valid key the endpoint still answers 200 with scope:"public"; that is logged MODEL_CATALOGUE_UNAUTHENTICATED and the last good copy is kept. Any other failure also keeps the last good copy. No copy at all = no opinion.
• Per model it reads metadata.capabilities.vision, metadata.limits.max_images / max_images_total and metadata.reasoning.supported_efforts.
• modelSupportsVision asks the catalogue first, then falls back to the configured LLM_VISION_MODEL or a known vision-family id pattern.
• Images attached to a chat turn are clamped to the answering model’s max_images per turn (chatEngine/attachments.ts); the transcripts of every image are still in the context.
• A configured id (chat, cheap, strong, vision, WARDEN_JUDGE_MODEL) missing from the customer-scope catalogue logs MODEL_NOT_IN_CATALOGUE model=… and turns /health degraded:

{"status":"degraded","reasons":["model deepseek-v4.1-flash not in provider catalogue"], …}

ai-service /health stays HTTP 200 when degraded (503 only when the provider is not configured) and reports ai_provider.catalogue_loaded_at. The warden relay’s /health says ok or degraded with the same reasons. The analyzer runs the same check (src/core/providers/modelCatalogue.ts) over its analysis model and its judge model; its /health turns degraded with reasons and carries modelCatalogue (loaded, models, loadedAt, lastError).

Same question, same answer (2026-10-07)

The provider documents no seed, so repeatable answers come from facts and caches, not sampling. src/core/consistency/cortexCache.ts stores the first verifier-clean answer under a key made of everything that can change it, and replays it byte for byte.

Store. Redis via src/shared/redis.ts, reading REDIS_URL, else WARDEN_REDIS_URL (e.g. redis://host:6379/1 — REDIS_URL wins so ai-service and the relay can share one server on different databases). Keys cortex:<namespace>:<sha256>, TTL 24 h. Redis is never on the correctness path: down or unconfigured, a per-process Map (500 entries) answers instead and the turn reports cache:"local".

Namespace	Variable	What it caches	Sets message_end.metadata.cache
agent	agentAnswerCache (chatEngine/v2/agentAnswerCache.ts)	agent-path answers — see eligibility below	yes
upload	uploadAnswerCache (chatEngine/v2/uploadAnswer.ts)	answers about an uploaded file, keyed on the question, the analysis, its facts_digest and the model. An “AI unavailable” fallback is never kept	yes
logfindings	findingsCache (services/logAnalysis/modelPassCall.ts)	the log-analysis model pass’s findings	no
open-ended	answerCache (synthesize/answerCache.ts)	open-ended answers. Stored only when verification is ok; follow-ups bypass it entirely	yes
deterministic	detCache (intent/dataQueries/answerCache.ts)	no-LLM answers (performance, activity, workspace performance)	yes
plan	planCache (synthesize/openEndedPlanner.ts)	the planner’s primitives and window for a question	only local
route	routeCache (intent/llmBackstop.ts)	the LLM backstop’s intent route	only local

agent, upload and logfindings were added on 2026-10-07; together they make the agent path, upload questions and the log-analysis model pass repeat their first verified result.

Which agent-path answers are stored. Only an answer that is all of:

• the first turn of a conversation;
• sent with no attachments;
• produced with every tool call succeeding;
• verifier ok, or verification off.

A turn that ran no tool is keyed without the data version, so a data-free answer survives new logs arriving. The knowledge-pack pages injected into the turn are part of the key. A turn that did read data is keyed on the data version of the pinned index, else of the caller workspace’s application-log and trace families (log-analytics, log-observability, log-cloud-platform, trace-analytics-raw — the set the empty-workspace guard reads). Until ai-service bfcf6b0 a Cortex or API-key turn, whose clientId is empty, resolved no patterns at all: every unpinned answer was keyed on none (data_version 140bedbf9c3f = sha256 of none, seen on both estates) and replayed for the rest of the day while new logs arrived.

The agent cache stored nothing before 2e07c8e

First-turn eligibility read the conversation history, which already held the question being asked — so no turn ever looked like a first turn and the agent cache never stored a single answer until ai-service 2e07c8e.

Key (cortexCacheKey, sha256 over canonical JSON):

• the canonical question — lower-cased, punctuation and whitespace collapsed;
• workspace, scope (with |pin: and the pinned index when one is pinned), analysis id;
• the relative window expression (last_24_hours), never the resolved from/to; a calendar expression (today, this week…) also carries its anchor date in the time zone (today@2026-10-06);
• timeZone (default UTC);
• the model id (deterministic-<intent> for the deterministic cache) and PROMPT_VERSION;
• dataVersion(...).key (src/core/consistency/dataVersion.ts): count and max(@timestamp) per index pattern read, cached 30 s. An unreadable pattern is count: null, never 0, so an outage cannot cache “you have no logs”. The plan and route caches pass an empty data version — they depend on the question alone;
• extras for the open-ended cache: verify mode, role, experiment key, and a sha256 of any attachment context.

message_end.metadata on every chat turn (messageEndMeta, src/core/consistency/turnMeta.ts):

Field	Meaning
handler	the branch that answered (falls back to the model label)
model	the model label the path reported
tier	standard, or flex when the flex lane answered
cache	hit, miss or local
facts_digest	sha256 of the canonical facts the answer was grounded on, or null
data_version	agent path only: the digest of the workspace data the answer was keyed on

Model output the answer never shows

The model sometimes answers in DeepSeek’s own tool-call markup (DSML) instead of the standard tool-call fields. When tools were offered, that markup is parsed into real tool calls; otherwise it is stripped, and the stream filter never shows it to the reader. A turn that offers no tools (the agent loop’s final pass) ends its prompt with a fixed note saying tools are unavailable, so the model answers in prose.

cortex_consistency.py

MasterControl ops/testing/cortex_consistency.py asks Cortex the same questions twice against a deployed estate and checks the answers carry the same facts. Stdlib only.

python3 ops/testing/cortex_consistency.py --estate dev [--repeat 2] [--json]
python3 ops/testing/cortex_consistency.py --estate prod --dry-run

Flag	Default
--estate	required	dev (workspace 3103, key from $LOGRITE_DEV_API_KEY) or prod (workspace 252, key from /home/deployd/.logrite-prod-api-key)
--repeat	2	asks per question; below 2 is refused
--time-zone	Asia/Kolkata	chat timeZone
--cache-dir	$XDG_CACHE_HOME/logrite or ~/.cache/logrite	where Hadoop_2k.log is kept
--json	off	machine-readable report
--dry-run	off	print the plan, no network

Reuse check (setup). It downloads Loghub Hadoop_2k.log (refused unless its sha256 matches) and uploads it twice to POST {ai}/api/v1/log-analytics/{workspace}?shape=batch. The second upload must answer status:"reused" with the same analysisId, datasetId and findings (ignoring the per-batch fileId).

Questions, each asked --repeat times in a fresh conversation through POST {ai}/api/v1/chatv2:

Id	Question
Q1	the Sandbox indexes preset — “List the indexes I can read.”
Q2	the recent-errors preset — “Show me the most recent error-level log lines and explain what they say.”
Q3	“What do the findings in {dataset} say?” (needs the upload)
Q4	“What went wrong in the Hadoop log I just uploaded?” (needs the upload)
Q5	“When did the errors peak in {dataset}?” (needs the upload)
Q6	the explain-trace preset (NullPointerException example trace)
Q7	the worksoft-cortex add-on’s first suggested prompt — skipped unless the add-on is enabled

Compared against repeat 1: handler, model, facts_digest, the “Likely trigger”, every number and every timestamp in the answer, and any error. cache and tier are printed but not compared — the second ask is supposed to be a hit, and flex is the same model.

Status per question:

Status	Meaning
MATCH	every repeat agrees with the first
MISMATCH	repeats disagree on the same data — a failure
MOVED	repeats differ and were keyed on different data_version values: the workspace data changed between the askings (a collector shipping every minute does this). Not a failure — new data is allowed a new answer
SKIP	the question does not apply (Q7 without the Worksoft add-on)
FAIL	a step failed — an error answer, or no uploaded dataset to ask about

Exit codes: 0 every repeat matched (or moved with its data); 1 a mismatch, a failed step, or the key appearing in a response; 2 no credential (before any network call).

Backend catalogue and registry (2026-10-07)

• service/byollm/ModelCatalog.java — the curated list for the BYO-LLM Settings page and the Warden provider catalog. NeuralWatt: deepseek-v4.1-flash, deepseek-v4.1-flash-flex, deepseek-v4-flash, glm-5.3-flash, qwen3.6-35b, kimi-k2.7-code.
• ByoLlmService defaults: byo.platform.provider = neuralwatt, byo.platform.model = deepseek-v4.1-flash.
• config/ModelConsolidationMigrator.java runs once on boot (marker model-consolidation-2026-10); resources/migrations/2026-10-model-consolidation.sql is its idempotent hand-run mirror. On an empty registry it does nothing (BillingCatalogSeeder writes the consolidated catalogue directly). Otherwise it:
  1. upserts ai_model deepseek-v4.1-flash (provider neuralwatt, capabilities jsonMode, toolUse, vision, reasoning, contextWindow 1,048,576), enabled;
  2. upserts its ai_model_rate: token basis, input 150,000, cached input 15,000, output 600,000 micro-USD per million tokens (0.15 / 0.015 / 0.60 USD per M), markup_bps 50000, path standard, effective 2026-10-07;
  3. upserts rank 0 of global roles analysis, chat, chat_cheap, chat_strong, code_fix, judge to the model. A new row gets requires_baseline only for analysis; an existing row keeps its requires_baseline as found, and its quality_baseline_digest is cleared only if the model actually changes. There is no vision role — vision uses the primary model;
  4. disables every global rank above 0 for those roles (the old judge rungs, rank 1 qwen3.6-35b-fast and rank 2 kimi-k2.7-code-fast), kept for the record, no longer served.

How to change the model

1. Check the provider serves it to this account. The new id must be in the customer-scope catalogue. After the change, /health on ai-service, warden and analyzer must not be degraded, and the logs must not show MODEL_NOT_IN_CATALOGUE. Check its supported_efforts include every reasoning_effort in the call-policy table, and vision if it will read images.

2. Set the env keys through the console (POST /api/env, see Environment variables), one request per service:

   Service	Keys
   ai-service	LLM_MODEL; LLM_VISION_MODEL if it is set explicitly (otherwise it follows LLM_MODEL)
   warden	LLM_MODEL, WARDEN_JUDGE_MODEL
   ingest-shim	LLM_MODEL, LLM_VISION_MODEL — it loads the same config module; no model call was found in src/shim/
   analyzer	LLM_MODEL; WARDEN_JUDGE_MODEL if set

   curl -s -X POST http://127.0.0.1:8090/api/env \
     -H 'Content-Type: application/json' \
     -d '{"host":"dev","service":"ai-service","changes":{"LLM_MODEL":"new-model-id"},"apply":true}'

   On "host":"prod" the recreate is refused unless the body also carries "confirm":"DEPLOY-TO-PRODUCTION". The request returns 202 with a job id.

3. Update the backend catalogue and registry — add the id to ModelCatalog’s NeuralWatt list, and write a migration in the shape of ModelConsolidationMigrator: ai_model row, ai_model_rate row, rank-0 ai_model_role rows. A service with no env override for a role takes the registry row; the analyzer refuses an analysis row with requires_baseline until the baselines are re-measured or ALLOW_UNVERIFIED_ANALYSIS_MODEL=true.

4. Bump PROMPT_VERSION and ANALYSIS_VERSION if the new model changes what an answer or an analysis says. The open-ended cache key already contains the model id, but the upload-reuse key does not: without an ANALYSIS_VERSION bump an identical re-upload replays the old model’s findings.

5. Re-run cortex_consistency.py against the estate and expect exit 0.

A new model is a new benchmark

The 2026-09-24 ranking was made with no output ceiling, and the model then returned empty answers under the production ceiling. Measure a candidate with the request production sends — same prompt, same max_tokens, same reasoning_effort — before pinning it.