[GH-ISSUE #6569] [Bug] Model summaries report no chat template for named-only GGUF templates #3958

New issue

Closed

opened 2026-10-31 00:45:35 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-10-31 00:45:35 +00:00

Owner

Originally created by @coygeek on GitHub (Oct 23, 2026).
Original GitHub issue: https://github.com/morluto/rea/issues/6569

Before submitting

I searched the open and closed issues for a related report.
I removed credentials, tokens, private paths, and sensitive target data.

Area

Evidence, artifacts, or reconstruction

Observed behavior

Source-only finding; runtime reproduction was not run. A GGUF containing tokenizer.chat_templates = ["tool_use"] and tokenizer.chat_template.tool_use = "{{ messages }}", without the default tokenizer.chat_template key, produces chat_template_present: false in the model summary even though its named template is retained in metadata. The same false flag appears in the compact model context attached to selected pages.

Reproduction

Proposed runtime check, not executed:

Produce a GGUF with a named template and no default, for example by calling GGUFWriter.add_chat_template([{"name": "tool_use", "template": "{{ messages }}"}]) in an otherwise valid GGUF-writing workflow.
Call MCP inspect_model_artifact with {"path":"/artifacts/named-template.gguf","detail":"summary"} in a normal REA session.
Inspect chat_template_present in the returned summary, then query the retained metadata for tokenizer.chat_template.tool_use.

The source proof follows that request through ModelArtifactService.inspect, the ordinary GGUF metadata reader, and summarizeModel; it does not depend on malformed input or a missing provider.

Expected behavior

The summary should report that a chat template exists when a named template is present. REA documents chat-template presence as part of the compact model summary, without restricting it to the default template. A model with neither default nor named templates should continue to report false.

Impact and scope

The default MCP summary gives a false negative for GGUF files with named templates only, which can cause an analyst or agent to skip existing prompt-format evidence. Full metadata remains available through complete detail or a retained metadata query. Runtime frequency and affected real-model population were not measured.

REA version

Source revision 46111dd091f2ae01baab9eb119b53b9bd309a05b on main; package version 6.7.0. Latest published npm release checked: 6.7.0.

Runtime and platform

Source inspection on Linux x86_64 with Node.js v26.12.1. REA and a model inference runtime were not executed.

Target and artifact context

GGUF v2/v3 with a valid string-array tokenizer.chat_templates entry and a corresponding string tokenizer.chat_template.tool_use entry, but no default template key.

Interface

MCP

Earliest failure stage

Transport or result projection

Relevant output or evidence

Summary projection computes the flag using only metadata.some((row) => row.key === "tokenizer.chat_template"). Both named-template keys fail this exact comparison. The same summary supplies page context.
Public inspection workflow projects summary requests through the retained-analysis summary.
GGUF metadata admission accepts and retains these keys and their values; there is no requirement for a default template.
Documented summary contract includes chat-template presence.
GGUF writer writes named template strings and their name array, then returns without writing a default template when none was supplied. This establishes a producer-supported trigger.

No CLI output, MCP response, or runtime fixture result is claimed.

Typed outcome or recovery expected

Keep the summary's presence flag consistent with retained template metadata. Direct metadata queries provide the existing workaround.

Analysis or suspected cause

Source-derived cause: the presence test recognizes only the default template key. The input reader retains named templates, so the incorrect result first appears during summary projection.

Additional context

Duplicate searches covered chat_template_present, named template, and chat template. Existing feature requests #3653, #3679, and #3682 concern broader tokenizer, prompt-package, or multi-file model analysis rather than this existing summary flag. A proposed restoration check should compare named-only, default-template, and no-template files without changing retained metadata.

Triage assessment

Impact level: P2
Assessment status: supported
Impact basis: A supported GGUF metadata combination deterministically produces a false summary flag according to the public call path and exact predicate; runtime population and frequency are unmeasured.
Workaround status: available
Workaround basis: The documented complete-detail and retained metadata queries expose the named template even when the compact flag is false.

Originally created by @coygeek on GitHub (Oct 23, 2026). Original GitHub issue: https://github.com/morluto/rea/issues/6569 ### Before submitting - [x] I searched the open and closed issues for a related report. - [x] I removed credentials, tokens, private paths, and sensitive target data. ### Area Evidence, artifacts, or reconstruction ### Observed behavior Source-only finding; runtime reproduction was not run. A GGUF containing `tokenizer.chat_templates = ["tool_use"]` and `tokenizer.chat_template.tool_use = "{{ messages }}"`, without the default `tokenizer.chat_template` key, produces `chat_template_present: false` in the model summary even though its named template is retained in metadata. The same false flag appears in the compact model context attached to selected pages. ### Reproduction Proposed runtime check, not executed: 1. Produce a GGUF with a named template and no default, for example by calling `GGUFWriter.add_chat_template([{"name": "tool_use", "template": "{{ messages }}"}])` in an otherwise valid GGUF-writing workflow. 2. Call MCP `inspect_model_artifact` with `{"path":"/artifacts/named-template.gguf","detail":"summary"}` in a normal REA session. 3. Inspect `chat_template_present` in the returned summary, then query the retained metadata for `tokenizer.chat_template.tool_use`. The source proof follows that request through `ModelArtifactService.inspect`, the ordinary GGUF metadata reader, and `summarizeModel`; it does not depend on malformed input or a missing provider. ### Expected behavior The summary should report that a chat template exists when a named template is present. REA documents chat-template presence as part of the compact model summary, without restricting it to the default template. A model with neither default nor named templates should continue to report false. ### Impact and scope The default MCP summary gives a false negative for GGUF files with named templates only, which can cause an analyst or agent to skip existing prompt-format evidence. Full metadata remains available through complete detail or a retained metadata query. Runtime frequency and affected real-model population were not measured. ### REA version Source revision `46111dd091f2ae01baab9eb119b53b9bd309a05b` on `main`; package version 6.7.0. Latest published npm release checked: 6.7.0. ### Runtime and platform Source inspection on Linux x86_64 with Node.js v26.12.1. REA and a model inference runtime were not executed. ### Target and artifact context GGUF v2/v3 with a valid string-array `tokenizer.chat_templates` entry and a corresponding string `tokenizer.chat_template.tool_use` entry, but no default template key. ### Interface MCP ### Earliest failure stage Transport or result projection ### Relevant output or evidence - [Summary projection](https://github.com/morluto/rea/blob/46111dd091f2ae01baab9eb119b53b9bd309a05b/src/domain/analysisView/modelArtifactView.ts#L73-L104) computes the flag using only `metadata.some((row) => row.key === "tokenizer.chat_template")`. Both named-template keys fail this exact comparison. The same summary supplies page context. - [Public inspection workflow](https://github.com/morluto/rea/blob/46111dd091f2ae01baab9eb119b53b9bd309a05b/src/application/model/ModelArtifactService.ts#L50-L97) projects summary requests through the retained-analysis summary. - [GGUF metadata admission](https://github.com/morluto/rea/blob/46111dd091f2ae01baab9eb119b53b9bd309a05b/src/model/GgufHeader.ts#L283-L323) accepts and retains these keys and their values; there is no requirement for a default template. - [Documented summary contract](https://github.com/morluto/rea/blob/46111dd091f2ae01baab9eb119b53b9bd309a05b/docs/model-artifacts.md#L81-L85) includes chat-template presence. - [GGUF writer](https://github.com/ggml-org/llama.cpp/blob/f5027fb21307eb264aed483242603d68c87b0fe3/gguf-py/gguf/gguf_writer.py#L1201-L1229) writes named template strings and their name array, then returns without writing a default template when none was supplied. This establishes a producer-supported trigger. No CLI output, MCP response, or runtime fixture result is claimed. ### Typed outcome or recovery expected Keep the summary's presence flag consistent with retained template metadata. Direct metadata queries provide the existing workaround. ### Analysis or suspected cause Source-derived cause: the presence test recognizes only the default template key. The input reader retains named templates, so the incorrect result first appears during summary projection. ### Additional context Duplicate searches covered `chat_template_present`, `named template`, and `chat template`. Existing feature requests #3653, #3679, and #3682 concern broader tokenizer, prompt-package, or multi-file model analysis rather than this existing summary flag. A proposed restoration check should compare named-only, default-template, and no-template files without changing retained metadata. ## Triage assessment - Impact level: P2 - Assessment status: supported - Impact basis: A supported GGUF metadata combination deterministically produces a false summary flag according to the public call path and exact predicate; runtime population and frequency are unmeasured. - Workaround status: available - Workaround basis: The documented complete-detail and retained metadata queries expose the named template even when the compact flag is false.

bsakaguchi closed this issue

2026-10-31 00:45:36 +00:00

bsakaguchi referenced this issue

2026-10-31 01:11:36 +00:00

[PR #4118] [MERGED] fix(apple): keep Interface Builder objects whose archived class is unresolved #6068