morluto/rea

Fork 0

mirror of https://github.com/morluto/rea.git synced 2026-11-05 06:45:22 +00:00

[PR #5781] [MERGED] perf(model): avoid decoding omitted GGUF strings #7055

New issue

Closed

opened 2026-11-01 21:45:02 +00:00 by bsakaguchi · 0 comments

bsakaguchi commented

2026-11-01 21:45:02 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/morluto/rea/pull/5781
Author: @gensokyo-dev
Created: 10/21/2026
Status: ✅ Merged
Merged: 10/21/2026
Merged by: @gensokyo-dev

Base: main ← Head: perf/5774-gguf-omitted-utf8

📝 Commits (4)

2f091cc perf(model): validate omitted GGUF strings without decoding
93daf90 docs(perf): record model validation costs and reconcile coverage
09dccb1 docs(perf): qualify integrated validation and source reproduction
3b2cc90 fix(perf): preserve failed model phase receipts through cleanup

📊 Changes

10 files changed (+1064 additions, -1 deletions)

View changed files

📝 experiments/analysis-efficiency/README.md (+8 -0)
➕ experiments/analysis-efficiency/coverage-review-20261021.md (+59 -0)
➕ experiments/analysis-efficiency/model-utf8-validation.json (+435 -0)
➕ experiments/analysis-efficiency/model-utf8-validation.md (+147 -0)
➕ experiments/analysis-efficiency/model-utf8-validation.mjs (+243 -0)
📝 experiments/analysis-efficiency/retained-capture-validation.md (+1 -1)
📝 experiments/analysis-efficiency/retained-query-finalization.mjs (+10 -0)
➕ experiments/analysis-efficiency/test_model_utf8_validation.py (+68 -0)
📝 src/model/GgufHeader.ts (+8 -0)
📝 tests/boundary/model/modelArtifact.test.ts (+85 -0)

📄 Description

Summary

Fixes #5774. GGUF arrays retain eight string previews, but previously decoded every omitted vocabulary entry into discarded text. Validate omitted single-chunk strings with Node's isUtf8; keep the fatal streaming decoder for longer values and all retained strings.

The owning parser still validates and hashes every admitted byte. Preview values/BOMs, offsets, coverage, partial progress, cancellation fairness, Evidence identity and complete retained access stay intact. CLI/MCP share the producer; no cache, API/default, dependency, installation or authority change. Existing read-only static-file scope; no new maintainer scope decision applies.

Executed evidence

Three real public models, 205 tensors / 41 metadata rows, match official gguf 0.19.0 / safetensors 0.8.0 readers and independent byte/hash checks. All before/after complete reports, selected answers, parent identities and catalog digests agree.
18 complete production stdio SDK tasks +18 provider processes, alternating three pairs per artifact; one CPU,4 GiB hard /3 GiB early-stop family bounds, no swap. Every payload exit/counter/cleanup qualifies, no limit events.
68 tests /4 files passed: producer formats/bounds, five new omitted-UTF-8 cases, partial cancellation, actual MCP and compiled CLI/export/retention. All 68 pass again after integration onto main 71b696582. Three real-model public CLI complete reports and three final SDK tasks also match the original reports.

Median	Baseline → candidate
Main GGUF first /repeat inspection	154.03 /53.58 →121.41 /37.19 ms
Seven-inspection family CPU	0.855 →0.729 s
Whole SDK task wall /CPU	6.667 /6.498 →6.567 /6.386 s
Whole SDK task peak family memory	771.0 →816.3 MiB

Phase gain is about21% first /31% repeated. No stable whole-task speedup or memory benefit claimed: the unchanged safetensors control also varies; main memory rises (769.3–799.0 →813.6–818.9 MiB, cause unmeasured). Whole catalog +all requests/replies estimates895,583 tokens for the main model in both builds (tiktoken0.12 cl100k_base), not actual model usage. No token saving. Fresh processes have unflushed OS caches; cgroup memory includes file cache. Terminal whole-scope counters remain unknown.

Measurements and reproduction pin original source/runtime/artifact/driver digests, all samples, controls and failed harness/test attempts. Later integration does not relabel the cohort. Coverage reconciliation maps46 directories /212 tools to existing evidence and narrower gaps, including prior model, IL2CPP, firmware and IDA performance work; it is not an all-tool runtime qualification. Raw binaries/reports/transcripts remain outside Git.

Validation performed

npm run check: typecheck, lint/module boundaries, format, metadata and compile pass; fails Knip on inherited g++ in src/windows/WindowsHandleClose.test.ts from #5772. Independent main source plus matching worker artifacts and uncached Knip reproduces only that finding. Left unchanged as unrelated.
npm run build:cached, npm run docs:check (5 tests), and separate npm run verify:architecture-guards: pass.
Exact sequential built-project runner:68 tests /4 files pass on integrated main. Real CLI/SDK verification passes for all three public models. Guard exit/counters/cleanup verified; no OOM. Development checks use6/8 GiB hard bounds; runtime tasks remain4 GiB.
Actual model inference/billing, larger vocabularies, Windows/macOS model parsing and cold disks unmeasured. Codex completed the original head with no findings. Greptile found a phase-receipt defect, fixed in 3b2cc90f7: 3 red→green failure regressions +11 shared receipt tests pass; real main-model MCP/provider reruns pass. The follow-up changes experimental receipt handling only; production bytes and original timing pins remain unchanged. Final head 3b2cc90f7: Codex has no findings, Greptile5/5, all review threads resolved; Static checks, Source tests, Build package and PR readiness pass. Merged 7d8e4e43 using authorized admin bypass after normal merge was refused by branch policy. At merge, Windows native controls and all8 full test shards remained queued/running; none had failed. These remaining jobs are not claimed as passed.

Author readiness

Human owner: @gensokyo-dev

The following owner declarations are transcribed at the human owner's explicit standing instruction in this session. This records that instruction, not an independently witnessed human review by the agent.

I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
I declare this PR ready for initial maintainer review.

Before merge

This PR addresses a concrete problem or an agreed enhancement.
Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
The PR has one focused outcome and the title follows type(scope): outcome.
Related issue is linked, or the reason for not linking one is stated above.
Tests cover changed observable behavior and meaningful failure paths.
Owning docs, contracts, and generated metadata are updated where needed.
User-visible CLI/MCP changes include representative output. Output is unchanged and verified against complete exports.
I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/morluto/rea/pull/5781 **Author:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Created:** 10/21/2026 **Status:** ✅ Merged **Merged:** 10/21/2026 **Merged by:** [@gensokyo-dev](https://github.com/gensokyo-dev) **Base:** `main` ← **Head:** `perf/5774-gguf-omitted-utf8` --- ### 📝 Commits (4) - [`2f091cc`](https://github.com/morluto/rea/commit/2f091ccc734d7bd945bd61432d41cf5ba68be12b) perf(model): validate omitted GGUF strings without decoding - [`93daf90`](https://github.com/morluto/rea/commit/93daf90b5b5e45354fb549809ca512bddb03641c) docs(perf): record model validation costs and reconcile coverage - [`09dccb1`](https://github.com/morluto/rea/commit/09dccb12fd229254462fe22901ee3936b39eae65) docs(perf): qualify integrated validation and source reproduction - [`3b2cc90`](https://github.com/morluto/rea/commit/3b2cc90f77e023a10f1b96e27eb3685c9022b017) fix(perf): preserve failed model phase receipts through cleanup ### 📊 Changes **10 files changed** (+1064 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `experiments/analysis-efficiency/README.md` (+8 -0) ➕ `experiments/analysis-efficiency/coverage-review-20261021.md` (+59 -0) ➕ `experiments/analysis-efficiency/model-utf8-validation.json` (+435 -0) ➕ `experiments/analysis-efficiency/model-utf8-validation.md` (+147 -0) ➕ `experiments/analysis-efficiency/model-utf8-validation.mjs` (+243 -0) 📝 `experiments/analysis-efficiency/retained-capture-validation.md` (+1 -1) 📝 `experiments/analysis-efficiency/retained-query-finalization.mjs` (+10 -0) ➕ `experiments/analysis-efficiency/test_model_utf8_validation.py` (+68 -0) 📝 `src/model/GgufHeader.ts` (+8 -0) 📝 `tests/boundary/model/modelArtifact.test.ts` (+85 -0) </details> ### 📄 Description ## Summary Fixes #5774. GGUF arrays retain eight string previews, but previously decoded every omitted vocabulary entry into discarded text. Validate omitted single-chunk strings with Node's `isUtf8`; keep the fatal streaming decoder for longer values and all retained strings. The owning parser still validates and hashes every admitted byte. Preview values/BOMs, offsets, coverage, partial progress, cancellation fairness, Evidence identity and complete retained access stay intact. CLI/MCP share the producer; no cache, API/default, dependency, installation or authority change. Existing read-only static-file scope; no new maintainer scope decision applies. ## Executed evidence - Three real public models, **205 tensors / 41 metadata rows**, match official gguf 0.19.0 / safetensors 0.8.0 readers and independent byte/hash checks. All before/after complete reports, selected answers, parent identities and catalog digests agree. - **18 complete production stdio SDK tasks +18 provider processes**, alternating three pairs per artifact; one CPU,4 GiB hard /3 GiB early-stop family bounds, no swap. Every payload exit/counter/cleanup qualifies, no limit events. - **68 tests /4 files passed**: producer formats/bounds, five new omitted-UTF-8 cases, partial cancellation, actual MCP and compiled CLI/export/retention. All 68 pass again after integration onto main `71b696582`. Three real-model public CLI complete reports and three final SDK tasks also match the original reports. | Median | Baseline → candidate | | --- | ---: | | Main GGUF first /repeat inspection |154.03 /53.58 →121.41 /37.19 ms| | Seven-inspection family CPU |0.855 →0.729 s| | Whole SDK task wall /CPU |6.667 /6.498 →6.567 /6.386 s| | Whole SDK task peak family memory |771.0 →816.3 MiB| Phase gain is about21% first /31% repeated. **No stable whole-task speedup or memory benefit claimed**: the unchanged safetensors control also varies; main memory rises (769.3–799.0 →813.6–818.9 MiB, cause unmeasured). Whole catalog +all requests/replies estimates895,583 tokens for the main model in both builds (tiktoken0.12 `cl100k_base`), not actual model usage. No token saving. Fresh processes have unflushed OS caches; cgroup memory includes file cache. Terminal whole-scope counters remain unknown. [Measurements and reproduction](https://github.com/morluto/rea/blob/3b2cc90f77e023a10f1b96e27eb3685c9022b017/experiments/analysis-efficiency/model-utf8-validation.md) pin original source/runtime/artifact/driver digests, all samples, controls and failed harness/test attempts. Later integration does not relabel the cohort. [Coverage reconciliation](https://github.com/morluto/rea/blob/3b2cc90f77e023a10f1b96e27eb3685c9022b017/experiments/analysis-efficiency/coverage-review-20261021.md) maps46 directories /212 tools to existing evidence and narrower gaps, including prior model, IL2CPP, firmware and IDA performance work; it is not an all-tool runtime qualification. Raw binaries/reports/transcripts remain outside Git. ## Validation performed - `npm run check`: typecheck, lint/module boundaries, format, metadata and compile pass; **fails Knip** on inherited `g++` in `src/windows/WindowsHandleClose.test.ts` from #5772. Independent main source plus matching worker artifacts and uncached Knip reproduces only that finding. Left unchanged as unrelated. - `npm run build:cached`, `npm run docs:check` (5 tests), and separate `npm run verify:architecture-guards`: pass. - Exact sequential built-project runner:68 tests /4 files pass on integrated main. Real CLI/SDK verification passes for all three public models. Guard exit/counters/cleanup verified; no OOM. Development checks use6/8 GiB hard bounds; runtime tasks remain4 GiB. - Actual model inference/billing, larger vocabularies, Windows/macOS model parsing and cold disks unmeasured. Codex completed the original head with no findings. Greptile found a phase-receipt defect, fixed in `3b2cc90f7`: 3 red→green failure regressions +11 shared receipt tests pass; real main-model MCP/provider reruns pass. The follow-up changes experimental receipt handling only; production bytes and original timing pins remain unchanged. Final head `3b2cc90f7`: Codex has no findings, Greptile5/5, all review threads resolved; Static checks, Source tests, Build package and PR readiness pass. Merged `7d8e4e43` using authorized admin bypass after normal merge was refused by branch policy. At merge, Windows native controls and all8 full test shards remained queued/running; none had failed. These remaining jobs are not claimed as passed. ## Author readiness Human owner: @gensokyo-dev The following owner declarations are transcribed at the human owner's explicit standing instruction in this session. This records that instruction, not an independently witnessed human review by the agent. - [x] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work. - [x] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps. - [x] I declare this PR ready for initial maintainer review. ## Before merge - [x] This PR addresses a concrete problem or an agreed enhancement. - [x] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff. - [x] The PR has one focused outcome and the title follows `type(scope): outcome`. - [x] Related issue is linked, or the reason for not linking one is stated above. - [x] Tests cover changed observable behavior and meaningful failure paths. - [x] Owning docs, contracts, and generated metadata are updated where needed. - [x] User-visible CLI/MCP changes include representative output. Output is unchanged and verified against complete exports. - [x] I checked the final diff for secrets, unrelated cleanup, and unsupported claims. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

bsakaguchi

2026-11-01 21:45:02 +00:00

closed this issue
added the
pull-request
label

bsakaguchi referenced this issue

2026-11-03 13:39:05 +00:00

[PR #7055] [CLOSED] fix(build): normalize release dependency patch directory for git apply #7608

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

morluto/rea#7055

No description provided.

Rows
Columns